fix: response-shape error classes and one vector codec path

docs/vectors.md

@@ -15,8 +15,8 @@ NodeDB's vector engine is built for production semantic search — not vectors s
 - **Quantization frontier**:
   - **SQ8** — Scalar quantization to 8-bit. ~4× memory reduction, minimal recall loss.
   - **PQ** — Classic product quantization codec. ~4-8× / ~16 bytes/vector for very large indexes.
-  - **OPQ** — PQ + learned random rotation; minor accuracy bump over plain PQ.
-  - **Binary** — Sign-bit Hamming for ultra-cold tiers (no rerank).
+  - **OPQ** — PQ after a learned (Procrustes-fitted) rotation; higher recall than PQ at the same code size.
+  - **Binary** — Sign-bit Hamming candidates, reranked by exact distance.
   - **Ternary** — BitNet 1.58 trit-pack `{-1, 0, +1}` with cold/hot pack and AVX-512 popcnt.
   - **RaBitQ** — 1-bit quantization with `O(1/√D)` error bound (SIGMOD 2024). Frontier 1-bit primary path.
   - **BBQ** — Centroid-asymmetric 1-bit + 14-byte corrective factors + oversample rerank.
@@ -75,13 +75,15 @@ LIMIT 10;
 | Argument            | Type   | Notes                                                                                       |
 | ------------------- | ------ | ------------------------------------------------------------------------------------------- |
 | `quantization`      | string | `none`, `sq8`, `pq`, `binary`, `ternary`, `rabitq`, `bbq`, `opq`                            |
-| `oversample`        | u8     | Candidates fetched before rerank. Default `3`. Final rerank set = `oversample × ef_search`. |
+| `oversample`        | u8     | Rerank candidates kept per requested row. Default `3`. Rerank set = `top_k × oversample`; the HNSW beam widens to at least that set. |
 | `query_dim`         | u32    | Coarse-to-fine on first-N dims of Matryoshka embeddings.                                    |
 | `meta_token_budget` | u8     | MetaEmbed multivec MaxSim / PLAID budget.                                                   |
-| `ef_search`         | u32    | HNSW / Vamana beam width. Default `64`.                                                     |
-| `target_recall`     | f32    | Adaptive recall target — cost-model planner picks `oversample` and `ef_search` to hit it.   |
+| `ef_search`         | u32    | HNSW beam width. Default `2 × LIMIT`.                                                       |
+| `target_recall`     | f32    | Recall target in `(0.0, 1.0]`. Above `0.80` it scales `ef_search` up to 5× and `oversample` up to √5×. |
 
-Set `target_recall` and let the planner do the rest unless you need a hard latency ceiling.
+Set `target_recall` to widen a search toward a recall target. Set `ef_search` and `oversample` for a hard latency ceiling.
+
+The `quantization` argument names the codec the index was created with. A value that names another codec is refused with a data exception. The codec is fixed at `CREATE`.
 
 ## Sparse Vectors
 
@@ -146,15 +148,15 @@ Default `primary='document'` is unchanged: classic `CREATE VECTOR INDEX ON ...`
 | `none`    | 32       | 100%          | Small index (< 1M vectors), latency not critical         |
 | `sq8`     | 8        | ~99%          | Balanced default for medium index sizes                  |
 | `pq`      | ~2       | ~95%          | Large memory-bound indexes; classic Product Quantization |
-| `opq`     | ~2       | ~96%          | PQ + learned random rotation                             |
+| `opq`     | ~2       | ~96%          | PQ after a learned rotation                              |
 | `rabitq`  | 1        | ~97%          | Frontier 1-bit with `O(1/√D)` error bound                |
 | `bbq`     | 1        | ~98%          | Centroid-asymmetric 1-bit + 14-byte corrective           |
-| `binary`  | 1        | ~85%          | Hamming-only, ultra-cold tiers                           |
+| `binary`  | 1        | ~85%          | Hamming candidates + exact rerank                        |
 | `ternary` | 1.58     | ~96%          | BitNet `{-1, 0, +1}` cold/hot pack                       |
 
 ## How It Works
 
-Vectors are indexed in the HNSW graph at full precision to maintain structural integrity. During search, the engine traverses quantized copies of the graph for speed, then re-ranks the top candidates against full-precision vectors for accuracy.
+Vectors are indexed in the HNSW graph at full precision to maintain structural integrity. Each sealed segment carries one codec. SQ8 and PQ read a beam of `ef_search` HNSW candidates and keep the `top_k × oversample` nearest by code. RaBitQ, BBQ, Binary, Ternary, and OPQ search their own quantized graph. Every codec path reranks `top_k × oversample` candidates by exact distance. Filtered and point-in-time searches rank exact vectors. `none` stores no codec. An omitted `quantization` selects `sq8`. A codec that fails to build leaves the segment's HNSW graph installed without it: the segment answers exactly and `builds_failed` counts the error. A codec build the memory budget refuses retries with backoff. An IVF-PQ index searches by PQ codes, so it accepts the `pq` search hint.
 
 When a query includes metadata filters (e.g., `WHERE category = 'ml'`), the engine builds a Roaring Bitmap of matching document IDs and selects the optimal filtering strategy automatically — if the filter is highly selective, it pre-filters before HNSW traversal; if broad, it post-filters after.
 

nodedb-cluster-tests/tests/common_suite/cases/single_node_calvin_returning.rs

@@ -14,6 +14,10 @@
 //! implicit-edge document with `RETURNING *` returns that document's row through
 //! the completion path.
 //!
+//! The same path evaluates a RETURNING expression as the direct path does: a
+//! shaping error keeps its own SQLSTATE and native code, and `nextval` reads
+//! the session's sequences.
+//!
 //! File name contains "calvin" within the cluster-tests crate so nextest applies
 //! the cluster test-group serialization.
 
@@ -50,11 +54,10 @@ fn collection_edge_bearing(node: &TestClusterNode, coll: &str) -> bool {
         .unwrap_or(false)
 }
 
-/// Flag ON: a `DELETE ... RETURNING` on an edge-bearing collection routes
-/// through the single-node Calvin OLLP path and returns the deleted row through
-/// the completion path (previously dropped as a bare tag).
-#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
-async fn calvin_returning_delete_surfaces_deleted_row() {
+/// Spawn a single-node Calvin server and seed `coll` with [`SOURCES`]
+/// implicit-edge documents `edge_0..`, each carrying `n: 0`. Returns once the
+/// collection is edge-bearing, so a PK delete on it routes through Calvin.
+async fn calvin_edge_collection(coll: &str) -> TestClusterNode {
     // 4 Data-Plane cores so the edge endpoints land on distinct vShards — the
     // doc-delete participant carries the RETURNING rows while the edge-delete
     // participants carry none, exercising the deposit gate.
@@ -71,10 +74,6 @@ async fn calvin_returning_delete_surfaces_deleted_row() {
     )
     .await;
 
-    // The collection name deliberately embeds "returning": it doubles as
-    // regression coverage that a `*_returning` identifier is not mistaken for the
-    // RETURNING keyword by the clause stripper.
-    let coll = "sncalvin_returning";
     node.client
         .simple_query(&format!(
             "CREATE COLLECTION {coll} WITH (engine='document_schemaless')"
@@ -95,7 +94,7 @@ async fn calvin_returning_delete_surfaces_deleted_row() {
     for i in 0..SOURCES {
         node.client
             .simple_query(&format!(
-                "INSERT INTO {coll} {{ id: 'edge_{i}', _from: 'src_{i}', _to: 'hub', _type: 'l', mark: 'keep' }}"
+                "INSERT INTO {coll} {{ id: 'edge_{i}', _from: 'src_{i}', _to: 'hub', _type: 'l', mark: 'keep', n: 0 }}"
             ))
             .await
             .unwrap_or_else(|e| panic!("insert implicit edge src_{i} -> hub: {e}"));
@@ -110,6 +109,29 @@ async fn calvin_returning_delete_surfaces_deleted_row() {
         || collection_edge_bearing(&node, coll),
     )
     .await;
+    node
+}
+
+/// Assert the sequencer admitted at least one transaction since `before`, so
+/// the statement took the Calvin path rather than the fast `PointDelete` path.
+fn assert_admitted_since(node: &TestClusterNode, before: u64, what: &str) {
+    let after = sequencer_admitted(node);
+    assert!(
+        after > before,
+        "{what} must be admitted to a Calvin epoch (before={before}, after={after})"
+    );
+}
+
+/// Flag ON: a `DELETE ... RETURNING` on an edge-bearing collection routes
+/// through the single-node Calvin OLLP path and returns the deleted row through
+/// the completion path (previously dropped as a bare tag).
+#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
+async fn calvin_returning_delete_surfaces_deleted_row() {
+    // The collection name deliberately embeds "returning": it doubles as
+    // regression coverage that a `*_returning` identifier is not mistaken for the
+    // RETURNING keyword by the clause stripper.
+    let coll = "sncalvin_returning";
+    let node = calvin_edge_collection(coll).await;
 
     let admitted_before = sequencer_admitted(&node);
 
@@ -126,12 +148,7 @@ async fn calvin_returning_delete_surfaces_deleted_row() {
 
     // Proof it traversed the sequencer→scheduler path (not the fast PointDelete
     // path that never touches Calvin): the delete was admitted to a Calvin epoch.
-    let admitted_after = sequencer_admitted(&node);
-    assert!(
-        admitted_after > admitted_before,
-        "the RETURNING delete must be admitted to a Calvin epoch \
-         (before={admitted_before}, after={admitted_after})"
-    );
+    assert_admitted_since(&node, admitted_before, "the RETURNING delete");
 
     let rows: Vec<&tokio_postgres::SimpleQueryRow> = msgs
         .iter()
@@ -174,3 +191,91 @@ async fn calvin_returning_delete_surfaces_deleted_row() {
 
     node.shutdown().await;
 }
+
+/// A RETURNING expression that fails to shape on the Calvin path fails the
+/// statement with its own class: `22012` over pgwire and
+/// `ErrorCode::DIVISION_BY_ZERO` over the native protocol. It never degrades
+/// to a bare command tag.
+#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
+async fn calvin_returning_shaping_error_keeps_its_sqlstate() {
+    let coll = "sncalvin_ret_err";
+    let node = calvin_edge_collection(coll).await;
+
+    let admitted_before = sequencer_admitted(&node);
+    let error = node
+        .client
+        .simple_query(&format!(
+            "DELETE FROM {coll} WHERE id = 'edge_0' RETURNING id, 10 / n AS q"
+        ))
+        .await
+        .expect_err("a RETURNING division by zero must fail the statement");
+    assert_admitted_since(&node, admitted_before, "the pgwire RETURNING delete");
+    let db_err = error
+        .as_db_error()
+        .unwrap_or_else(|| panic!("expected a DbError, got: {error}"));
+    assert_eq!(
+        db_err.code().code(),
+        "22012",
+        "expected division_by_zero, got {}: {}",
+        db_err.code().code(),
+        db_err.message()
+    );
+
+    let admitted_before = sequencer_admitted(&node);
+    let error = node
+        .native_client()
+        .query(&format!(
+            "DELETE FROM {coll} WHERE id = 'edge_1' RETURNING id, 10 / n AS q"
+        ))
+        .await
+        .expect_err("a native RETURNING division by zero must fail the statement");
+    assert_admitted_since(&node, admitted_before, "the native RETURNING delete");
+    assert_eq!(
+        error.code(),
+        nodedb_types::error::ErrorCode::DIVISION_BY_ZERO,
+        "expected DIVISION_BY_ZERO, got {}: {}",
+        error.code(),
+        error.message()
+    );
+
+    node.shutdown().await;
+}
+
+/// A RETURNING `nextval` on the Calvin path reads the session's sequences, as
+/// the direct path does.
+#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
+async fn calvin_returning_nextval_evaluates() {
+    let coll = "sncalvin_ret_seq";
+    let node = calvin_edge_collection(coll).await;
+    node.client
+        .simple_query("CREATE SEQUENCE sncalvin_seq")
+        .await
+        .expect("CREATE SEQUENCE");
+
+    let admitted_before = sequencer_admitted(&node);
+    let msgs = node
+        .client
+        .simple_query(&format!(
+            "DELETE FROM {coll} WHERE id = 'edge_2' RETURNING id, nextval('sncalvin_seq') AS n"
+        ))
+        .await
+        .expect("a RETURNING nextval routed through Calvin must complete");
+    assert_admitted_since(&node, admitted_before, "the RETURNING nextval delete");
+
+    let rows: Vec<&tokio_postgres::SimpleQueryRow> = msgs
+        .iter()
+        .filter_map(|m| match m {
+            tokio_postgres::SimpleQueryMessage::Row(r) => Some(r),
+            _ => None,
+        })
+        .collect();
+    assert_eq!(rows.len(), 1, "the delete returns its one row");
+    assert_eq!(rows[0].get("id"), Some("edge_2"));
+    assert_eq!(
+        rows[0].get("n"),
+        Some("1"),
+        "the first nextval of a fresh sequence is 1"
+    );
+
+    node.shutdown().await;
+}

nodedb-codec/README.md

@@ -2,7 +2,7 @@
 
 > Compression codecs: ALP, FastLanes, FSST, Gorilla, LZ4, vector quantization
 
-Per-column codec implementations used by the columnar / timeseries / spatial engines, plus the vector quantization frontier (SQ8, PQ, OPQ, RaBitQ, BBQ, Ternary BitNet 1.58, Binary). Each codec is a free-standing module — pick what you need.
+Per-column codec implementations used by the columnar / timeseries / spatial engines, plus the vector quantization codecs OPQ, RaBitQ, BBQ, and Ternary (BitNet 1.58), with the shared quantized-vector layout and Hamming kernel. SQ8, PQ, and Binary live in `nodedb-vector`. Each codec is a free-standing module — pick what you need.
 
 ## Status
 

nodedb-codec/src/error.rs

@@ -30,6 +30,10 @@ pub enum CodecError {
         limit: usize,
     },
 
+    /// A codec was asked to train or encode on input it cannot accept.
+    #[error("invalid codec input: {detail}")]
+    InvalidInput { detail: String },
+
     /// The codec stored in metadata doesn't match the expected codec.
     #[error("codec mismatch: expected {expected}, found {found}")]
     CodecMismatch { expected: String, found: String },