Skip to content

fix: response-shape error classes and one vector codec path - #444

Merged
farhan-syah merged 3 commits into
mainfrom
fix/error-classes-vector
Nov 3, 2026
Merged

farhan-syah merged 3 commits into
mainfrom
fix/error-classes-vector

Conversation

@farhan-syah

Copy link
Copy Markdown
Member

Closes #338.
Closes #395.
Closes #396.
Closes #397.

Error classes (#338)

  • A response cell of the wrong shape raises 22000 (data exception), not XX000.
  • A RETURNING row that cannot be decoded or shaped fails the statement with its own SQLSTATE. No fallback row is sent.
  • Calvin RETURNING shapes its rows once per batch, with the session's sequences. nextval() in a RETURNING list evaluates.
  • Native SQL Calvin RETURNING honours the RETURNING list.
  • HTTP NDJSON error lines carry the error code.

Vector codecs (#395, #396, #397)

  • Each sealed segment carries one codec: none, SQ8, PQ, or a graph codec (RaBitQ, BBQ, binary, ternary, OPQ).
  • The codec builds on the builder thread with the graph.
  • A codec build error installs the graph without a codec and reports the error. A memory-budget error retries with backoff, from 1 s to 60 s, 8 times at most.
  • DDL refuses a PQ, OPQ or graph-codec shape the codec cannot build, with 22023.
  • Ternary and OPQ indexes are searchable.
  • One rerank rule on every path: the pool is top_k × oversample, the beam is max(ef_search, pool), then exact rerank.

Behaviour changes

  • oversample defaults to 3.
  • target_recall widens the beam. It must be in (0, 1].
  • A search-time quantization hint that differs from the index codec raises 22000.
  • The default VectorQuantization is Sq8.

Breaking changes for embedders (nodedb-lite)

  • CodecName and VectorQuantization gain variants. VectorQuantization is no longer #[non_exhaustive], so exhaustive matches need the new arms.
  • BuildRequest, BuiltSegment, complete_build, complete_rebuild and seal change signatures. complete_build and complete_rebuild return InstallOutcome.
  • OpqCodec::train returns Result.
  • IVF search takes an oversample argument.

Licence headers

  • 28 files in Apache-2.0 crates carried a BUSL-1.1 SPDX header. They now say Apache-2.0.

Tests

  • Full suite: stage 1 24268/24268, stage 2 535/535.
  • New tests cover cell-shape SQLSTATEs, Calvin RETURNING errors and nextval(), codec build errors and retry, compaction id remapping, and the code-stage pool.

Thread the session sequence accessor through the Calvin fold so
RETURNING expressions resolve identically on the Calvin path and the
single-shard path. Report a streaming NDJSON error as a terminal line,
and simplify the materialized RETURNING shaping outcome.
Unify codec build, checkpoint, segment, and sealed search under a
shared codec layer, replacing the separate dispatch and sidecar build
modules. Add OPQ and ternary rerank codecs, reject unbuildable codec
shapes at CREATE time, and default vector quantization to SQ8.
@farhan-syah farhan-syah added the run-ci Opt this PR into the full test suite; re-add to force a re-run label Nov 3, 2026
@farhan-syah
farhan-syah merged commit 30a3c35 into main Nov 3, 2026
16 checks passed
@farhan-syah
farhan-syah deleted the fix/error-classes-vector branch November 3, 2026 20:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run-ci Opt this PR into the full test suite; re-add to force a re-run

Projects

None yet

1 participant