feat(txn): serializable snapshot isolation for transaction blocks by farhan-syah · Pull Request #442 · NodeDB-Lab/nodedb

farhan-syah · 2026-11-01T23:50:17Z

Why

[0.6.2] Interactive transactions lose updates: no write-write conflict detection, no stable snapshot (SHOW transaction_isolation = read committed) #430: two transactions read and wrote one row. Both committed, and one update was lost.
pgwire: no way to pin a read snapshot across pages — multi-page readers can observe half-applied producer batches; add pg_snapshot_xmin / SHOW SNAPSHOT #370: paging inside BEGIN ... COMMIT saw different data on each page.

What changed

Transactions are serializable. They read at a point pinned by the first statement.
COMMIT checks the read and write sets on every replica. A conflict fails with 40001.
Engines keep old row versions until no open transaction needs them.
A transaction open past txn_snapshot_max_age_secs fails with 40001.
A write that cannot wait for COMMIT fails inside a transaction with 25001.
Every format version is 1. Old data does not open, and all compat code is removed.
Quarantine is per tenant and per FTS field, for the process lifetime. [quarantine_storage] is removed.
Also fixed: KV TTL replication, UPDATE ... FROM / MERGE NULLs, INSERT ... SELECT rollback, LAST_VALUE cache, detect_i64_codec panic, Calvin leader restart, restore of old versions.

Lite breaks: new plan-op fields and MetaOp arms, TileEntry.placement, required sync fields (checksum 0 refused), strict ErrorDetails / TextFields decoding, RecordType::GraphNodeCascade removed.

How to check

Wire tests txn_serializable_conflicts, txn_snapshot_reads, version_cleanup_horizon.
Full suite passes with --retries 0.

Closes #430
Closes #370

Remove the ShardVersion and WriteVersion types. Columnar WAL records, sync wire types, and shared identifiers no longer carry per-write version stamps. Transactions read a snapshot cut instead.

Record type 65 names no record type. Readers and writers drop the foreign version header fields.

Add version chains, cuts, and per-segment version tables. Mutations record history, install rows at a version, and purge history. Readers select blocks by cut. Row materialization returns every stored version.

Add a node filter that hides live nodes written after a cut and admits soft-deleted nodes ended after it. Add versioned search, export, and insert operations to the collection.

Remove the home-version meta plan. Add restore key and version purge meta plans. Read and write plans carry snapshot cuts.

…tions Nodes exchange their oldest held snapshot on the health ping to derive the cleanup horizon. Raft groups carry ordinal floors and a sequencer cut floor. The Calvin sequencer forwards entries and wakes catch-up waiters. Sequencer types carry an incarnation.

…reads

…tes to the core loop

… floors

… suites

The test seeds a stored user with an empty password to check that CredentialStore::open refuses empty derived credentials. No production credential comes from it.

…g propose test The message now states the expected outcome in plain words.

The test overwrites 100 rows a few times instead of one row hundreds of times. The writes take few statements, so they finish inside the history window on a slow runner before any pass trims them.

farhan-syah added the run-ci Opt this PR into the full test suite; re-add to force a re-run label Nov 2, 2026

farhan-syah added 29 commits November 2, 2026 15:52

feat(types): drop write versions and version columnar wire records

0eb6274

Remove the ShardVersion and WriteVersion types. Columnar WAL records, sync wire types, and shared identifiers no longer carry per-write version stamps. Transactions read a snapshot cut instead.

feat(wal): remove the graph node cascade record and version stamps

2536466

Record type 65 names no record type. Readers and writers drop the foreign version header fields.

fix(codec): tighten format detection and Gorilla decoding

21e67cb

feat(raft): carry an ordinal floor on leader messages and state

45fdf61

feat(array): align segment format and retention with snapshot cuts

a33850b

feat(columnar): keep row version chains and read at a cut

1c0be25

Add version chains, cuts, and per-segment version tables. Mutations record history, install rows at a version, and purge history. Readers select blocks by cut. Row materialization returns every stored version.

feat(vector): admit nodes by system-time cut

6ddccbc

Add a node filter that hides live nodes written after a cut and admits soft-deleted nodes ended after it. Add versioned search, export, and insert operations to the collection.

feat(spatial): keep R-tree history for reads at a cut

761d6b4

refactor(graph,query,crdt): drop write-version plumbing

692ba47

feat(physical): plan snapshot reads, restore keys, and version purge

9106942

Remove the home-version meta plan. Add restore key and version purge meta plans. Read and write plans carry snapshot cuts.

feat(sql): plan temporal search and snapshot cuts

4cc0fce

test(support): read snapshot rows in the cluster harness

f2a25fb

build: watch the branch ref when tracking the current commit

5bf7fbe

feat(nodedb): version storage types, errors, and config for snapshot …

acd9d93

…reads

feat(wal): replay versioned writes and write groups in the engine WAL

0661739

feat(engine/kv): store versions and read at a cut

88e1082

feat(engine/sparse): store versions and read at a cut

796666f

feat(engine/document): store versions and read at a cut

c43c118

feat(engine/graph): store versions and read at a cut

97d5711

feat(engine/array): store versions and read at a cut

a6bce23

feat(engine/crdt): store versions and read at a cut

9741476

feat(engine/vector): store versions and read at a cut

abc2a02

feat(engine/timeseries): read at a cut and rebuild last values

4ff97df

feat(data): add newest ordinals, transaction validation, and write no…

ed4f219

…tes to the core loop

feat(data): validate and apply snapshot transactions

d718823

feat(data): read documents, points, and joins at a cut

011e384

feat(data): version columnar reads and writes

e5ee713

feat(data): version KV, timeseries, and control handlers

b001ed0

farhan-syah added 15 commits November 2, 2026 15:52

feat(data): version graph, vector, spatial, and text handlers

2ef5d5f

feat(data): complete executor changes

a2773a1

feat(security): store catalog entries per version

db1ff99

feat(planner): plan snapshot cuts, Calvin writes, and RLS injection

6703b5a

feat(cluster): apply Calvin, Raft, and snapshot installs with ordinal…

ad991d9

… floors

feat(replication): encode and decode versioned WAL replication records

ffbcc7b

feat(backup): restore and verify backups with version keys

cdd0a74

feat(control): add read cuts, version cleanup, and KV expiry

dcb0c55

feat(server): run transaction blocks on one snapshot

7f18740

feat(control): complete control plane changes

d3858e6

feat(event): carry versioned write events

92898bb

test(nodedb): cover snapshot transactions in inproc, wire, and native…

def8795

… suites

test(cluster): cover Calvin failover and snapshot installs

6d461ba

docs: describe serializable snapshot transactions

e4367f8

ci: update the warm-tier storage gate

3dd4d31

farhan-syah force-pushed the fix/txn-snapshot-isolation branch from 3aa68e9 to 3dd4d31 Compare November 2, 2026 07:53

farhan-syah added 3 commits November 2, 2026 18:33

ci: suppress the hard-coded credential alert in a credential store test

544abaa

The test seeds a stored user with an empty password to check that CredentialStore::open refuses empty derived credentials. No production credential comes from it.

test(control): replace a debug-format assertion message in the catalo…

3196139

…g propose test The message now states the expected outcome in plain words.

test(wire): overwrite many rows in the version cleanup horizon test

1e3e3ee

The test overwrites 100 rows a few times instead of one row hundreds of times. The writes take few statements, so they finish inside the history window on a slow runner before any pass trims them.

farhan-syah merged commit f8c478f into main Nov 2, 2026
16 checks passed

farhan-syah deleted the fix/txn-snapshot-isolation branch November 2, 2026 18:25

farhan-syah mentioned this pull request Nov 2, 2026

pgwire: no way to pin a read snapshot across pages — multi-page readers can observe half-applied producer batches; add pg_snapshot_xmin / SHOW SNAPSHOT #370

Closed

5 of 8 tasks

Navigation Menu

Repository navigation

feat(txn): serializable snapshot isolation for transaction blocks - #442

feat(txn): serializable snapshot isolation for transaction blocks#442
farhan-syah merged 47 commits into
mainfrom
fix/txn-snapshot-isolation

farhan-syah commented Nov 1, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

farhan-syah commented Nov 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What changed

How to check

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

farhan-syah commented Nov 1, 2026 •

edited

Loading