[frankmcsherry]

Stacked on #1071: the first commit here is #1071's, and only the second is new, since upstream can't use the fork's ddir-chunk-bounds branch as a base.

CorgiChunk::restrict(needles) returns an in-core chunk holding every row whose identifier is among a sorted, distinct set of needles, and possibly other rows.
Today it returns the chunk itself, an Rc clone, so behavior is unchanged.
The reduce now restricts its prior input and its output once per retire, reads those chunks in every window, and releases them at finish.
This is the one call readers need once a chunk is a list of independently decompressed pages: a reader that touches a few rows of a large chunk will bring in only the pages its keys can be in, as an ordinary in-core chunk, and keep searching within it as it does today.

What changes

• chunk.rs: CorgiChunk::restrict(&self, needles: &[i64]) -> CorgiChunk, documented with that contract. It debug-asserts that the needles are sorted and distinct, and returns self.clone().
• reduce.rs:
  • At a retire's first window, each prior input chunk and each output chunk is restricted to the retire's keys. The novel chunks define the keys and stay as they are.
  • Retire keeps the chunk lists: in_chunks (prior then novel), prior, and out_chunks. next_window no longer rebuilds its lists from the instance on every call; the key search, the value pools, the key columns, the output matches, the colliding-key checks and finish all read these.
  • finish releases the retire's chunks, both pools, the output keys and the key columns. Before, they stayed until the next retire replaced them.
  • search, present and split_real_keys take &[CorgiChunk] rather than &[&CorgiChunk], as the retire owns its chunks.
• The join and half join are left alone. Proxy join: columnar bridges and block matches, no owned times on the common path #1069 rewrites the join's staging and the half-join PR rewrites half_join/backend.rs, so wiring restrict into them now would conflict; they come after those land.

Why positions stay consistent

Every position into a chunk indexes Retire's lists: the input matches found at the first window, the pools' (column, row) references, the output keys, and the key rows emit records for finish.
The lists are built once, at the first window, and nothing replaces them until finish.
Once restrict builds new chunks, calling it in each window could give each window different chunks, and an output id recorded in one window would name a row of the wrong chunk in the next; building the lists once rules that out.

Measurements

One run each, mini-01, one worker, diagnostics off, against 16bf76f9 (#1071) built the same way.

	base	this
LDBC SF1 one-off, 808 ticks: instructions	2,073.2 G	2,072.9 G
LDBC: server CPU	117.07 s	116.68 s
LDBC: peak footprint	4,931 MiB	4,779 MiB
SCC (LiveJournal 250k) first tick: instructions / CPU	57.17 G / 3.20 s	57.07 G / 3.22 s
SCC, 50 churn ticks: instructions / CPU	34.11 G / 2.02 s	34.12 G / 2.01 s
Reachability first tick: instructions / CPU	6.61 G / 0.361 s	6.60 G / 0.359 s
Reachability, 50 churn ticks: instructions / CPU	5.35 G / 0.342 s	5.36 G / 0.343 s

Instructions and CPU are within noise.
The LDBC pair ran three times, as another job shared the machine during the first two base runs; the table has the third pair, on an idle machine.
LDBC's peak footprint was lower in all three pairs, by 41 to 152 MiB; releasing the pools at finish is the only change that could lower it, but I have not measured what the difference consists of.
SCC's and reachability's peak footprints moved by under 1.2%, in opposite directions.
Answer and physical digests are identical on all 808 LDBC ticks in every run, and SCC's and reachability's initial and final peeks are identical.

Tests

• windows_partition_a_retire now also checks that finish leaves the backend holding no chunks and no columns.
• cargo test -p differential-dataflow -p differential-dogs3 -p interactive: 459 passed, 0 failed, 22 ignored; no warnings.
• The default difftest, seeds 0..1000: 1000 passed, 0 refused, 0 failed.

Implemented and measured by a Claude Opus subagent; reviewed and opened by Claude Code.

🤖 Generated with Claude Code

https://claude.ai/code/session_01H6wB4H99TLw4tqu4x48pXL