Repository navigation
fix(stage2): keep a tumbling-window chain at ingestion time only on its readers' intervals (#1173) - #1178
Merged
Conversation
…ts readers' intervals A tumbling-window chain built at ingestion time keeps windows on the grid of its width anchored at the epoch. Stage 2's ingestion check skipped readers with no declared phase, so a 15 s chain could serve a query evaluated every 10 s and answer from windows ending up to 10 s early (#1173). Every reader at a fixed interval now needs an interval the width divides (IngestionPruneReason::PanesOffInterval); the chain stays a query-time candidate otherwise. Regression tests: the same query every 10 s and every 15 s answers as the query-time reference at every evaluation over one minute (runtime_install), and a 1-min chain read every 90 s is not maintained at ingestion time. Closes #1173
Review: builds_one_pane_per_evaluation decides keeping panes across evaluations, not the phase-0 assumption.
zzylol
added a commit
that referenced
this pull request
Nov 2, 2026
zzylol
added a commit
that referenced
this pull request
Nov 2, 2026
…ding (#1163, part 2) (#1176) * refactor: rename every remaining pane name to the tumbling-window wording (#1163, part 2) Helper types, functions, modules, files, test names, comments and docs: Pane* becomes TumblingWindow*, pane_* becomes tumbling_window_*, and the word "pane" becomes "tumbling window" (e.g. PaneWidthDomain -> TumblingWindowWidthDomain, pane_chain.rs -> tumbling_window_chain.rs, expand_pane_chains -> expand_tumbling_window_chains). The planning report's pane_width_domain and pane_widths_pending become tumbling_window_* too, and the DAG viewer upgrades them in older documents. The deployed wire contract keeps its names: TumblingWindowLayout still reads and writes pane_width_ms and pane_origin_ms, WindowEdgeCoverage pane_aligned, TumblingWindowCoverageError ZeroPaneWidth / UnknownPaneOrigin / pane_phase_ms, and WindowEdgeCompatibility RequiresAlignedPanePhaseOrExactBoundaryResidual. No behavior change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor: apply review of #1163 part 2 - viewer: expanded tumbling-window nodes keep one CSS class (tumblingWindow) that app.js styles, with a test; JS data fields are camelCase; - historical docs, artifact paths and plan ids keep their recorded names; - code names inside comments and labels keep identifier form; - wording: doubled words, hyphenated compound modifiers, contrasted terms; - snapshot labels updated to the new display and cost text; - test constants name what they hold (TUMBLING_WINDOW_COUNT, ...); - the coverage errors' wire names are tested; cargo fmt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: snapshot keys in sorted order after the tumbling_window_retention_ms rename Part 1 renamed pane_retention_ms in the snapshot JSON in place; the snapshots store keys sorted, so the renamed key now sorts after proven_optimal. The regenerated snapshots equal the checked-in ones as JSON (22 of 22). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor: rename pane names that main added since part 2 started (#1163) Reruns the part 2 rename over the files main changed (#1174, #1178, #1180, #1182) and reapplies the review's wording fixes there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
#1173: when the same exact query runs every 10 s and every 15 s, the planner selects one
SummaryTumblingWindowChainof 4 × 15000 ms, built at ingestion time, and both copies read it. The backend builds each tumbling window once, on the 15 s grid anchored at the epoch. The 10 s copy evaluates at multiples of 10 s, so at +10, +20, +40 and +50 s its newest window ends up to 10 s before the evaluation, and it answers from the wrong 60 s. On a deployment that keeps no raw data, that is a wrong answer for an exact query.What
PanesOffInterval, and stays a query-time candidate.AllowedOutputsfor the chain then lists only the readers whose interval it divides. Stage 3 enforces that list: today's engine throughreach[q][c], and feat(plan-selection): group selection MILP over execution contexts as Stage 3's default search (#1095) #1144's group MILP throughforbid_reach.How
The fix is in
ingestion_eligibility's chain branch (crates/physical-asap-optimization/src/lib.rs).plan_pane_phase). AFixedIntervalreader has no phase, so the loopcontinued, taking the reader to run on the chain's boundaries.QueryRecurrence::fixed_interval_ms()must be a multiple oftumbling_window_width_msbefore the existing phase check runs.This mirrors the whole-window branch below it, which already requires
interval_ms.is_multiple_of(window_ms)(WindowOffEpochGrid). The module doc's ingestion rule now states both conditions.Key code interfaces added / modified
None public.
IngestionPruneReasonis crate-private (pub(crate)); it gains one variant:crates/physical-asap-optimization/src/lib.rs
Usage. A workload with
sum by (job) (rate(m[1m]))twice,"fixed_interval": 10000and"fixed_interval": 15000, over 6 series ofm, planned withstage_pipeline --deployment-capabilities '{"raw_data_retained": false}':PanesOffInterval) and the 10000 ms chain for the 15 s copy.width_ms10000 (end offsets 0 … 50000), read by the 10 s copy;width_ms15000 (end offsets 0 … 45000), read by the 15 s copy.Major algorithm changes
Stage 2 ingestion eligibility for tumbling-window chains
rate(m[1m])every 10 s and every 15 s, Stage 2 emittedAllowedOutputs { candidate: C21 (15000 ms chain), outputs: [0, 1] }, and Stage 3 selected it for both outputs.PanesOffInterval;PanesOffPhase), and aScheduledreader with no common phase is rejected.EstimatedRatereaders, which have no fixed interval, are still taken to run on the chain's boundaries, as before.PaneWidthOffRangeStep).Before this PR
A chain at ingestion time could serve a reader whose interval its width does not divide. Verified on main c83aa19 with
plan_installableandruntime_install.rs'sIngestionBuilds::Tumblingruntime: the 10 s copy differs from the query-time reference at 4 of 7 evaluations, for bothrateandsum_over_time.After this PR
Every evaluation of both copies equals the query-time reference. A chain built at ingestion time is shared only by readers whose intervals it divides. The planner keeps a 10 s chain and a 15 s chain for the example above. With
raw_data_retained: falseit could instead share one 5 s chain; it picks the cheaper of the two.Verification
cargo test -p asap-planner --test planner -- queries_at_two_intervals_read_chains_on_their_own_grids. Before the fix it fails atsum by (job) (rate(m[1m])), copy 0 (every 10000 ms) at 3610000.cargo test -p asap-physical-asap-optimization --lib -- a_tumbling_window_chain_is_maintained_only_on_its_readers_intervals. Before the fix it fails withleft: Ok(()) right: Err(PanesOffInterval).cargo test -p asap-physical-asap-optimization --lib: 31 passed.cargo test -p asap-planner --test planner -- runtime_install::: 41 passed.cargo fmt --checkon both crates: clean.cargo test --release -p asap-devtools --test corpus_snapshots -- --ignoredreports 22 snapshots (12 corpus, 10 sketch-plan) as differing. Each regenerated snapshot equals the committed one after JSON parsing with sorted keys, so no selected plan changed. The line diffs are key order only, e.g.tumbling_window_retention_msandreductionplaced differently in the committed files, which this PR does not touch. I did not rerun this test on main.Follow-up