Skip to content

fix(stage2): keep a tumbling-window chain at ingestion time only on its readers' intervals (#1173) - #1178

Merged
zzylol merged 2 commits into
mainfrom
fix/1173-chain-reader-interval
Nov 2, 2026
Merged

zzylol merged 2 commits into
mainfrom
fix/1173-chain-reader-interval

Conversation

@zzylol

@zzylol zzylol commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Why

#1173: when the same exact query runs every 10 s and every 15 s, the planner selects one SummaryTumblingWindowChain of 4 × 15000 ms, built at ingestion time, and both copies read it. The backend builds each tumbling window once, on the 15 s grid anchored at the epoch. The 10 s copy evaluates at multiples of 10 s, so at +10, +20, +40 and +50 s its newest window ends up to 10 s before the evaluation, and it answers from the wrong 60 s. On a deployment that keeps no raw data, that is a wrong answer for an exact query.

What

  • Stage 2 rule. A tumbling-window chain is eligible for ingestion time only if every reader with a fixed interval has an interval that is a multiple of the chain's width. This applies with or without a declared phase.
  • Otherwise the chain is pruned at ingestion time with the new reason PanesOffInterval, and stays a query-time candidate.
  • Outputs. Stage 2's AllowedOutputs for the chain then lists only the readers whose interval it divides. Stage 3 enforces that list: today's engine through reach[q][c], and feat(plan-selection): group selection MILP over execution contexts as Stage 3's default search (#1095) #1144's group MILP through forbid_reach.

How

The fix is in ingestion_eligibility's chain branch (crates/physical-asap-optimization/src/lib.rs).

  • Before: it checked only readers with a known phase (plan_pane_phase). A FixedInterval reader has no phase, so the loop continued, taking the reader to run on the chain's boundaries.
  • After: each reader's QueryRecurrence::fixed_interval_ms() must be a multiple of tumbling_window_width_ms before the existing phase check runs.

This mirrors the whole-window branch below it, which already requires interval_ms.is_multiple_of(window_ms) (WindowOffEpochGrid). The module doc's ingestion rule now states both conditions.

Key code interfaces added / modified

None public. IngestionPruneReason is crate-private (pub(crate)); it gains one variant:

crates/physical-asap-optimization/src/lib.rs

/// Why a summary build cannot be materialized at ingestion time (S4). Stays.
pub(crate) enum IngestionPruneReason {
    // other variants unchanged
    /// A tumbling-window chain read at a fixed interval that its width does
    /// not divide (e.g. 15 s tumbling windows read every 10 s): built once
    /// each on the grid anchored at the epoch, its tumbling windows cannot
    /// end at every evaluation. Workload-dependent (not a bug); the chain
    /// stays a query-time candidate.
    PanesOffInterval, // new
}

Usage. A workload with sum by (job) (rate(m[1m])) twice, "fixed_interval": 10000 and "fixed_interval": 15000, over 6 series of m, planned with stage_pipeline --deployment-capabilities '{"raw_data_retained": false}':

  • Stage 1 offers chains at several widths, including 10000, 15000 and 5000 ms.
  • Stage 2 rejects the 15000 ms chain at ingestion time for the 10 s copy (PanesOffInterval) and the 10000 ms chain for the 15 s copy.
  • Stage 3 now selects two ingestion-time chains, one per copy. The selected plan's materialized views:
    • 6 views of width_ms 10000 (end offsets 0 … 50000), read by the 10 s copy;
    • 4 views of width_ms 15000 (end offsets 0 … 45000), read by the 15 s copy.

Major algorithm changes

Stage 2 ingestion eligibility for tumbling-window chains

  • Problem: see Why. For rate(m[1m]) every 10 s and every 15 s, Stage 2 emitted AllowedOutputs { candidate: C21 (15000 ms chain), outputs: [0, 1] }, and Stage 3 selected it for both outputs.
  • Before / after: for each repeated reader of a chain of width w:
    1. (new) if the reader has a fixed interval i and i mod w ≠ 0, return PanesOffInterval;
    2. as before, a reader with a known phase must have that phase on the w-grid (PanesOffPhase), and a Scheduled reader with no common phase is rejected.
  • Correctness: a chain built once per tumbling window on the epoch grid can answer evaluation time t only if t is a multiple of w. A reader at interval i with phase p evaluates at times ≡ p (mod i). All of them are multiples of w iff w divides i and w divides p. Step 1 checks the first condition; the existing phase check covers the second.
    • Tests: the regression test and the unit test listed under Verification.
  • Complexity: one modulo per reader per chain candidate.
  • Assumptions and limits:

Before this PR

A chain at ingestion time could serve a reader whose interval its width does not divide. Verified on main c83aa19 with plan_installable and runtime_install.rs's IngestionBuilds::Tumbling runtime: the 10 s copy differs from the query-time reference at 4 of 7 evaluations, for both rate and sum_over_time.

After this PR

Every evaluation of both copies equals the query-time reference. A chain built at ingestion time is shared only by readers whose intervals it divides. The planner keeps a 10 s chain and a 15 s chain for the example above. With raw_data_retained: false it could instead share one 5 s chain; it picks the cheaper of the two.

Verification

  • Tests that fail before the fix and pass after:
    • cargo test -p asap-planner --test planner -- queries_at_two_intervals_read_chains_on_their_own_grids. Before the fix it fails at sum by (job) (rate(m[1m])), copy 0 (every 10000 ms) at 3610000.
    • cargo test -p asap-physical-asap-optimization --lib -- a_tumbling_window_chain_is_maintained_only_on_its_readers_intervals. Before the fix it fails with left: Ok(()) right: Err(PanesOffInterval).
  • Related suites:
    • cargo test -p asap-physical-asap-optimization --lib: 31 passed.
    • cargo test -p asap-planner --test planner -- runtime_install::: 41 passed.
    • cargo fmt --check on both crates: clean.
  • Plan snapshots: cargo test --release -p asap-devtools --test corpus_snapshots -- --ignored reports 22 snapshots (12 corpus, 10 sketch-plan) as differing. Each regenerated snapshot equals the committed one after JSON parsing with sorted keys, so no selected plan changed. The line diffs are key order only, e.g. tumbling_window_retention_ms and reduction placed differently in the committed files, which this PR does not touch. I did not rerun this test on main.

Follow-up

…ts readers' intervals

A tumbling-window chain built at ingestion time keeps windows on the grid of
its width anchored at the epoch. Stage 2's ingestion check skipped readers
with no declared phase, so a 15 s chain could serve a query evaluated every
10 s and answer from windows ending up to 10 s early (#1173). Every reader
at a fixed interval now needs an interval the width divides
(IngestionPruneReason::PanesOffInterval); the chain stays a query-time
candidate otherwise.

Regression tests: the same query every 10 s and every 15 s answers as the
query-time reference at every evaluation over one minute (runtime_install),
and a 1-min chain read every 90 s is not maintained at ingestion time.

Closes #1173
@zzylol zzylol added bug Something isn't working optimizer Workload optimization and plan selection window Window functions, frames, panes, or temporal semantics labels Nov 2, 2026
Review: builds_one_pane_per_evaluation decides keeping panes across
evaluations, not the phase-0 assumption.
@zzylol
zzylol merged commit 2c99451 into main Nov 2, 2026
1 check passed
zzylol added a commit that referenced this pull request Nov 2, 2026
Reruns the part 2 rename over the files main changed (#1174, #1178, #1180,
#1182) and reapplies the review's wording fixes there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Nov 2, 2026
…ding (#1163, part 2) (#1176)

* refactor: rename every remaining pane name to the tumbling-window wording (#1163, part 2)

Helper types, functions, modules, files, test names, comments and docs: Pane*
becomes TumblingWindow*, pane_* becomes tumbling_window_*, and the word
"pane" becomes "tumbling window" (e.g. PaneWidthDomain ->
TumblingWindowWidthDomain, pane_chain.rs -> tumbling_window_chain.rs,
expand_pane_chains -> expand_tumbling_window_chains). The planning report's
pane_width_domain and pane_widths_pending become tumbling_window_* too, and
the DAG viewer upgrades them in older documents.

The deployed wire contract keeps its names: TumblingWindowLayout still reads
and writes pane_width_ms and pane_origin_ms, WindowEdgeCoverage pane_aligned,
TumblingWindowCoverageError ZeroPaneWidth / UnknownPaneOrigin / pane_phase_ms,
and WindowEdgeCompatibility RequiresAlignedPanePhaseOrExactBoundaryResidual.
No behavior change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor: apply review of #1163 part 2

- viewer: expanded tumbling-window nodes keep one CSS class (tumblingWindow)
  that app.js styles, with a test; JS data fields are camelCase;
- historical docs, artifact paths and plan ids keep their recorded names;
- code names inside comments and labels keep identifier form;
- wording: doubled words, hyphenated compound modifiers, contrasted terms;
- snapshot labels updated to the new display and cost text;
- test constants name what they hold (TUMBLING_WINDOW_COUNT, ...);
- the coverage errors' wire names are tested; cargo fmt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: snapshot keys in sorted order after the tumbling_window_retention_ms rename

Part 1 renamed pane_retention_ms in the snapshot JSON in place; the snapshots
store keys sorted, so the renamed key now sorts after proven_optimal. The
regenerated snapshots equal the checked-in ones as JSON (22 of 22).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor: rename pane names that main added since part 2 started (#1163)

Reruns the part 2 rename over the files main changed (#1174, #1178, #1180,
#1182) and reapplies the review's wording fixes there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol
zzylol deleted the fix/1173-chain-reader-interval branch November 3, 2026 02:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working optimizer Workload optimization and plan selection window Window functions, frames, panes, or temporal semantics

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant