Skip to content

refactor(types): properties and requirements from #1207; chains as time-indexed state (#1207 PR 1) - #1227

Merged
zzylol merged 1 commit into
mainfrom
refactor/1207-properties
Nov 10, 2026
Merged

zzylol merged 1 commit into
mainfrom
refactor/1207-properties

Conversation

@zzylol

@zzylol zzylol commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Problem: PR 0 left every property check of the concepts doc, section 2, as todo!(): logical properties and the θ lists, physical properties, α^phys, delivers_on over execution contexts; a chain had no time index and TimeRange/TimeShift rejected state.
  • Change: this PR implements them against the 2026-11-08 design: θ is structure only and comes from a fixed list per algorithm (parameter_list), contexts are evaluation grids, and a SummaryTumblingWindowChain is a time-indexed state (window_end, keys…, state).
  • Effect: for a stored X = sum_over_time(m[5m]) read every 60 s and every 120 s (grid 60 s @ 0), delivers_on now answers true for 60 s @ 0, 120 s @ 0 and 120 s @ 60 s and false for 30 s @ 0 and 60 s @ 15 s, where it panicked before; KLL gets 7 listed θ (k = 26 … 2863), Count-Min 28, UnivMon 40.

Review guide

  • Position: base main, after refactor: one crate per stage (#1207 PR 0b) #1229 (merged as 6bcf3984b); feat(stage0, types): memo hash-consing, group properties (column_sign, depends_on_evaluation_time, aggregate_coverage), @ end()/@ start() above their subtree (#1207 PR 2) #1230 (PR 2, memo) builds on it. One commit, head b65c3964c. Diff vs base: 22 files, +1,532 / −243. The stack's PRs merge into main one by one after review.
  • Restack (2026-11-10): rebuilt on main after refactor: define every stage's interfaces from #1207 and delete the ones they replace (#1207 PR 0) #1226's and refactor: one crate per stage (#1207 PR 0b) #1229's file layouts. The commit's content is unchanged; each item is in its scope file: LogicalProperties::new in stages/logical/candidate.rs, parameter_list and the θ const lists in summary_type/parameter_lists.rs, delivers_on and producer_satisfies_physical_requirements in stages/physical/candidate.rs, PhysicalProperties::new in stages/physical/properties.rs, QueryRecurrence::evaluation_grid_ms in workload/query.rs, the estimator changes in accuracy/src/guarantee/. Previous head 13063ab21 (backup backup/1207-layout-properties-13063ab21).
  • Reading order:
    1. types/src/stages/logical/candidate.rs: LogicalProperties::new; types/src/summary_type/parameter_lists.rs: parameter_list and the const lists.
    2. types/src/stages/physical/candidate.rs: delivers_on, producer_satisfies_physical_requirements (their tests in stages/physical/memo_candidates.rs, which can build a PhysicalASAPMemoCandidates with its private fields); types/src/stages/physical/properties.rs: PhysicalProperties::new.
    3. types/src/ir/operator/asap.rs (chain_schema, summary_build, selected_from, summary_type_of_operator) and types/tests/tumbling_window_chain.rs.
    4. types/src/workload/query.rs: QueryRecurrence::evaluation_grid_ms.
  • Mechanical: the const θ tables (KLL_PARAMETERS … UNIVMON_PARAMETERS, built by with_depths, with_heap_sizes, univmon_parameters); asap-accuracy sizing made #[cfg(test)]; propagation.rs reads merged windows through summary_build.
  • Careful: the delivers_on table (chain width vs context period/phase; stored results on their group's grid); the chain schema and the rule that a SummaryMerge over a time-indexed state reads exactly one TimeRange; the Count Sketch depths (median-of-rows model).

Behavior: before → after

No workload reaches these functions end to end yet (Stage 0, Pass 1, Pass 2 and Stage 2 are still todo!() in PRs 2–5); the example is the concepts doc's section 5 flow, Q_A = quantile_over_time(0.99, latency[1m]) every 10 s at phase 0 (FixedIntervalAt), checked by the unit tests in Verification. It serves the design need that every later stage reads one set of property checks.

Call Before (PR 0b) After (this PR)
parameter_list(SketchAlgorithm::Kll) todo!() [k=26, 52, 132, 269, 547, 1404, 2863]
LogicalProperties::new(chain[KLL, w=10 s], Some(Kll{k:547})) todo!() Ok; k = 270 → ParametersNotInList; no θ → SketchBuildWithoutParameters
PhysicalProperties::new(IngestionTime, OneEvaluation) todo!() Err; every other pair Ok
producer_satisfies_physical_requirements(IT AcrossEvaluations chain, QueryTime) todo!() true
chain(w=10 s).delivers_on(grid 10 s @ 0 / 30 s @ 0 / 10 s @ 5 s) todo!() true / true / false (Pass 2 offers w = 5 s for the last)
schema of SummaryTumblingWindowChain the plain build's (no time index) (window_end, keys…, state), unique key (window_end, keys)
TimeRange / TimeShift over a chain rejected keep the chain's schema; instant TimeRange over a chain rejected

Interfaces

crates/types/src/stages/logical/candidate.rs · crates/types/src/summary_type/parameter_lists.rs

 impl Memo {
     /// The candidate `id` (body filled in; `delivers_on` reads a chain's width through it).
     pub fn candidate(&self, id: CandidateForMemoGroupId) -> &CandidateForMemoGroup;
 }
 /// p^log = θ_v; the summary type and a chain's width are operator arguments. The type stays.
 pub struct LogicalProperties {
     /// θ_v from the algorithm's list; None for an exact operator.
     parameters: Option<SketchParams>,
 }
 impl LogicalProperties {
     /// Rejects θ missing on a sketch build (a chain included), present on an operator that builds no
     /// sketch, or not in `parameter_list(algorithm)` (body filled in; signature unchanged).
     pub fn new(
         expression: &CandidateExpression,
         parameters: Option<SketchParams>,
     ) -> Result<Self, LogicalPropertiesError>;
     /// θ_v.
     pub fn parameters(&self) -> Option<&SketchParams>;
 }
 /// Why `LogicalProperties::new` rejects; bad input from a candidate generator (a planner bug). The type stays.
 pub enum LogicalPropertiesError {
     /// θ on an operator that builds no sketch.
     ParametersOnExactOperator,
     /// A sketch build (a chain included) without θ.
     SketchBuildWithoutParameters,
     /// θ not in its algorithm's list (also another algorithm's θ).
     ParametersNotInList,
 }
 /// The algorithm's predefined θ list, structures only, smallest first; empty for an algorithm
 /// without an error model (body filled in; signature unchanged).
 pub fn parameter_list(algorithm: SketchAlgorithm) -> &'static [SketchParams];

crates/types/src/stages/physical/candidate.rs · crates/types/src/stages/physical/properties.rs

 impl PhysicalCandidate {
     /// Whether a consumer running in `context` can read this producer: None everywhere, OneEvaluation
     /// only in `Evaluation` contexts, an AcrossEvaluations chain when its width divides the context's
     /// period and phase, any other stored result on its group's grid (body filled in; signature PR 0's).
     pub fn delivers_on(
         &self,
         candidates: &PhysicalASAPMemoCandidates,
         context: &ExecutionContext,
     ) -> bool;
 }
 /// α^phys: the timing × lifetime table only (body filled in).
 pub fn producer_satisfies_physical_requirements(
     producer: &PhysicalCandidate,
     consumer_timing: ExecutionTiming,
 ) -> bool;
 impl PhysicalProperties {
     /// (τ, ℓ), unless (IngestionTime, OneEvaluation) (body filled in).
     pub fn new(
         timing: ExecutionTiming,
         lifetime: Lifetime,
     ) -> Result<Self, PhysicalPropertiesError>;
 }

crates/types/src/ir/operator/asap.rs · crates/types/src/ir/operator/tumbling_window_chain.rs

-/// The states `node` selects from when it is a `Filter` that keeps whole summary states by their plain columns.
+/// The states `node` selects from when it keeps whole summary states: a `Filter` by their plain
+/// columns, or a `TimeRange` or `TimeShift` over a time-indexed state, which selects windows.
 pub fn selected_from<I: OperatorInput>(node: &I) -> Option<I>;
-/// The build a summary state carries: a build itself, or the first input's build under `SummaryMerge`s.
+/// The build a summary state carries: a build itself, the chain under the `TimeRange` and
+/// `TimeShift`s that select its windows, or the first input's build under `SummaryMerge`s.
 pub fn summary_build<I: OperatorInput>(state: &I) -> Option<ASAPOp<I>>;
+/// The summary type a build operator (`SummaryAgg` or `SummaryTumblingWindowChain`) builds; None otherwise.
+/// `LogicalProperties::new` and later stages read the algorithm from it.
+pub fn summary_type_of_operator<I>(operator: &Operator<I>) -> Option<&FieldDataType>;
-/// The `SummaryTumblingWindowChain` a summary state carries merged.
-pub fn merged_chain<I: OperatorInput>(state: &I) -> Option<I>;

merged_chain: replaced by summary_build / selected_from through the window selection.

crates/types/src/workload/query.rs

+impl QueryRecurrence {
+    /// The epoch-anchored grid every evaluation falls on; None when the times are not known,
+    /// including `FixedInterval` without a phase. Pass 2's chain widths and Stage 3's grids read it.
+    pub fn evaluation_grid_ms(&self, range_evaluation: Option<RangeEvaluation>) -> Option<u64>;
+}
-impl QueryWorkloadEntry {
-    /// The grid of the entry's evaluations.
-    pub fn evaluation_grid_ms(&self) -> Option<u64>;
-}

crates/accuracy/src/sizing.rs · crates/accuracy/src/test_support.rs

-/// Today's sizing of `kind` for (ε, δ).
-pub(crate) fn default_size_params(
-    kind: SketchAlgorithm,
-    intent: &AggIntent,
-    eps: f64,
-    delta: f64,
-) -> SketchParams;
-/// `default_size_params` for tests.
-pub fn default_size_params(
-    kind: SketchAlgorithm,
-    intent: &AggIntent,
-    eps: f64,
-    delta: f64,
-) -> SketchParams;
-/// Count-Min width for ε.
-pub fn cms_width(eps: f64) -> u32;

No caller remains: the lists replace sizing; the sizing helpers only the list test reads are #[cfg(test)] (kll_k, kmv_k, count_sketch_depth, UnivMon's distinct_heap and its constants).

Usage, end to end (the flow PRs 2–7 run for the doc's section 5 example with the calls this PR implements; each step is checked by a unit test named in Verification at the linked commit):

// What the user writes: Q_A = quantile_over_time(0.99, latency[1m]) every 10 s at phase 0 (FixedIntervalAt).
// Pass 1 (PR 3): one KLL build per listed θ for G3 = SummaryAgg[KLL](TimeRange(1m, G1)).
// `kll_list`: KLL's θ list, [k=26, 52, 132, 269, 547, 1404, 2863]; Pass 1 offers one candidate per entry.
let kll_list = parameter_list(SketchAlgorithm::Kll);
// Pass 2 rule 1 (PR 4): `chain_expression` = Gw = SummaryTumblingWindowChain[KLL, w = 10 s](G1),
// output (window_end, state); G3's merge over TimeRange(1m, Gw) reads it.
let chain_expression = CandidateExpression { operator: Operator::ASAP(ASAPOp::SummaryTumblingWindowChain {
    child: g1, summary_type: FieldDataType::Sketch(SketchAlgorithm::Kll),
    input: SummaryUpdate::column(ColumnRef::SampleValue), reduction: Reduction::PerEntity,
    filter: None, tumbling_window_width_ms: 10_000 }) };
// `k_547`: the chain's logical properties; k = 547 is listed, k = 270 is ParametersNotInList.
let k_547 = LogicalProperties::new(&chain_expression, Some(kll_list[4].clone()))?;
// G3's second candidate SummaryMerge(TimeRange(1m, Gw)) has G3's schema, so it joins G3.

// Stage 2 (PR 5): `chain` = Gw at ingestion time, materialized across evaluations.
let chain = PhysicalCandidate { logical_candidate_id: gw_candidate,
    physical_properties: PhysicalProperties::new(ExecutionTiming::IngestionTime,
        Lifetime::AcrossEvaluations(StorageMedium::Memory))? };
// α^phys for G3's merge, which runs at query time: an IT chain materialized across evaluations serves it.
assert!(producer_satisfies_physical_requirements(&chain, ExecutionTiming::QueryTime));

// Stage 3 (PRs 6–7): `kappa_10` = Q_A's grid; `kappa_30` = Q_C's (every 30 s).
let kappa_10 = ExecutionContext::Evaluation { grid: EvaluationGrid { period_ms: 10_000, phase_ms: 0 } };
let kappa_30 = ExecutionContext::Evaluation { grid: EvaluationGrid { period_ms: 30_000, phase_ms: 0 } };
assert!(chain.delivers_on(&candidates, &kappa_10)); // 10 s divides 10 s and 0
assert!(chain.delivers_on(&candidates, &kappa_30)); // 10 s divides 30 s and 0
// What the user gets back at Stage 3: a query every 10 s at phase 5 s reads no 10 s chain (Pass 2 offers w = 5 s).
assert!(!chain.delivers_on(&candidates, &ExecutionContext::Evaluation {
    grid: EvaluationGrid { period_ms: 10_000, phase_ms: 5_000 } }));

The planner's text output does not change in this PR: planning still stops at Stage 0's todo!().

Algorithm changes

None in planning results: no stage that would reach these checks runs yet. The new checks:

θ lists (parameter_list)

  • Problem: PR 0 left the lists todo!(); θ is structure only (2026-11-08), so the lists hold no sized-for fields.

  • After: const arrays built by two const fns: with_depths(widths, depths, shape) crosses 7 widths with 4 depths (CMS_DEPTHS or COUNT_SKETCH_DEPTHS), with_heap_sizes(shapes) crosses those 28 with 10 heap sizes; univmon_parameters() builds the 40 UnivMon shapes. The match lists every SketchAlgorithm (no _), so a new sketchlib algorithm fails to compile until it gets a list.

    Algorithm Entries List
    KLL 7 k = 26, 52, 132, 269, 547, 1404, 2863 (no δ variants: KLL's δ is not a parameter)
    DDSketch 7 alpha = 0.1 … 0.001
    HLL 4 precision 13, 15, 17, 18
    KMV 4 k = 1118, 4323, 26 656, 106 263 (ε down to 0.01)
    Count-Min 28 widths 28, 55, 136, 272, 544, 1360, 2719, each with depths 5, 7, 10, 12 (δ = e^−d ≤ 1e-2, 1e-3, 1e-4, 1e-5)
    Count Sketch 28 widths 300 … 3 000 000, each with depths 47, 81, 117, 155 (median-of-rows δ = P(Binomial(d, 1/3) ≥ (d+1)/2) ≤ 1e-2 … 1e-5)
    CMS / Count Sketch with heap 280 each each width × depth with heap sizes 10, 20, 50, …, 10 000
    UnivMon 40 7 L2 shapes, 4 distinct-count shapes, the 28 combinations, and the baseline (no distinct_failure_probability)
    Hydra and other algorithms 0 no error model yet (Planner: error models and parameter lists for Hydra (and other unsized families), so Pass 1 can offer them #1228), so no build of them is a valid candidate
  • Correctness: the widths, ks, precisions and heap sizes are today's sizing over ε = 0.1 … 0.001 (smallest first); asap-accuracy's parameter_list_is_the_sizing_over_the_error_grid keeps them equal to it (that sizing is test-only; PR 4 replaces the test with a per-entry guarantee test).

  • Complexity: &'static slices, no allocation.

  • Assumptions and limits: no parameter floors (floor_for_cap, strongest, weakest, dominates, combine): a readout's input carries a RequiredAccuracy, checked against the producer's guarantee (PR 4).

delivers_on over execution contexts

  • Problem: PR 0's todo!(); it read QueryRecurrence/RangeEvaluation from the context before.

  • After: match ℓ; for AcrossEvaluations read the candidate's operator from candidates.logical_candidates.memo. A chain needs its tumbling_window_width_ms w to divide the context's (period, phase) (the grid's, or the refresh window's (width, phase)), so w divides every time phase + k·period. Any other result delivers everywhere when its group's depends_on_evaluation_time is false; else every context time must lie on evaluation_grid_of_group[group]: its period divides the context's and the phases agree modulo it.

    Producer Evaluation { grid } IngestionPipeline(RefreshWindow) Materialized(v)
    ℓ = None true true true
    ℓ = OneEvaluation true false false
    chain of width w, ℓ = AcrossEvaluations w divides period_ms and phase_ms w divides width_ms and phase_ms todo!() (PR 7: v's own schedule)
    other group, AcrossEvaluations, not depends_on_evaluation_time true true true
    other group, AcrossEvaluations, time-dependent, grid g = evaluation_grid_of_group[group] g's period divides period_ms and phase_ms ≡ g's phase (mod g's period) the same with width_ms todo!() (PR 7)
    the same without a known grid (None) false false todo!() (PR 7)
  • Correctness: delivers_on_by_lifetime_without_windows, delivers_on_a_chain_when_its_width_divides_the_context_times, delivers_on_a_stored_result_on_its_group_grid.

  • Complexity: O(1) per call (one memo lookup).

  • Assumptions and limits: delivers_on does not shift the context's times by a TimeShift on the reading path: evaluation_grid_of_group is the grid of the times the group itself is evaluated at, so Pass 2 (PR 4) must fold the queries' shifts into it.

α^phys (producer_satisfies_physical_requirements)

producer consumer QueryTime consumer IngestionTime
QT, None / OneEvaluation / AcrossEvaluations ✓ ✗
IT, None ✗ ✓
IT, AcrossEvaluations ✓ ✓

Chains as time-indexed state (schema derivation)

  • Before: a chain had the plain build's schema; PR 0 deleted the count-based chain and left finalizing/rolling up merged windows todo!().
  • After: chain_schema derives the build's schema, removes its time column if it has one, inserts a computed window_end (the input time column's type) at column 0 as the time index, and sets the unique key to (window_end, plain columns). TimeRange and TimeShift accept a time-indexed state and keep its schema (rejected over any other state; an Instant TimeRange over a chain is a validation error). A SummaryMerge over a time-indexed state reads exactly one TimeRange over a chain; its schema is summary_agg_schema(chain input, summary type, reduction), the chain's build over that range, so it joins the original state group. FinalizeExactAccumulator and a rollup over merged windows derive as over that build.
  • Correctness: types/tests/tumbling_window_chain.rs (five tests, below).

Decisions and open questions

Needs reviewer decision: None (P1–P3 approved 2026-11-09).

Decided:

Item Decision Date Rejected
θ is structure only lists without sized-for fields; derived equality 2026-11-08 sized-for fields (KLL δ, HLL/KMV relative error, UnivMon distinct_failure_probability)
Count-Min depths 5, 7, 10, 12 per width (δ grid 1e-2 … 1e-5) 2026-11-08
Count Sketch depths (P1) 47, 81, 117, 155 per width (same δ grid, median-of-rows model) 2026-11-09 the CMS depths 5/7/10/12 (δ 0.21/0.17/none/none under the median model)
UnivMon list 40 entries: 7 L2, 4 distinct-count, 28 combined, baseline 2026-11-08 single-purpose shapes only
No floors no floor_for_cap/strongest/weakest/dominates/combine; RequiredAccuracy instead 2026-11-08 θ dominance floors (this PR's earlier version)
Summary type FieldDataType::Sketch(SketchAlgorithm); summary_type_of_operator (was family_of_operator); sketch_algorithm_of deleted 2026-11-08
producer_satisfies_physical_requirements(producer, consumer_timing) consumer's timing only 2026-11-08 (PR 7 J3)
OneEvaluation in delivers_on only in Evaluation contexts 2026-11-08 (PR 7 J3 b)
delivers_on for IngestionPipeline(RefreshWindow) / Materialized(v) implemented; Materialized(v) stays todo!() for PR 7 2026-11-08 (PR 7 K3)
delivers_on(&PhysicalASAPMemoCandidates, context) (P2) the signature is PR 0's; this PR reads the memo through it 2026-11-09 a context that cannot read v
GroupProperties::depends_on_evaluation_time, LogicalASAPMemoCandidates::evaluation_grid_of_group (P3) types in PR 0 (derived in PR 2, filled in PR 4); delivers_on reads them here 2026-11-09
One chain schema; instant TimeRange over a chain rejected as above 2026-11-08
SummaryMerge over a time-indexed state reads exactly one TimeRange 2026-11-08
selected_from/summary_build see through TimeRange/TimeShift no merged-chain helper 2026-11-08 merged_chain
evaluation_grid_ms None for FixedInterval without a phase; the QueryWorkloadEntry wrapper deleted 2026-11-07 / 2026-11-08
A sketch build without θ; θ outside the list rejected 2026-11-06
Test-only sizing in asap-accuracy kept here; PR 4 deletes it with a per-entry guarantee test 2026-11-08

Deleted besides the interfaces above: this PR's earlier strongest, weakest, dominates, combine and their tests (strongest_and_weakest_combine_one_family_per_parameter, a_listed_univmon_dominates_an_l2_and_a_distinct_floor, hydra_dominance_compares_every_size_of_one_kind); its earlier producer_satisfies_logical_requirements(properties, requirement), LogicalRequirements::new and logical_requirement_new_accepts_any_floor (replaced by asap_accuracy::producer_satisfies_logical_requirements(memo, producer, consumer, input_index, evidence), PR 0 interface, PR 4 body, and LogicalRequirements { required_accuracy }); a_configuration_satisfying_a_listed_floor_is_no_less_accurate; KLL, HLL and KMV list entries' sized-for fields and UnivMon's distinct_failure_probability; PR 0's new_rejects_one_evaluation_at_ingestion, new_accepts_each_valid_pair (replaced by new_over_every_timing_and_lifetime).

Verification

Restack check (2026-11-10, head b65c3964c vs previous head 13063ab21): cargo test -p asap-types -p asap-accuracy --all-features --no-fail-fast: 225 passed / 0 failed / 2 ignored on both, the same tests by name (the two ignored are the delivers_on tests below); cargo check --workspace --tests with no warnings, cargo clippy --workspace --all-targets -- -D warnings and cargo fmt --check clean. PR 1 touches only asap-types and asap-accuracy, so the other crates were not rerun; the table below is from the earlier head.

cargo test -p <crate> --all-features --no-fail-fast at 8030d3fe6; cargo fmt --all -- --check and cargo clippy --workspace --all-targets --all-features -- -D warnings clean; crate_graph: 6 passed. By test name against the base PR 0b (058ea4234): no test that passed fails; 12 new tests pass; 0 newly failing. Two new tests (delivers_on_a_chain_when_its_width_divides_the_context_times, delivers_on_a_stored_result_on_its_group_grid) build their memo fixture with find_or_add_group, which is todo!() until PR 2, so they are #[ignore = "filled in by #1207 PR 2 (#1230)"] here; PR 2 removes the attribute and they pass there. At the head c2e717bf9 (only change: those two attributes) cargo test -p asap-types --no-fail-fast: 164 passed, 0 failed, 2 ignored (the two above); PR 0's two PhysicalProperties::new tests (failing at its todo!()) are replaced. Every failure panics at a later PR's todo!() or is the stale fixture (PR 9).

Scope Passed / failed / ignored Newly failing vs base Notes (fails at)
asap-types 164 / 0 / 2 0 2 ignored: the two delivers_on tests, filled in by PR 2
asap-accuracy 60 / 0 / 0 0
asap-cost 31 / 1 / 0 0 PR 6: check θ, which the exported node carries: 1
asap-frontend-common 49 / 0 / 0 0
asap-sql-function-catalog 10 / 0 / 0 0
metricsql_common 3 / 0 / 0 0
asap-frontend-promql 217 / 0 / 0 0
asap-frontend-metricsql 15 / 0 / 0 0
asap-frontend-sql 173 / 0 / 0 0
asap-executor 210 / 63 / 1 0 PR 6: θ on the exported node 32, Hydra grid sizes 13, retention_ms = $h_v$ 8, weighted-frequency θ 7 + 2; PR 2: Stage 0 over Memo::find_or_add_group 1
asap-planner 8 / 18 / 1 0 PR 2: Stage 0 over Memo::find_or_add_group: 18
asap-devtools 37 / 56 / 3 0 PR 2: Stage 0: 54; output.status.success(): 2
asap-integration-tests 83 / 51 / 4 0 PR 2: Stage 0: 47; PR 6 retention_ms: 3; old-wire fixture ("invalid type: sequence, expected string or map"): 1
asap-stage3-plan-selection 3 / 0 / 0 0
asap-stage0-memo 0 / 0 / 0 0
asap-stage2-physical-optimization 0 / 0 / 0 0
asap-stage1-pass2-reuse-candidates 0 / 0 / 0 0
asap-stage1-pass1-per-computation-candidates 11 / 0 / 0 0
Total 1074 / 189 / 11 0

Against the previous head 3c2825903: only the 36 tests PR 0 now deletes with pass1/composition.rs (P11) are gone (35 failed, 1 passed there); no test changed result.

New tests:

  • stages::logical::candidate::tests::logical_properties_new_over_every_operator_and_parameters: 5 operators (KLL build, Sketch(HydraCms) build, exact Sum, KLL chain, readout) × 5 θ (none, KLL 269, CMS, Hydra CMS, unlisted KLL 270).
  • stages::logical::candidate::tests::summary_type_of_operator_reads_the_builds_summary_type (it builds a CandidateExpression); summary_type::parameter_lists::tests::parameter_list_holds_its_algorithm_without_repeats (one type per list, empty exactly for algorithms without an error model, the depth sets, sizes 28 / 280 / 40).
  • stages::physical::memo_candidates::tests: delivers_on_by_lifetime_without_windows (None in every context; OneEvaluation only in Evaluation); delivers_on_a_chain_when_its_width_divides_the_context_times (grids 10 s @ 0, @ 5 s, @ 2 s, 7 s @ 0; refresh windows 5 s @ 0, 10 s @ 5 s, 10 s @ 2.5 s; widths 5 s and 10 s); delivers_on_a_stored_result_on_its_group_grid (X = the scan's last 5 minutes, grid 60 s @ 0, against 60 s @ 0, 120 s @ 0, 120 s @ 60 s, 30 s @ 0, 60 s @ 15 s, refresh 60 s @ 0, and no grid); producer_satisfies_physical_requirements_over_every_valid_timing_and_lifetime.
  • stages::physical::properties::tests::new_over_every_timing_and_lifetime.
  • types/tests/tumbling_window_chain.rs: a_chain_is_a_state_with_a_time_index, an_instant_time_range_over_a_chain_is_rejected, time_range_and_time_shift_select_a_chains_windows, merged_windows_are_the_build_over_the_selected_range, a_merge_over_a_chain_reads_one_time_range.
  • workload::query::tests::range_evaluation_is_optional_carried_and_validated: FixedInterval without a phase has no grid; FixedIntervalAt keeps its grids.
  • asap-accuracy guarantee::tests::parameter_list_is_the_sizing_over_the_error_grid: every list is today's sizing over the ε grid at δ = 0.01, in order (UnivMon has 28 more: the combined shapes).

End-to-end evidence: not applicable (planning stops at Stage 0's todo!() until PR 2).

Self-review: no over-abstraction (delivers_on reads the two new fields directly; the (period, phase) pair is local; the two const fns exist because the lists are const, replacing hand-written 28- and 280-entry arrays). Duplication removed: one with_depths takes the depth array; the chain and stored-result branches share the context-times match; sketch_algorithm_of, family_of_operator and the sizing wrappers are gone. The lists duplicate today's sizing as constants by design, kept equal by one test. QueryRecurrence::evaluation_grid_ms has no caller in this PR (Pass 2's chain widths and Stage 3's grids use it in PRs 4 and 7).

Follow-ups

@zzylol zzylol added optimizer Workload optimization and plan selection refactor labels Nov 6, 2026
@zzylol
zzylol force-pushed the refactor/1207-properties branch 4 times, most recently from 5ef1ef3 to d9d64df Compare November 6, 2026 15:37
@zzylol
zzylol force-pushed the refactor/1207-properties branch from d9d64df to 2a62109 Compare November 6, 2026 15:47
@zzylol
zzylol force-pushed the refactor/1207-properties branch from 2a62109 to b87b7c0 Compare November 6, 2026 16:30
@zzylol
zzylol changed the base branch from refactor/1207-interfaces to refactor/1207-crates-per-stage November 6, 2026 16:30
@zzylol
zzylol force-pushed the refactor/1207-properties branch 3 times, most recently from 460e9c7 to ba60db8 Compare November 6, 2026 17:12
@zzylol
zzylol force-pushed the refactor/1207-crates-per-stage branch from 4d55373 to ae4843e Compare November 6, 2026 18:56
@zzylol
zzylol force-pushed the refactor/1207-properties branch from ba60db8 to 60a16ee Compare November 6, 2026 18:56
@zzylol
zzylol force-pushed the refactor/1207-crates-per-stage branch from ae4843e to 8d020b0 Compare November 6, 2026 19:10
@zzylol
zzylol force-pushed the refactor/1207-properties branch from 60a16ee to 107a77b Compare November 6, 2026 19:12
@zzylol
zzylol force-pushed the refactor/1207-crates-per-stage branch from 8d020b0 to d1feaad Compare November 6, 2026 21:29
@zzylol
zzylol force-pushed the refactor/1207-properties branch from 107a77b to c7a0bc9 Compare November 6, 2026 21:31
@zzylol
zzylol force-pushed the refactor/1207-crates-per-stage branch from d1feaad to 9a3bb19 Compare November 6, 2026 21:38
@zzylol
zzylol force-pushed the refactor/1207-properties branch from c7a0bc9 to 9e5ce62 Compare November 6, 2026 21:38
@zzylol
zzylol force-pushed the refactor/1207-crates-per-stage branch from 9a3bb19 to 345b5df Compare November 6, 2026 21:57
@zzylol
zzylol force-pushed the refactor/1207-properties branch from 9e5ce62 to 96d3457 Compare November 6, 2026 22:11
@zzylol
zzylol force-pushed the refactor/1207-crates-per-stage branch from cbd0359 to 4445d45 Compare November 6, 2026 22:46
@zzylol
zzylol force-pushed the refactor/1207-properties branch from 96d3457 to b250852 Compare November 6, 2026 22:46
@zzylol
zzylol force-pushed the refactor/1207-crates-per-stage branch from 4445d45 to 84cf80d Compare November 7, 2026 17:47
@zzylol
zzylol force-pushed the refactor/1207-properties branch from b250852 to 7ceb7c3 Compare November 7, 2026 18:04
@zzylol zzylol changed the title refactor(types): properties and requirements from #1207 (#1207 PR 1) refactor(types): properties and requirements from #1207; chains as time-indexed state (#1207 PR 1) Nov 7, 2026
@zzylol
zzylol force-pushed the refactor/1207-crates-per-stage branch from 84cf80d to 69fed95 Compare November 7, 2026 20:51
@zzylol
zzylol force-pushed the refactor/1207-properties branch from 7ceb7c3 to 12a4b96 Compare November 7, 2026 20:55
@zzylol
zzylol force-pushed the refactor/1207-crates-per-stage branch from 69fed95 to dfd6a5b Compare November 8, 2026 15:06
@zzylol
zzylol force-pushed the refactor/1207-properties branch from 12a4b96 to cf93b37 Compare November 8, 2026 15:10
@zzylol
zzylol force-pushed the refactor/1207-crates-per-stage branch from dfd6a5b to b768e8d Compare November 8, 2026 21:29
@zzylol
zzylol force-pushed the refactor/1207-properties branch from cf93b37 to 1ca33f1 Compare November 8, 2026 21:42
@zzylol
zzylol force-pushed the refactor/1207-crates-per-stage branch from b768e8d to 0e0f75a Compare November 9, 2026 03:33
@zzylol
zzylol force-pushed the refactor/1207-properties branch from 1ca33f1 to 3c28259 Compare November 9, 2026 03:45
@zzylol
zzylol force-pushed the refactor/1207-crates-per-stage branch from 0e0f75a to 058ea42 Compare November 9, 2026 04:31
@zzylol
zzylol force-pushed the refactor/1207-properties branch 2 times, most recently from 8030d3f to c2e717b Compare November 10, 2026 01:26
@zzylol
zzylol force-pushed the refactor/1207-crates-per-stage branch from 058ea42 to a4290ee Compare November 10, 2026 13:37
@zzylol
zzylol force-pushed the refactor/1207-properties branch from c2e717b to 13063ab Compare November 10, 2026 13:37
@zzylol
zzylol force-pushed the refactor/1207-crates-per-stage branch from a4290ee to cadf5d5 Compare November 10, 2026 15:01
@zzylol
zzylol changed the base branch from refactor/1207-crates-per-stage to main November 10, 2026 15:31
- LogicalProperties::new(expression, parameters): a sketch build (a chain
  included) needs θ from parameter_list(algorithm) of its
  FieldDataType::Sketch(algorithm); an exact operator has none.
- parameter_list(algorithm): structures only. Count-Min lists each width
  with depths 5, 7, 10 and 12, Count Sketch with 47, 81, 117 and 155 (its
  median-of-rows δ levels 1e-2 … 1e-5); heap variants each of those
  with the 1-2-5 heap sizes; UnivMon 40 entries; KLL, DDSketch, HLL, KMV
  one entry per ε level. A test keeps the lists equal to today's sizing,
  which is test-only.
- PhysicalProperties::new rejects (IngestionTime, OneEvaluation);
  producer_satisfies_physical_requirements(producer, consumer_timing).
- PhysicalCandidate::delivers_on(candidates, context): None in every
  context, OneEvaluation only in Evaluation contexts, a chain with
  AcrossEvaluations when its width divides the grid's period and phase
  (or the refresh window's width and phase), any other AcrossEvaluations
  result everywhere when its group does not depend on the evaluation
  time, else when its group's evaluation grid holds the context's times;
  Materialized(v) contexts stay todo!() for PR 7.
- Chains: one schema (window_end at column 0, keys, state), TimeRange and
  TimeShift over a chain select windows (an instant TimeRange is
  rejected), a SummaryMerge over them finalizes and rolls up as the
  chain's build over that range (the two PR 0 todos).
- summary_type_of_operator; QueryRecurrence::evaluation_grid_ms is None
  for a fixed interval without a phase.
@zzylol
zzylol force-pushed the refactor/1207-properties branch from 13063ab to b65c396 Compare November 10, 2026 15:41
@zzylol
zzylol marked this pull request as ready for review November 10, 2026 15:43
@zzylol
zzylol merged commit 14437bc into main Nov 10, 2026
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

optimizer Workload optimization and plan selection refactor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant