Repository navigation
feat(stage0, types): memo hash-consing, group properties (column_sign, depends_on_evaluation_time, aggregate_coverage), @ end()/@ start() above their subtree (#1207 PR 2) - #1230
Merged
Conversation
zzylol
force-pushed
the
refactor/1207-properties
branch
2 times, most recently
from
November 6, 2026 19:12
60a16ee to
107a77b
Compare
zzylol
force-pushed
the
refactor/1207-memo
branch
from
November 6, 2026 19:14
9e7e404 to
7ea9152
Compare
zzylol
force-pushed
the
refactor/1207-properties
branch
2 times, most recently
from
November 6, 2026 21:38
c7a0bc9 to
9e5ce62
Compare
zzylol
force-pushed
the
refactor/1207-memo
branch
from
November 6, 2026 21:38
54d8b30 to
60c5cfe
Compare
zzylol
force-pushed
the
refactor/1207-properties
branch
from
November 6, 2026 22:11
9e5ce62 to
96d3457
Compare
zzylol
force-pushed
the
refactor/1207-memo
branch
from
November 6, 2026 22:11
60c5cfe to
4915a82
Compare
zzylol
force-pushed
the
refactor/1207-properties
branch
from
November 6, 2026 22:46
96d3457 to
b250852
Compare
zzylol
force-pushed
the
refactor/1207-memo
branch
from
November 6, 2026 22:46
4915a82 to
e710906
Compare
This was referenced Nov 6, 2026
zzylol
force-pushed
the
refactor/1207-properties
branch
2 times, most recently
from
November 7, 2026 20:55
7ceb7c3 to
12a4b96
Compare
zzylol
force-pushed
the
refactor/1207-memo
branch
from
November 7, 2026 21:38
e710906 to
ef0e774
Compare
zzylol
force-pushed
the
refactor/1207-properties
branch
from
November 8, 2026 15:10
12a4b96 to
cf93b37
Compare
zzylol
force-pushed
the
refactor/1207-memo
branch
from
November 8, 2026 15:18
30670f8 to
fe0ff33
Compare
zzylol
force-pushed
the
refactor/1207-properties
branch
3 times, most recently
from
November 9, 2026 04:33
3c28259 to
8030d3f
Compare
zzylol
force-pushed
the
refactor/1207-memo
branch
from
November 9, 2026 04:40
fe0ff33 to
8095a51
Compare
zzylol
force-pushed
the
refactor/1207-memo
branch
from
November 9, 2026 16:33
8095a51 to
49a145c
Compare
…ookups before derivation (#1207 PR 2 review) 1. Aggregate layout: aggregate_output_schema and the new aggregate_output_columns derive the schema and, per output column, its OutputColumnSource (InputColumn, Measure, AddedLabel) in one pass; the column signs read the sources, and a summary state's layout is summary_agg_columns, beside summary_agg_schema. Fixes a `without` count_values aggregate getting a sign for a label column its schema does not have (count_values_without_has_one_sign_per_output_column). 2. find_or_add_group looks the expression up before deriving its properties; add_candidate checks for a duplicate first and then only the node rules (check_candidate_expression), not the group properties. 3. A chain's coverage is of_summary_agg over the chain itself. 4. combined_with: one constraint intersection for the lifted columns. 5. of_window_selection walks to the chain with selected_from. 6. SourceColumn deleted: the memo's declared signs are keyed by source, then column, declared by Stage 0 as its walk reaches each source's Scan (Memo::declare_column_signs replaces Memo::new). 7. tests/memo_fixtures.rs: scan, filter, range, shift and state shared by the two memo test files; group_properties_come_from_input_groups moves to memo_group_properties.rs without its unshareable-LIMIT assertion. a. A group's serialized operator is computed once and stored with its hash bucket entry. b. depends_on_evaluation_time checks the input groups once. c. The SQL timestamp cast target is dtype_to_arrow(&DataType::Timestamp).
zzylol
force-pushed
the
refactor/1207-memo
branch
from
November 10, 2026 16:28
36312cb to
77fc121
Compare
… properties; tests of the aggregate layout (#1207 PR 2 review) - SummaryCoverage::derive_from_inputs and the walk helpers it uses move from ir/properties/summary_coverage.rs to the new stages/logical/aggregate_coverage.rs, next to column_properties.rs; the SummaryCoverage type and its other methods stay in ir/properties. The test of constraint_of moves with it. - aggregate_schema.rs tests aggregate_output_columns for by, without, count_values, absent and a per-entity reduction.
zzylol
commented
Nov 10, 2026
zzylol
marked this pull request as ready for review
November 10, 2026 17:15
zzylol
commented
Nov 10, 2026
zzylol
commented
Nov 10, 2026
…cts; groups record the sources they scan (#1207 PR 2 review) - Memo::new(data) keeps the data workload's facts per source, fixed at construction; a Scan group's column signs read them, a histogram's or summary's series through its family (series_facts, now shared with DataWorkload::series_facts) and a counter's value non-negative. Memo::declare_column_signs and Stage 0's declared_column_signs are deleted. - GroupProperties::scanned_sources replaces reads_only_time_series, with scans_only_time_series(); OperatorInput::reads_only_time_series and its impls are deleted. One rule, scanned_sources in ir/operator/input.rs, derives it: stored per memo group, applied down a node tree by the top-k schema. Pass 1 reads the aggregate group's property. Source derives Ord.
…1207 PR 2 review) check_node_rules -> validate_expression_over_input_groups (an expression over input groups is valid when its output schema derives, its inputs pass validation, and a merge reads only states it may merge). Its one other caller, check_candidate_expression, is inlined into Memo::add_candidate. Docs, the CandidateError::Schema message and a Stage 0 test name say what is checked instead of "node rules".
zzylol
force-pushed
the
refactor/1207-memo
branch
from
November 10, 2026 17:30
c2cc5a1 to
0fef100
Compare
…e names, one layout pass, one equality (#1207 PR 2 review) - D1: OperatorInput::scanned_sources (a memo group's stored set, recomputed for node trees by sources_scanned_by); one scans_only_time_series over a set, read by the top-k schema and Pass 1; topk.rs's own walk is deleted. - D2: SummaryCoverage -> AggregateCoverage, SummaryStateDefinition -> AggregateDefinition, CoverageError::NotSummary -> NoCoverage; ir/properties/aggregate_coverage.rs; tests/memo_aggregate_coverage.rs. - D4: constraint_of and flip beside Constraint in ir/properties; dedup and interval private again. - M1: one expression equality (same_serialized_expression). - M2: OutputColumnSource::Measure { index, input_column }; the signs read the measure's input column from the layout. - M3: summary_agg_layout replaces summary_agg_schema and summary_agg_columns. - M4: validate_expression_over_input_groups returns the schema and derives coverage only to check a merge. - M5, M6: stale docs and doc links. - M7: column_sign -> column_signs; CandidateError::Schema -> NotValidOverInputGroups; depends_on_it inlined. - M8: MemoGroupAsInput, its fields, group_properties, output_column_sign and derive_from_inputs are pub(super). - M9: Stage 0's hash-consing tests move to stages/logical/memo/tests.rs, built through find_or_add_group. - M10: Memo has no Default; Memo::new(&DataWorkload::default()) in tests.
…s derivation, one serialization per expression (#1207 PR 2 review) - D1: AggregateCoverage, AggregateDefinition and CoverageError move to stages/logical/aggregate_coverage.rs with the derivation, now the free function aggregate_coverage; ir/properties/selection.rs keeps the selection algebra (SelectionBox, Constraint, ColumnIdentity, IntervalConstraintLimit, constraint_of). - 1, 2: an expression is serialized once, with its group ids; the hash reads that string; a group is equal by typed equality, then the cached strings; a candidate by typed equality first. - 3: validate_expression_over_input_groups returns a merge's coverage, which derive_group_properties reuses. - 4: OutputColumnSource is crate-private; aggregate_output_columns deleted. - 5, 6: ranked_row_projection's doc restored; stale docs fixed. - 7: ValueSign and the sign derivation (column_signs) in column_properties.rs. - 9: MemoGroup's parents and queries are pub(super); Memo::queries(). - 10: the memo fixtures are in test_support::memo_fixtures, shared by the memo's unit tests and the integration tests. - 11: a measure's input column is computed once. - 12: Source::is_time_series.
…ge quantiles; arithmetic signs - ValueSign documents its meaning: NonNegative means every non-NaN value is >= 0, NonPositive every non-NaN value is <= 0. - A point count or top-k read from a Count Sketch or UnivMon state (and their variants) is Unknown: a median of signed counters can be negative over non-negative weights. SummaryTypeCapability::estimates_may_flip_sign holds the per-algorithm fact; Count-Min keeps the state's sign. - A quantile with q outside [0, 1] is Unknown, for both AggIntent::Quantile and SketchStatistic::Quantile. - Vector BinaryOp arithmetic and ScalarExpr arithmetic share one rule (arithmetic_sign): a sum of two non-negatives (non-positives) is non-negative (non-positive); a product or quotient of same signs is non-negative and of opposite signs non-positive; anything else Unknown. - PromqlVectorFromScalar(expr)'s value has expression_sign(expr).
- The memo compares coverages by AggregateDefinition == (it holds memo group ids); AggregateCoverage::definition_id and its test_support helper are deleted, and the memo coverage test compares definitions only. - ViewCoverage::definition_id is documented as the plan-scope content id: the definition with every input group expanded recursively to its expression down to the Scans, with no group or node ids, canonically serialized and hashed. coverage_of (still todo for PR 6) returns the ViewCoverage and names that content id. - Design docs (asap-primitive-schema.md, physical-planning-and-deployment.md) say the memo compares definitions and the plan carries the content id. - Constraint::Interval documents that a full-range interval is still a constraint: it excludes NULL and NaN rows.
This was referenced Nov 10, 2026
…the plan node - ValueSign is the IEEE 754 sign bit: NonNegative means every non-NaN value has its sign bit clear, NonPositive set. sign_of_number reads the sign bit (-0.0 is NonPositive, NaN Unknown), so `x / -0` over a non-negative x is NonPositive, no longer NonNegative. - SummaryTypeCapability::point_counts_may_flip_sign (was estimates_may_flip_sign) is pub(crate) and an exhaustive match over SketchAlgorithm. - A readout's sign rule takes the state column's type (readout_signs's value(sign, dtype)); the separate state-field lookup is gone. - PhysicalASAPDAGNode::view_coverage carries a materialized view's coverage into the plan; the views code reads it. coverage_of, the IR it needed and coverage_by_node are deleted; operator_nodes is test-only. Stage 3's todo names where it is filled (the content id of the expanded definition). Serde default, so plan files and fixtures are unchanged. - The content id is defined once, on ViewCoverage::definition_id; the design docs link to it. - Test rows without "(was Unknown)" notes.
…rding; shared memo test helpers - root_groups and queries_rooted_at are Memo methods (root_groups(&self), queries_rooted_at(&self, group)) instead of free functions over &[MemoQuery]; the planner CLI calls the method. - sources_scanned_by (ir/operator/input.rs) is Operator::scanned_sources in ir/operator/node.rs, with the other rules generic over the input. - "summary coverage" wording left over from the rename now says aggregate coverage (crate docs, the merge-validation comment, a SQL lowering test comment, the architecture doc and the design doc's section 6.6 row). - aggregate, aggregate_where and readout are in test_support::memo_fixtures, used by the memo unit tests and memo_group_properties instead of their own copies.
…lan's content id by expansion - AggregateCoverage<R>, AggregateDefinition<R> (input_group renamed input) and CoverageError move to ir/properties/aggregate_coverage.rs with the derivation, now Operator::<I: OperatorInput>::aggregate_coverage, generic over OperatorInput::Reference (how a definition names an input). OperatorInput gains `type Reference` and `reference()`; node trees use `()` and store no coverage; the memo instantiates R = MemoGroupId. AggregateCoverage::map_references maps the references. Nothing under ir/ imports crate::stages. - ViewCoverage moves beside PhysicalASAPDAGNode. stages::view_coverage expands a group's coverage (every input group to its expression, down to the Scans) and hashes its JSON (FNV-1a 128) into definition_id; the selection is the memo's. Stage 3's todo names it. - Tests: equal definition_id across memos with different group ids, another source gives another id, no coverage gives None. - asap-primitive-schema.md: the IR type and derivation, map_references, the content id by expansion; derive_from_inputs and input_group gone.
This was referenced Nov 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Memomethods, Stage 0 and the group propertiestodo!(), so every planning test stopped at Stage 0; andm[5m] @ end()lowered in place was a group of its own, sharing nothing with the same selector without@(Range queries: evaluate@ end()/@ start()subtrees once per request and share them #1251).find_or_add_group), candidate storage (add_candidate),add_querywith each query's accuracy target andcolumn_signs,depends_on_evaluation_time,scanned_sourcesandaggregate_coveragederived once per group; PromQL@ end()/@ start()lifted above the subtree they anchor.sum by (host) (rate(requests_total[1m]))over a declared counter now gets aNonNegativesum column,sum by (host) (gauge)anUnknownone (so Stage 2 later prunes its Count-Min heap top-k); a query overmwith 1000 declared series enters withnumber_of_values_per_query_evaluation == Some(1000);latency[5m] @ end()shares thelatency[5m]group with the instant query. Planning now passes Stage 0 and stops at Pass 1's target discovery (PR 3).Review guide
14437bc47); refactor(types, frontends): column lineage replaces table qualifiers (#1207 PR 2b) #1233 (PR 2b, lineage) builds on it. Twelve commits, headbba5e221c: the restacked PR 2 (3cabdc3b5), the review fixes (77fc121d3), the coverage move with the aggregate-layout tests (c71be0331), the source facts and scanned sources (f458196ba), the expression check named by what it checks (0fef10001), the second review (1ae8acf91: D1, D2, D4, M1–M10), the third review (f68d414aa: D1 option A, items 1–12), the sign rules (c2f23d760: theValueSigndefinition and sign items 1–5), the two-scope definition identity (73785fa07), the sign bit, view coverage on the plan node and review cleanups (288cd70d6), the memo query methods,Operator::scanned_sources, coverage wording and shared test helpers (0de33fd94), and coverage in the IR with the plan's content id by expansion (bba5e221c). This PR changes only core data structures, interfaces and definitions; case-specific fixes from review go to follow-up issues and PRs. Diff vs base: 67 files, +5,846 / −1,684 (most of it the coverage type and derivation moving out ofsummary_coverage.rs).36312cbaf, backupbackup/1207-layout-memo-36312cbaf) in the scope layout, content unchanged:Memomethods instages/logical/memo/mod.rs, hash-consing instages/logical/memo/hash_consing.rs, group-property derivation instages/logical/group.rs, the sign derivation instages/logical/column_properties.rs(PR 2b adds lineage there), the memo unit tests instages/logical/memo/tests.rs. The second commit applies the 2026-11-10 review (items 1–7, a–c under Decided); the third moves the coverage derivation tostages/logical/aggregate_coverage.rs(B1) and adds theaggregate_output_columnstests (B4), moves only. Review the three apart.types/src/stages/logical/memo/mod.rs:Memo::new,find_or_add_group,add_candidate,add_query;types/src/stages/logical/memo/hash_consing.rs.types/src/stages/logical/group.rs(validate_expression_over_input_groups,derive_group_properties) and the newtypes/src/stages/logical/column_properties.rs;types/src/ir/schema/aggregate_schema.rs(OutputColumnSource).stage0-memo/src/lib.rs:memo_of_queries.types/src/stages/logical/aggregate_coverage.rs(AggregateCoverage,aggregate_coverage,time_constraint,of_aggregate),types/src/ir/properties/selection.rs(SelectionBox,Constraint,constraint_of) andfrontend-promql/src/promql.rs(anchored,selector).frontend-common/frontend-sql; the top-k schema and Pass 1 readOperatorInput::scanned_sources/GroupProperties::scanned_sourcesinstead of their own walks; test fixtures; concepts doc andasap-primitive-schema.md§4.2.2 updates.same_serialized_expression) and the shareability rule; the per-operator sign table; the coverage time intervals and where the walk stops (TimeShift { at: End | Start }); counter states merging only over intervals disjoint in time.Behavior: before → after
Before (on PR 1),
memo_of_queries,find_or_add_group,add_candidateandadd_querypanic attodo!();m[5m] @ end()lowered asTimeRange(TimeShift(@ end())(Scan))in a range query and asm[5m]in an instant one, through arange_evaluatedflag (previous revision). It serves the design need that rules add groups and candidates only through these methods and derive every group property once.sum by (host) (rate(requests_total[1m])), a counterNonNegative,depends_on_evaluation_timetruesum by (host) (gauge), nothing declaredUnknownlatencydeclared{"value": "non_negative"},quantile_over_time(0.99, latency[1m])NonNegativem[5m] @ 1700000000depends_on_evaluation_timefalse; coveragets ∈ (1699999700000, 1700000000000]latency[5m] @ end()TimeShift(@ end())over the group oflatency[5m], shared with instant queriesmwith 1000 declared seriesMemoQuery::number_of_values_per_query_evaluation == Some(1000)quantile_over_time(0.9, latency[1h]) / quantile_over_time(0.5, latency[1h]),latencynon-negativeNonNegative(Unknownatf68d414aa)Interfaces
Where PR 2's code is (scope layout)
Memo(with the data workload's source facts) andMemoQuery, the memo operations (new,find_or_add_group,add_candidate,add_query) and the hash-consing teststypes/src/stages/logical/memo/mod.rstypes/src/stages/logical/memo/hash_consing.rsMemomethod unit teststypes/src/stages/logical/memo/tests.rsGroupProperties(incl.scanned_sources),MemoGroup(parents,queriescrate-private behind their accessors), the expression check (validate_expression_over_input_groups),MemoGroupAsInput,derive_group_properties,depends_on_evaluation_timetypes/src/stages/logical/group.rsValueSignand thecolumn_signsderivationtypes/src/stages/logical/column_properties.rs(PR 2b adds lineage)AggregateCoverage,AggregateDefinition,CoverageErrorand their derivation (aggregate_coverageand its walk)types/src/stages/logical/aggregate_coverage.rsSelectionBox,Constraint,ColumnIdentity,IntervalConstraintLimit,constraint_oftypes/src/ir/properties/selection.rsOutputColumnSource,aggregate_output, both crate-private) and its teststypes/src/ir/schema/aggregate_schema.rssummary_agg_layout)types/src/ir/operator/asap.rsOperatorInput::scanned_sources, the rulesources_scanned_by(aScan's source, else the union over its inputs)types/src/ir/operator/input.rs;Source::is_time_seriesinir/operator/arguments.rsseries_facts, shared byDataWorkload::series_facts)types/src/workload/data.rsmemo_of_queries, the declared signs, its tests)stage0-memo/src/lib.rstypes/tests/memo_group_properties.rs,memo_aggregate_coverage.rs;types/src/test_support/memo_fixtures.rs(featuretest-support)crates/types/src/stages/logical/memo/mod.rs·crates/types/src/stages/logical/memo/hash_consing.rscrates/types/src/stages/logical/group.rs·crates/types/src/stages/logical/column_properties.rs/// What every candidate of a group produces, derived once from its expression and input groups. The type stays. pub struct GroupProperties { /// Output schema. pub schema: Schema, /// Output kind. pub result_kind: OperatorResultKind, /// Per output column: the sign of its values, from the input groups' and, for a `Scan`, the declared ones /// (now derived; was `todo!()`). - pub column_sign: Vec<ValueSign>, + pub column_signs: Vec<ValueSign>, /// Whether the result changes with the evaluation time: a `TimeRange`, `time()`/`now()`; false under `@ T` /// (now derived; was `todo!()`). pub depends_on_evaluation_time: bool, /// What an aggregate group covers: a summary state, a chain, a `TimeRange`/`TimeShift` over a chain or a /// merge over one, or an exact `Aggregate`; None otherwise (chains, window selections, exact Aggregates added). - pub aggregate_coverage: Option<SummaryCoverage>, + pub aggregate_coverage: Option<AggregateCoverage>, /// Whether two equal expressions return the same rows (not an unordered `LIMIT`). pub shareable_when_equal: bool, + /// The sources the `Scan`s below this group read: a `Scan`'s own, else the union over its input groups. + pub scanned_sources: BTreeSet<Source>, } pub enum CandidateError { - Schema(SchemaDerivationError), + /// The expression is not valid over its input groups. + NotValidOverInputGroups(SchemaDerivationError), OutputDiffersFromGroup { group: MemoGroupId }, } +/// Whether `operator` is a valid expression over its input groups: its output schema derives, its inputs pass +/// validation, and a merge reads only states it may merge. Returns the output schema. +pub(super) fn validate_expression_over_input_groups(operator: &Operator<MemoGroupAsInput<'_>>) + -> Result<Schema, SchemaDerivationError>;crates/types/src/ir/operator/input.rs·crates/types/src/ir/operator/arguments.rs/// An input of an operator: a node of a tree or a memo group. The trait stays. pub trait OperatorInput: Sized + Clone { /// Its schema. fn schema(&self) -> &Schema; /// Its result kind. fn result_kind(&self) -> OperatorResultKind; /// The input's own operator, its children as inputs again. fn operator(&self) -> Operator<Self>; /// What the input covers when it is an aggregate group in the memo; None otherwise. - fn aggregate_coverage(&self) -> Option<&SummaryCoverage>; + fn aggregate_coverage(&self) -> Option<&AggregateCoverage>; + /// The sources the `Scan`s at or below the input read: a memo group's stored set, recomputed for a node tree. + fn scanned_sources(&self) -> BTreeSet<Source>; } +/// A `Scan`'s source, else the union over its inputs' `scanned_sources`. +pub(crate) fn sources_scanned_by<I: OperatorInput>(operator: &Operator<I>) -> BTreeSet<Source>; -#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)] +#[derive(Debug, Clone, PartialEq, Eq, Hash, PartialOrd, Ord, Serialize, Deserialize)] pub enum Source { TimeSeries { metric: String }, Table { table_ref: String } } +impl Source { + /// Whether it is a time series (PromQL rows); the top-k schema and Pass 1 read `.all(Source::is_time_series)`. + pub fn is_time_series(&self) -> bool; +} +// workload/data.rs: `DataWorkload::series_facts` over a sources map; the method delegates to it. +pub(crate) fn series_facts<'a>(sources: &'a BTreeMap<String, SourceFacts>, series: &str) + -> Option<(&'a SourceFacts, Option<PrometheusMetricType>)>;crates/stage0-memo/src/lib.rs/// Stage 0: the memo groups of the roots, query `i` with `accuracy_targets[i]` and its n_q (`values_per_result`); /// `Scan`s read `data`'s declared signs. Fails with the first node whose expression is not valid over its children's groups (a frontend bug) /// (body filled in). pub fn memo_of_queries( roots: &[QueryRoot], accuracy_targets: &[Option<AccuracyTarget>], data: &DataWorkload, ) -> Result<Memo, SchemaDerivationError>;PromQL lowering has no interface change; its output shape for
@ end()/@ start()changes (TimeShift { at }above the selector). The top-k schema's and Pass 1's private walks for PromQL rows are replaced byscanned_sources.Usage, end to end (Stage 0 tests
a_scan_reads_the_declared_column_signs,each_query_enters_with_the_values_one_evaluation_returns; types testa_top_k_ranks_values_of_its_inputs_sign):And
@ end()(promql lib testat_start_and_at_end_are_lifted_above_the_selector_they_anchor, types testat_end_and_at_start_share_the_subtree_they_anchor_and_stay_unlifted):crates/types/src/ir/schema/aggregate_schema.rs·crates/types/src/ir/operator/asap.rs·crates/types/src/stages/logical/aggregate_coverage.rs·crates/types/src/ir/properties/selection.rsAlgorithm changes
Stage 0 hash-consing
cse::same_node, deleted in PR 0) and the group lookup the rules need were two equalities that could disagree (on a guarantee, for one).Rctable, then numbered and paired with groups. After: 1.group_ofmaps each node by address to its group (a diamond is visited once), children first, callingMemo::find_or_add_group(operator over child groups); it derives the properties, hashes the operator's arguments with the input group ids (hash_consing::expression_hash) and compares the bucket withsame_expression(only shareable expressions are indexed). 2.add_queryper root, with its target andvalues_per_result(root, series_read(root, data), data)(Nonewhendatabounds nothing).add_candidatechecks before changing anything, then appends, adds parents, and walks down from the inputs inserting the group's queries in sorted position (stopping below a group that already had them).declared_column_signswalks the roots once for the scanned sources and readsDataWorkload::series_facts(SourceFacts::column_signs), so a histogram's_sum/_countseries get the family's declarations and a declared counter'svalueis non-negative without a declaration.memo_tests10.Aggregate coverage with time intervals
todo!();@ Tand SQL timestamp filters had no representation; exact aggregates had no coverage.TimeRangegave a separate relative time field and a chain expanded to a build over count × width. After,time_constraintsums the offsets of theTimeShifts on the path, takes the one range width and the one@ T, and gives(anchor − o − W, anchor − o]on the timestamp column withIntervalConstraintLimitends (left-open, right-closed):m[5m] offset 1h→(t − 65m, t − 60m],@ T→ fixed timestamps. A chain is a build over its input with time unconstrained, and a window selection over it adds the interval, soSummaryMerge(TimeRange(TimeShift(Chain[w](X))))covers whatSummaryAgg(TimeRange(TimeShift(X)))does andadd_candidateaccepts it in that group.DisjointInTimecompares the timestamp column (counter states merge only over intervals disjoint in time). An exactAggregateis a top unlifted operator. SQL folds a timestamp cast of a literal and resolution drops a cast to the operand's own type, sots >= '2026-11-01'lifts. The walk inof_aggregatestops atTimeShift { at: End | Start }: it is the definition'sinput_group.TimeRange's window iff its timestamp is in the interval; equal coverage means equal rows. Tests:memo_aggregate_coverage.rs(13).TimeRanges or@ Ts.Value sign, evaluation-time dependence, time-series reads
output_column_sign(operator, output, source_facts)per the doc's "Value sign" table: Scan: declared sign, elseUnknown; Filter/TimeRange/TimeShift/Sort/Limit/Dedup/column Project: the input's;absNonNegative, negation flips, numeric literals and casts; sum/avg/min/max/quantile/top-k/first/last keep the value's sign; counts and spreadsNonNegative;rate/irate/increaseNonNegativeover aNonNegativeinput; a state records the sign of its update values; a merge keeps the sign its inputs agree on; a readout reads the state's (a distinct count, L2 norm or entropy isNonNegative); anything elseUnknown. It follows each operator's output layout (aggregate_output_schema's;chain_schema's for a chain:window_endUnknown, the build's time column dropped; a merge over a chain's windows reads the chain's build); an unknown layout isUnknowneverywhere; adebug_assertchecks one sign per output column.depends_on_evaluation_time: true for aTimeRangeand for a scalar expression readingtime()/now(); false forTimeShift { at: Some(Timestamp(T)) }; otherwise true when an input group's is.reads_only_time_series: aScanof a time series, or every input group reading only time series;OperatorInput::reads_only_time_seriescomputes the same rule over node trees (one rule,ir::operator::input::reads_only_time_series).memo_group_properties.rs(10 tests; sign rules under their own section), Stage 0'sa_scan_reads_the_declared_column_signs.Sign rules (commit
c2f23d760)x / 1000,x * 8and a ratio of quantiles lost a knownNonNegative.NonNegativemeans every non-NaN value is ≥ 0,NonPositiveevery non-NaN value is ≤ 0 (documented onValueSign); a NaN does not break either.f68d414aa)c2f23d760)CountSketch,CountSketchFold,CountSketchWithHeap,CountSketchWithL2Norm,CountSketchWithSpaceSaving,HydraCountSketch,UnivMon,UnivMonQ,UnivMonTwoTier,HydraUnivMon)UnknownAggIntent::Quantile/SketchStatistic::Quantilewith q outside [0, 1]UnknownBinaryOparithmetic andScalarExpr::Arithmetic(one rule,arithmetic_sign)Unknown+: bothNonNegative→NonNegative, bothNonPositive→NonPositive;*and/: same signs →NonNegative, opposite →NonPositive;-,%,^,atan2and anything unknown →UnknownPromqlVectorFromScalar(expr)(vector(s))Unknownexpression_sign(expr)(vector(1000)→NonNegative)SummaryTypeCapability::estimates_may_flip_sign(summary_type)next to the other summary-type facts; no new trait.quantile_signserves both quantile forms.x / 1000lowers to aProjectofvalue / 1000(ScalarExpr::Arithmetic), SQLvalue * 8likewise, so both use the shared rule. Tests: the five sign tests listed under Verification inmemo_group_properties.rs.@ end()/@ start()loweringPromqlLowerer::range_evaluated). After, the selector or subquery X is lowered without the@(its offset and an@ Tstay in X), andTimeShift { offset_ms: 0, at: Some(End | Start) }is put above it; instant and range-evaluated queries lower alike.at_start_and_at_end_are_lifted_above_the_selector_they_anchor,at_end_and_at_start_share_the_subtree_they_anchor_and_stay_unlifted, conformanceat_modifier_lowers_to_a_time_shift.Decisions and open questions
Needs reviewer decision: None.
Decided:
Coverage design, 2026-11-10 (approved; commit
bba5e221c):AggregateCoverage<R>,AggregateDefinition<R>(fieldinput_grouprenamedinput;unlifted_operatorskept) andCoverageErrorare inir/properties/aggregate_coverage.rswith the derivation,Operator::<I: OperatorInput>::aggregate_coverage, an IR rule likeoutput_schema.OperatorInputgainstype Reference: Clone + PartialEqandreference(), and keepsaggregate_coverage()(stored coverage).reference()returnsOption<Self::Reference>: a rule's temporary build has none, so the wrapperInputOrTemporaryBuild<I>keepsReference = I::Referenceand still delegates stored coverage to the wrapped input. Node trees:Reference = (), no stored coverage. The memo:MemoGroupAsInput::Reference = MemoGroupId,GroupProperties::aggregate_coverage: Option<AggregateCoverage<MemoGroupId>>; merges still compare definitions with==.SchemaDerivationError::Coverage(CoverageError)stays in ir, now ir-internal. Nothing underir/importscrate::stages(two doc links remain, toPhysicalPropertiesandPhysicalCandidate::properties).ViewCoverageis besidePhysicalASAPDAGNodeinplan/physical_asap_dag.rs(the planner re-export keeps its name);stages::view_coverage(memo, group)(stages/logical/aggregate_coverage.rs, the only memo-scope glue left there) maps the group's coverage references to each input group's expression, recursively down to theScans (ExpandedExpression, no ids), and setsdefinition_idto the 128-bit FNV-1a hash, in hex, of that definition's JSON (serde_f64for non-finite floats; no new dependency); the selection is the memo's. Stage 3 setting the node field staystodo!inselect_plan, which namesview_coverage(memo, its group).Layout, 2026-11-10 (approved; commit
0de33fd94; items 1–3 of that review were replaced by the coverage design below, commitbba5e221c):root_groupsandqueries_rooted_atareMemomethods; the planner CLI's memo JSON callsmemo.queries_rooted_at(group).sources_scanned_by(ir/operator/input.rs) isOperator::scanned_sourcesinir/operator/node.rs, in the block of rules generic overOperatorInput;GroupProperties::scanned_sourcesand bothOperatorInputimpls call it.types/src/lib.rs, theSummaryMergevalidation comment inir/operator/asap.rs,frontend-sql/tests/sql_lowering.rs,docs/develop_docs/asap-aware-mapping-architecture.md, and the §6.6 row ofasap-primitive-schema.md's section table. Left:docs/user_guide_docs/run-a-query.md's "Check summary coverage" (which summary candidates Pass 1 offers, another meaning) andplanner_fuzz'scampaign_writes_summary_coverage_and_drafts(the campaign's summary file).aggregate,aggregate_whereandreadoutare intest_support::memo_fixtures; the memo unit tests (quantile_readout,quantile_agg) andmemo_group_properties.rs(aggregate,readout) use them.quantile_aggwithout a filter now buildsfilters: vec, asmemo_group_propertiesdid; its tests compare groups built the same way.SketchAlgorithmvariants have no arms inpoint_counts_may_flip_sign:asap-typeshas noexperimentalfeature (kept as is).Sign bit, readout dtype, view coverage, 2026-11-10 (approved; commit
288cd70d6):ValueSignis the IEEE sign bit.NonNegative: every non-NaN value has its sign bit clear;NonPositive: set.sign_of_number: NaN →Unknown,is_sign_negative()→NonPositive, elseNonNegative(-0.0isNonPositive). Rules checked under it:+(two clear or two set sign bits keep it),*and/(XOR of sign bits, exact in IEEE), negation (flips),abs(clears), min / max / sum / avg / quantile interpolation (combine values of one sign bit),rate/irate/increaseover a non-negative counter (x − xis+0): all hold, none adjusted. The concepts doc (docs(concepts): memo groups, properties, requirements, per-stage interfaces and rules #1224) states the old "≥ 0" definition and needs the sign-bit wording when it is synced.SummaryTypeCapability::point_counts_may_flip_sign(wasestimates_may_flip_sign),pub(crate), an exhaustivematchoverSketchAlgorithmwithout_, same results. No arms forasap_sketchlib'sexperimentalvariants (CmsHll,MicroCm,UniformSampling): the planner never buildsasap_sketchlibwithexperimental, andasap-typeshas no such feature, so#[cfg(feature = "experimental")]there is an unknown-cfg warning (an error under-D warnings); adding the feature toasap-typesmakes--all-featuresbuilds enable it and breaks the other exhaustive matches (parameter_list,asap-accuracy's guarantee table). The doc says so, asparameter_listalready does by omission.PhysicalASAPDAGNode::view_coverage: Option<ViewCoverage>(serde default, skipped whenNone), set when the plan is built from the memo (Stage 3, where the node's group is known);select_plan'stodo!names it: "… each materialized view node's view_coverage from its memo group's aggregate_coverage, with definition_id the content id of the expanded definition". The views code reads it (ViewWithoutCoveragewhen absent);coverage_of, theoperator_nodes()IR built for it andcoverage_by_nodeare deleted;operator_nodesis now test-only (query_rootsreads it). No wire-version bump: the field is optional and absent from every fixture.readout_signspasses the state column's type to its rule (value(sign, &field.dtype)); the separate state-field lookup is deleted.Unknown)" notes.ViewCoverage::definition_id;asap-primitive-schema.mdandphysical-planning-and-deployment.mdlink to it.pub enum ValueSign { - /// Every non-NaN value is ≥ 0. + /// Every non-NaN value has its sign bit clear (`+0.0`, positive, `+inf`). NonNegative, - /// Every non-NaN value is ≤ 0. + /// Every non-NaN value has its sign bit set (`-0.0`, negative, `-inf`). NonPositive, Unknown, } impl SummaryTypeCapability { - pub fn estimates_may_flip_sign(summary_type: &FieldDataType) -> bool; + pub(crate) fn point_counts_may_flip_sign(summary_type: &FieldDataType) -> bool; } pub struct PhysicalASAPDAGNode { … #[serde(skip)] pub selected_physical_candidate: Option<PhysicalCandidateId>, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub view_coverage: Option<ViewCoverage>, } impl PhysicalASAPDAG { - pub(crate) fn operator_nodes(&self) -> Result<HashMap<PhysicalASAPNodeId, Rc<OperatorNode>>, PhysicalASAPDAGValidationError>; + #[cfg(any(test, feature = "test-support"))] + fn operator_nodes(&self) -> Result<HashMap<PhysicalASAPNodeId, Rc<OperatorNode>>, PhysicalASAPDAGValidationError>; } -fn coverage_of( - id: PhysicalASAPNodeId, - inputs: &MaterializedViewInputs, - ir: &HashMap<PhysicalASAPNodeId, Rc<OperatorNode>>, - coverage_by_node: &HashMap<PhysicalASAPNodeId, ViewCoverage>, -) -> Result<ViewCoverage, CoverageError>;Definition identity, 2026-11-10 (approved; commit
73785fa07): two scopes, two identities.AggregateDefinition==(it holdsMemoGroupIds); nothing in the memo needs an id.AggregateCoverage::definition_id()(an FNV hash of the definition'sDebugtext, ids included) andtest_support::definition_idare deleted;states_with_equal_definitions_and_disjoint_selections_mergecompares the definitions only.ViewCoverage::definition_idis the content id of what the view computes: its definition with every input group expanded recursively to its expression down to theScans, with no group or node ids, canonically serialized and hashed; equal across plans and processes that compute the same thing, different when they do not.coverage_of(private,todo!for PR 6) now returns theViewCoverage: "Tracking: implement the optimization formulation (memo candidates, properties, edges y_{u,c,i}, MILP) as named code per stage #1207 PR 6: the view's coverage from its memo group's aggregate_coverage, with definition_id the content id of the expanded definition". Design docs updated:asap-primitive-schema.md(theAggregateCoverageinterface twice, worked example case 2, the fields list) andphysical-planning-and-deployment.md(the read table, the matching paragraph, the materialized-views mapping, the store-keyed test row).materialized_view_fixtures/*.jsonholddefinition_idas opaque hex strings that tests only use as keys and compare; they stay.Constraint::Intervaldocuments that(Unbounded, Unbounded)is still a constraint (it excludes NULL and NaN rows), not the absence of one.Sign rules, 2026-11-10 (approved; commit
c2f23d760):NonNegative: every non-NaN value is ≥ 0;NonPositive: every non-NaN value is ≤ 0.Unknown; Count-Min keeps the state's sign. The fact isSummaryTypeCapability::estimates_may_flip_sign.Unknown, forAggIntent::QuantileandSketchStatistic::Quantile.BinaryOpArithmetic: the rule in the table under Algorithm changes, one shared fn (arithmetic_sign).PromqlVectorFromScalar(expr): the value column's sign isexpression_sign(expr, &[]).ScalarExprarithmetic inexpression_signusesarithmetic_sign.Third review, 2026-11-10 (approved; commit
f68d414aa; D2, deletingtopological_order,set_candidate_input_requirementsandroot_groups, rejected: later PRs use them, firstset_candidate_input_requirementsin PR 3 (#1232) and PR 4,topological_orderin PR 4 (Pass 2 requirements) and PR 6 (Stage 3 cost),root_groupsin PR 4 (dead candidates) and PR 5 (pruning)):bba5e221c).AggregateCoverage,AggregateDefinitionandCoverageErrorare instages/logical/aggregate_coverage.rswith their derivation, the free functionaggregate_coverage(operator)(pub(super)).ir/properties/selection.rs(renamed fromsummary_coverage.rs) keeps only the IR selection algebra:SelectionBox,Constraint,ColumnIdentity,IntervalConstraintLimit,constraint_of/flip,union/intersect/disjoint. Dependency direction:ir/operator/input.rs(OperatorInput::aggregate_coverage),ir/schema/error.rs(SchemaDerivationError::Coverage) andplan/materialized_view.rsimport the type fromstages(ir → stages, as forMemoGroupIdalready);stages/logical/aggregate_coverage.rsimports the selection algebra fromir/properties/selection.rs.serialized_expression,.expect); the hash reads that string; a group is equal by typed==, then the cached strings (thesame_valuesemantics forNaNand-0).operator_and_arguments, the extrasame_expressionpath, the separate children hashing and theOptionplumbing are deleted.==first; the serializations are compared only when typed-equal.validate_expression_over_input_groupsreturns a merge's coverage, whichderive_group_propertiesreuses: a merge's coverage is derived once.OutputColumnSourceispub(crate)and no longer re-exported;aggregate_output_columnsis deleted; callers useaggregate_output.ranked_row_projection's doc restored; stale docs:find_or_add_group,ir/properties/mod.rs, the coverage error message,input.rs,materialized_view.rs, the design doc's §4.2 and §6.6 titles,AggregateDefinition's field docs (also exactAggregate), thememo_aggregate_coverage.rsheader.ValueSignis incolumn_properties.rswith its derivation,column_signs(wasoutput_column_sign).MemoGroup::parentsand::queriesarepub(super)(read throughparent_groups()andreached_by_queries());Memo::queriesis private, read throughMemo::queries()(Stage 0, Pass 1, devtools and integration tests updated).types/src/test_support/memo_fixtures.rs(featuretest-support, andcfg(test)), shared bystages/logical/memo/tests.rsand the memo integration tests; the unit tests'scanandkll_statecopies are deleted. Kept: the unit tests'service_scanand Stage 0'smscan, whose[ts, service, value, latency]column order the sharedscan([ts, value, columns…]) does not produce.group_properties_come_from_input_groupsuses its file's helpers.aggregate_schema.rscomputes a measure's input column once per measure on both paths.Source::is_time_seriesreplacesscans_only_time_series; the top-k schema and Pass 1 read.iter().all(Source::is_time_series).Second review, 2026-11-10 (approved; commit
1ae8acf91; D3, renamingcolumn_signaway, rejected because PRs 4, 5 and 8 read it):OperatorInput::scanned_sources()(likeaggregate_coverage()): a memo group returns its storedGroupProperties::scanned_sources; a node tree and a rule's temporary build compute it withsources_scanned_by. Onescans_only_time_series, a free function over a set inir/operator/input.rs, read by the top-k schema and Pass 1;GroupProperties::scans_only_time_seriesand the top-k schema's own walk are deleted.SummaryCoverage→AggregateCoverage,SummaryStateDefinition→AggregateDefinition,CoverageError::NotSummary→NoCoverage; the type's file isir/properties/aggregate_coverage.rsand the derivation'sstages/logical/aggregate_coverage.rs(same name, two scopes: the type in the IR's properties, its derivation over memo groups besidecolumn_properties.rs);tests/memo_aggregate_coverage.rs.constraint_ofandflipare besideConstraintinir/properties/aggregate_coverage.rs;dedupandintervalare private again. Crate visibility left:constraint_of,union,Constraint::intersect,Constraint::disjoint,SelectionBox::disjoint,AggregateCoverage::definition_id.same_serialized_expression(typed equality confirmed by the serialized operator, each computed once);same_expressionis it with both sides serialized.OutputColumnSource::Measure { index, input_column }: the signs read the measure's input column from the layout; the re-derived probe column and measure input are deleted.summary_agg_layout(schema and sources in one pass) replacessummary_agg_schemaandsummary_agg_columns.validate_expression_over_input_groupsreturns the schema and derives coverage only for aSummaryMerge(its check);derive_group_propertiesderives the coverage itself.asap-primitive-schema.md(@ Tlifted,@ start()/@ end()not), thememo_group_properties.rsheader, this body; doc links toGroupProperties::shareable_when_equalby full path (cargo docchecked).column_sign→column_signs;CandidateError::Schema→NotValidOverInputGroups;depends_on_itinlined.MemoGroupAsInput, its fields,group_properties(),output_column_signandderive_from_inputsarepub(super)(onlystages/logicalreads them).stages/logical/memo/tests.rs, built throughfind_or_add_group;a_scan_reads_the_declared_column_signskeeps one assert thatmemo_of_queriespasses the data's facts through.Memohas noDefault; tests useMemo::new(&DataWorkload::default()).Layout decisions, 2026-11-10:
memo/mod.rs:40).Memoholds the data workload's facts per source, fixed at construction (Memo::new(data)) and read-only afterwards, instead of signs declared during Stage 0's walk. Reason: the memo's data says what it is (the data workload's source facts), not a by-product of Stage 0's walk, and nothing mutates it after construction; Stage 0 needs no walk for it at all. AScangroup's signs read the facts where they are derived, a histogram's or summary's series through its family (series_facts, the one lookupDataWorkload::series_factsalso uses) and a declared counter'svalueNonNegative. Stage 0'sdeclared_column_signsandMemo::declare_column_signsare deleted. Compared with PR 8's final shape: the field is the same (source_facts: BTreeMap<String, SourceFacts>); PR 8 builds it in Stage 0 keyed by the scanned series name (family already resolved, counters'valuealready filled), and takes the map (Memo::new(source_facts)). Here the map isDataWorkload::sourceskeyed by family and resolved at theScan, so at restack PR 8's readers ofsource_facts(max_distinct_per_evaluation, the lineage-keyed sign lookup) go throughseries_facts.group.rs:83).GroupProperties::scanned_sources: BTreeSet<Source>withscans_only_time_series()replacesreads_only_time_series;OperatorInput::reads_only_time_seriesand its three impls are deleted. One rule derives it,scanned_sourcesinir/operator/input.rs: a memo group stores it from its input groups' (group.rs), and the top-k schema inir/schema/topk.rs, which also derives node trees' schemas (no group property there), applies the same rule down the tree. Pass 1 reads the aggregate group'sscans_only_time_series().SourcegainsPartialOrd, Ord(twoStringvariants) for the set. For the restack: PR 8'sMemo::max_distinct_per_evaluationwalks the scans below a group; it could sum overscanned_sourcesinstead (note: the walk counts eachScangroup, the set each source once).SummaryCoverage's derivation over memo groups (derive_from_inputsand its walk helpers) is instages/logical/aggregate_coverage.rs, next tocolumn_properties.rs: like the column signs, it is a group property read from the input groups. TheSummaryCoveragetype and its other methods stay inir/properties/summary_coverage.rs; the test ofconstraint_ofmoves with it. Visibility the move forces:interval,dedup,union,Constraint::intersect,Constraint::disjointandSelectionBox::disjointarepub(crate); the unusedstages::MemoGroupAsInputre-export is deleted.aggregate_schema.rstestsaggregate_output_columnsforby,without,count_values,absentand a per-entity reduction, each checking one source per output column.group.rs:113).check_node_rulesisvalidate_expression_over_input_groups: it checks that an expression is valid over its input groups (its output schema derives, its inputs pass validation, a merge reads only states it may merge); "node rules" named neither what nor over what.check_candidate_expressionhad one caller, so it is inlined intoMemo::add_candidate. TheCandidateError::Schemamessage and docs say what is checked; Stage 0's test ismemo_of_queries_returns_the_error_of_a_node_invalid_over_its_input_groups.group.rs) and B3 (thedelivers_ontests' location): no change.Review fixes, 2026-11-10 (approved; commit
77fc121d3):column_properties.rsre-derived theby/without/per-entity/count_values/absentlayoutaggregate_schema.rsowns, faked aSummeasure for summary states, and re-derived the chain layout. Nowaggregate_output_columns(one pass withaggregate_output_schema) gives each output column'sOutputColumnSource, which the signs read; a summary state's layout issummary_agg_columnsbesidesummary_agg_schema; the chain keeps the build's layout without its time column, afterwindow_end. A measure's input value is its first input column, else the columnaggregate_output_schematypes it by (the sample value, else the first column not kept). Bug fixed:count_values without (host) ("version", v)got a sign for a label column awithoutaggregate does not have (one sign too many, adebug_assertpanic); regression testcount_values_without_has_one_sign_per_output_column(failed before the fix, passes after).find_or_add_grouplooks the expression up first and derives properties only on a miss (a hit equals a group that was validated; only shareable groups are listed, and an equal expression is as shareable).add_candidatechecks for a duplicate first, then only validates the expression over its input groups (validate_expression_over_input_groups: schema, input validation, merge coverage), not the group properties.of_summary_aggtakes aSummaryAggor aSummaryTumblingWindowChain; the chain arm that rebuilt aSummaryAggis deleted.combined_with: one constraint intersection for the three lifted-column sites.of_window_selectionwalks to the chain withselected_from; aFilteron the way still gives no coverage.SourceColumndeleted (it was never serialized; supersedes Q1 below): the declared signs are keyed by source, then column (the shape ofSourceFacts::column_signs), and Stage 0 declares a source's signs as its one walk (group_of) reaches the source'sScan(Memo::declare_column_signsreplacesMemo::new); the separate walk is deleted.tests/memo_fixtures.rsholdsscan(metric, columns, predicates),filter,range(width_ms, child),shift(offset_ms, at, child)andstate(summary_type, child)formemo_group_properties.rsandmemo_summary_coverage.rs(each keeps a one-line wrapper for its source,series/latency).group_properties_come_from_input_groupsmoved tomemo_group_properties.rs, without its unshareable-LIMITassertion. The unit tests instages/logical/memo/tests.rskeep their localscan,kll_stateandunordered_limit: unit tests insrc/cannot reach the integration tests' module.depends_on_evaluation_timematches the operator's own reads, then checks the input groups once.dtype_to_arrow(&DataType::Timestamp).LogicalProperties::parametersandMemo::source_factsarepub(super)(read bymemo/hash_consing.rs,memo/tests.rsandgroup.rs),same_expression_and_logical_propertiesispub(super)inmemo/hash_consing.rs.Result; one validity check of an expression over its input groups for groups and candidatesAggregatecoverage, time as an interval withIntervalConstraintLimit,@ Tfixed, SQL timestamp filters,DisjointInTime, chain coverage and the merge over a chain's windows(t − r, t]@ end()/@ start()lifted above the anchored subtree asTimeShift { at }(X), X without@;range_evaluateddeleted; coverage leaves theTimeShift(@)unliftedValueSign/column_signderived per operator, Scan from declared source factsdepends_on_evaluation_timederived in Stage 0Memo::new, privatedeclared_column_signs)MemoQuery::accuracy_target: Option<_>(P17),memo_of_queries(roots, accuracy_targets, data)(P5)SourceColumn { pub source: String, pub column: String }stays besideSource, re-exported by the planner facade as theSelectionBoxkeyNonNegative, rates only overNonNegativeinput, comparisons, readouts of distinct count / L2 norm / entropy)time()/now()setdepends_on_evaluation_time@lifted above the selector onlyGroupProperties::reads_only_time_series: bool, derived once inderive_group_properties;OperatorInput::reads_only_time_seriesthe same rule over node trees; the recursive walks inir/schema/topk.rsand Pass 1 deleted. Example: PromQLScan latency→ true;TimeRange(1m, Scan latency)→ true; SQLScan requests→ false;Join(requests, hosts)→ false (a SQL join reads relations only, so no join of a time series and a table exists to test)reads_only_time_seriesfunction with its own recursionDeleted besides the interfaces above:
PromqlLowerer::range_evaluatedand therange_evaluatedparameter oflower_query,vs_parts,time_shift,extract_matrix(previous revision; replaced byanchored/selector); testsat_end_and_at_start_share_coverage_only_in_an_instant_query(types),at_start_and_at_end_stay_only_in_a_range_query(promql lib) and Stage 0'sdiffering_accuracy_targets_are_two_groups_over_one_scan_group(measures carry no target since PR 0; replaced byqueries_that_differ_only_in_their_accuracy_target_are_one_group); thetodo!()s forcolumn_sign,depends_on_evaluation_time, Stage 0,add_query, and#[allow(unreachable_code)]onderive_group_properties.Verification
Coverage design (
bba5e221c): 680 / 0 / 0, the same tests by name as0de33fd94plus three new ones inmemo_aggregate_coverage.rs:a_view_coverage_is_equal_across_memos_whatever_the_group_ids(thelatencyscan is a different group id in each memo, the ids are equal),a_view_coverage_over_another_source_has_another_definition_id,a_group_without_aggregate_coverage_has_no_view_coverage. No test moved; adapted:memo_aggregate_coverage.rsimports the coverage types fromir::propertiesand readsdefinition().input.cargo check --workspace --all-features --tests,fmt --check,cargo clippy --workspace --all-features --all-targets -D warningsclean;cargo doc -p asap-typesreports only the four older links.Layout (
0de33fd94): 677 / 0 / 0, the same tests by name as288cd70d6(no test moved; the memo unit tests andmemo_group_propertiescall the shared fixtures).cargo check --workspace --all-features --tests,fmt --check,cargo clippy --workspace --all-features --all-targets -D warningsclean;cargo doc -p asap-typesreports only the four older links.Sign bit, view coverage (
288cd70d6): 677 / 0 / 0, the same tests by name as73785fa07. New case (failing first):scalar_arithmetic_follows_its_operands_signsrow G7,latency / -0— on73785fa07left[Unknown, NonNegative, Unknown], right[Unknown, NonPositive, Unknown]; passes. Adapted: the same test's andvector_arithmetic_follows_its_operands_signs's tables (history notes dropped); everyPhysicalASAPDAGNodeliteral in tests (types/tests/physical_export.rs, 12 executor test files) getsview_coverage: None.cargo check --workspace --all-features --tests,fmt --checkandcargo clippy --workspace --all-features --all-targets -D warningsclean;cargo doc -p asap-typesreports only the four older links. View-path tests:integration::materialized_views::*andkll_tumbling_window_execution::*give the same results as on73785fa07(1 passed; 4 failing on both, on the retentiontodo!and a fixture's deserialization, before the views code).Definition identity (
73785fa07): 677 / 0 / 0, the same tests by name asc2f23d760; adapted:memo_aggregate_coverage::states_with_equal_definitions_and_disjoint_selections_merge(thedefinition_idequality assert removed, the definition==assert kept).cargo check --workspace --testswith no warnings,fmt --checkandclippy --all-targets -D warnings(the test-set crates plusasap-integration-tests, which readsViewCoverage::definition_id) clean.Sign rules (
c2f23d760): 677 / 0 / 0, the same tests by name asf68d414aaplus four new ones inmemo_group_properties.rs, each failing onf68d414aawith only the test file applied:a_count_sketch_point_count_has_no_sign_a_count_min_one_keeps_it(item 1): before, leftNonNegative, rightUnknown; passes.a_quantile_outside_the_unit_interval_has_no_sign(item 2): before, leftNonNegative, rightUnknown; passes.vector_arithmetic_follows_its_operands_signs(items 3, 4: the ratio of quantiles,+,-,vector(1000),vector(-1), products): before, the ratio's value leftUnknown, rightNonNegative; passes.scalar_arithmetic_follows_its_operands_signs(item 5: PromQLx / 1000,x * 8, SQLvalue * 8,x - x,x * -8): before,x / 1000left[Unknown, Unknown, Unknown], right[Unknown, NonNegative, Unknown]; passes.scans_read_declared_signs_and_row_operators_keep_or_set_them: its G9 (value * 2over non-negativelatency) now expectsNonNegative(wasUnknown), and a G10 row checksvalue - valueisUnknown.cargo check,fmt --check,clippy --all-targets -D warningsclean;cargo doc -p asap-typesreports only the four links older than this PR.Third review (
f68d414aa): 673 / 0 / 0, the same tests by name as1ae8acf91(a_value_comparison_is_an_interval_between_fixed_valuesand the selection tests run fromir::properties::selection);cargo check,fmt,clippy -D warningsclean;cargo doc -p asap-typeswith-D warningsreports only links older than this PR. Second review (1ae8acf91): 673 / 0 / 0, the same tests as0fef10001by name exceptequal_keyless_nodes_that_return_the_same_rows_are_one_group→equal_keyless_expressions_that_return_the_same_rows_are_one_group; moved from Stage 0 tostages::logical::memo::tests(same names):aggregates_that_differ_in_an_argument_are_different_groups,dedup_is_one_group_only_on_every_column,unordered_limits_and_their_parents_are_different_groups,limits_over_a_sort_with_their_partitions_are_one_group,signed_zero_and_non_finite_literals_are_different_groups;memo_summary_coverage::*→memo_aggregate_coverage::*.cargo check,fmt,clippy -D warningsclean;cargo doc -p asap-typeswith-D warningsreports only links older than this PR.Restack and review fixes (2026-11-10):
cargo test -p asap-types -p asap-stage0-memo -p asap-stage1-pass1-per-computation-candidates -p asap-stage1-pass2-reuse-candidates -p asap-frontend-common -p asap-frontend-promql -p asap-frontend-sql --all-features --no-fail-fast: previous head36312cbaf667 / 0 / 0; restacked commit3cabdc3b5667 / 0 / 0, the same tests by name (the twodelivers_ontests run, no longer ignored); head77fc121d3668 / 0 / 0: the same pluscount_values_without_has_one_sign_per_output_column, which failed on3cabdc3b5(one sign per output column of Aggregate, left 2, right 1) and passes;group_properties_come_from_input_groupsruns from its new file. Head0fef10001(expression check renamed,check_candidate_expressioninlined): 673 / 0 / 0, the same tests by name exceptmemo_of_queries_returns_the_error_of_a_node_that_breaks_the_node_rules→memo_of_queries_returns_the_error_of_a_node_invalid_over_its_input_groups. Headf458196ba(source facts, scanned sources): 673 / 0 / 0, the same tests by name,a_group_reads_only_time_series_when_every_scan_below_reads_onerenameda_group_scans_the_sources_of_the_scans_below_it(it now also checks the sets); Stage 0'sa_scan_reads_the_declared_column_signs(histogram family and counter) passes overMemo::new(data). Headc71be0331(B1, B4): 673 / 0 / 0, the same plus the fiveaggregate_output_columnstests; the moveda_value_comparison_is_an_interval_between_fixed_valuesruns fromaggregate_coverage.rs.cargo check --workspace --testswith no warnings,cargo clippy --workspace --all-targets -- -D warningsandcargo fmt --checkclean at each head. The table below is from the earlier head.cargo test -p <crate> --all-features --no-fail-fast. The full table below was measured at8095a5145(this PR before N7 addedreads_only_time_series); at the head49a145c26,asap-typesandasap-stage0-memowere rerun: all pass, includinga_group_reads_only_time_series_when_every_scan_below_reads_one. Every failure panics at a later PR'stodo!()or is the stale fixture (PR 9); planning now stops at Pass 1's target discovery (PR 3). By test name against the base PR 1 (8030d3fe6): 0 newly failing; 45 new tests pass (asap-types30,asap-stage0-memo15, one each inasap-frontend-promqlandasap-frontend-sql); PR 1'sdelivers_on_a_chain_when_its_width_divides_the_context_timesanddelivers_on_a_stored_result_on_its_group_gridare ignored there (their memo fixture needs this PR'sfind_or_add_group); this PR removes the two#[ignore]attributes and they run and pass: at the head43f81fe01(restacked onto PR 1c2e717bf9; tree identical to49a145c26)cargo test -p asap-types --no-fail-fast: 195 passed, 0 failed, 0 ignored. Tests that failed at PR 2'stodo!()on PR 1 now fail at PR 3's (target discovery) or PR 6's.cargo clippy --workspace --all-targets --all-features -- -D warningsandcargo fmt --all --checkclean;crate_graph: 6 passed.retention_ms=asap_plannersubprocess stopping at atodo!(): 2retention_ms: 3; stale old-wire fixture (PR 9): 18095a5145New or rewritten tests:
memo_group_properties.rs(6): sign table rows for row operators (scans_read_declared_signs_and_row_operators_keep_or_set_them), aggregates (aggregates_keep_their_keys_and_derive_their_values), states/merges/readouts (states_record_their_input_sign_and_readouts_read_it), the doc's top-k example (a_top_k_ranks_values_of_its_inputs_sign),depends_on_evaluation_timetrue/false (a_result_depends_on_the_evaluation_time_under_a_time_range_or_now),a_group_reads_only_time_series_when_every_scan_below_reads_one.at_end_and_at_start_share_the_subtree_they_anchor_and_stay_unlifted(types); promql libat_start_and_at_end_are_lifted_above_the_selector_they_anchor; conformanceat_modifier_lowers_to_a_time_shift(updated).each_query_enters_with_the_values_one_evaluation_returns,a_scan_reads_the_declared_column_signs,queries_that_differ_only_in_their_accuracy_target_are_one_group,memo_of_queries_returns_the_error_of_a_node_that_breaks_the_node_rules.stages::logical::memo::tests(wasmemo_testsincandidates.rs):find_or_add_group_*(equal expression, different inputs, unshareable, parents, no schema) andadd_candidate_*(duplicate, parents and queries, output differs, inputs fail validation).memo_aggregate_coverage.rs(13): coverage intervals,@ T, chains, merges over windows, counter disjointness.End-to-end evidence: not applicable (planning stops at Pass 1's target discovery until PR 3).
Self-review: no trait or wrapper added (new:
Memo::new(data)and privatesource_facts, privatecolumn_properties.rsfunctions,MemoGroupAsInput::group_properties()(pub(super), shared by the two derivations), and from the reviewOutputColumnSourcewithaggregate_output_columns); the sign derivation is not generic overOperatorInputbecause only memo groups have signs. The output-column layouts areaggregate_schema.rs's alone (review 1); PR 2b'scolumn_lineageshould read the sameOutputColumnSource. No dead code (clippy-D warnings; therange_evaluatedplumbing and stale#1207 PR 2comments are gone). Fixed in this round: the coverage walk passedTimeShift { at: End | Start }and returned no interval for the whole path; it now stops there.Follow-ups
column_lineageshould read the sameOutputColumnSourceascolumn_signs(refactor(types, frontends): column lineage replaces table qualifiers (#1207 PR 2b) #1233).column_signs: PR 5 (refactor(stage2): physical expansion (decision-tree levels 3–4), physical edges, pruned alternatives; Count-Min weights via column_signs (#1207 PR 5) #1235).TimeRanges or@ Ts.@ end()/@ start()asRepeatAcrossSteps: Range queries: evaluate@ end()/@ start()subtrees once per request and share them #1251.depends_on_evaluation_time— SQL table scans and@ end()under@ Tare marked independent; aTimeRangeover@ Tis marked dependent.CAST(<integer> AS TIMESTAMP)as milliseconds.TimeShiftmoved past a subqueryFilter, and chain windows or merges that read an evaluation-time-dependent group.window_end).Unknown.NonNegative, but a single-item entropy is-0.0under the sign-bit definition.definition_id(expanded definition, canonical serialization and hash): PR 6, withcoverage_of.