Skip to content

refactor: define every stage's interfaces from #1207 and delete the ones they replace (#1207 PR 0) - #1226

Merged
zzylol merged 22 commits into
mainfrom
refactor/1207-interfaces
Nov 10, 2026
Merged

refactor: define every stage's interfaces from #1207 and delete the ones they replace (#1207 PR 0)#1226
zzylol merged 22 commits into
mainfrom
refactor/1207-interfaces

Conversation

@zzylol

@zzylol zzylol commented Nov 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Merge unit. PR 0 of #1207's merge unit (PRs 0–9). The PRs merge into main together, after PR 9; each is stacked on the one before it for review. This PR compiles, but planning panics at the first todo!(); tests that reach a todo!() fail until the PR that fills it in (see Verification).

Summary

  • Problem: the planner's stage interfaces did not match the concepts doc (docs(concepts): memo groups, properties, requirements, per-stage interfaces and rules #1224): a sketch type was Sketch(SketchKind, GroupingStrategy) with θ recording the error it was sized for, accuracy was a θ floor and a pluggable AccuracyModel checked after Stage 3, and Stage 3 decoded a selected-candidate DAG that Stage 2 realized into a RealizedPlan.
  • Change: defines every data structure and interface of the doc, with todo!() bodies where the behavior lands in a later PR, and deletes the interfaces they replace (420 files, +6,122 / −79,960 lines), so no later PR needs an adapter between an old and a new interface.
  • Effect: for quantile_over_time(0.99, latency[1m]) / quantile_over_time(0.5, latency[1m]) within relative 0.03, the readout's input now carries RequiredAccuracy { metric: Relative, max_error, failure_probability } instead of a θ floor, θ is structure only (Kll { k }), and Stage 3 returns (PhysicalASAPDAG, Selection); PlanOutput holds only plan. Test status at the head: 1,078 passed / 217 failed / 9 ignored, every failure at a todo!() except one stale fixture.

Review guide

  • Position: base main. First code PR of the Tracking: implement the optimization formulation (memo candidates, properties, edges y_{u,c,i}, MILP) as named code per stage #1207 stack, after the concepts doc docs(concepts): memo groups, properties, requirements, per-stage interfaces and rules #1224; refactor: one crate per stage (#1207 PR 0b) #1229 (PR 0b, one crate per stage) builds on it. Head 7030d68de.
  • Reading order:
    1. types/src/ir/schema/mod.rs and types/src/summary_type/{parameters.rs, parameter_lists.rs, statistic.rs, capability.rs}: summary types and θ.
    2. types/src/stages/logical/{memo/mod.rs, group.rs, candidate.rs, requirements.rs, edge.rs, memo_candidates.rs}, types/src/stages/pruning.rs and types/src/stages/physical/{execution_context.rs, candidate.rs, edge.rs, memo_candidates.rs, properties.rs}: the memo, requirements, contexts, by scope (memo → group → candidate → candidate input → edge → stage output).
    3. accuracy/src/{evidence.rs, requirements.rs, error_propagation.rs, guarantee/mod.rs}: AccuracyEvidence, producer_satisfies_logical_requirements, target_limit, error_propagation, theoretical_guarantee.
    4. Stage entry points: logical-asap-optimization/src/{stage0.rs, pass1.rs, pass2.rs}, plan-selection/src/{select.rs, report.rs}.
    5. Facade: planner/src/{pass.rs, run.rs, installable.rs}.
  • Mechanical: the deletions (plan-selection's old engine, Pass 2's old CSE, the realizer, the window functions, label tooling); renames (family → summary_type, FamilyCapability → SummaryTypeCapability, NodePriceSource → NodeCostSource, PlanRanking::Price → Cost, family_name → summary_type_name); Serialize on PruneReason, PhysicalProperties, OriginOfCandidateForMemoGroup, RootPruneReason; docs (asap-primitive-schema §1 and §6, architecture, library-API, mapping-contracts, extension, input-output, accuracy docs) using the new names; the checked-in version-10 fixture's DAG migrated; workload fixtures and the viewer read column_signs.
  • Rebase onto main (2026-11-10): the last commit drops the removed accuracy argument from lower_sql_dialect in main's new ClickHouse tests (frontend-sql/tests/clickbench/clickbench.rs, sql_lowering.rs) and writes AggIntent::Count as a unit variant there. The argument is gone because this PR moves the accuracy target from the operator (AggIntent::Count { accuracy }) to the query (MemoQuery::accuracy_target).
  • File layout (2026-11-10 review, R1–R3, L1–L6, E1–E3): the last two commits only move and rename (no logic change); the table under Interfaces lists every move. git diff -M 61541f748 7030d68de shows them as renames where a file moved whole.
  • Careful: that each todo!() names the PR that fills it; the public signatures below (the later PRs fill them without changing them); the deletion table (each deleted item's replacement).
  • Private MILP variables (doc §3–4), not shown below: SelectionVariables::t: Vec<Option<Variable>> (only on strongly connected components); AccuracyVariables::{error_bound, error_bound_of_input} per metric ($E_{c,m}$, $F_{c,i,m}$).

Behavior: before → after

The design need: one model from the concepts doc that every stage PR fills in behind fixed interfaces. Example workload: p99 / p50 of latency (declared non-negative), relative error 0.03, every 10 s.

Before (main) After (this PR)
Sketch state type Sketch(SketchKind { Kll, Kll { k, failure_probability } }, GroupingStrategy); Hydra a grouping axis; op field family Sketch(Kll); Hydra an algorithm (HydraKll, HydraCms, HydraCountSketch); op field summary_type; θ Kll { k } on the candidate
Accuracy on a measure Quantile { col, q, accuracy } Quantile { col, q }; the target enters the memo with the query (MemoQuery::accuracy_target) with $n_q$
Accuracy on an edge θ floor; AccuracyModel::satisfies before and after Stage 3 LogicalRequirements { required_accuracy: Option<RequiredAccuracy> } on a readout's input; α^log compares the producer's measured-or-theoretical guarantee at δ_r (PR 4); nothing checked after Stage 3
Pass 1 prunes summaries no query can use for accuracy prunes nothing for accuracy; Pass 2's dead-candidate elimination (PR 4) replaces it
Stage 3 output Selection { selected_dag, cost, search_report, .. } realized by Stage 2 into a RealizedPlan; PlanOutput::plans per query select_plan(..) -> (PhysicalASAPDAG, Selection); each node names its physical candidate; PlanOutput { plan }
Source sign values_non_negative: true "column_signs": {"value": "non_negative"}
Plan file wire version 13 wire version 14: ops carry summary_type, a sketch state is {"Sketch": "Kll"}, no grouping; selected_physical_candidate not written
Running it plans panics at Stage 0's todo!() until PR 2

Interfaces

Permalinks at the PR head 7030d68de; diff against main (82165ae2b). The interface list as of the earlier head 7e83db97d (decisions of 2026-11-06 and 2026-11-07, with its deletion tables) is kept in this comment; the 2026-11-07 detail in the interfaces comment.

types/src/ir/schema/mod.rs · types/src/summary_type/ (parameters.rs, parameter_lists.rs, statistic.rs, capability.rs)

 /// A column's type: a plain value or a summary type (family + algorithm). Stays.
 pub enum FieldDataType {
     /// An ordinary readable value.
     Plain(DataType),
     /// Exact, mergeable accumulator state.
     ExactAggregate(ExactKind, ExactParams),
-    /// Sketch state: algorithm and θ bundled, plus per-group or shared (Hydra) grouping.
-    Sketch(SketchKind, GroupingStrategy),
+    /// Sketch state of one algorithm, the `Hydra*` algorithms included; θ is not part of the type.
+    Sketch(SketchAlgorithm),
     /// A summary sample (the rows a sampling summary keeps).
     SummarySample(SamplingKind, SamplingParams),
     /// Wavelet summary state.
     Wavelet(WaveletKind, WaveletParams),
     /// Statistical-model summary state.
     StatModel(StatModelKind, StatModelParams),
 }
 /// θ: the structure's sizes, from `parameter_list(algorithm)`; no sized-for error fields,
 /// so equality (now derived) compares every field. Stays.
 pub enum SketchParams {
     /// UnivMon.
     UnivMon {
         heap_size: u32,
         sketch_rows: u32,
         sketch_cols: u32,
         layers: u8,
-        distinct_failure_probability: Option<f64>,
     },
     /// KLL.
     Kll {
         k: u32,
-        failure_probability: Option<f64>,
     },
     /// Count-Min.
     Cms {
         width: u32,
         depth: u32,
     },
     /// HyperLogLog.
     Hll {
         precision: u8,
-        relative_error: Option<f64>,
     },
     /// DDSketch.
     DDSketch {
         alpha: f64,
     },
     /// Count-Min with a top-k heap.
     CmsWithHeap {
         width: u32,
         depth: u32,
         heap_size: u32,
     },
     /// KMV.
     Kmv {
         k: u32,
-        relative_error: Option<f64>,
     },
     /// Count Sketch.
     CountSketch {
         width: u32,
         depth: u32,
     },
     /// Count Sketch with a top-k heap.
     CountSketchWithHeap {
         width: u32,
         depth: u32,
         heap_size: u32,
     },
+    /// The sizes of a Hydra structure, for the `Hydra*` algorithms.
+    Hydra(HydraParams),
 }
-/// An algorithm and its θ, validated as a pair.
-pub struct SketchKind {
-    algorithm: SketchAlgorithm,
-    params: SketchParams,
-}
-/// The Hydra variants of the shared grouping.
-pub enum HydraKind {
-    HydraKll,
-    HydraCms,
-    HydraCountSketch,
-}
-/// One sketch per group, or one shared structure for all groups.
-pub enum GroupingStrategy {
-    PerSubpopulationInstance,
-    SharedMultiSubpopulation {
-        kind: HydraKind,
-        params: HydraParams,
-    },
-}
-/// What one family can do. Renamed below.
-pub struct FamilyCapability {
+/// What one summary type can do: its row of the capability table.
+pub struct SummaryTypeCapability {
     /// How the inputs of a merge must relate; `None` without a sound merge.
     pub merge_inputs: Option<MergeInputRule>,
     /// Each statistic one state answers, with the error of that readout from a merged state.
     pub(crate) readouts: &'static [(StatisticKind, MergedError)],
 }
-impl FamilyCapability {
+impl SummaryTypeCapability {
     /// Each statistic one state answers, with its merged-state error.
     pub fn readouts(&self) -> &'static [(StatisticKind, MergedError)];
     /// Whether a rollup may merge different groups' states (not a Hydra's: its groups share one state).
-    pub fn merges_groups(family: &FieldDataType) -> bool;
+    pub fn merges_groups(summary_type: &FieldDataType) -> bool;
+    /// Whether `summary_type` is a Hydra algorithm; read by `merges_groups` and the executor's kernel check.
+    pub fn shares_one_state_across_groups(summary_type: &FieldDataType) -> bool;
     /// Whether one state answers `statistic`.
     pub fn answers(&self, statistic: &SketchStatistic) -> bool;
     /// The error of `statistic` from a merged state; `None` without a sound merge.
     pub fn merged_error(&self, statistic: &SketchStatistic) -> Option<MergedError>;
-    pub fn of_per_instance_sketch(algorithm: &SketchAlgorithm) -> Self;
     /// The row of `summary_type`; types not listed fail closed.
-    pub fn of(family: &FieldDataType) -> Self;
+    pub fn of(summary_type: &FieldDataType) -> Self;
 }
 /// One summary type a deployment's executor can build, with its readouts.
 pub struct SupportedSummary {
-    pub family: SketchAlgorithm,
+    /// The summary type the executor can build (JSON field `summary_type`).
+    pub summary_type: SketchAlgorithm,
     /// The statistics it answers from one state.
     pub readouts: Vec<StatisticKind>,
 }

of_per_instance_sketch: of(&Sketch(a)) covers it once Hydra is an algorithm. Module family_capability is now asap_types::summary_type (capability.rs).

types/src/ir/operator/asap.rs · agg_intent.rs

 /// The ASAP summary operators. Stays; three variants change.
 pub enum ASAPOp<C = Rc<OperatorNode>> {
     /// Build one summary state per group (or one shared Hydra structure).
     SummaryAgg {
         child: C,
-        family: FieldDataType,
+        summary_type: FieldDataType,
         input: SummaryUpdate,
         reduction: Reduction,
-        grouping: GroupingStrategy,
         #[serde(default)]
         filter: Option<crate::ir::scalar::Predicate<C>>,
     },
     /// Read a statistic from a state (unchanged).
     SummaryEstimate { .. },
     /// Finalize an exact accumulator (unchanged).
     FinalizeExactAccumulator { .. },
     /// Maintain a population (unchanged).
     MaintainPopulation { .. },
     /// Evaluate a population (unchanged).
     EvaluatePopulation { .. },
     /// Merge states (unchanged).
     SummaryMerge { .. },
     /// A chain of tumbling windows of states; no window count (the width only).
     SummaryTumblingWindowChain {
         child: C,
-        family: FieldDataType,
+        summary_type: FieldDataType,
         input: SummaryUpdate,
         reduction: Reduction,
-        grouping: GroupingStrategy,
         #[serde(default)]
         filter: Option<crate::ir::scalar::Predicate<C>>,
         #[serde(alias = "pane_width_ms")]
         tumbling_window_width_ms: u64,
-        #[serde(alias = "pane_count")]
-        tumbling_window_count: u32,
     },
     /// Subtract states (unchanged).
     SummarySubtract { .. },
     /// Delete from a state (unchanged).
     SummaryDelete { .. },
     /// Join two states on a key.
     SummaryJoin {
         outer: C,
         inner: C,
         key: ColumnId,
-        family: FieldDataType,
+        summary_type: FieldDataType,
     },
     /// An extension operator (unchanged).
     Extension { .. },
 }
 /// What an aggregate computes. Stays; measures carry no accuracy (the query's target does, N1).
 pub enum AggIntent<C = ColumnId> {
-    Count {
-        accuracy: AccuracyTarget,
-    },
+    Count,
     Sum { col: Option<C> },
     Min { col: Option<C> },
     Max { col: Option<C> },
     Avg { col: Option<C> },
     StdDev { col: Option<C>, population: bool },
     Variance { col: Option<C>, population: bool },
     PearsonCorr { left: C, right: C },
     Quantile {
         col: Option<C>,
         #[serde(with = "crate::serde_f64")]
         q: f64,
-        accuracy: AccuracyTarget,
     },
     TopK {
         k: usize,
-        accuracy: AccuracyTarget,
     },
     Cardinality {
         cols: Vec<C>,
-        accuracy: AccuracyTarget,
     },
     FrequencyL2 {
         col: Option<C>,
-        accuracy: AccuracyTarget,
     },
     FrequencyEntropy {
         col: Option<C>,
-        accuracy: AccuracyTarget,
     },
     Rate,
     IRate,
     Increase,
     Changes,
     Delta,
     IDelta,
     Deriv,
     Resets,
     PredictLinear { seconds: f64 },
     DoubleExpSmoothing { smoothing: f64, trend: f64 },
     HistogramCount,
     HistogramSum,
     HistogramAvg,
     HistogramStdDev,
     HistogramStdVar,
     HistogramFraction { lower: f64, upper: f64 },
     HistogramQuantile { q: f64, le: C },
     Math(MathFunc),
     Absent { labels: BTreeMap<String, String> },
     AbsentOverTime,
     PresentOverTime,
     TimeFn(TimeFunc),
     Group,
     CountValues { label: String },
     LastOverTime,
     FirstOverTime,
     MadOverTime,
     TsOfMinOverTime,
     TsOfMaxOverTime,
     TsOfFirstOverTime,
     TsOfLastOverTime,
     Extension { ext_kind: String, payload: serde_json::Value },
 }

types/src/stages/logical/ (memo/mod.rs, group.rs, candidate.rs, requirements.rs, edge.rs, memo_candidates.rs) · types/src/stages/pruning.rs · types/src/accuracy/

+/// What every candidate of a group produces, derived from its expression and input groups.
+pub struct GroupProperties {
+    pub schema: Schema,
+    pub result_kind: OperatorResultKind,
+    /// Per output column: the sign of its values (derived in PR 2).
+    pub column_sign: Vec<ValueSign>,
+    /// Whether the result changes with the evaluation time (derived in PR 2): true under a
+    /// relative `TimeRange`, false under a fixed `@ T`; a stored copy of a false one delivers on every grid.
+    pub depends_on_evaluation_time: bool,
+    /// What an aggregate group covers; `None` for other groups.
+    pub aggregate_coverage: Option<SummaryCoverage>,
+    /// Whether equal expressions return the same rows, so they may be one group.
+    pub shareable_when_equal: bool,
+}
+/// The sign of one column's values; a group property derived like lineage; a `Scan`'s are declared per column.
+pub enum ValueSign {
+    NonNegative,
+    NonPositive,
+    Unknown,
+}
 /// The memo: groups with their candidates, and each query over them. Stays.
 pub struct Memo {
-    pub groups: Vec<MemoGroup>,
-    pub query_roots: Vec<QueryRoot<MemoGroupId>>,
-    pub candidates: Vec<Vec<CandidateForMemoGroup>>,
+    /// The groups, by id; private, added only through `find_or_add_group`.
+    groups: Vec<MemoGroup>,
+    /// The queries, by index in the workload.
+    pub queries: Vec<MemoQuery>,
 }
+/// One query of the memo: its root, $A_q$ (`None`: accuracy not enforced, P17) and $n_q$ (`None`: not bounded).
+pub struct MemoQuery {
+    pub root: QueryRoot<MemoGroupId>,
+    pub accuracy_target: Option<AccuracyTarget>,
+    /// Pass 2 turns a whole-result δ into a per-value one with it.
+    pub number_of_values_per_query_evaluation: Option<u64>,
+}
 impl Memo {
+    /// The groups, by id.
+    pub fn groups(&self) -> &[MemoGroup];
+    /// Adds `query` and returns its index; Stage 0 adds each query once.
+    pub fn add_query(&mut self, query: MemoQuery) -> usize;
 }
 /// The groups some query's root reads, ascending.
-pub fn root_groups(query_roots: &[QueryRoot<MemoGroupId>]) -> BTreeSet<MemoGroupId>;
+pub fn root_groups(queries: &[MemoQuery]) -> BTreeSet<MemoGroupId>;
 /// The queries whose root reads `group`, ascending.
-pub fn queries_rooted_at(query_roots: &[QueryRoot<MemoGroupId>], group: MemoGroupId) -> Vec<usize>;
+pub fn queries_rooted_at(queries: &[MemoQuery], group: MemoGroupId) -> Vec<usize>;
+/// $r^{\log}_{c,i}$: only a readout's input carries one.
+pub struct LogicalRequirements {
+    /// `None` on every non-readout input, and on a readout's input no target reaches.
+    pub required_accuracy: Option<RequiredAccuracy>,
+}
+/// The accuracy a readout's input needs: the producer's guarantee at `failure_probability`
+/// is at most `max_error` in `metric`; also the readout's own output error.
+pub struct RequiredAccuracy {
+    pub metric: AccuracyMetric,
+    /// $\varepsilon_r$, in the metric's units.
+    pub max_error: f64,
+    /// $\delta_r$, per value; 1 when the target sets none.
+    pub failure_probability: f64,
+}
+/// The algorithm's predefined θ list (structures only).
+pub fn parameter_list(algorithm: SketchAlgorithm) -> &'static [SketchParams];
 /// Stage 1, Pass 2's output. Stays.
 pub struct LogicalASAPMemoCandidates {
     /// The memo after the last rule.
     pub memo: Memo,
+    /// Every $(u, c, i)$ between surviving candidates, grouped by $(c, i)$.
+    pub edges: Vec<LogicalEdge>,
+    /// The candidates dead-candidate elimination removed.
+    pub pruned: Vec<PrunedLogicalCandidate>,
+    /// Per group: the coarsest grid holding every evaluation time of the queries reaching it
+    /// (Pass 2, PR 4); `None` when not on one known grid.
+    pub evaluation_grid_of_group: Vec<Option<EvaluationGrid>>,
 }
+/// Why the edge at the end of a prune path fails, or why the candidate has no edge.
+#[derive(Serialize)]
+pub enum RootPruneReason {
+    /// The producer's guarantee at the required δ exceeds the required error.
+    BelowRequiredAccuracy {
+        consumer: CandidateForMemoGroupId,
+        input: usize,
+        required: RequiredAccuracy,
+    },
+    /// The consumer's error contract carries none of the producer's metrics.
+    MetricRule {
+        consumer: CandidateForMemoGroupId,
+        input: usize,
+        producer_metric: AccuracyMetric,
+        consumer_carries: Vec<AccuracyMetric>,
+    },
+    /// Never had a consumer and is not a query output.
+    NoConsumer,
+}
 /// Where a candidate came from. Stays; now `Serialize`.
+#[derive(Serialize)]
 pub enum OriginOfCandidateForMemoGroup {
     Pass1,
-    SharedBuild,
-    SummaryResult,
     SummaryInput,
     TumblingWindows,
     Rollup,
     TopKReuse,
     WidenedScans,
 }
 /// Why Stage 2 or Stage 3 removed a candidate. Stays; now `Serialize`.
+#[derive(Serialize)]
 pub enum PruneReason {
-    RequiresExact,
-    NullableAvgInput,
-    NoRelationalCountSketchRealization,
-    UnsupportedByDeployment,
-    QuantileOutsideUnitInterval,
-    TopKHeapOverOneRowPerItem,
-    NoErrorBoundForQueryTarget,
-    PerEntityAggregateNotExecutable,
+    /// Stage 2: runs at ingestion time, and the source has no ingestion pipeline.
+    NoIngestionPipeline,
     /// Stage 2: scans raw data at query time, and the deployment retains none.
     RawDataNotRetained,
+    /// Stage 2: a query-time result materialized, and the deployment retains no query-time results.
+    QueryTimeResultsNotRetained,
     /// Stage 2: an input has no buildable producer.
     InputHasNoBuildableProducer,
     /// Stage 2: no query output can read it.
     ReachesNoQueryOutput,
-    TooFewQueryReaders,
+    /// Stage 2 (PR 5): a Count-Min build whose weight column's `column_sign` is not `NonNegative`.
+    CountMinWeightsNotProvenNonNegative,
     /// Stage 3: another candidate is at least as good in every respect.
     DominatedBy(PhysicalCandidateId),
     /// Stage 3: `asap-cost` could not cost it.
     UsageNotCostable,
 }

types/src/stages/physical/ (execution_context.rs, candidate.rs, edge.rs, memo_candidates.rs, properties.rs) · types/src/plan/physical_asap_dag.rs · types/src/workload/

+/// κ: executions at the same times share one run.
+pub enum ExecutionContext {
+    /// Evaluations at the times of `grid`.
+    Evaluation { grid: EvaluationGrid },
+    /// One ingestion pipeline's refresh window.
+    IngestionPipeline(RefreshWindow),
+    /// The executions of materialized candidate v.
+    Materialized(PhysicalCandidateId),
+}
+/// The evaluation times phase_ms + k·period_ms.
+pub struct EvaluationGrid {
+    pub period_ms: u64,
+    pub phase_ms: u64,
+}
 /// Index of a physical candidate; moved from `graph.rs` to `memo_candidates.rs`, now `Serialize`.
 pub struct PhysicalCandidateId(pub usize);
+impl PhysicalCandidate {
+    /// Whether a consumer running in `context` can read this producer (alignment table; PR 1;
+    /// the `Materialized(v)` branch PR 7).
+    pub fn delivers_on(&self, candidates: &PhysicalASAPMemoCandidates, context: &ExecutionContext) -> bool;
+}
+/// α^phys: timing and lifetime only.
+pub fn producer_satisfies_physical_requirements(
+    producer: &PhysicalCandidate,
+    consumer_timing: ExecutionTiming,
+) -> bool;
-pub struct SelectedNodeId(pub usize);
-pub struct SelectedEdge {
-    pub producer: SelectedNodeId,
-    pub consumer: SelectedNodeId,
-    pub input: usize,
-}
-pub struct DagOfSelectedCandidates {
-    pub candidate_of_node: Vec<PhysicalCandidateId>,
-    pub edges: Vec<SelectedEdge>,
-    pub node_of_output_reference: Vec<SelectedNodeId>,
-}
-pub trait AssignmentRealizer { /* realize a selected assignment */ }
-pub struct RealizedPlan { /* the realized plan */ }
-pub enum PhysicalCandidateError { /* realizer errors */ }
-pub struct PhysicalWorkloadPlan { /* per-query physical plans */ }
 /// One node of the plan. Stays.
 pub struct PhysicalASAPDAGNode {
     pub id: PhysicalASAPNodeId,
     pub payload: PhysicalASAPOperatorPayload,
     pub output_state: ExecutionDataState,
-    pub materialization: Materialization,
+    /// How long and where its output is materialized.
+    pub lifetime: Lifetime,
+    /// θ of a summary build; `None` otherwise.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub parameters: Option<SketchParams>,
     pub output_schema: Schema,
     pub guarantee: Option<AccuracyGuarantee>,
     /// $h_v$; Stage 3 sets it.
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub retention_ms: Option<u64>,
+    /// The physical candidate it was built from; not in the plan file; `None` for a custom pass.
+    #[serde(skip)]
+    pub selected_physical_candidate: Option<PhysicalCandidateId>,
 }
 /// Why a DAG is invalid. Stays.
 pub enum PhysicalASAPDAGValidationError {
-    MaterializationTimingMismatch {
-        node: PhysicalASAPNodeId,
-        timing: ExecutionTiming,
-        materialization: Materialization,
-    },
+    /// A node whose timing and lifetime cannot go together: a planner bug.
+    PhysicalProperties {
+        node: PhysicalASAPNodeId,
+        #[source]
+        reason: PhysicalPropertiesError,
+    },
+    /// An unmaterialized summary build not at query time: a planner bug.
+    UnmaterializedSummaryBuildNotAtQueryTime { node: PhysicalASAPNodeId },
     QueryDependencyInIngestion { producer: PhysicalASAPNodeId, consumer: PhysicalASAPNodeId },
     UnexpectedRetention(PhysicalASAPNodeId),
     DuplicateNodeId(PhysicalASAPNodeId),
     MissingRoot(PhysicalASAPNodeId),
     MissingEdgeEndpoint(PhysicalASAPNodeId),
     EdgeSchemaMismatch { producer: PhysicalASAPNodeId, consumer: PhysicalASAPNodeId },
     EdgeDataStateMismatch { producer: PhysicalASAPNodeId, consumer: PhysicalASAPNodeId },
     EdgeRoleMismatch { consumer: PhysicalASAPNodeId },
     NoRoots,
     ScalarRootPosition(usize),
     Cycle,
     UnreachableNode(PhysicalASAPNodeId),
-    SummaryFamilySchemaMismatch { node: PhysicalASAPNodeId },
-    SummaryGroupingMismatch { node: PhysicalASAPNodeId },
+    SummaryTypeSchemaMismatch { node: PhysicalASAPNodeId },
 }
 impl PhysicalASAPDAG {
+    /// Test support (P9): the plan's query roots as IR, in entry order.
+    #[cfg(any(test, feature = "test-support"))]
+    pub fn query_roots(&self) -> Result<Vec<crate::ir::QueryRoot>, PhysicalASAPDAGValidationError>;
-    pub fn scan_window(
-        &self,
-        scan: PhysicalASAPNodeId,
-        kept_chains: &HashSet<PhysicalASAPNodeId>,
-    ) -> ScanWindow;
-    pub fn scan_windows(
-        &self,
-        scans: impl IntoIterator<Item = PhysicalASAPNodeId>,
-        kept_chains: &HashSet<PhysicalASAPNodeId>,
-    ) -> HashMap<PhysicalASAPNodeId, ScanWindow>;
-    pub fn view_window(&self, node: PhysicalASAPNodeId) -> ScanWindow;
-    pub fn view_windows(
-        &self,
-        nodes: impl IntoIterator<Item = PhysicalASAPNodeId>,
-    ) -> HashMap<PhysicalASAPNodeId, ScanWindow>;
 }
-pub enum ScanWindow { /* a scan's read window */ }
 /// What the deployment declares about one source. Stays.
 pub struct SourceFacts {
     pub series_count: Option<u64>,
     pub metric_type: Option<PrometheusMetricType>,
-    #[serde(default, skip_serializing_if = "std::ops::Not::not")]
-    pub values_non_negative: bool,
+    /// Each column's declared sign, e.g. `{"value": "non_negative"}`; default none (P7).
+    #[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
+    pub column_signs: BTreeMap<String, ValueSign>,
     pub sample_interval: Option<DurationMs>,
     pub stream_norms: Option<StreamNorms>,
     pub max_distinct_per_evaluation: Option<u64>,
     pub distinct_values_per_label: BTreeMap<String, u64>,
     pub distinct_values_per_label_set: Vec<LabelSetDistinctValues>,
     pub rows_held: Option<u64>,
 }

Materialization is replaced by PhysicalProperties { timing, lifetime } with Lifetime. The window functions: a Scan is an ordinary node, its retention its retention_ms ($h_v$); the executor keeps the window code privately (executor/src/sources/window.rs) until PR 6 (P10).

evidence.rs · requirements.rs · error_propagation.rs · guarantee/mod.rs · lib.rs

+/// The deployment's measured accuracy, per summary configuration.
+pub trait AccuracyEvidence {
+    /// The measured error bound at `failure_probability`; `None` without a measurement or when θ cannot fail that rarely.
+    fn measured_guarantee(
+        &self,
+        summary_type: &FieldDataType,
+        parameters: &SketchParams,
+        statistic: &SketchStatistic,
+        failure_probability: f64,
+    ) -> Option<AccuracyGuarantee>;
+}
+/// No measurement: every guarantee is the theoretical one; `PlanningModels::builtin()`'s evidence.
+pub struct NoMeasuredGuarantees;
+/// The algorithm's error bound at `failure_probability` (`todo!()`, PR 4).
+pub fn theoretical_guarantee(
+    summary_type: &FieldDataType,
+    parameters: &SketchParams,
+    statistic: &SketchStatistic,
+    failure_probability: f64,
+) -> Option<AccuracyGuarantee>;
+/// α^log of one edge (`todo!()`, PR 4); here, not in `asap-types`, because it needs the guarantee (P3).
+pub fn producer_satisfies_logical_requirements(
+    memo: &Memo,
+    producer: CandidateForMemoGroupId,
+    consumer: CandidateForMemoGroupId,
+    input_index_of_consumer: usize,
+    evidence: &dyn AccuracyEvidence,
+) -> bool;
+/// The comparison limit with the tolerance: limit · (1 + 1e-9) + f64::EPSILON.
+pub fn target_limit(limit: f64) -> f64;
-pub trait AccuracyModel {
-    fn satisfies(&self, guarantee: &AccuracyGuarantee, target: &AccuracyTarget) -> bool;
-    fn accepts_output_regardless_of_guarantee(&self, _output: &QueryRoot, _target: &AccuracyTarget) -> bool;
-    fn may_accept_regardless_of_guarantee(&self) -> bool;
-    fn metric_can_meet(&self, metric: AccuracyMetric, target: &AccuracyTarget) -> bool;
-}
-pub struct BuiltinAccuracyModel;
-pub struct IgnoreAccuracyGuarantees;
-pub trait AccuracyEvidenceProvider {
-    fn operand_domain(&self, _operand: &OperatorNode) -> Option<OperandDomain>;
-    fn output_guarantee(&self, _output: &QueryRoot) -> Option<AccuracyGuarantee>;
-    fn may_provide_evidence(&self) -> bool;
-}
-pub struct NoAccuracyEvidence;
-pub struct OperandDomain { /* operand facts */ }
-pub fn with_failure_probability(params: &SketchParams, delta: f64) -> SketchParams;
-pub fn size_for_all_readers(params: &[&SketchParams]) -> Option<SketchParams>;
-pub fn sizing_target(intent: &AggIntent) -> Option<AccuracyTarget>;
-pub fn accuracy_budget(accuracy: &AccuracyTarget) -> (f64, f64);
-/// test support
-pub fn satisfies(guarantee: &AccuracyGuarantee, target: &AccuracyTarget) -> bool;
-/// test support
-pub fn local_guarantee(family: &FieldDataType, query: &SketchStatistic) -> Option<AccuracyGuarantee>;
-/// test support
-pub const DEFAULT_DELTA: f64;

Also deleted: metric_meets, scope_meets, native_target, AccuracyScope::covers (only satisfies used them).

Stages: stage0.rs · pass1.rs · pass2.rs · plan-selection/src/select.rs · report.rs

-pub fn generate_dag_of_logical_memo_groups(roots: &[QueryRoot]) -> DagOfLogicalMemoGroups;
+/// Stage 0 (`todo!()`, PR 2): each query with its target and $n_q$; signs from `data` (P5).
+pub fn memo_of_queries(
+    roots: &[QueryRoot],
+    accuracy_targets: &[Option<AccuracyTarget>],
+    data: &DataWorkload,
+) -> Result<Memo, SchemaDerivationError>;
-pub fn generate_per_computation_logical_memo_candidates(
-    dag_of_logical_memo_groups: DagOfLogicalMemoGroups,
-    accuracy_targets: &[AccuracyTarget],
-    accuracy: &dyn AccuracyModel,
-    evidence: &dyn AccuracyEvidenceProvider,
-    data: &DataWorkload,
-    supported_summaries: Option<&[SupportedSummary]>,
-) -> Result<PerComputationLogicalMemoCandidates, PerComputationLogicalMemoCandidatesError>;
+/// Pass 1: every way to compute each group; no accuracy pruning.
+pub fn pass1_memo(
+    memo: Memo,
+    supported_summaries: Option<&[SupportedSummary]>,
+) -> Result<Memo, Pass1Error>;
 /// Pass 2: reuse rules, then α^log edges reading the evidence.
 pub fn generate_logical_asap_memo_candidates(
-    pass1: PerComputationLogicalMemoCandidates,
+    pass1: Memo,
     entries: &[QueryWorkloadEntry],
+    evidence: &dyn AccuracyEvidence,
 ) -> Result<LogicalASAPMemoCandidates, Pass2RuleError>;
 /// Pass 2's local window composition step.
 pub fn pass2_local_window_composition(
-    pass1: PerComputationLogicalMemoCandidates,
+    pass1: Memo,
     entries: &[QueryWorkloadEntry],
-) -> Memo;
+) -> Result<Memo, Pass2RuleError>;
 /// Pass 2 after the local window composition.
 pub fn pass2_after_local_window_composition(
     mut memo: Memo,
+    evidence: &dyn AccuracyEvidence,
 ) -> Result<LogicalASAPMemoCandidates, Pass2RuleError>;
 /// Stage 3 (`todo!()`, PR 6): selects and builds the plan.
 pub fn select_plan(
     physical: &PhysicalASAPMemoCandidates,
     entries: &[QueryWorkloadEntry],
     data: &DataWorkload,
-    accuracy_checks: &AccuracyCheckInputs<'_>,
     cost_inputs: &CostInputs,
-    realizer: &dyn AssignmentRealizer,
-) -> Result<Selection, ErrorSelectingPlan>;
+) -> Result<(PhysicalASAPDAG, Selection), ErrorSelectingPlan>;
 /// Stage 3's report about its own run. Stays.
 pub struct Selection {
-    pub selected_dag: DagOfSelectedCandidates,
-    pub nodes_with_proven_non_negative_weights: Vec<SelectedNodeId>,
-    pub cost: PlanCost,
     /// Per query: the accepted error's metric and the sketches it reads.
     pub accepted_accuracy_per_query: Vec<AcceptedAccuracy>,
-    pub pruned_candidates: Vec<PrunedCandidate<PhysicalCandidateId>>,
-    pub search_report: SearchReport,
+    /// The candidates Stage 3 pruned, with their reason.
+    pub pruned_candidates: Vec<PrunedCandidate<PhysicalCandidate>>,
+    /// MILP size and timings (were in `SearchReport`).
+    pub milp_variables: usize,
+    pub milp_constraints: usize,
+    pub milp_construction_ms: u64,
+    pub milp_solve_ms: u64,
 }
 /// One query's accepted accuracy. Stays.
 pub struct AcceptedAccuracy {
-    pub target_metric: Option<AccuracyMetric>,
     pub accepted_metric: Option<AccuracyMetric>,
-    pub accepted_regardless_of_guarantee: bool,
-    pub guarantee_check_rejection: Option<RejectionCause>,
     pub sketches: Vec<String>,
 }
-pub struct AccuracyCheckInputs<'a> { /* .. */ }
-pub struct SearchReport { /* .. */ }
-pub struct InexactObjective { /* .. */ }
-pub enum SearchStatus { /* .. */ }
 /// Why selection failed. Stays.
 pub enum ErrorSelectingPlan {
     InvalidSettings(AnalyticalCostError),
     EntryCount { candidate: String, roots: usize, entries: usize },
     Cost { candidate: String, node: PhysicalASAPNodeId, error: AnalyticalCostError },
     NoValidCandidate(Vec<Rejection>),
-    RealizerRejected(String),
     Solver(String),
     UnsupportedCostModel(String),
-    ObjectiveDiffersFromPlanCost { objective_per_second: f64, plan_cost_per_second: f64 },
-    SelectedPlanFailedCheck { cause: RejectionCause, reason: String },
 }
 /// Why a candidate lost. Stays.
 pub enum RejectionCause {
     AccuracyTargetMissed,
-    CountMinWeightsNotProvenNonNegative,
     NoAccuracyGuarantee,
     LatencyBoundExceeded,
     LatencyOverRepeatInterval,
     OverClusterCapacity,
     CostNotRepresentable,
     Costlier,
     Slower,
     MoreMemory,
     TiedLostOnEnumerationOrder,
     Infeasible,
 }

Facade and tools: planner/src/pass.rs · run.rs · installable.rs · cost/src · frontend-sql/src/lib.rs · frontend-metricsql/src/lib.rs

 /// Planning logic and models, split by stage. Stays.
 pub struct PlanningModels<'a> {
-    pub accuracy: &'a dyn AccuracyModel,
-    pub evidence: &'a dyn AccuracyEvidenceProvider,
+    /// The deployment's measured accuracy, read by Pass 2; none by default.
+    pub evidence: &'a dyn AccuracyEvidence,
     /// The deployment's cost inputs.
     pub cost_inputs: &'a CostInputs,
     /// What the deployment's executor can run and keep.
     pub deployment_capabilities: &'a DeploymentCapabilities,
     /// Do not enforce accuracy: the facade gives every query no target before Pass 2.
     pub ignore_accuracy_guarantees: bool,
 }
 impl<'a> PlanningModels<'a> {
-    pub fn new(
-        accuracy: &'a dyn AccuracyModel,
-        evidence: &'a dyn AccuracyEvidenceProvider,
-    ) -> Self;
+    /// This evidence with the default settings.
+    pub fn new(evidence: &'a dyn AccuracyEvidence) -> Self;
-    pub fn accuracy_check_inputs(&self) -> AccuracyCheckInputs<'a>;
-    pub fn with_accuracy(mut self, accuracy: &'a dyn AccuracyModel) -> Self;
-    pub fn with_evidence(mut self, evidence: &'a dyn AccuracyEvidenceProvider) -> Self;
+    /// Replaces the evidence.
+    pub fn with_evidence(mut self, evidence: &'a dyn AccuracyEvidence) -> Self;
 }
-pub struct QueryPlan {
-    pub entry_index: usize,
-    pub root: Rc<OperatorNode>,
-}
 /// The selected plan: one `PhysicalASAPDAG`, the only plan structure. Stays.
 pub struct PlanOutput {
-    pub plans: Vec<QueryPlan>,
-    pub scalar_roots: Vec<(usize, asap_types::ir::ScalarExpr)>,
-    pub scalar_plans: Vec<QueryPlan>,
-    pub selection: Option<Selection>,
-    pub tumbling_window_retention_ms: BTreeMap<PhysicalASAPNodeId, u64>,
+    pub plan: PhysicalASAPDAG,
 }
 impl PlanOutput {
-    pub fn new(plans: Vec<QueryPlan>) -> Self;
+    pub fn new(plan: PhysicalASAPDAG) -> Self;
 }
 /// Why a pass failed. Stays.
 pub enum OptimizeError {
     Input(#[from] OptimizationInputError),
     PerComputationCandidates(#[from] PerComputationCandidateError),
     Selection(#[from] SelectionError),
     /// The pass returned something the contract forbids: a pass defect.
     ContractViolation { pass: &'static str, detail: String },
+    /// The pass returned an invalid DAG: a pass defect (P12).
+    InvalidPlan {
+        pass: &'static str,
+        error: PhysicalASAPDAGValidationError,
+    },
 }
 /// Every stage's output for one workload. Stays.
 pub struct PlanningRun {
-    pub per_computation_candidates: Option<PerComputationLogicalMemoCandidates>,
+    /// Pass 1's memo.
+    pub pass1_memo: Option<Memo>,
     pub logical_asap_candidates: Option<LogicalASAPMemoCandidates>,
     pub physical_asap_candidates: Option<PhysicalASAPMemoCandidates>,
-    pub selection: Result<Selection, SelectionError>,
-    pub selected_physical_plan: Option<RealizedPlan>,
-    pub selected_plan: Option<Vec<QueryRoot>>,
+    /// Stage 3's plan and its report.
+    pub selection: Result<(PhysicalASAPDAG, Selection), SelectionError>,
     pub report: PlanningReport,
 }
 /// The installable plan of `plan` with the report of its cost (P13).
 pub fn installable_plan_of(
-    roots: &[QueryRoot],
-    selection: Option<&Selection>,
-    tumbling_window_retention_ms: &BTreeMap<PhysicalASAPNodeId, u64>,
+    plan: &PhysicalASAPDAG,
+    cost: Option<&PlanCost>,
     entries: &[QueryWorkloadEntry],
 ) -> Result<(InstallablePlan, Option<SelectedPlanReport>), InstallablePlanError>;
 /// The selected plan's cost, for reporting. Stays.
 pub struct SelectedPlanReport {
     pub cost_per_second: Option<f64>,
     pub predicted_usage: PredictedUsage,
-    pub search_report: SearchReport,
-    pub queries_accepted_without_guarantee: Vec<usize>,
 }
 /// Which costs one node's CPU came from (M8).
-pub enum NodePriceSource {
+pub enum NodeCostSource {
     Analytical,
     Measured,
 }
 /// How a cost model orders valid plans (M8).
 pub enum PlanRanking {
-    Price,
+    Cost,
     Latency,
     Memory,
 }
 /// The plan's cost.
 pub fn plan_cost(
     dag: &PhysicalASAPDAG,
     nodes: &NodeMap<'_>,
-    kept_chains: &HashSet<PhysicalASAPNodeId>,
     recurrence: &[QueryRecurrence],
     range_evaluations: &[Option<RangeEvaluation>],
     data: &DataWorkload,
     settings: &CostInputs,
     formula_scale: f64,
 ) -> Result<(PlanCost, CalibrationFit), (PhysicalASAPNodeId, AnalyticalCostError)>;
 /// SQL front end: no accuracy argument (measures carry none).
 pub async fn lower_sql(
     query: &str,
     catalog: &SqlCatalog,
-    accuracy: AccuracyTarget,
 ) -> Result<Rc<OperatorNode>, SqlError>;
 pub async fn lower_sql_dialect(
     query: &str,
     catalog: &SqlCatalog,
     dialect: SqlDialect,
-    accuracy: AccuracyTarget,
 ) -> Result<Rc<OperatorNode>, SqlError>;
 /// MetricsQL front end: no accuracy argument.
 pub fn lower_metricsql_query(
     query: &str,
-    accuracy: AccuracyTarget,
     sample_interval: Duration,
 ) -> Result<asap_types::ir::QueryRoot, MetricsqlError>;

Also: NodeCost::cost_source (was price_source); summary_type_name (was family_name); PlanCost::cost_unit and COST_UNIT deleted; InstallablePlanError::TumblingWindowRetentionOfUnknownChain deleted; INSTALLABLE_PLAN_WIRE_VERSION 14 (was 13). Tools: the asap_planner document has no L1/P1 candidate lists (they were the realized plan); stage3_plan_selection holds plan and the new Selection fields; sketch_coverage loses --max-error; the fuzz planner's plan read and the peak-memory warning are todo!() (PR 9, PR 6).

File layout and renames (2026-11-10 review)

Pure moves and renames: no logic change, no alias for an old path. Public asap_types::stages::X paths are unchanged; every user of a moved module path is updated, docs included.

 pub struct MemoQuery {
-    pub values_per_evaluation: Option<u64>,
+    pub number_of_values_per_query_evaluation: Option<u64>,
 }
 impl Memo {
-    pub fn set_input_requirements(&mut self, id: CandidateForMemoGroupId, input_requirements: Vec<LogicalRequirements>);
+    pub fn set_candidate_input_requirements(&mut self, id: CandidateForMemoGroupId, input_requirements: Vec<LogicalRequirements>);
 }
-impl LogicalRequirements {
-    pub fn error_propagation(consumer: &Operator<MemoGroupId>, metric: AccuracyMetric) -> Option<ErrorPropagation>;
-}
-pub struct asap_types::stages::ErrorPropagation { .. }
+// asap-accuracy, error_propagation.rs
+pub fn asap_accuracy::error_propagation(consumer: &Operator<MemoGroupId>, metric: AccuracyMetric) -> Option<ErrorPropagation>;
+pub struct asap_accuracy::ErrorPropagation { .. }
-pub fn asap_types::stages::parameter_list(algorithm: SketchAlgorithm) -> &'static [SketchParams];
+pub fn asap_types::summary_type::parameter_list(algorithm: SketchAlgorithm) -> &'static [SketchParams];
Was Now What
types/src/stages/candidates.rs (Memo, MemoQuery) and memo.rs (root_groups, queries_rooted_at) types/src/stages/logical/memo/mod.rs memo scope (R3)
types/src/stages/memo.rs (MemoGroupId, MemoGroup, GroupProperties, ValueSign, MemoGroupAsInput) and candidates.rs (derive_group_properties) types/src/stages/logical/group.rs group scope
types/src/stages/candidates.rs (CandidateForMemoGroupId, CandidateForMemoGroup, CandidateExpression, OriginOfCandidateForMemoGroup, CandidateError, LogicalProperties(Error)) types/src/stages/logical/candidate.rs candidate scope
types/src/stages/candidates.rs (LogicalRequirements, RequiredAccuracy) types/src/stages/logical/requirements.rs candidate-input scope
types/src/stages/logical_memo_candidates.rs (LogicalEdge) types/src/stages/logical/edge.rs edge scope
types/src/stages/logical_memo_candidates.rs (LogicalASAPMemoCandidates) types/src/stages/logical/memo_candidates.rs Stage 1 output
types/src/stages/logical_memo_candidates.rs (PrunedLogicalCandidate, PruneCause, RootPruneReason) types/src/stages/pruning.rs every stage's removed candidates
types/src/stages/physical/memo_candidates.rs (PhysicalCandidate(Id), producer_satisfies_physical_requirements; PhysicalEdge(Id)) types/src/stages/physical/candidate.rs; edge.rs candidate and edge scope
types/src/physical/properties.rs (module asap_types::physical deleted) types/src/stages/physical/properties.rs (asap_types::stages::{PhysicalProperties, Lifetime, StorageMedium, PhysicalPropertiesError}) E1
types/src/stages/candidates.rs (parameter_list) types/src/summary_type/parameter_lists.rs L2, E1
types/src/ir/schema/sketch.rs types/src/summary_type/parameters.rs (θ types, EXACT_ACCUMULATOR_STATE_BYTES, KLL_MAX_K) and statistic.rs (SketchStatistic, SummaryUpdate, SummaryInputExpr, WeightDomain, NonNegativeWeightProof, EntityIdentity); SketchAlgorithm re-exported in summary_type/mod.rs E1 (asap_types::ir::schema::X → asap_types::summary_type::X)
types/src/ir/schema/summary_type_capability.rs types/src/summary_type/capability.rs E1
types/src/ir/schema/schema_support.rs types/src/ir/schema/series_identity.rs E1
types/src/ir/operator/operator_properties.rs types/src/ir/operator/arguments.rs E1
types/src/ir/properties/{accuracy,guarantee}.rs types/src/accuracy/{target,guarantee}.rs (asap_types::accuracy::X) E1
types/src/ir/physical_export.rs, ir/materialized_view.rs types/src/plan/physical_asap_dag.rs, plan/materialized_view.rs E1
types/src/workload/mod.rs workload/query.rs, data.rs, planning.rs; mod.rs re-exports (paths unchanged) E1
types/src/stages/candidates.rs (LogicalRequirements::error_propagation, ErrorPropagation) accuracy/src/error_propagation.rs (free fn) L3
accuracy/src/estimators/ accuracy/src/guarantee/ E2
accuracy/src/model.rs evidence.rs (AccuracyEvidence, NoMeasuredGuarantees) and requirements.rs (producer_satisfies_logical_requirements) E2
accuracy/src/lib.rs (target_limit, meets) accuracy/src/requirements.rs; lib.rs re-exports only E2
accuracy/src/evidence.rs (PropagationStats) accuracy/src/propagation_stats.rs renamed so the new evidence.rs does not collide
cost/src/lib.rs (cost models, CostInputs, CalibrationFit, cost-source consts) cost/src/model.rs E3
cost/src/lib.rs (ResourceUsage), resources.rs cost/src/resources/mod.rs E3
cost/src/usage.rs (node_usage, NodeUsage, ProducerOutput, node_map, summary_type_name) and lib.rs (NodeCost) cost/src/node/usage.rs E3
cost/src/{analytical,state_bytes,operator_statistics}.rs cost/src/node/ (asap_cost::node::analytical, ::node::state_bytes) E3
cost/src/usage.rs (plan_cost) and lib.rs (PlanCost, QueryLatency) cost/src/plan/cost.rs E3
cost/src/usage.rs (tumbling-window retention, query_reach, maintained_window_ms) cost/src/plan/retention.rs E3
cost/src/{execution_demand,ranking}.rs cost/src/plan/ (asap_cost::plan::execution_demand) E3
logical-asap-optimization/src/pass1/logical_memo_candidates.rs (pass1_memo, Pass1Error) pass1.rs L5 (before PR 0b the passes are modules)
logical-asap-optimization/src/pass1/per_computation_candidates(_tests).rs pass1/computation_targets(_tests).rs L5
logical-asap-optimization/src/pass2/logical_memo_candidates.rs (generate_logical_asap_memo_candidates and its two steps) pass2.rs L6

Removed items and their replacements

Deleted Replaced by
SketchKind, GroupingStrategy, HydraKind, ops' grouping, SummaryGroupingMismatch FieldDataType::Sketch(SketchAlgorithm) with the Hydra* algorithms
FamilyCapability, of_per_instance_sketch, family_capability.rs SummaryTypeCapability::of (summary_type/capability.rs)
sized-for θ fields, hand PartialEq, with_failure_probability (×3), KMV/HLL sized ε structure-only θ; the δ levels are list entries (PR 1) and the guarantee is evaluated at δ_r (PR 4)
parameter floor, LogicalRequirements::new, RootPruneReason::BelowAccuracyFloorRequirement (earlier PR 0 rounds) required_accuracy: Option<RequiredAccuracy>, BelowRequiredAccuracy
asap_types::stages::producer_satisfies_logical_requirements(producer, requirement) (earlier round) asap_accuracy::producer_satisfies_logical_requirements(memo, producer, consumer, input, evidence)
AccuracyModel, BuiltinAccuracyModel, IgnoreAccuracyGuarantees, AccuracyEvidenceProvider, NoAccuracyEvidence, OperandDomain, AccuracyCheckInputs, PlanningModels::{accuracy, accuracy_check_inputs, with_accuracy} AccuracyEvidence (Pass 2); ignore mode = no targets before Pass 2
local_guarantee theoretical_guarantee
pub struct AccuracyEvidence {} (earlier round) trait AccuracyEvidence + NoMeasuredGuarantees
Memo::{query_roots, query_accuracy_targets, query_values_per_evaluation} Memo::queries: Vec<MemoQuery>
satisfies, metric_meets, scope_meets, native_target, AccuracyScope::covers target_limit; the metric and scope rules: no replacement (α^log compares one metric at δ_r, PR 4)
SupportedSummary::family SupportedSummary::summary_type
Pass 1 accuracy pruning (without_summaries_no_query_can_use, output_can_meet_target, sizing_target, accuracy_budget, DEFAULT_DELTA, PerComputationCandidateError::InvalidAccuracy) Pass 2 dead-candidate elimination (PR 4)
AggIntent measures' accuracy; front ends' accuracy parameter the query's target in the memo (MemoQuery::accuracy_target)
SourceFacts::values_non_negative SourceFacts::column_signs (P7)
ExecutionContext::Evaluation { recurrence, range_evaluation }, RefreshWindowId (earlier rounds) Evaluation { grid: EvaluationGrid }, IngestionPipeline(RefreshWindow)
RealizedPlan, PhysicalWorkloadPlan, DagOfSelectedCandidates, SelectedEdge, SelectedNodeId, AssignmentRealizer, Stage2AssignmentRealizer, PhysicalCandidateError, SelectionError::Realization, ErrorSelectingPlan::RealizerRejected Stage 3 builds PhysicalASAPDAG; selected_physical_candidate
SearchReport, SearchStatus, InexactObjective, relative_gap, ObjectiveDiffersFromPlanCost, SelectedPlanFailedCheck, Selection::{selected_dag, nodes_with_proven_non_negative_weights, cost, search_report}, PlanCost::cost_unit, COST_UNIT, AcceptedAccuracy::{target_metric, accepted_regardless_of_guarantee, guarantee_check_rejection}, RejectionCause::CountMinWeightsNotProvenNonNegative Selection::{milp_*}; plan_cost(&plan); (K) has every bound; PruneReason::CountMinWeightsNotProvenNonNegative
with_confirmed_weights(_at), confirmed_weights, WeightDomain::confirmed none: their only consumer applied Stage 3's weight proofs (column_sign, PR 4–5)
PlanOutput::{plans, scalar_roots, scalar_plans, selection, tumbling_window_retention_ms}, QueryPlan, PlanningRun::{selected_plan, selected_physical_plan}, PlanningReport::search_report, InstallablePlanError::TumblingWindowRetentionOfUnknownChain PlanOutput { plan }, PlanningRun::selection; retention is on the nodes
ScanWindow, union, scan_window(s), view_window(s), time_range_window, offset_below (types); materialized_tumbling_window_chains, trailing_window, widest_window_read_ms, node_usage's scan_window, plan_cost's chain set (cost) Scan reads and retention_ms ($h_v$) (PR 6–7); executor-private copy until PR 6 (P10). In asap-cost a query-time Scan's read and retention are todo!() (PR 7); set_retention, raw_data_retention and refresh_windows are todo!() (PR 6)
NodePriceSource, price_source, PlanRanking::Price, family_name NodeCostSource, cost_source, PlanRanking::Cost, summary_type_name
pass1/composition.rs (compose_logical_candidate and its helpers), test_support::{cms_summary, weighted, estimate_over}, planning::{explicit_local_candidates, explicit_summary_candidate}, the PromQL tests' logical_candidates/logical_plan/local_computation_candidates, the executor tests' computation_candidates (P11) none: planning adds every way to compute to the memo (Pass 1, PR 3), which re-creates Pass 1 coverage over the memo
ShownLogicalCandidate, logical_candidates(), physical_candidates(), candidate_labels, physical_label and its helpers, usage_json, sketch_coverage --max-error none (R10); the document's stage 3 holds plan

Usage, end to end (the bodies are todo!() until the PRs named, so this is the call flow, not a run):

// What the user writes: p99 / p50 of `latency` (declared non-negative), relative error 0.03, every 10 s.
let workload: PlanningWorkload = serde_json::from_str(r#"{
  "query_workload": {"language": "promql", "repeating_queries": [{
    "query": "quantile_over_time(0.99, latency[1m]) / quantile_over_time(0.5, latency[1m])",
    "demand": {"fixed_interval_at": {"interval": 10000, "evaluation_phase": 0}},
    "requirements": {"accuracy": {"accuracy_metric": "relative", "max_error": 0.03}},
    "predictability": {"predictable": {"known_at": null}}}]},
  "data_workload": {"arrival": {"ingesting": {"rows_per_second": 1000.0}},
    "sources": {"latency": {"series_count": 1, "column_signs": {"value": "non_negative"}}}}}"#)?;
// The front end lowers the query with no accuracy on its measures (Quantile { col, q }).
// run_planning: Stage 0 adds the query with Some(target) and n_q = 1 (memo_of_queries, PR 2);
// Pass 1 adds KLL and DDSketch builds, one per listed θ (pass1_memo, PR 3);
// Pass 2 gives each readout a RequiredAccuracy level and computes α^log with
// producer_satisfies_logical_requirements(memo, build, readout, 0, &NoMeasuredGuarantees) (PR 4):
// a KLL readout is cut by MetricRule, a DDSketch build with alpha 0.05 by
// BelowRequiredAccuracy { required: relative 0.02 }.
// Stage 3 returns (plan, selection) (select_plan, PR 6): plan.nodes[i].selected_physical_candidate
// names each node's candidate.
// `output`: what the user gets back, PlanOutput { plan }; its cost is plan_cost(&plan).
let output = e2e_plan(UserInput::new(&workload, FrontendInput::Promql, PlanningModels::builtin())).await?;

Algorithm changes

None: the bodies of the new interfaces are todo!(). One behavior is removed: Pass 1 no longer prunes summaries no query could use for accuracy (without_summaries_no_query_can_use); Pass 2's dead-candidate elimination (PR 4) replaces it. The executor's window derivation is unchanged code in a new place (P10).

Decisions and open questions

Needs reviewer decision: None.

Decided (all approved; nothing pending):

  • 2026-11-10 (review of this PR): R1 MemoQuery::values_per_evaluation → number_of_values_per_query_evaluation; R2 Memo::set_input_requirements → set_candidate_input_requirements; R3 Memo/MemoQuery live in the memo module. L1 types/src/stages/ is organized by scope, logical vs physical first, then memo → group → candidate → candidate input → edge → stage output, with pruning.rs for every stage's removed candidates. L2 parameter_list beside the θ types. L3 error propagation in asap-accuracy as the free function error_propagation. L4 HowToComputeAggregateMeasure leaves asap-types once only Pass 1 uses it. L5 Pass 1's entry in its root module, computation_targets.rs for choosing targets. L6 Pass 2's entry in its root module. E1–E3 asap-types, asap-accuracy and asap-cost organized by scope the same way (summary types, accuracy, plan, workload split; guarantee/evidence/requirements; model, resources, node, plan). Pure moves and renames; the move table is under Interfaces.
  • 2026-11-10, choices made applying them (no behavior change):
    • L4 does not apply in this PR: ComputationTarget and FusedComputation, still in types/src/stages/per_computation_logical_memo_candidates.rs, hold HowToComputeAggregateMeasure, and devtools and the SQL front end's tests read it, so the file stays until PR 3 empties it.
    • L3 keeps this PR's signature error_propagation(consumer, metric); PR 4 adds the memo argument.
    • L5: this PR has no add_aggregate_candidates yet, so all of Pass 1's target code is computation_targets.rs; PR 3 adds candidates.rs. The #[cfg(any(test, feature = "test-support"))] that sat on the old mod logical_memo_candidates is gone with the file (the crate root re-exported its items unconditionally, so cargo test -p asap-logical-asap-optimization without that feature did not build before; it does now).
    • asap_types::physical::ExecutionTiming was a re-export of ir::properties::ExecutionTiming; its users now name ir::properties::ExecutionTiming.
    • accuracy/src/evidence.rs held PropagationStats (deleted in PR 4), so it is renamed propagation_stats.rs and the new evidence.rs holds AccuracyEvidence. The modules that glob-imported lib.rs's imports (composition.rs, budget.rs, guarantee/mod.rs, propagation_stats.rs) now import what they use.
    • summary_type_name stays with node_usage in cost/src/node/usage.rs, its only reader. cost_source is pub(crate) there because plan_cost reads it too; derive_group_properties is pub(super) in group.rs because the memo calls it; MAX_RANGE_EVALUATION_STEPS and validate_rate are pub(super) in workload/ because the query and data files share them.
    • Time aliases TimestampMs and DurationMs are in workload/query.rs, their main user; workload/data.rs imports DurationMs.
    • Tests stay inline (mod tests) in the scope file of what they test; workload/mod.rs's tests are split over query.rs, data.rs and planning.rs.
  • 2026-11-08: Terminology: family = FieldDataType variant, algorithm = one structure (SketchAlgorithm, Hydra included), summary type = both (one FieldDataType value), θ = sizes. Accuracy is a requirement on a readout's input, not a θ floor: one rule for every algorithm; θ records no error. The accuracy source is decided up front (measured guarantee, else the formula), so nothing is checked after Stage 3: AccuracyModel, the post-solve check and the "accept regardless" mode go. One plan structure: Stage 3 builds the PhysicalASAPDAG directly; the selected-candidate DAG, the realizer and the plan wrappers go. N1: measures carry no accuracy. M8: cost names. R4: Serialize on the prune and property types. R10: the label tooling deleted.
  • 2026-11-09 (P1, P2): AccuracyEvidence is a trait with one method, its bound at δ; NoMeasuredGuarantees is the default.
  • 2026-11-09 (P18): the bound at δ is an AccuracyGuarantee whose failure_probability is the δ asked for. Rejected: a new type.
  • 2026-11-09 (P3): producer_satisfies_logical_requirements in asap-accuracy (it needs the guarantee; asap-types cannot depend on asap-accuracy).
  • 2026-11-09 (P4): one Vec<MemoQuery>, replacing query_roots and the two parallel vectors. (P17): MemoQuery::accuracy_target: Option<AccuracyTarget>; None carries ignore_accuracy_guarantees and "no target reaches it". Rejected: a non-optional AccuracyTarget.
  • 2026-11-09 (P5): memo_of_queries(roots, accuracy_targets, data): Stage 0 computes $n_q$ (series_read, values_per_result kept for it) and reads the declared signs from data.
  • 2026-11-09 (P6): the Memo stores the declared signs (Memo::new(declared_column_signs), private field; PR 2), because rules add groups too (a rollup's finest common Scan).
  • 2026-11-09 (P7): column_signs; the workload JSON in other repositories changes in one PR.
  • 2026-11-09 (P8): SummaryTypeCapability::shares_one_state_across_groups, one Hydra list read by merges_groups and the executor's kernel check (it was duplicated); the doc's Hydra capability (subpopulation readout) comes when Hydra is in scope.
  • 2026-11-09 (P9): PhysicalASAPDAG::query_roots() (test support) and integration-tests' plan_cost_per_second: tests that inspected selected_plan or Selection::cost read the plan instead.
  • 2026-11-09 (P10): ScanWindow executor-private until PR 6.
  • 2026-11-09 (P11): pass1/composition.rs deleted in this PR, with its fixtures and the 74 tests that use them. Rejected: keeping it behind test-support until PR 3 (no interim code).
  • 2026-11-09 (P12): OptimizeError::InvalidPlan { pass, error }: the pass contract validates the returned DAG and that its roots and scalar roots answer every entry.
  • 2026-11-09 (P13): installable_plan_of(plan, cost: Option<&PlanCost>, entries); SelectedPlanReport { cost_per_second, predicted_usage } (the report is the plan's cost, not the selection's).
  • 2026-11-09 (P14): summary_type (also SupportedSummary, JSON summary_type; the viewer reads it). (P15): WeightDomain deleted in PR 5, Count-Min reads column_sign. (P16): satisfies deleted.
  • 2026-11-09 (PR 1's P2, P3): delivers_on(&self, candidates: &PhysicalASAPMemoCandidates, context) so a Materialized(v) context can read v; GroupProperties::depends_on_evaluation_time and LogicalASAPMemoCandidates::evaluation_grid_of_group. Example: X = sum_over_time(m[5m]) read every 60 s and every 120 s has grid 60 s @ 0, so a stored X delivers on 60 s and 120 s grids, not on 30 s.
  • Not in PR 0, and later PRs must not add them: family_of_operator/summary_type_of_operator and sketch_algorithm_of (PR 1 adds them; with Sketch(SketchAlgorithm) the latter is a match); floor_for_cap, strongest, weakest, dominates, combine (PR 1/PR 4); ExportedNodeAssignment, physical_asap_dag_with_assignments (PR 6); AccuracyCheckInputs::query_roots (the struct is deleted).

Verification

cargo test -p <crate> --all-features --no-fail-fast at the head; cargo clippy --workspace --all-targets --all-features -- -D warnings and cargo clippy --workspace -- -D warnings clean (no new #[allow]; derive_group_properties carries #[allow(unreachable_code)] until PR 2 derives column_sign); cargo fmt --check clean; crate_graph passes (no stage crate depends on another). Every failure except the stale fixture panics at a todo!(); the notes name it and the PR that fills it in. Newly failing vs main: not measured by test name; main has none of these todo!()s, so the failures below are expected until their PRs.

Scope Passed / failed / ignored Newly failing vs base Notes
asap-types 153 / 2 / 0 not measured vs main reject the invalid combination of section 2 (PR 1): 2
asap-accuracy 59 / 0 / 0 —
asap-cost 31 / 1 / 0 not measured vs main check θ, which the exported node carries (PR 6): 1
asap-frontend-common 49 / 0 / 0 —
asap-sql-function-catalog 10 / 0 / 0 —
metricsql_common 3 / 0 / 0 —
asap-frontend-promql 218 / 1 / 0 not measured vs main Stage 0 (PR 2): 1
asap-frontend-metricsql 17 / 0 / 0 —
asap-frontend-sql 173 / 1 / 0 not measured vs main Stage 0 (PR 2): 1
asap-logical-asap-optimization 25 / 24 / 0 not measured vs main Stage 0 (PR 2): 23; the distinct-count guarantee of θ (PR 4): 1
asap-physical-asap-optimization 0 / 0 / 0 —
asap-plan-selection 3 / 0 / 0 —
asap-executor 210 / 63 / 1 not measured vs main θ from the exported node (PR 6): 32; Hydra grid sizes are θ (PR 6): 13; retention_ms is $h_v$, set by Stage 3 (PR 6): 8; weighted-frequency θ (PR 6): 7 + 2; Stage 0 (PR 2): 1
asap-planner 8 / 18 / 1 not measured vs main Stage 0 (PR 2): 18
asap-devtools 37 / 56 / 3 not measured vs main Stage 0 (PR 2): 54; assertion failed: output.status.success(): 2
asap-integration-tests 82 / 51 / 4 not measured vs main Stage 0 (PR 2): 47; retention_ms (PR 6): 3; stale fixture materialized_views_equal_the_stored_outputs_main_listed (old wire format; PR 9 re-baselines it): 1
Total 1078 / 217 / 9

By test name against earlier heads of this PR:

  • vs 61541f748 (before the 2026-11-10 file layout), asap-types, asap-accuracy, asap-cost and asap-logical-asap-optimization (feature test-support): 269 passed / 27 failed / 0 ignored on both heads; the same tests by name (only module paths and the workload doctests' file changed); cargo check --workspace --tests with no warnings, cargo clippy --workspace --all-targets -- -D warnings and cargo fmt --check clean; cargo doc with -D warnings reports the same pre-existing link errors as before. The other crates' tests were not rerun for this move (they only change import paths and compile).
  • vs be6ea1d6a (before P11): the only differences are the 74 tests deleted with pass1/composition.rs (70 failed, 3 passed, 1 ignored there); no other test changed result.
  • vs 7708f3668: no test that passed fails now. Differences: 8 deleted tests and the added target_limit_admits_only_floating_point_noise (passes). Passed 1088 → 1081, failed 287 → 287.

New tests:

  • accuracy tests::target_limit_admits_only_floating_point_noise: the limit admits only floating-point noise above the target.
  • contract_rejects_a_scalar_root_keyed_beyond_the_entries: the pass contract rejects a scalar root keyed beyond the entries.

Deleted tests (with why):

File Deleted Why
accuracy/src/guarantee/mod.rs (was estimators/mod.rs) enlarging_meets_the_failure_probability_at_no_larger_error, a_distinct_count_univmon_grows_its_heap, local_guarantee_needs_a_statistic_the_family_answers δ resizing and local_guarantee deleted
accuracy/src/guarantee/kll.rs sizing_meets_a_failure_probability_below_one_percent, sizing_below_one_percent_names_a_distinct_contract, an_invalid_failure_probability_gives_no_guarantee KLL's sized-for failure_probability deleted; PR 4 evaluates the guarantee at δ_r
accuracy/src/guarantee/cardinality.rs kmv_sized_for_a_failure_probability_meets_it, kmv_k_follows_the_failure_probability KMV's sized-for ε deleted
types/src/ir/schema/sketch.rs (now types/src/summary_type/) sketch_kind_keeps_a_valid_algorithm_and_params_pair, sketch_kind_rejects_params_from_another_algorithm, grouping_strategy_default_is_per_subpopulation_instance SketchKind, GroupingStrategy deleted
types/tests/physical_export.rs a_scan_window_is_the_union_of_its_readers_windows, a_view_window_is_the_union_of_the_windows_below_it, a_scan_read_under_an_at_anchor_has_an_unbounded_window the DAG's window functions deleted (P10)
logical-asap-optimization/src/pass1/per_computation_candidates.rs (now pass1/computation_targets.rs) count_min_estimate_over_raw_data_samples_is_gated_on_the_metric compared with local_guarantee without θ; replaced by column_sign (PR 4)
planner/src/pass.rs contract_rejects_a_scalar_reference_without_its_plan, contract_checks_scalar_plan_order_entry_and_count scalar_plans deleted
devtools/src/candidate_labels.rs connected_labels_describe_selected_states, connected_labels_name_actual_shared_state, connected_labels_name_exact_results the L1 labels deleted
frontend-promql/tests/promql_lowering.rs, frontend-sql/tests/sql_lowering.rs accuracy_target_flows_into_quantile_intent, median_threads_the_accuracy_target measures carry no accuracy
integration-tests/tests/promql_numeric_regressions.rs logical_candidates_fails_on_a_candidate_that_does_not_compose ShownLogicalCandidate deleted
integration-tests/tests/joint_search_equivalence.rs custom_accuracy_policy_is_respected AccuracyModel deleted
accuracy/src/lib.rs satisfies_is_fail_closed_on_unknowns_and_exact, satisfies_requires_the_targets_metric, a_target_without_delta_ignores_the_failure_probability, per_value_guarantees_meet_whole_result_targets_over_g_values, a_record_share_target_is_met_over_one_over_share_groups, scope_matters_only_with_a_failure_probability, a_target_without_a_metric_accepts_any_metric satisfies deleted
types/src/ir/properties/accuracy.rs (now types/src/accuracy/target.rs) a_wider_scope_covers_a_narrower_one AccuracyScope::covers deleted

Renamed, with the accuracy assertion dropped: count_maps_to_count (was count_maps_to_count_and_inherits_accuracy), distinct_over_time_lowers_to_cardinality_over_nested_windows, asap_input_is_rejected (its invalid-accuracy half deleted with InvalidAccuracy). Tests that read PlanOutput::plans, selected_plan or Selection::cost now read plan.query_roots() and plan_cost_per_second (P9). The estimator tests that called satisfies (CMS, Count Sketch, HLL, KLL, UnivMon, composition) check the evaluated error and δ against target_limit through a test-only meets, with their metric asserted beside it.

P11: 74 tests deleted, all composing whole-workload candidates through pass1/composition.rs (names in this comment): frontend-metricsql/tests/external_corpus.rs 1, frontend-promql/tests/count_planning.rs 4, .../observability/metrics_observability.rs 2, .../observability/promql_corpus.rs 1, frontend-promql/tests/univmon_candidates.rs 1, integration-tests/tests/exact_composition.rs 1, per_computation_candidates_sql.rs 4, promql_numeric_regressions.rs 7, sketch_rollup.rs 3, sql_to_post_asap.rs 7, stage1_logical_asap_optimization.rs 13, pass1/per_computation_candidates.rs 19, pass1/per_computation_candidates_tests.rs 10, pass1/realization.rs 1. Files left without a test are deleted with their registrations: frontend-promql/tests/univmon_candidates.rs, frontend-promql/tests/observability/metrics_observability.rs (test target), integration-tests/tests/{exact_composition, promql_numeric_regressions, sketch_rollup, stage1_logical_asap_optimization}.rs.

End-to-end evidence: not applicable (planning panics at Stage 0 until PR 2; no snapshot or plan comparison applies).

Self-review (over-abstraction, duplication, dead code):

  • Duplication fixed: the pass contract's scalar-key check repeated PhysicalASAPDAG::validate's ScalarRootPosition (the contract counts the roots, then validates); the Hydra list was in merges_groups and the executor's kernel check (one shares_one_state_across_groups, P8); of_per_instance_sketch duplicated of; the satisfaction tolerance formula lives once, in target_limit.
  • Dead after the deletions, deleted: InvalidAccuracy and its mapping; computation_candidates_for_intent's never-failing Result; sources_read; the weight-proof application; the chain-window helpers; payloads_by_id, offset_below; TumblingWindowRetentionOfUnknownChain; OperandDomain; DEFAULT_DELTA; the label helpers (for_queries, query_names, duration_name, build_label, state_label, dag_readers) and usage_json; parse_max_error; metric_meets, scope_meets, native_target, AccuracyScope::covers, the static NO_ACCURACY_EVIDENCE; test helpers window_over, exported_with_scan, series_scan, approximate, error_and_delta, delta_and_error; the empty mod stream_norms {} (Pass 1 tests) and two empty mod tests {} (devtools planning_stages_text.rs, candidate_labels.rs).
  • Unused parameters removed: plan_cost's and each_node_usage's chain set, each_node_usage's root_scan_lookback_ms, node_usage's scan_window, devtools' lower_promql_with_sample_interval's accuracy, test helpers' accuracy.
  • Test-only code marked: PerComputationCandidates::{roots, aggregate} behind test-support.
  • Left, and why: target_limit has no production caller until PR 4 (α^log) and PR 8 ((K)); root_groups none until PR 5–6; AccuracyGuarantee::over_scope none (PR 4/8 state a whole-result δ); NoMeasuredGuarantees is the doc's default evidence; series_read, values_per_result wait for Stage 0's $n_q$ (PR 2); depends_on_evaluation_time is a second unreachable todo!() in derive_group_properties (PR 2); RejectionCause, Rejection and ErrorSelectingPlan::NoValidCandidate are never constructed (PR 6 decides); the #[allow(dead_code)] markers from earlier rounds (θ-based estimators and sizing, milp/model.rs, valid_matrix, GroupStates::Keyed) stay until PRs 4–8; the front-end test helper lower_promql(query, accuracy) still takes a target (it fills the workload entry, not the measures).
  • Over-abstraction: none added; the bound at δ reuses AccuracyGuarantee (P18).

Follow-ups

@zzylol zzylol added CSE optimizer Workload optimization and plan selection refactor labels Nov 6, 2026
This was referenced Nov 6, 2026
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
@zzylol
zzylol force-pushed the refactor/1207-interfaces branch 3 times, most recently from 4772efb to f57ae08 Compare November 6, 2026 13:22
@zzylol

zzylol commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

Per-file test deletions in #1226 (origin/main → PR head). D = file deleted, M = modified; counts are #[test] functions.

File main now deleted What it covered Restored by
cost/src/state_bytes.rs M 8 4 4 held memory / refresh runs over hand-assigned materialized DAGs; state bytes per θ PR 6–7
cost/tests/held_memory.rs D 1 0 1 held memory / refresh runs over hand-assigned materialized DAGs; state bytes per θ PR 6–7
cost/tests/ingestion_refresh_runs.rs D 7 0 7 held memory / refresh runs over hand-assigned materialized DAGs; state bytes per θ PR 6–7
devtools/src/candidate_labels.rs M 4 3 1 reference pricing (memo_pricing), stage text, labels PR 6 (Appendix A check), PR 9 (tools)
devtools/src/planning_stages_text.rs M 1 0 1 reference pricing (memo_pricing), stage text, labels PR 6 (Appendix A check), PR 9 (tools)
devtools/tests/asap_planner.rs M 46 43 3 reference pricing (memo_pricing), stage text, labels PR 6 (Appendix A check), PR 9 (tools)
devtools/tests/memo_pricing.rs D 6 0 6 reference pricing (memo_pricing), stage text, labels PR 6 (Appendix A check), PR 9 (tools)
executor/src/physical_planner/mod.rs M 3 1 2 executor runs of planner-built DAGs; kernels per θ PR 6
executor/src/summary_kernels/factory.rs M 7 5 2 executor runs of planner-built DAGs; kernels per θ PR 6
executor/tests/deployment_computation.rs M 16 3 13 executor runs of planner-built DAGs; kernels per θ PR 6
executor/tests/plan_properties.rs M 4 2 2 executor runs of planner-built DAGs; kernels per θ PR 6
executor/tests/precompute_candidates.rs D 2 0 2 executor runs of planner-built DAGs; kernels per θ PR 6
executor/tests/state_column_updater.rs M 7 5 2 executor runs of planner-built DAGs; kernels per θ PR 6
executor/tests/weighted_topk_binding.rs M 2 1 1 executor runs of planner-built DAGs; kernels per θ PR 6
frontend-promql/tests/univmon_candidates.rs M 2 1 1 planner-built plans exported and run PR 6
frontend-sql/tests/pearson_corr.rs M 7 6 1 planner-built plans exported and run PR 6
integration-tests/tests/declared_group_counts.rs D 3 0 3 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/deployment_capabilities.rs D 2 0 2 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/exact_composition.rs M 5 1 4 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/finer_aggregate_reuse.rs M 10 2 8 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/frequency_execution.rs D 1 0 1 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/having_shared_aggregate.rs D 1 0 1 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/materialization_e2e.rs D 4 0 4 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/materialized_views.rs M 4 2 2 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/memo_enumeration.rs D 3 0 3 per-group enumeration counts PR 6 (Appendix A)
integration-tests/tests/operator_design_examples.rs M 6 5 1 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/planner_layering_example1.rs D 31 0 31 #509 example acceptance suite over the reference pricer (WholePlanSelection) PR 6 (baseline), PR 7 (materialization), PR 8 (accuracy)
integration-tests/tests/planner_layering_example2.rs D 35 0 35 #509 example acceptance suite over the reference pricer (WholePlanSelection) PR 6 (baseline), PR 7 (materialization), PR 8 (accuracy)
integration-tests/tests/planner_layering_example3.rs D 22 0 22 #509 example acceptance suite over the reference pricer (WholePlanSelection) PR 6 (baseline), PR 7 (materialization), PR 8 (accuracy)
integration-tests/tests/planner_layering_example4.rs D 17 0 17 #509 example acceptance suite over the reference pricer (WholePlanSelection) PR 6 (baseline), PR 7 (materialization), PR 8 (accuracy)
integration-tests/tests/precompute_raw_samples.rs D 3 0 3 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/production_rollup_workloads.rs M 4 3 1 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/promql_to_post_asap.rs M 13 9 4 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/range_evaluation.rs D 6 0 6 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/reuse_rules.rs D 7 0 7 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/rollup_controlled.rs M 3 1 2 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/runtime_handoff.rs D 8 0 8 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/sharing_issue_regressions.rs M 12 6 6 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/sketch_rollup_planning.rs D 3 0 3 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/sql_group_bound.rs D 7 0 7 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/sql_to_physical.rs D 1 0 1 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/sql_trailing_window.rs D 3 0 3 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/sql_two_percentiles.rs D 4 0 4 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/stage1_logical_asap_optimization.rs M 14 13 1 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/stage2_physical_asap_optimization_sql.rs D 4 0 4 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/tumbling_window_merge.rs D 14 0 14 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
logical-asap-optimization/src/pass1/logical_memo_candidates.rs M 3 0 3 candidate population with private nodes, budget-share sizing, Pass 1 pruned reasons, θ in nodes PR 3; pruned reasons: #1205; budget split: no replacement planned
logical-asap-optimization/src/pass1/memo_population.rs D 13 0 13 candidate population with private nodes, budget-share sizing, Pass 1 pruned reasons, θ in nodes PR 3; pruned reasons: #1205; budget split: no replacement planned
logical-asap-optimization/src/pass1/per_computation_candidates.rs M 56 42 14 candidate population with private nodes, budget-share sizing, Pass 1 pruned reasons, θ in nodes PR 3; pruned reasons: #1205; budget split: no replacement planned
logical-asap-optimization/src/pass1/per_computation_candidates_tests.rs M 35 31 4 candidate population with private nodes, budget-share sizing, Pass 1 pruned reasons, θ in nodes PR 3; pruned reasons: #1205; budget split: no replacement planned
logical-asap-optimization/src/pass2/asap_aware_cse.rs D 27 0 27 node-level Pass 2 rewrites (windows, rollups, top-k, scan widening) PR 4
logical-asap-optimization/src/pass2/memo/rules/rollup.rs M 1 0 1 node-level Pass 2 rewrites (windows, rollups, top-k, scan widening) PR 4
logical-asap-optimization/src/pass2/memo/rules/shared_configuration_readers.rs D 2 0 2 rule 9 shared configurations no replacement planned (deleted by design)
logical-asap-optimization/src/pass2/memo/shared_configurations.rs D 6 0 6 rule 9 configuration enumeration no replacement planned
logical-asap-optimization/src/pass2/memo/state_groups.rs D 23 0 23 step 10 per-size state groups no replacement planned (deleted by design)
logical-asap-optimization/src/pass2/memo/window_candidates.rs M 2 0 2 node-level Pass 2 rewrites (windows, rollups, top-k, scan widening) PR 4
physical-asap-optimization/src/catalog/realize.rs M 1 0 1 Stage 2 catalog, materialization options, timing split, realization PR 5 (expansion), PR 6 (realization)
physical-asap-optimization/src/lib.rs M 24 0 24 Stage 2 catalog, materialization options, timing split, realization PR 5 (expansion), PR 6 (realization)
physical-asap-optimization/src/split_by_timing.rs D 4 0 4 Stage 2 catalog, materialization options, timing split, realization PR 5 (expansion), PR 6 (realization)
plan-selection/src/groups/accuracy_positions.rs D 1 0 1 #1144 MILP, result slots, dominance, whole-plan selector and its checks PR 6 (A)–(F)/decode, PR 7 contexts/(R), PR 8 (K); latency checks: no replacement planned
plan-selection/src/groups/tests.rs D 11 0 11 #1144 MILP, result slots, dominance, whole-plan selector and its checks PR 6 (A)–(F)/decode, PR 7 contexts/(R), PR 8 (K); latency checks: no replacement planned
plan-selection/src/lib.rs M 104 0 104 #1144 MILP, result slots, dominance, whole-plan selector and its checks PR 6 (A)–(F)/decode, PR 7 contexts/(R), PR 8 (K); latency checks: no replacement planned
plan-selection/src/milp/model/tests.rs D 20 0 20 #1144 MILP, result slots, dominance, whole-plan selector and its checks PR 6 (A)–(F)/decode, PR 7 contexts/(R), PR 8 (K); latency checks: no replacement planned
plan-selection/src/select/tests.rs D 4 0 4 #1144 MILP, result slots, dominance, whole-plan selector and its checks PR 6 (A)–(F)/decode, PR 7 contexts/(R), PR 8 (K); latency checks: no replacement planned
plan-selection/src/selector.rs D 1 0 1 #1144 MILP, result slots, dominance, whole-plan selector and its checks PR 6 (A)–(F)/decode, PR 7 contexts/(R), PR 8 (K); latency checks: no replacement planned
planner/src/installable.rs M 4 2 2 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
planner/src/pass.rs M 7 6 1 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
planner/tests/deployment_accuracy_model.rs D 3 0 3 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
planner/tests/per_sample_updater.rs D 2 0 2 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
planner/tests/runtime_install.rs M 42 5 37 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
planner/tests/sql_per_group_topk.rs D 6 0 6 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
planner/tests/sql_table_scans.rs M 11 3 8 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
planner/tests/summary_sharing.rs M 13 4 9 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
types/src/ir/properties/timing.rs D 23 0 23 timing pass over the side map PR 6
types/src/ir/schema/sketch.rs M 6 5 1 KLL equality across δ (θ in the type) PR 1
types/src/stages/physical/producers.rs D 2 0 2 producer-set interning no replacement planned
types/tests/materialized_view.rs D 10 0 10 export and materialized views with non-default physical properties; node timing PR 6
types/tests/physical_export.rs M 19 12 7 export and materialized views with non-default physical properties; node timing PR 6
types/tests/schema_rebuilding.rs M 9 7 2 export and materialized views with non-default physical properties; node timing PR 6
types/tests/structure_contract.rs M 7 6 1 export and materialized views with non-default physical properties; node timing PR 6
types/tests/summary_coverage.rs M 33 32 1 export and materialized views with non-default physical properties; node timing PR 6
types/tests/summary_merge_structure.rs M 9 8 1 export and materialized views with non-default physical properties; node timing PR 6
types/tests/tumbling_window_chain.rs M 14 10 4 export and materialized views with non-default physical properties; node timing PR 6
total 642

Restored in the 2026-11-06 revision through the test-only asap_types::test_support::physical_asap_dag_at_query_time(_with_node_ids) (every node at query time, unmaterialized, no θ): executor time_series_window, node_ids, current_series_heap, planspace_series_identity_heap, deployment_computation (3 of 15); integration filtered_aggregates, kll_tumbling_window_execution, per_series_exact_execution, sql_cardinality, sql_frequency, exact_composition, and parts of sketch_rollup, stage1_logical_asap_optimization, operator_design_examples, operator_sharing, promql_numeric_regressions, sql_to_post_asap, promql_to_post_asap; types physical_export; frontend count_planning, pearson_corr. Tests whose DAG came from the planner stay deleted (PR 6 restores them). Restored sketch tests fail at the executor's todo!() (θ from the node's parameters) until PR 6.

@zzylol
zzylol force-pushed the refactor/1207-interfaces branch from f57ae08 to 2966355 Compare November 6, 2026 13:24
@zzylol zzylol mentioned this pull request Nov 6, 2026
Merged
@zzylol zzylol self-assigned this Nov 6, 2026
This was referenced Nov 6, 2026
Merged
Merged
@zzylol
zzylol force-pushed the refactor/1207-interfaces branch 2 times, most recently from 341c98b to 3a75e1e Compare November 6, 2026 21:37
@zzylol

zzylol commented Nov 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Key code interfaces, continued (moved here from the PR body, which GitHub limits to 65,536 characters). Links point at the head.

crates/types/src/stages/logical_memo_candidates.rs

/// $(u, c, i)$ over logical candidates: `producer_candidate_id` is a
/// candidate of $G_{c,i}$, the group input `input_index_of_consumer` of
/// `consumer_candidate_id` reads.
pub struct LogicalEdge {
    pub producer_candidate_id: CandidateForMemoGroupId,
    pub consumer_candidate_id: CandidateForMemoGroupId,
    pub input_index_of_consumer: usize,
    /// $\alpha^{\log}_{u,c,i}$.
    pub satisfies_logical_requirement: bool,
}

/// Stage 1, Pass 2's output.
pub struct LogicalASAPMemoCandidates {
    /// The memo after the last rule.
    pub memo: Memo,
    /// Every $(u, c, i)$ with $u \in G_{c,i}$ between candidates that
    /// survive dead-candidate elimination, grouped by $(c, i)$.
    pub edges: Vec<LogicalEdge>,
    /// The candidates dead-candidate elimination removed. // new 2026-11-07
    pub pruned: Vec<PrunedLogicalCandidate>,
}

/// A candidate eliminated by Pass 2. // new 2026-11-07
pub struct PrunedLogicalCandidate {
    pub candidate: CandidateForMemoGroupId,
    /// Every cause: one per failing edge it is cut off by.
    pub causes: Vec<PruneCause>,
}

/// One path from the candidate, through consumers that are themselves
/// eliminated, to the edge that fails. One shortest path per failing edge. // new 2026-11-07
pub struct PruneCause {
    /// The eliminated consumers on the path, nearest first.
    pub through_pruned_consumers: Vec<CandidateForMemoGroupId>,
    pub root: RootPruneReason,
}

// new 2026-11-07
pub enum RootPruneReason {
    /// The producer's parameters are weaker than the consumer input's floor.
    BelowAccuracyFloorRequirement { consumer: CandidateForMemoGroupId, input: usize, floor: SketchParams },
    /// The consumer's error contract carries none of the producer's metrics.
    MetricRule { consumer: CandidateForMemoGroupId, input: usize, producer_metric: AccuracyMetric, consumer_carries: Vec<AccuracyMetric> },
    /// The candidate never had a consumer and is not a query output.
    NoConsumer,
}

crates/physical-asap-optimization/src/catalog/mod.rs (Stage 2 entry, 2026-11-07)

/// Stage 2. Costs nothing, and cannot fail. // was (logical, entries, data, query_time_results_retained, raw_data_retained) -> Result<_, ErrorGeneratingPhysicalASAPMemoCandidates>
pub fn generate_physical_asap_memo_candidates(
    logical: &LogicalASAPMemoCandidates,
    data: &DataWorkload,
    capabilities: &DeploymentCapabilities, // moved from asap-logical-asap-optimization into asap_types::deployment_capabilities
) -> PhysicalASAPMemoCandidates;

crates/types/src/stages/pruning.rs (2026-11-07)

/// One candidate a stage removed, and why: Stage 2's `PhysicalCandidate`,
/// Stage 3's `PhysicalCandidateId`. // loses `group`
pub struct PrunedCandidate<C> {
    pub candidate: C,
    pub reason: PruneReason,
}

pub enum PruneReason {
    NoIngestionPipeline,          // new
    RawDataNotRetained,
    QueryTimeResultsNotRetained,  // new
    InputHasNoBuildableProducer,
    ReachesNoQueryOutput,
    DominatedBy(PhysicalCandidateId),
    UsageNotCostable,
    // deleted: PerEntityAggregateNotExecutable (no executor-support check), TooFewQueryReaders (no consumer-count policies)
}

crates/types/src/workload/mod.rs (excerpt, 2026-11-07)

pub enum DataArrival {
    AtRest,
    Ingesting { rows_per_second: f64, max_out_of_order_ms: Option<u64> }, // max_out_of_order_ms new
    Mixed { rows_per_second: f64, max_out_of_order_ms: Option<u64> },     // max_out_of_order_ms new
    Unknown,
}

crates/types/src/physical/properties.rs

/// Where a materialized output is stored.
pub enum StorageMedium {
    Memory,
    Disk,
}

/// $\ell_v$: how long a candidate's output lives. Only a materialized output
/// has a medium.
pub enum Lifetime {
    /// Discarded after its consumer's execution completes.
    #[default]
    None,
    /// Materialized until every consumer within one evaluation finishes.
    OneEvaluation(StorageMedium),
    /// Materialized across evaluations, so later evaluations reuse it.
    AcrossEvaluations(StorageMedium),
}

/// A timing and lifetime that cannot go together (the invalid combination
/// of the concepts doc, section 2).
pub enum PhysicalPropertiesError {
    #[error("an ingestion-time output cannot be materialized for one evaluation: there is no evaluation at ingestion time")]
    OneEvaluationAtIngestion,
}

/// $p^{\mathrm{phys}}_v = (\tau, \ell)$. [`Self::new`] rejects the invalid
/// combination (concepts doc, section 2).
pub struct PhysicalProperties {
    timing: ExecutionTiming,
    lifetime: Lifetime,
}

impl PhysicalProperties {
    /// The properties `(timing, lifetime)`, unless they cannot go together.
    pub fn new(
        timing: ExecutionTiming,
        lifetime: Lifetime,
    ) -> Result<Self, PhysicalPropertiesError>;

    /// $\tau_v$.
    pub fn timing(&self) -> ExecutionTiming;

    /// $\ell_v$.
    pub fn lifetime(&self) -> Lifetime;

    /// Whether $\ell_v \ne$ None. Derived, not stored.
    pub fn is_materialized(&self) -> bool;
}

crates/types/src/stages/physical/memo_candidates.rs

/// Index into [`PhysicalASAPMemoCandidates::edges`].
pub struct PhysicalEdgeId(pub usize);

/// One physical alternative of a logical candidate: what Stage 3 selects.
pub struct PhysicalCandidate {
    /// $e_v$ and $p^{\log}_v$.
    pub logical_candidate_id: CandidateForMemoGroupId,
    /// $p^{\mathrm{phys}}_v$.
    pub physical_properties: PhysicalProperties,
}

impl PhysicalCandidate {
    /// Whether a consumer running in `context` can read this producer (the
    /// concepts doc's alignment table, section 2). True for $\ell$ = None,
    /// and for `OneEvaluation` in the same context. A chain producer with
    /// $\ell$ = `AcrossEvaluations`, with $w$ the chain operator's width in
    /// its expression in `memo`, needs every evaluation time of the context on its
    /// grid: $w$ divides the context's evaluation grid (interval, range step
    /// and phase), each scheduled time, or a one-off query's time; false
    /// when the times are not known (an interval without a phase, an
    /// estimated rate, an unknown recurrence).
    pub fn delivers_on(&self, memo: &Memo, context: &ExecutionContext) -> bool;
}

/// $\alpha^{\mathrm{phys}}_{u,c,i}$: timing and lifetime only (the concepts
/// doc's table, section 2). Alignment is checked per context in Stage 3 with
/// [`PhysicalCandidate::delivers_on`].
pub fn producer_satisfies_physical_requirements(producer: &PhysicalCandidate, consumer: &PhysicalCandidate) -> bool;

/// $(u, c, i)$ over physical candidates.
pub struct PhysicalEdge {
    pub producer_candidate_id: PhysicalCandidateId,
    pub consumer_candidate_id: PhysicalCandidateId,
    pub input_index_of_consumer: usize,
    /// $\alpha^{\log}_{u,c,i}$.
    pub satisfies_logical_requirement: bool,
    /// $\alpha^{\mathrm{phys}}_{u,c,i}$.
    pub satisfies_physical_requirement: bool,
}

/// Stage 2's output, which Stage 3 selects from.
pub struct PhysicalASAPMemoCandidates {
    /// Pass 2's output: the memo, its candidates' $e_v$, $p^{\log}_v$,
    /// $r^{\log}_{c,i}$, and the logical edges.
    pub logical_candidates: LogicalASAPMemoCandidates,
    /// $\mathcal C$, indexed by [`PhysicalCandidateId`].
    pub physical_candidates: Vec<PhysicalCandidate>,
    /// The alternatives Stage 2 pruned, each as the `PhysicalCandidate` it
    /// would have been. // was PrunedCandidate<PhysicalCandidateKey>; query_roots deleted (read logical_candidates.memo.query_roots)
    pub pruned_candidates: Vec<PrunedCandidate<PhysicalCandidate>>,
    /// $\mathcal E$, grouped by (consumer, input); indexed by
    /// [`PhysicalEdgeId`].
    edges: Vec<PhysicalEdge>,
    /// $P_{c,i}$ after $\alpha$: the admissible edges into each (consumer, input).
    admissible_edges_into_input: HashMap<(PhysicalCandidateId, usize), Vec<PhysicalEdgeId>>,
    /// The admissible edges out of each candidate, by [`PhysicalCandidateId`].
    admissible_edges_out_of_candidate: Vec<Vec<PhysicalEdgeId>>,
}

impl PhysicalASAPMemoCandidates {
    /// Stage 2's output over `physical_candidates` and `edges` (grouped by
    /// (consumer, input)), with the admissible-edge indexes built once.
    pub fn new(
        logical_candidates: LogicalASAPMemoCandidates,
        physical_candidates: Vec<PhysicalCandidate>,
        edges: Vec<PhysicalEdge>,
        pruned_candidates: Vec<PrunedCandidate<PhysicalCandidate>>,
    ) -> Self; // 2026-11-07: query_roots dropped, pruned_candidates added

    /// $\mathcal E$: every edge, in (consumer, input) order.
    pub fn edges(&self) -> impl Iterator<Item = (PhysicalEdgeId, PhysicalEdge)> + '_;

    /// $P^{\mathrm{adm}}_{c,i}$: the admissible edges into input
    /// `input_index_of_consumer` of `consumer`
    /// ($\alpha^{\log} = \alpha^{\mathrm{phys}} = 1$).
    pub fn admissible_edges_into(
        &self,
        consumer: PhysicalCandidateId,
        input_index_of_consumer: usize,
    ) -> &[PhysicalEdgeId];

    /// The admissible edges out of `producer`: every consumer that may share it.
    pub fn admissible_edges_out_of(&self, producer: PhysicalCandidateId) -> &[PhysicalEdgeId];
}

/// The output references of `query_roots`: each operator reference, in
/// order (a scalar root's in `map_operator_refs` order), with the index of
/// the query whose root it is.
pub fn output_references<C: Clone>(query_roots: &[QueryRoot<C>]) -> Vec<(usize, C)>;

crates/types/src/stages/realizer.rs

/// Why a selected assignment could not be realized. Only rejected
/// assignments are safe to exclude from the finite search; adapter or
/// identity failures must stop with diagnostics, not become no-good cuts.
pub enum PhysicalCandidateError {
    #[error("selected physical assignment is infeasible: {0}")]
    RejectedAssignment(String),
    #[error("invalid or unsupported physical reconstruction: {0}")]
    InvalidOrUnsupported(String),
}

impl From<String> for PhysicalCandidateError {
    fn from(message: String) -> Self;
}
impl From<&str> for PhysicalCandidateError {
    fn from(message: &str) -> Self;
}

/// A realized selection (was `SelectedPhysicalCandidate`): the physical
/// plan Stage 2 built for it, how long each materialized tumbling-window chain of that plan
/// keeps its tumbling windows, and the operator node each selected node became.
pub struct RealizedPlan {
    /// The selection rebuilt as one physical plan, one operator node per
    /// selected node, each with its candidate's timing and lifetime.
    pub plan: PhysicalWorkloadPlan,
    /// For each tumbling-window chain of `plan` materialized beyond one evaluation
    /// (maintained at ingestion time, or built at query time and materialized
    /// across evaluations), by node id in `plan.dag`: how far back from the
    /// newest evaluation the runtime must keep its completed tumbling windows, in
    /// milliseconds. It is how far its tumbling windows reach back: the offset its
    /// newest tumbling window ends before the evaluation (`[60m] offset 10m`: 10 min),
    /// plus the chain's window, plus the longest range-evaluation span of
    /// its queries, so the earliest step of the next request still finds
    /// its tumbling windows; without an offset or range evaluations, just the window.
    /// Empty when the plan keeps no tumbling windows.
    pub tumbling_window_retention_ms: BTreeMap<PhysicalASAPNodeId, u64>,
    /// The operator node of `plan` each selected node became, indexed by
    /// [`SelectedNodeId`](super::SelectedNodeId).
    pub operator_node_of_selected_node: Vec<Rc<OperatorNode>>,
}

/// Realizes a [`DagOfSelectedCandidates`] through Stage 2, one plan node
/// per selected node. Implemented by the
/// Stage 2 crate, which holds the planning context (the workload's
/// original query roots, its entries and data, and whether query-time
/// results are retained); the facade builds it, passes it to Stage 3, and
/// calls the same method to build the selected plan.
///
/// Guarantees: a returned result has one operator node per selected node, each
/// rebuilt over its selected producers with its identity kept, guarantees
/// recomposed, and every output contract (result kind and schema) met; a
/// realized assignment also passes Stage 2's timing and dependency rules
/// and the physical DAG validator, with every node keeping its
/// candidate's timing and lifetime.
///
/// It checks structure and execution legality only. It does not check cost,
/// and it does not check accuracy targets: Stage 3 does both.
pub trait AssignmentRealizer {
    fn realize(
        &self,
        physical: &PhysicalASAPMemoCandidates,
        selected: &DagOfSelectedCandidates,
    ) -> Result<RealizedPlan, PhysicalCandidateError>;
}

crates/types/src/stages/physical/execution_context.rs (new)

/// A refresh window of the workload's ingestion pipelines, by index.
pub struct RefreshWindowId(pub usize);

/// Index of an [`ExecutionContext`] in the workload's list of distinct
/// contexts. Contexts are deduplicated by equality: [`QueryRecurrence`]
/// holds an `f64` (`EstimatedRate`), so a context cannot be hashed.
pub struct ExecutionContextId(pub usize);

/// $\kappa$: a class of executions with a common owner and schedule, where
/// a candidate with $\ell$ = None or `OneEvaluation` runs.
pub enum ExecutionContext {
    /// The evaluation of every query with this recurrence and range
    /// evaluation, the workload's own description of when it runs: queries
    /// with equal ones are evaluated together and share its `OneEvaluation`
    /// results. A and B both every 10 s at phase 0 give one context; a
    /// one-off query's context is its `OneTime` recurrence.
    Evaluation {
        recurrence: QueryRecurrence,
        range_evaluation: Option<RangeEvaluation>,
    },
    /// $I_r$: the ingestion pipeline of a refresh window.
    IngestionPipeline(RefreshWindowId),
    /// $\kappa_m$: inside an `AcrossEvaluations` candidate's own computation.
    Materialized(PhysicalCandidateId),
}

crates/plan-selection/src/milp/model.rs

/// A query, by index in the workload.
pub struct QueryId(pub usize);

/// The baseline variables (section 3).
struct SelectionVariables {
    /// $x_v$, by [`PhysicalCandidateId`].
    x: Vec<Variable>,
    /// $y_{u,c,i}$, on admissible edges only.
    y: HashMap<PhysicalEdgeId, Variable>,
    /// $z_{q,v}$, for $v \in G_q$.
    z: HashMap<(QueryId, PhysicalCandidateId), Variable>,
    /// $t_v$, by [`PhysicalCandidateId`].
    t: Vec<Variable>,
}

/// The execution-context variables (section 5), which replace $x$ and the
/// unindexed $y$ of the baseline.
struct ExecutionContextVariables {
    /// $x^{\mathrm{mat}}_v$, for an `AcrossEvaluations` candidate.
    x_materialized: HashMap<PhysicalCandidateId, Variable>,
    /// $x_{\kappa,v}$, for a candidate with $\ell$ = None or `OneEvaluation`.
    x_in_execution_context: HashMap<(ExecutionContextId, PhysicalCandidateId), Variable>,
    /// $y_{\kappa,u,c,i}$, indexed by the consumer's context.
    y_in_execution_context: HashMap<(ExecutionContextId, PhysicalEdgeId), Variable>,
    /// $h_v$, for an `AcrossEvaluations` producer.
    retention_amount: HashMap<PhysicalCandidateId, Variable>,
}

/// The accuracy variables of constraint (K) (section 4), all continuous.
struct AccuracyVariables {
    /// $E_c$, by [`PhysicalCandidateId`].
    error_bound: Vec<Variable>,
    /// $F_{c,i}$: the chosen producer's error bound on input $i$ of $c$.
    error_bound_of_input: HashMap<(PhysicalCandidateId, usize), Variable>,
    /// $D_c$, by [`PhysicalCandidateId`].
    failure_probability: Vec<Variable>,
    /// $H_{c,i}$: the chosen producer's failure probability on input $i$ of $c$.
    failure_probability_of_input: HashMap<(PhysicalCandidateId, usize), Variable>,
}

/// (A) $\sum_{u \in P_{c,i}} y_{u,c,i} = x_c$.
fn input_satisfaction_constraints(
    physical: &PhysicalASAPMemoCandidates,
    variables: &SelectionVariables,
) -> Vec<Constraint>;

/// (B) $y_{u,c,i} \le x_u$.
fn producer_selection_constraints(
    physical: &PhysicalASAPMemoCandidates,
    variables: &SelectionVariables,
) -> Vec<Constraint>;

/// (C) $\sum_{v \in G_q} z_{q,v} = 1$; (D) $z_{q,v} \le x_v$.
fn query_coverage_constraints(
    physical: &PhysicalASAPMemoCandidates,
    variables: &SelectionVariables,
) -> Vec<Constraint>;

/// (E) $t_u + 1 \le t_c + |\mathcal C|(1 - y_{u,c,i})$.
fn acyclicity_constraints(
    physical: &PhysicalASAPMemoCandidates,
    variables: &SelectionVariables,
) -> Vec<Constraint>;

/// (F) $x_v \le \sum_{(v,c,i)} y_{v,c,i} + \sum_q z_{q,v}$.
fn utilization_constraints(
    physical: &PhysicalASAPMemoCandidates,
    variables: &SelectionVariables,
) -> Vec<Constraint>;

/// (R) $h_v \ge \operatorname{need}_{\kappa,v,c,i}\, y_{\kappa,v,c,i}$.
///
/// Assumptions (decided 2026-11-06): an edge's need is the time horizon
/// the queries it serves read, a constant per edge (a 5m range over 10 s
/// windows needs 30 windows), so (R) is linear; query latency is ignored
/// until the latency design: a result counts as read when its evaluation
/// starts.
fn retention_constraints(
    physical: &PhysicalASAPMemoCandidates,
    variables: &ExecutionContextVariables,
) -> Vec<Constraint>;

/// (K): $E_c$ and $D_c$ per candidate, big-M on selected edges, and the
/// root checks $E_v \le \varepsilon_q + M_q(1 - z_{q,v})$,
/// $D_v \le \delta_q + (1 - z_{q,v})$.
fn accuracy_constraints(
    physical: &PhysicalASAPMemoCandidates,
    selection: &SelectionVariables,
    accuracy: &AccuracyVariables,
) -> Vec<Constraint>;

crates/logical-asap-optimization/src/pass2/memo/rules.rs

pub(crate) mod finer_aggregate;
pub(crate) mod rollup;
pub(crate) mod scan_labels;
pub(crate) mod summary_inputs;
pub(crate) mod topk;

/// Why a Pass 2 rule failed on a memo.
pub enum Pass2RuleError {
    #[error("memo contains an out-of-range group reference")]
    InvalidGroupReference,
    #[error("memo group capacity exceeds the group identifier range")]
    TooManyGroups,
    #[error("memo dependencies contain a cycle")]
    CyclicDependencies,
}

crates/types/src/ir/schema/mod.rs (excerpt)

/// What a schema field carries (the type stays).
pub enum FieldDataType {
    /// An ordinary, readable value.
    Plain(DataType), // unchanged
    /// Exact, mergeable accumulator state.
    ExactAggregate(ExactKind, ExactParams), // unchanged
    /// Approximate sketch state: its family and grouping layout. θ is a
    /// candidate's `LogicalProperties::parameters`.
    Sketch(SketchAlgorithm, GroupingStrategy), // was Sketch(SketchKind, GroupingStrategy)
    /// A summary sample (e.g. a reservoir).
    SummarySample(SamplingKind, SamplingParams), // unchanged
    /// Wavelet-transform summary state.
    Wavelet(WaveletKind, WaveletParams), // unchanged
    /// Fitted statistical-model summary state.
    StatModel(StatModelKind, StatModelParams), // unchanged
}

crates/types/src/ir/physical_export.rs (excerpt)

pub struct PhysicalASAPDAGNode {
    // id, payload, output_state, lifetime, output_schema, guarantee, retention_ms: unchanged
    /// θ of a summary build: its candidate's `LogicalProperties::parameters`.
    /// `None` on every other node and for an exact build. Filled in PR 6.
    #[serde(default, skip_serializing_if = "Option::is_none")]
    pub parameters: Option<SketchParams>, // new
}
pub const INSTALLABLE_PLAN_WIRE_VERSION: u32 = 13; // was 11 (in crates/planner/src/installable.rs)

crates/types/src/ir/schema/sketch.rs (excerpt)

pub enum SketchParams {
    // Kll, Cms, CountSketch, …: unchanged
    Hydra(HydraParams), // new: Hydra's sizes are θ
}
pub enum GroupingStrategy {
    PerSubpopulationInstance,
    SharedMultiSubpopulation { kind: HydraKind }, // was { kind, params: HydraParams }
}
impl HydraParams {
    pub fn per_subpopulation(&self) -> SketchParams; // new
}

crates/executor/src/summary_kernels/factory.rs and crates/types/src/test_support.rs (excerpts)

pub fn create_planner_accumulator(
    family: &SummaryFamilyType,
    parameters: Option<&SketchParams>, // new: the exported node's θ
    input: &SummaryUpdate,
    grouping: &GroupingStrategy,
) -> Result<Box<dyn AccumulatorUpdater>, String>;

// feature `test-support`: every node at query time, Lifetime::None, parameters None
pub fn physical_asap_dag_at_query_time(root: &Rc<OperatorNode>)
    -> Result<PhysicalASAPDAG, PhysicalASAPDAGValidationError>;
pub fn physical_asap_dag_at_query_time_with_node_ids(roots: &[QueryRoot])
    -> Result<(PhysicalASAPDAG, Vec<Rc<OperatorNode>>), PhysicalASAPDAGValidationError>;

Ambiguities and questions (answered)

The 2026-11-06 answers are applied above: θ out of the operator, Pass 1 returns Memo, export via realize, memo_of_queries, x_materialized, Hydra sizes as θ, θ on the wire, the Pass 1 renames, deleting expand_tumbling_window_chains_in_workload, and tests that build PhysicalASAPDAG directly. The schedule σ is dropped (decided 2026-11-06): Schedule and its error variants are deleted, producer_satisfies_physical_requirements checks timing x lifetime only, and delivers_on is Stage 3's per-context alignment check. Also decided 2026-11-06: the renames agreed in review (#1224): LogicalPropertiesError, PhysicalPropertiesError, LogicalRequirements (parameter_floor_for_required_accuracy), producer_satisfies_logical_requirements, producer_satisfies_physical_requirements, StorageMedium, ExecutionContext/ExecutionContextId (x_in_execution_context, y_in_execution_context), and the fields logical_candidate_id, physical_properties, producer_candidate_id, consumer_candidate_id, logical_candidates, physical_candidates; a sketch build needs θ from its family's list; a requirement's floor may be outside the list; producer_satisfies_physical_requirements takes only the producer and consumer; an evaluation context is the workload's recurrence and range evaluation (ExecutionContext::Evaluation { recurrence, range_evaluation }), one-off queries included. No open questions; the items marked "pending" in the table above await approval.

Moved from the PR body (2026-11-07, body size limit)

crates/types/src/ir/operator/input.rs (new)

/// An input of an operator, as the rules read it.
pub trait OperatorInput: Sized + Clone {
    fn schema(&self) -> &Schema;
    fn result_kind(&self) -> OperatorResultKind;
    /// The input's own operator, its children as inputs again.
    fn operator(&self) -> Operator<Self>;
    /// What the input covers when it is an aggregate group in the memo;
    /// `None` for any other input. // renamed 2026-11-07
    fn aggregate_coverage(&self) -> Option<&SummaryCoverage>;
}

/// A front-end node tree. Node trees carry no summary coverage.
impl OperatorInput for Rc<OperatorNode> { … }

/// An input of a rule: a real input `I`, or a build the rule constructs
/// only while it derives.
pub(crate) enum InputOrTemporaryBuild<I> {
    Input(I),
    TemporaryBuild(Rc<TemporaryBuild<I>>),
}

/// A build a rule constructs only while it derives (the build a rollup stands for, the build a selection of
/// states stands for): an operator over `I`s with its derived schema and
/// kind. Never stored in the memo. A rule never builds one over another
/// rule's temporary build, so the type does not nest.
pub(crate) struct TemporaryBuild<I> {
    operator: Operator<InputOrTemporaryBuild<I>>,
    schema: Schema,
    result_kind: OperatorResultKind,
}
impl<I: OperatorInput> OperatorInput for InputOrTemporaryBuild<I> { … }

crates/types/src/ir/operator/node.rs (excerpt)

/// The derivation rules, one set for every kind of input. // was impl Operator (node trees only)
impl<I: OperatorInput> Operator<I> {
    pub(crate) fn output_schema(&self) -> Result<Schema, SchemaDerivationError>;
    pub(crate) fn output_kind(&self) -> OperatorResultKind;
    pub(crate) fn validate_inputs(&self) -> Result<(), SchemaDerivationError>;
    pub(crate) fn is_shareable_when_equal(&self) -> bool; // new: an ASAP operator, or NonASAPOp::is_shareable_when_equal
}
// The same for NonASAPOp<I>, ASAPOp<I> and ScalarExpr<C>::scalar_type.

crates/types/src/ir/properties/summary_coverage.rs (excerpt)

/// What a summary state computes, with every lifted selection removed. // new
pub struct SummaryStateDefinition {
    /// The group the unlifted operators read: the first input under the
    /// SummaryAgg that is not Filter / range TimeRange / TimeShift without @
    /// / direct-column Project. `None` when that input is a `Scan` with a
    /// liftable predicate (a PromQL label matcher): it is lifted too, and the
    /// `Scan` with the rest of its predicates is the last unlifted operator,
    /// so states over one source with different matchers may merge.
    pub input_group: Option<MemoGroupId>,
    /// The operators not lifted into the selection, listed top down from the
    /// SummaryAgg, each with only its residual arguments. `None` marks an
    /// operator's main input (the next unlifted operator, or input_group);
    /// `Some(g)` is another input group it reads, such as the group a scalar
    /// subquery in a Filter reads. Two states may merge only if input_group
    /// and these operators are equal.
    pub unlifted_operators: Vec<Operator<Option<MemoGroupId>>>,
}

pub struct SummaryCoverage {
    pub(crate) definition: SummaryStateDefinition, // was Rc<OperatorNode>
    /// Union of boxes over the output rows of the unlifted operators' input.
    pub(crate) selection: Vec<SelectionBox>,
}

impl SummaryCoverage {
    /// The coverage of a summary state a memo group computes with `operator`
    /// over its input groups. // was derive(node), derive_cached(node, cache)
    pub(crate) fn derive_from_inputs<'m>(
        operator: &Operator<MemoGroupAsInput<'m>>,
    ) -> Result<Self, CoverageError>;
    /// 128-bit FNV-1a of the definition; equal definitions of one memo get
    /// equal ids.
    pub(crate) fn definition_id(&self) -> String;
}

/// A conjunction of per-column constraints. The timestamp column is one of
/// the columns; there is no separate time field. // 2026-11-07: relative_time deleted
pub struct SelectionBox {
    pub(crate) columns: BTreeMap<ColumnIdentity, Constraint>,
}

pub enum Constraint {
    In(Vec<ScalarValue>),
    NotIn(Vec<ScalarValue>),
    /// `m[5m]`: (Excluded(EvaluationTimePlusMs(-300_000)), Included(EvaluationTimePlusMs(0))]; // 2026-11-08: (t − r, t]
    /// `price > 10`: lower Excluded(FixedValue(Float64(10.0))), upper Unbounded. // 2026-11-07
    Interval { lower: Bound<IntervalConstraintLimit>, upper: Bound<IntervalConstraintLimit> },
}

/// An end of an interval; limits compare only within one variant.
/// // new 2026-11-07, replaces TimeBound
pub enum IntervalConstraintLimit {
    FixedValue(ScalarValue),      // `@ T`, SQL timestamp literals: Timestamp(ms since the epoch)
    EvaluationTimePlusMs(i64),    // relative to the evaluation time t
}

pub enum CoverageError {
    // … unchanged variants; TumblingWindowChainExpansion deleted 2026-11-07
}

Major algorithm change: summary coverage over memo groups (2026-11-06)

Summary coverage over memo groups.

  • Problem. Coverage was derived from a node tree, so a memo group needed a rebuilt tree to know what its state covers, and a merge check walked the whole input tree.
  • Before. SummaryCoverage::derive(node) walked down from the SummaryAgg node, rebuilt the residual nodes into a definition: Rc<OperatorNode>, and cached coverage per node address (NodeToSummaryCoverageCache); OperatorNode::new checked every SummaryMerge.
  • After. derive_from_inputs(operator over MemoGroupAsInputs) runs the same walk over the input groups' expressions (OperatorInput::operator()), the same lifting rules (residual, constraint_of, column_at_top, time windows), and records the definition as input_group plus the unlifted operators (Operator<Option<MemoGroupId>>: None for each operator's main input, Some(g) for a group a scalar subquery reads). A merge reads its inputs' stored coverage and compares definitions by value. A chain, a merge of newest windows, a rollup and a selection of states use the same builds as before, as InputOrTemporaryBuild::TemporaryBuilds.
  • Correctness. Equal definitions read one group through equal operators, so they compute the same thing; the selection rules are unchanged. Kept behaviors: Scan predicates (PromQL matchers) are lifted, so matcher states still merge. PR 2's tests check the doc's examples.
  • Complexity. The walk visits only the liftable operators below the SummaryAgg; a merge compares n definitions in O(n · |unlifted_operators|) instead of walking node trees.

Other algorithms are deleted (see "Deleted"); later PRs re-implement them behind these interfaces.

Deleted tests, 2026-11-06 detail

Deleted with node-tree coverage, the node-tree merged_window_build and size_for_all_readers (decided 2026-11-06), 48 more:

File Tests Restored by
types/tests/summary_coverage.rs (file deleted) 32: coverage derived on node trees (lifting, merges, time windows, rollups, definition ids) PR 2's types/tests/memo_summary_coverage.rs (8) over memo groups; the rest as later PRs need them
integration-tests/tests/summary_coverage_e2e.rs (file deleted) 8: query string → planned state → coverage() no replacement planned: it read node-tree coverage of front-end trees
types/tests/summary_merge_structure.rs idempotent_families_merge_overlapping_states (node merge check) PR 2's states_with_overlapping_selections_do_not_merge
types/tests/tumbling_window_chain.rs a_rollup_of_the_newest_tumbling_windows_covers_their_window; the coverage asserts of 5 more PR 2's a_tumbling_window_chain_covers_its_whole_window
types/tests/tumbling_window_chain.rs a_merge_of_the_newest_tumbling_windows_covers_their_window (read the build through the node-tree merged_window_build) PR 4, with the merged-windows guarantee over groups
devtools/src/bin/asap_planner.rs memo_node_table_stores_a_shared_node_once (with MemoNodes::add) PR 9
accuracy/src/estimators/mod.rs, mod shared_size_tests 4: size_for_all_readers no replacement: deleted with the function

The coverage asserts in integration-tests/tests/sketch_rollup.rs (2 tests) and logical-asap-optimization's per_computation_candidates_tests.rs (1 test) are removed; the tests stay.

Kept by building the DAG directly (decided 2026-11-06). Tests that hand-built operator roots and ran them now export them with the test-only physical_asap_dag_at_query_time(_with_node_ids) instead of the deleted roots-based export. Restored: executor time_series_window, node_ids, current_series_heap, planspace_series_identity_heap, 3 of deployment_computation's tests; integration filtered_aggregates, kll_tumbling_window_execution, per_series_exact_execution, sql_cardinality, sql_frequency, exact_composition, and parts of sketch_rollup, stage1_logical_asap_optimization, operator_design_examples, operator_sharing, promql_numeric_regressions, sql_to_post_asap, promql_to_post_asap; types physical_export; frontend count_planning, pearson_corr. types/tests/physical_export.rs::untimed_plan_is_rejected (the +1 above, deleted after the comment was posted) is deleted: the builder always assigns query time, and an untimed export is no longer possible. Restored sketch tests fail at the executor's todo!() until PR 6, because the builder leaves parameters: None. Tests whose DAG came from the planner (runtime_install's planning tests, sql_per_group_topk, sql_table_scans, per_sample_updater, the planner-built deployment_computation tests, and others) stay deleted; PR 6 restores them.

Moved from the PR body (2026-11-08, body size limit)

2026-11-07 round, 10 more:

File Tests Restored by
types/tests/tumbling_window_chain.rs 6: merged_tumbling_windows_cover_the_whole_window, each_tumbling_window_is_one_shifted_minute, expansion_merges_the_tumbling_windows, a_merge_of_the_newest_tumbling_windows_reads_one_chain_and_fewer_than_all_tumbling_windows, newest_tumbling_windows_serializes_only_when_set, expansion_keeps_a_rollup_of_the_tumbling_windows (window count, newest windows, expansion) PR 1 (TimeRange over a chain), PR 6 (executor by window index)
types/src/ir/properties/summary_coverage.rs 3: large_reverse_ordered_tumbling_window_union_is_one_exact_interval, interval_fast_path_matches_general_union_for_permutations, sorted_union_preserves_gaps_boundaries_and_original_group_order (relative_time union) PR 2 (timestamp-column intervals)
planner/tests/runtime_install.rs tumbling_windows_of_an_hour_share_one_definition_with_distinct_times (expanded windows' relative_time) and its now-unused helpers plan_json, plan_json_for PR 6

@zzylol zzylol mentioned this pull request Nov 6, 2026
Merged
@zzylol
zzylol force-pushed the refactor/1207-interfaces branch from 633460c to 22caa51 Compare November 7, 2026 17:16
This was referenced Nov 10, 2026
Open
Merged
zzylol added 20 commits November 10, 2026 13:17
@zzylol
refactor: define every stage's interfaces from #1207 and delete the o…
1df3d47 
…nes they replace (#1207 PR 0)
@zzylol
refactor(types): CandidateForMemoGroupId.index -> index_in_group
99245c1
@zzylol
refactor(types): drop the physical schedule σ; PhysicalProperties { t…
940c88b 
…iming, lifetime }

Deletes Schedule and the σ-only InvalidPhysicalProperties variants; the only
invalid combination left is (IngestionTime, OneEvaluation). satisfies_physical
checks timing x lifetime only; delivers_on is the per-context alignment check
(w | I and w | φ). A OneEvaluation result runs in one evaluation context, so
x_{κ,v} covers it as for ℓ = None.
@zzylol
refactor(types): theta must be listed, floors need not be; satisfies_…
cb6e9e1 
…physical drops memo

InvalidLogicalProperties gains SketchBuildWithoutParameters and renames
FloorNotInParameterList to ParametersNotInList, returned by
LogicalProperties::new. LogicalRequirement::new no longer checks list
membership (a floor is any value of the family's parameter type, e.g.
heap_size >= k), so it returns Self. satisfies_physical takes only the
producer and consumer.
@zzylol
refactor(plan-selection): one evaluation context per schedule
ed44fc2 
Context::QueryEvaluation(QueryId) becomes Context::Evaluation(EvaluationSchedule):
queries with the same (interval, phase) are evaluated together and share
their OneEvaluation results. evaluation_schedule returns that schedule.
QueryId stays for z_{q,v}. A non-periodic query has no context yet.
@zzylol
refactor(types): rename the #1207 interfaces as agreed (2026-11-06)
9416f44 
InvalidLogicalProperties -> LogicalPropertiesError, InvalidPhysicalProperties
-> PhysicalPropertiesError, LogicalRequirement -> LogicalRequirements
(parameter_floor -> parameter_floor_for_required_accuracy), satisfies_logical
/ satisfies_physical -> producer_satisfies_logical_requirements /
producer_satisfies_physical_requirements, Medium -> StorageMedium,
Context / ContextId -> ExecutionContext / ExecutionContextId
(x_in_context / y_in_context -> x_in_execution_context /
y_in_execution_context). PhysicalCandidate and PhysicalCandidateKey fields
-> logical_candidate_id, physical_properties; edge fields ->
producer_candidate_id, consumer_candidate_id; PhysicalASAPMemoCandidates
fields -> logical_candidates, physical_candidates.
@zzylol
refactor(types): execution contexts reuse the workload's recurrence
b05cec1 
Deletes EvaluationSchedule and ExecutionContext::evaluation_schedule. An
evaluation context is Evaluation { recurrence, range_evaluation }: queries
with equal ones share it and its OneEvaluation results, one-off queries
included. ExecutionContext, ExecutionContextId and RefreshWindowId move to
asap_types::stages so PhysicalCandidate::delivers_on can take the context.
Contexts are deduplicated by equality (QueryRecurrence holds an f64).
@zzylol
refactor: naming sweep for the #1207 unit
868d2f8 
PhysicalASAPDAGValidationError::InvalidPhysicalProperties becomes
PhysicalProperties { node, reason }. In the files this unit edits, "kept"
describing a materialized result or retained data becomes "materialized"
or "retained" (cost, materialized views, realizer, installable plan,
deployment capabilities; locals kept/raw_kept_rows -> retained/
raw_retained_rows), and "reader" becomes "query" where it is a query
(execution demand, ScheduleGroup::queries, cost usage, labels, tests) or
"consumer" where it is a consuming candidate.
@zzylol
refactor(types): Memo methods return Result; CandidateError (#1207 PR 0)
f4c0397 
find_or_add_group returns the node rules' SchemaDerivationError and
add_candidate a CandidateError (the node rules, or an output that differs
from the group's). Bodies stay todo!() until PR 2.
@zzylol
refactor: Memo groups private behind groups(); Stage 0 through the Me…
98f4eb2 
…mo (#1207 PR 0)

Memo::groups is private, read through groups(); groups are added only
through find_or_add_group and queries through add_query (todo!() until
PR 2). memo_of_queries returns Result and its body is todo!() until PR 2,
which builds Stage 0 on find_or_add_group; Stage 0's node-level CSE (cse.rs),
its test_support export and the dead share_identical helper are deleted.
@zzylol
refactor(types): a memo group is its expression and GroupProperties; …
f104f6f 
…OperatorInput; coverage only in the memo (#1207 PR 0)

MemoGroup { expression, group_properties: GroupProperties, parents, queries,
candidates } replaces logical_node and child_groups; GroupProperties {
schema, result_kind, summary_coverage,
shareable_when_equal } is derived once by derive_group_properties, which
applies the existing rules to the expression over MemoGroupAsInputs.

The rules (output_schema, output_kind, validate_inputs,
is_shareable_when_equal, summary builds, window chains, scalar subqueries,
top-k, populations) are generic over OperatorInput, implemented for
Rc<OperatorNode> and MemoGroupAsInput; only how they read an input changed.
BuiltOrInput carries the builds a rule makes while deriving (one build over
a chain's window, the build a rollup or a selection stands for).

SummaryCoverage { definition: SummaryStateDefinition { input_group,
unlifted_operators }, selection } is derived only from memo groups
(derive_from_inputs) and stored in GroupProperties; node-tree coverage,
NodeToSummaryCoverageCache, derive_cached and OperatorNode::coverage are
deleted, so node trees have no merge check. Memo::logical_node_trees is
deleted: Pass 1 target discovery (PR 3), devtools group labels (PR 9) and
the export's view coverage (PR 6) are todo!(). find_or_add_group and
add_candidate derive the properties, then todo!() until PR 2.
@zzylol
refactor(accuracy): delete size_for_all_readers and its shared-size h…
0fe71dd 
…elpers (#1207 PR 0)

One state's θ is no longer sized for several consumers: θ comes from
parameter_list per candidate. size_for_all_readers had no non-test caller;
min_recorded, kll::shared_k and univmon::shared_heap served only it.
@zzylol
refactor(types): InputOrTemporaryBuild; subquery groups in unlifted o…
2a04c9f 
…perators; delete merged_window_build_node (#1207 PR 0)

- BuiltOrInput / BuiltInput renamed InputOrTemporaryBuild / TemporaryBuild:
  a build a rule constructs only while it derives, never stored in the memo.
- SummaryStateDefinition::unlifted_operators is Vec<Operator<Option<MemoGroupId>>>:
  None marks an operator's main input, Some(g) another input group it reads
  (a scalar subquery's). States whose definitions read subqueries compare and
  merge; CoverageError::SubqueryInDefinition deleted.
- merged_window_build_node deleted with its test; the accuracy guarantee of a
  read of merged tumbling windows is todo!() until PR 4 rewrites it over groups.
@zzylol
refactor(types): 2026-11-07 interfaces: chains without a window count…
ac7273d 
…, TimeBound, Pass 2 pruned list, Stage 2 entry (#1207 PR 0)

- Delete WindowLayout, LogicalProperties::window_layout and its two error
  variants, SummaryMerge::newest_tumbling_windows and
  SummaryTumblingWindowChain::tumbling_window_count, with the count-based
  chain builds and expansion; behaviour that read the count is todo!() for
  PR 1, 2, 6, 7 or 9.
- SelectionBox { columns } only, and TimeBound { At, FromEvaluationTime };
  lifting TimeRange/TimeShift is todo!() for PR 2.
- PrunedLogicalCandidate, PruneCause, RootPruneReason and
  LogicalASAPMemoCandidates::pruned.
- error_propagation(consumer, metric) -> Option<ErrorPropagation>;
  Memo::set_input_requirements.
- PhysicalCandidateKey deleted (PrunedCandidate<PhysicalCandidate>),
  PrunedCandidate::group and PhysicalASAPMemoCandidates::query_roots
  dropped; new(logical, physical, edges, pruned).
- Stage 2 entry takes &DeploymentCapabilities (moved into asap-types) and
  cannot fail; PruneReason follows the doc's Stage 2 table.
- DataArrival::{Ingesting, Mixed} gain max_out_of_order_ms.
- GroupProperties::summary_coverage -> aggregate_coverage.
@zzylol
refactor(types): IntervalConstraintLimit ends for Constraint::Interva…
8a98866 
…l; ScalarValue::Timestamp (#1207 PR 0)

Replace TimeBound by IntervalConstraintLimit { FixedValue(ScalarValue),
EvaluationTimePlusMs(i64) }: Constraint::Interval holds
Bound<IntervalConstraintLimit> on every column, so the timestamp column
and value columns share one interval type. Doc comments carry the
examples (m[5m], offset 1h, @ T, SQL ts >= literal, price > 10).
ScalarValue::Timestamp(ms since the epoch) is the fixed value of an
@ T or a SQL timestamp literal.
@zzylol
refactor(types): PromQL ranges are (t - r, t] in the interval example…
31f25bb 
…s; delete ParametersOfAnotherFamily (#1207 PR 0)

A range m[r] evaluated at t reads (t - r, t]: IntervalConstraintLimit and
Constraint::Interval examples, the chain window doc and the primitive-schema
table now use Excluded lower and Included upper ends. A parameter list holds
only its own family's parameters, so ParametersNotInList also rejects another
family's parameters.
@zzylol
refactor(types): 2026-11-08 interfaces: summary types, required accur…
0d992a1 
…acy, the plan is one PhysicalASAPDAG (#1207 PR 0)

- Terminology: ops' `family` is `summary_type`; FieldDataType::Sketch(SketchAlgorithm)
  (SketchKind, GroupingStrategy, HydraKind and the ops' `grouping` deleted);
  FamilyCapability is SummaryTypeCapability; parameter_list(algorithm).
- θ is structure only: SketchParams derives equality; KLL failure_probability,
  UnivMon distinct_failure_probability and HLL/KMV relative_error deleted.
- ValueSign and GroupProperties::column_sign; SourceFacts::column_signs
  replaces values_non_negative.
- LogicalRequirements { required_accuracy: Option<RequiredAccuracy> };
  RootPruneReason::BelowRequiredAccuracy; Memo::add_query takes the target
  and n_q; AggIntent measures carry no accuracy and front ends take none.
- asap-accuracy: AccuracyEvidence::measured_guarantee, theoretical_guarantee
  (was local_guarantee), producer_satisfies_logical_requirements(memo,
  producer, consumer, input_index, evidence); the AccuracyModel trait and
  evidence provider, Pass 1's accuracy pruning and AccuracyCheckInputs deleted.
- Physical: ExecutionContext::Evaluation { grid: EvaluationGrid },
  IngestionPipeline(RefreshWindow); RefreshWindowId deleted;
  producer_satisfies_physical_requirements(producer, consumer_timing);
  PruneReason::CountMinWeightsNotProvenNonNegative.
- Plan: PlanOutput { plan: PhysicalASAPDAG }; PhysicalASAPDAGNode::
  selected_physical_candidate; select_plan returns (PhysicalASAPDAG,
  Selection); Selection { accepted_accuracy_per_query, pruned_candidates,
  milp_* }; RealizedPlan, PhysicalWorkloadPlan, DagOfSelectedCandidates,
  SelectedEdge, SelectedNodeId, AssignmentRealizer, ScanWindow and the
  scan/view window functions, SearchReport, SearchStatus, InexactObjective,
  ObjectiveDiffersFromPlanCost, SelectedPlanFailedCheck and
  PlanCost::cost_unit deleted.
- Plan file wire version 14 (summary_type, Sketch(algorithm), no grouping).
- Cost names: NodeCostSource, cost_source, PlanRanking::Cost; Serialize on
  PruneReason, PhysicalProperties, OriginOfCandidateForMemoGroup and
  RootPruneReason.
@zzylol
refactor(types): guarantees at a failure probability, AccuracyEvidenc…
d82c149 
…e trait, MemoQuery, evaluation grids of groups (#1207 PR 0)

2026-11-09 decisions:
- theoretical_guarantee and AccuracyEvidence::measured_guarantee take the
  failure probability and return the error bound that holds at it (None
  when θ cannot fail that rarely); bodies stay todo!() for PR 4.
- AccuracyEvidence is a trait; NoMeasuredGuarantees has no measurement.
  PlanningModels::evidence and Pass 2 take &dyn AccuracyEvidence.
- The memo holds its queries in one Vec<MemoQuery> { root,
  accuracy_target, values_per_evaluation }; add_query(query).
- SupportedSummary::family is summary_type.
- satisfies is deleted with metric_meets, scope_meets, native_target and
  AccuracyScope::covers, which only it used; target_limit is the
  comparison limit with the tolerance.
- PhysicalCandidate::delivers_on(&PhysicalASAPMemoCandidates, context).
- GroupProperties::depends_on_evaluation_time (derived in PR 2) and
  LogicalASAPMemoCandidates::evaluation_grid_of_group (filled in PR 4).
@zzylol
refactor: delete Pass 1's test-support whole-workload composition and…
5587607 
… the tests that use it (#1207 PR 0)

pass1/composition.rs composed one whole-workload candidate DAG from one
per-computation candidate per target, for tests only; planning adds every
way to compute to the memo instead. It and the 74 tests and fixture helpers
that compose through it are deleted (no interim code for PR 3).
@zzylol
test(sql): lower main's new ClickHouse tests without the removed accu…
61541f7 
…racy argument (#1207 PR 0)
zzylol
zzylol commented Nov 10, 2026
View reviewed changes
Comment thread crates/types/src/stages/candidates.rs Resolved
@zzylol
zzylol force-pushed the refactor/1207-interfaces branch from 2ed6ec0 to 61541f7 Compare November 10, 2026 13:37
zzylol
zzylol commented Nov 10, 2026
View reviewed changes
Comment thread crates/types/src/stages/candidates.rs Resolved
zzylol
zzylol commented Nov 10, 2026
View reviewed changes
Comment thread crates/types/src/stages/candidates.rs Resolved
zzylol
zzylol commented Nov 10, 2026
View reviewed changes
Comment thread crates/frontend-sql/tests/clickbench/clickbench.rs Resolved
@zzylol
refactor: organize the stage types by scope; name the memo query and …
fe1473c 
…requirement setters by what they hold (#1207 PR 0)

- asap_types::stages: logical/ (memo/, group, candidate, requirements,
  edge, memo_candidates) and physical/ (execution_context, candidate,
  edge, memo_candidates); pruning.rs holds every stage's removed
  candidates. Public asap_types::stages paths are unchanged.
- parameter_list moves to asap_types::ir::schema (sketch_parameter_lists.rs).
- Error propagation moves to asap-accuracy (error_propagation.rs) as the
  free function error_propagation.
- MemoQuery::values_per_evaluation -> number_of_values_per_query_evaluation;
  Memo::set_input_requirements -> set_candidate_input_requirements.
- Pass 1: its entry moves into pass1.rs; per_computation_candidates.rs ->
  computation_targets.rs. Pass 2: its entry moves into pass2.rs.
@zzylol
refactor: organize asap-types, asap-accuracy and asap-cost by scope (#…
7030d68 
…1207 PR 0)

asap-types:
- summary_type/: parameters.rs, parameter_lists.rs, statistic.rs and
  capability.rs (were ir/schema/sketch.rs, sketch_parameter_lists.rs and
  summary_type_capability.rs).
- accuracy/: target.rs and guarantee.rs (were ir/properties/accuracy.rs
  and guarantee.rs).
- plan/: physical_asap_dag.rs and materialized_view.rs (were
  ir/physical_export.rs and ir/materialized_view.rs).
- stages/physical/properties.rs (was physical/properties.rs; the physical
  module is deleted).
- ir/operator/arguments.rs (was operator_properties.rs);
  ir/schema/series_identity.rs (was schema_support.rs).
- workload/mod.rs split into query.rs, data.rs and planning.rs; mod.rs
  re-exports them.

asap-accuracy:
- guarantee/ (was estimators/): each summary type's theoretical guarantee.
- evidence.rs: AccuracyEvidence, NoMeasuredGuarantees (was model.rs).
- requirements.rs: producer_satisfies_logical_requirements (was model.rs),
  target_limit and meets (were lib.rs).
- The old evidence.rs (PropagationStats) is renamed propagation_stats.rs
  so the new evidence.rs does not collide with it.
- lib.rs holds module declarations and re-exports only; the modules that
  glob-imported its imports now import what they use.

asap-cost:
- model.rs: the cost models and the deployment's cost inputs (were lib.rs).
- resources/mod.rs: ResourceUsage (was lib.rs) with the resource
  dimensions (was resources.rs).
- node/: usage.rs (one node's usage and cost: node_usage, NodeCost,
  ProducerOutput, node_map; was usage.rs and lib.rs), analytical.rs,
  state_bytes.rs and operator_statistics.rs.
- plan/: cost.rs (plan_cost, PlanCost, QueryLatency), retention.rs (the
  tumbling windows a plan retains; was usage.rs), execution_demand.rs and
  ranking.rs.
- lib.rs holds module declarations and re-exports only. Users of
  asap_cost::analytical, ::state_bytes and ::execution_demand use their
  new paths.

Every user's path is updated, docs included; no alias for an old path is
kept.
@zzylol
zzylol merged commit 828a2aa into main Nov 10, 2026
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@milindsrivastava1997 milindsrivastava1997 milindsrivastava1997 left review comments

Assignees

@zzylol zzylol

Labels

CSE optimizer Workload optimization and plan selection refactor

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@zzylol @milindsrivastava1997