Skip to content

refactor(stage3): constraint (K) per execution and metric; sign-conditioned DDSketch guarantee, classic HLL certificate from the source's distinct bound (#1207 PR 8) - #1238

Merged
zzylol merged 4 commits into
mainfrom
refactor/1207-accuracy
Nov 16, 2026
Merged

zzylol merged 4 commits into
mainfrom
refactor/1207-accuracy

Conversation

@zzylol

@zzylol zzylol commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Problem: Stage 3's constraint (K) is todo!() on main: no plan checks a query's composed accuracy at its root. B = p99 / p50 at relative 0.03 may read two readouts each required at 0.02 (each meets 0.03 alone; composed 1.111 × 0.04 = 0.044). DDSketch's quantile guarantee is also taken over values of any sign, and an HLL distinct count never serves a query with a target, even when the source declares its distinct bound.
  • Change: (K) per execution and per error metric: $E_{\kappa,c,m}$ and $D_{\kappa,c}$ composed along the selected edges through each candidate's error contract (validity caps and the metric rule included), by one composition rule (AccuracyRules::compose), and checked at each query output in the metrics its root allows, with the target's δ per output value. DDSketch's quantile guarantee needs values of one sign (guarantee/ddsketch.rs); classic HLL's distinct count is certified over the scanned sources' max_distinct_per_evaluation. The report's accepted metric is the readout's requirement's.
  • Effect: on the two-query example (A: p95 at rank 0.01; B: p99/p50 at relative 0.03, 1,000 non-negative series) the plan reads Kll { k: 269 } for A and DDSketch { alpha: 0.01 } for B (composed 0.022); with accuracy ignored one KLL (k = 547) answers all three readouts and the report says "no accuracy target (Kll)". On p90, p50 and p90 / p50 sharing readouts at relative 0.05, main reads α = 0.05 everywhere (the ratio composes 0.111); this PR adds α ≤ 0.02 readouts for the ratio.

Review guide

Round 1 fixes

Review round 1 (abstraction and layering, plus the functional/efficiency review) of 55f0ef67d, fixed in b8d6ffb04 (f3e7aab15 after the rebase onto 79d99326e):

Id Finding Fix
A1 (K)'s composition written three times AccuracyRules::compose (own error + Σ β·input error, metric rule, validity cap, β = 0 skipped, D = δ + Σ H); selection_meets_targets calls it with the selected producer, the ceilings with the largest; accuracy_constraints writes the same rule as rows
A2 ceiling fixpoint over executions one pass in memo.topological_order() (AccuracyRules::executions_producers_first); a debug_assert that no selectable edge stays in one group
A3 report re-derived the readout metric CandidateForMemoGroup::required_accuracy(), read by (K) and accepted_accuracy; readout_error_metric is pub(crate) again
A4 sign lookups duplicated Stage 2 reads state_column_sign(); ValueSign::is_one_sign used by the error contracts and DDSketch's guarantee
A5 METRIC_COUNT, metric_index copied AccuracyMetric::COUNT, AccuracyMetric::index
A6 unreachable "1 without δ" failure_probability_per_output_value is None without a δ; the .and is gone
A7 own error recomputed three times AccuracyRules::own_error_per_execution
A8 ErrorModel with Option<&AccuracyRules> solve_selection builds the model only with rules; AccuracyVariables::is_empty deleted
A9 theoretical_guarantee ignored the sign the sign is folded in: it takes state_sign and returns None for a DDSketch quantile over values of unknown sign
L1 copy key compared proxies; query_roots_with_target called twice copies_in_equivalent_contexts(inputs, kept, rules) keys each output on its RootLimits (hashable by bits): exactly what the root rows read; queries differing only in $n_q$ without a whole-result δ now share (test contexts_whose_queries_differ_only_in_values_not_split_by_k_are_copies, failing before)
L2 two δ rules failure_probability_per_output_value; readout_caps::query_levels is min(per_output_value, δ / R), bit-identical to main's δ / max(V, R); a comment at the H rows on fan-in (see "With #1360")
L3 tolerance 1e-10 unprincipled tried row scaling with default tolerances: exact but RangeHeavy-8u took 99 s against 26 s; kept the derived tolerance min(1e-7, 1e-3 · smallest positive limit), at least 1e-10 (HiGHS's minimum), set only when below 1e-7: 1e-9 on RangeHeavy-8u, 13.5 s against the port's 12.3 s measured the same way (round-2 review); replaced in round 2 (see "Round 2 fixes"); test k_bounds_a_per_value_failure_probability_below_the_default_solver_tolerance fails without it; comment on #1356
L4 sign check outside the guarantee module in guarantee/ddsketch.rs (unknown failure probability over values of unknown sign), applied inside guarantee_before_failure_probability_check; cost-model follow-up commented on #1287 (already open)
L5 two folds over declared facts workload::bounds::declared_max_distinct_per_evaluation beside declared_value_magnitude_range, with why their lookup rules differ; Memo::max_distinct_per_evaluation delegates
L6 comments cited a missing doc accuracy-models.md gains "Sign conditions" and "Checking each query's target in selection: constraint (K)" (variables, rows, constant errors, ceilings, tolerance, copies, example); comments point there; PR numbers removed from new comments
L7 names reached_by_query_with_target, selectable_edges_per_input, error_is_constant, ErrorTerm { Constant, Variable { variable, ceiling } }, RootLimits::{tolerated_error_per_metric, tolerated_failure_probability_per_output_value}, roots keyed by QueryId
L8 accepted_metric is the first readout's documented: (K) bounds each metric apart, so with a target naming no metric this is the first readout's
B1, B2 Design scope incomplete; title and body drifted this body: title names the HLL certificate, base and commits in "Review guide", complete Design scope
Functional no workload where (K) decides the optimum regression test a_ratio_sharing_its_readouts_with_its_operands_still_meets_its_target (the reviewer's k2): fails with (K) disabled ([0.05, 0.05, 0.05])

Round 2 fixes

Review round 2 (verification of b8d6ffb04), fixed in 593ef4163:

Id Finding Fix
F1 the derived tolerance's 1e-10 floor let (K) pass a plan missing a per-value δ below 1e-7 (δ = 0.01 over 1e10 values: two readouts at 5.4e-13 selected against 1e-12), with no error after the solve, AccuracyRules::selection_meets_targets checks the selection exactly in f64; a miss is ErrorSelectingPlan::AccuracyTargetBelowSolverPrecision { query, failure_probability_per_output_value }, never a plan. selection_meets_targets returns Result<(), QueryId> to name the query. Test a_per_value_failure_probability_below_the_solver_precision_is_an_error (1e-10 and 1e-12 per value) fails without the check: the solver selects (2, 0)
F2 the derived tolerance alone made RollupHeavy-8u 2.8× slower than the port primal and MIP feasibility tolerance 1e-10 whenever (K) has rows, as in the port; feasibility_tolerance and smallest_positive_limit deleted. RollupHeavy-8u: 6.7 s against the round-1 rule's 11.5 s and the port's 11.5 s (see Verification). RangeHeavy-8u got slower instead: see "Decisions and open questions"
F3 #1360's body lost its "With #1238" section; open question 2 stale #1360's body has the section; its open question 2 and Design scope row now follow it
F4 wrong L3 timings; wrong cause for h01q/h03n L3 row corrected; h01q/h03n below
#1239 note RootPruneReason::NoGuarantee doc missed two cases it names a DDSketch quantile over values of unknown sign and a sketch readout under an exact target

h01q / h03n: both have only limits of 0.05 or more, so under the round-1 rule HiGHS ran at its defaults. Their 1.4e-9 and 1.3e-11 gaps are main's behaviour, not (K): the round-2 review made both optimal at tolerance 1e-9 or 1e-10, which this head uses (#1356, likely the MIP feasibility tolerance as the margin for pruning against the best plan found). On 79d99326e, this head and the round-1 rule select the same plan for both (see Verification).

Round 3 fixes

Reviewer decision on the solver tolerance (option (c), see "Decisions and open questions"), in 9a67a5560:

Id Change Test
T1 solve_selection solves at HiGHS's default tolerances; the selection is checked exactly (selection_meets_targets); a selection that misses a target is solved once more, from the same HiGHS model, at primal and MIP feasibility tolerance LEAST_HIGHS_FEASIBILITY_TOLERANCE (1e-10) and checked again; a miss after that is AccuracyTargetBelowSolverPrecision, as before. The retry is a loop inside solve_selection, no new function or type a_selection_missing_its_target_at_the_default_tolerance_is_solved_again_at_the_least_tolerance (p99 / p50, 1e-6 per value, readouts 5.4e-7 or 4e-7: the first solve selects two cheap readouts, 1.08e-6, the check fails, the second selects one of each, 9.4e-7; two solves). It failed before the change with Ok((1, 1, 1)): one solve at 1e-10
T2 Selection::milp_solve_count (1 or 2), printed by asap_planner as milp_solve_count; milp_solve_ms sums both solves a_selection_meeting_its_target_at_the_default_tolerance_is_solved_once (readouts 4e-7 or 3e-7: two cheap meet 1e-6 at the default; one solve)
T3 a_per_value_failure_probability_below_the_solver_precision_is_an_error unchanged in what it asserts: at 1e-10 and 1e-12 per value both solves select two cheap readouts and the error names the query and its per-value δ passes
T4 k_bounds_a_per_value_failure_probability_below_the_default_solver_tolerance renamed to T1's test, which also asserts the solve count

The retry is a real case, not a seam: the 1e-6-per-value ratio (δ = 0.01 over 1e4 values) misses its target at HiGHS's defaults and meets it at 1e-10.

Behavior: before → after

Integration test accuracy_at_query_roots::each_query_meets_its_target_at_its_root: over latency (1,000 series, 1 sample/s, ingested, values declared non-negative), both queries every 10 s, no raw data kept. A: quantile_over_time(0.95, latency[1h]) at rank 0.01; B: quantile_over_time(0.99, latency[1h]) / quantile_over_time(0.5, latency[1h]) at relative 0.03.

Before (main) After (this PR)
Root check none (K): composed error per metric, δ per output value, at each targeted output
B's readouts three DDSketch { alpha: 0.02 } builds: composed (0.02 + 0.02) / 0.9 = 0.044 > 0.03 DDSketch { alpha: 0.01 }, composed (0.01 + 0.01) / 0.9 = 0.022 ≤ 0.03
A Kll { k: 269 } Kll { k: 269 }
p90, p50, p90 / p50 at relative 0.05 (shared readouts) α = 0.05 only (ratio composed 0.111) adds α ≤ 0.02 readouts for the ratio
DDSketch quantile over values of unknown sign guarantee holds NoGuarantee (unknown failure probability)
COUNT(DISTINCT u) from HLL p = 18, relative 0.05, δ 0.01, source bounded at 9,636 NoGuarantee holds
Report, query without a target "accepted exact (Kll)" "no accuracy target (Kll)"
p99 / p50 at relative 0.03, δ = 0.01 over 1e4 values (1e-6 per value), readouts failing with 5.4e-7 (cheap) or 4e-7 no root check the solve at HiGHS's defaults selects two cheap (1.08e-6) and fails the exact check; the second solve, at 1e-10, selects one of each (9.4e-7); milp_solve_count 2
p99 / p50 at relative 0.03, δ = 0.01 over 1e10 values (1e-12 per value), readouts failing with 5.4e-13 (cheap) or 4e-13 no root check both solves select two cheap (1.08e-12) and fail the exact check: AccuracyTargetBelowSolverPrecision naming the query and 1e-12; no plan

Interfaces

crates/types/src/accuracy/target/mod.rs · stages/logical/candidate/mod.rs · stages/logical/group/ · stages/logical/memo/mod.rs · workload/bounds.rs

 impl AccuracyMetric {
+    /// ALL's length: the size of an array of one value per metric.
+    pub const COUNT: usize;
+    /// The position of `self` in ALL (Relative is 3).
+    pub fn index(self) -> usize;
 }
 impl AccuracyTarget {
+    /// δ split evenly over the values its scope covers when one evaluation returns
+    /// `values_per_evaluation` ($n_q$); None without a δ, or for a whole-result δ with
+    /// $n_q$ unknown. δ = 0.01 over 100 series → 0.0001 per output value.
+    pub fn failure_probability_per_output_value(&self, values_per_evaluation: Option<u64>) -> Option<f64>;
 }
 impl CandidateForMemoGroup {
+    /// A readout's input requirement, also the error it adds; None for any other candidate.
+    pub fn required_accuracy(&self) -> Option<RequiredAccuracy>;
 }
 impl ValueSign {
+    /// NonNegative or NonPositive.
+    pub fn is_one_sign(self) -> bool;
 }
 impl GroupProperties {
+    /// The `column_signs` entry of a state group's state column; None without one.
+    pub fn state_column_sign(&self) -> Option<ValueSign>;
 }
 impl Memo {
+    /// declared_max_distinct_per_evaluation over the sources scanned below `group`.
+    pub fn max_distinct_per_evaluation(&self, group: MemoGroupId) -> Option<u64>;
 }
+/// Sum of the declared `max_distinct_per_evaluation` of `scanned`, each looked up as a
+/// PromQL selector reads it (a family's bound covers its series); None when one declares none.
+pub(crate) fn declared_max_distinct_per_evaluation<'a>(
+    sources: &BTreeMap<String, SourceFacts>,
+    scanned: impl IntoIterator<Item = &'a str>,
+) -> Option<u64>;

crates/accuracy/src/lib.rs · guarantee/mod.rs · guarantee/ddsketch.rs · guarantee/hll/mod.rs

 pub fn theoretical_guarantee(
     summary_type: &FieldDataType,
     parameters: &SketchParams,
     statistic: &SketchStatistic,
     failure_probability: f64,
+    state_sign: Option<ValueSign>,
+    max_distinct_per_evaluation: Option<u64>,
 ) -> Option<AccuracyGuarantee>;
 pub(crate) fn guarantee_before_failure_probability_check(
     ...,
     failure_probability: f64,
+    state_sign: Option<ValueSign>,
+    max_distinct_per_evaluation: Option<u64>,
 ) -> Option<AccuracyGuarantee>;
-pub(super) fn guarantee(params: &SketchParams) -> Option<AccuracyGuarantee>; // ddsketch
+/// α, failing never over one-signed values; unknown failure probability otherwise.
+pub(super) fn guarantee(params: &SketchParams, state_sign: Option<ValueSign>) -> Option<AccuracyGuarantee>;
+/// The largest value `within_limit` admits for `limit` (limit · (1 + 1e-9) + ε); (K)'s rows use it.
+pub fn tolerated_limit(limit: f64) -> f64;
-pub(crate) fn query_roots_with_target(memo: &Memo)
+pub fn query_roots_with_target(memo: &Memo)
     -> Vec<(usize, MemoGroupId, &AccuracyTarget, Vec<AccuracyMetric>)>;
 impl ErrorContract {
-    pub(crate) fn input_error_reaches_output(&self, input: usize) -> bool;
+    pub fn input_error_reaches_output(&self, input: usize) -> bool;
 }
 // crate-private, in `guarantee/hll`:
+fn classic_guarantee_at(precision: u8, failure_probability: f64, max_distinct: u64) -> Option<AccuracyGuarantee>;
+struct ClassicHllConfidence { max_distinct: u32, relative_error: f64 }

crates/stage3-plan-selection/src/milp/accuracy.rs (crate-private) · equivalent_contexts.rs · upper_bound.rs · report.rs

+/// An execution's error per metric (by AccuracyMetric::index) and failure probability.
+pub(super) struct ExecutionError { error_bound_per_metric: [f64; AccuracyMetric::COUNT], failure_probability: f64 }
+/// What composing one execution's error gives.
+struct Composition { error: ExecutionError, inputs_admitted: bool }
+/// What (K) reads of the memo, once per selection with some target.
+pub(super) struct AccuracyRules {
+    error_metrics: ErrorMetricsOfMemo,
+    candidate_of_execution: Vec<CandidateForMemoGroupId>,
+    own_error_per_execution: Vec<ExecutionError>,
+    reached_by_query_with_target: Vec<bool>,
+    executions_producers_first: Vec<usize>,
+    roots: HashMap<(QueryId, MemoGroupId), RootLimits>,
+}
+/// What a root check bounds one output by, widened by tolerated_limit; Eq/Hash/Ord by bits.
+pub(super) struct RootLimits {
+    tolerated_error_per_metric: [f64; AccuracyMetric::COUNT],
+    tolerated_failure_probability_per_output_value: Option<f64>,
+}
+impl AccuracyRules {
+    pub(super) fn of(inputs: &SelectionInputs<'_>) -> Option<Self>;
+    pub(super) fn root_limits(&self, query: QueryId, group: MemoGroupId) -> Option<&RootLimits>;
+    /// (K)'s rule for one execution from the error read on each input.
+    fn compose(&self, c: usize, inputs_read: impl IntoIterator<Item = (usize, ExecutionError)>) -> Composition;
+    /// Exactly in f64; Err names the first query whose output misses its target.
+    pub(super) fn selection_meets_targets(&self, inputs: &SelectionInputs<'_>, edges: &[usize], answering: &[(QueryId, usize)]) -> Result<(), QueryId>;
+}
-struct AccuracyVariables {
-    error_bound: HashMap<(PhysicalCandidateId, AccuracyMetric), Variable>,
-    error_bound_of_input: HashMap<(PhysicalCandidateId, usize, AccuracyMetric), Variable>,
-    failure_probability: Vec<Variable>,
-    failure_probability_of_input: HashMap<(PhysicalCandidateId, usize), Variable>,
-}
+/// E per (execution, metric) and D per execution; F and H are made in `accuracy_constraints`.
+pub(super) struct AccuracyVariables {
+    error_bound: HashMap<(usize, usize), Variable>,
+    failure_probability: HashMap<usize, Variable>,
+}
+/// Per execution: rows or not, selectable edges per input, the big-M ceilings, and whether its
+/// error is constant (its inputs carry none: a readout), so it has no variable.
+pub(super) struct ErrorModel<'a> {
+    rules: &'a AccuracyRules,
+    has_rows: Vec<bool>,
+    selectable_edges_per_input: Vec<Vec<Vec<(usize, usize)>>>,
+    ceiling: Vec<ExecutionError>,
+    error_is_constant: Vec<bool>,
+}
+/// An execution's error in one metric (or its failure probability) as (K)'s rows read it.
+enum ErrorTerm { Constant(f64), Variable { variable: Variable, ceiling: f64 } }
 fn accuracy_constraints(
-    physical: &PhysicalASAPMemoCandidates,
-    selection: &SelectionVariables,
-    accuracy: &AccuracyVariables,
+    inputs: &SelectionInputs<'_>,
+    model: &ErrorModel<'_>,
+    accuracy: &AccuracyVariables,
+    y: &BTreeMap<usize, Variable>,
+    z: &BTreeMap<(QueryId, usize), Variable>,
+    shared: &SharedVariables,
+    problem: &mut ProblemVariables,
 ) -> Vec<Constraint>;
-pub(crate) fn dearer_than_a_feasible_plan(inputs: &SelectionInputs<'_>) -> Vec<bool>;
+pub(crate) fn dearer_than_a_feasible_plan(inputs: &SelectionInputs<'_>, rules: Option<&AccuracyRules>) -> Vec<bool>;
-pub(crate) fn copies_in_equivalent_contexts(inputs: &SelectionInputs<'_>, kept: &[bool]) -> SharedVariables;
+pub(crate) fn copies_in_equivalent_contexts<'a>(inputs: &SelectionInputs<'a>, kept: &[bool], rules: Option<&'a AccuracyRules>) -> SharedVariables;
 struct QuerySignature<'a> {
     answering_executions_per_output: Vec<Vec<ExecutionInContext>>,
     exact_only_groups: Option<&'a BTreeSet<MemoGroupId>>,
     range_span_ms: u64,
     accuracy_target_index: usize,
+    root_limits_per_output: Vec<Option<&'a RootLimits>>,
 }
 pub(crate) enum SelectionMilpError {
     Infeasible,
     Solver(String),
+    /// The solver's selection misses `query`'s target when checked exactly.
+    TargetBelowSolverPrecision { query: QueryId, failure_probability_per_output_value: Option<f64> },
 }
+/// The smallest primal and MIP feasibility tolerance HiGHS accepts; the second solve's.
+const LEAST_HIGHS_FEASIBILITY_TOLERANCE: f64 = 1e-10;
 pub(crate) struct SolvedSelection {
     ...
+    /// The time of every HiGHS solve, summed.
     pub(crate) solve_ms: u64,
+    /// 1, or 2 when the selection at HiGHS's default tolerances missed a target.
+    pub(crate) solve_count: usize,
 }
 pub struct Selection {
     ...
     pub milp_solve_ms: u64,
+    /// How many times HiGHS solved the MILP: 1, or 2 when the selection at HiGHS's default
+    /// feasibility tolerances (1e-7) missed an accuracy target and was solved again at 1e-10.
+    pub milp_solve_count: usize,
 }
 pub enum ErrorSelectingPlan {
     ...
     Solver(String),
+    /// The MILP's selection misses query `query`'s target when checked exactly, so no plan
+    /// is returned. Example: δ 1e-12 per output value, below the solver's least tolerance 1e-10.
+    AccuracyTargetBelowSolverPrecision { query: usize, failure_probability_per_output_value: Option<f64> },
     UnsupportedCostModel(String),
 }
 pub struct AcceptedAccuracy {
+    /// Whether the query has an accuracy target.
+    pub has_accuracy_target: bool,
     /// The metric of the first sketch readout's requirement (was: the target's).
     pub accepted_metric: Option<AccuracyMetric>,
     pub sketches: Vec<String>,
 }

Algorithm changes

(K), per execution $c$ a targeted query reaches and that is its own original (full statement in accuracy-models.md, "Checking each query's target in selection: constraint (K)"):

  • A readout's error is constant, its requirement $(\varepsilon_r, \delta_r)$: no variable. An input read only from constant producers contributes $\sum_u \varepsilon_{u,m} y_{u,c,i}$ (at most one $y$ per input is 1, (A)); otherwise $F_{c,i,m} \ge$ that sum and $F_{c,i,m} \ge E_{u,m} - M_{u,m}(1 - y_{u,c,i})$ per producer with a variable.
  • $E_{c,m} \ge \varepsilon_c[m = m_c] + \sum_i \beta_{c,i,m} F_{c,i,m}$, $F_{c,i,m} \le b_{c,i}$, $E_{u,m} \le M_{u,m}(1 - y_{u,c,i})$ (a constant producer: $y = 0$) where $c$ has no contract in $m$; β = 0 passes nothing.
  • $D_c \ge \delta_c + \sum_i H_{c,i}$, $H_{c,i}$ as $F$ (a union bound over the selected producers; Pass 2's levels already split δ over the readout values an output depends on).
  • Per output of a targeted query and execution $v$ answering it: $E_{v,m} \le \varepsilon_q[m \text{ allowed}] + (M - \varepsilon_q)(1 - z_{q,v})$, $D_v \le \delta_q / V + (M - \cdot)(1 - z_{q,v})$, limits widened by tolerated_limit; a constant $v$ above its limit gets $z_{q,v} = 0$.
  • $M$: one pass in the memo's topological order, each input taking its largest producer's ceiling, by compose.
  • HiGHS solves at its default tolerances (1e-7, absolute), under which a row may pass its limit by up to 1e-7. After the solve selection_meets_targets checks the selection exactly; a miss is solved once more at primal and MIP feasibility tolerance 1e-10, the least HiGHS accepts, and checked again; a second miss is AccuracyTargetBelowSolverPrecision, never a plan.

With no target (ignore mode, most corpus workloads) AccuracyRules::of returns None and nothing is built.

With #1360

#1360 (draft, on main) adds AccuracyTarget::failure_probability_per_readout_value(n_q, ReadoutValueCounts) = δ / min(R, V · fan_in) and rewrites readout_caps::query_levels on it. This PR adds failure_probability_per_output_value(n_q) = δ / V (None without a δ) and keeps its query_levels change minimal: min(per_output_value, δ / R), bit-identical to main's δ / max(V, R).

How they combine, whichever merges second:

  • failure_probability_per_readout_value is defined on the per-output-value one: δ / min(R, V · fan_in) = max(δ / R, per_output_value / fan_in), an unknown term dropping out, and 1 without a δ. query_levels then calls it alone (this PR's min line is replaced).
  • (K)'s root rows keep δ per output value (RootLimits::tolerated_failure_probability_per_output_value): $D_v$ is one output value's failure. The H rows add each selected producer's δ once; that is sound because Pass 2's levels split δ over the readout values an output depends on (the comment at the H rows says so). If (K) ever counts fan-in itself, only RootLimits and the H rows change, and the copy key follows RootLimits with no edit.

Decisions and open questions

Decided (round 3): the solver tolerance, option (c). HiGHS solves at its default tolerances; the selection is checked exactly; only a selection that misses a target is solved once more at 1e-10 and checked again; a second miss is AccuracyTargetBelowSolverPrecision. Why: exact correctness is kept by the post-check, not by the tolerance, since no plan that fails the check is ever returned; and the stricter tolerance, which made RangeHeavy-8u 6× slower than the port, is paid for only when the check needs it. Both paper workloads meet their targets at the defaults and solve once.

The measurements behind it (round 2; serial, one pinned core, one run each):

HiGHS tolerances RangeHeavy-8u RollupHeavy-8u
primal and MIP 1e-10 (593ef4163) 75.9 s, 9.67289259858e-5 6.6 s, 9.86298808867e-3
primal 1e-10, MIP default 10.8 s, 9.67289257283e-5 13.3 s, 9.86298808872e-3
defaults (this head's first solve) 10.8 s, 9.67289257283e-5 13.4 s, 9.86298808872e-3

Rejected: (a) 1e-10 always (RangeHeavy-8u 76 s, and a dearer plan); (b) primal 1e-10 with MIP default (as fast, but still pays a stricter tolerance on every solve and needs the check anyway). Trade-off accepted: RollupHeavy-8u loses the 6.6 s it had at 1e-10 and plans in about the port's time (see Verification).

Decided (port and round 1):

  • (K) versus context copies: copies share; (K) rows only for originals; the copy key is each output's RootLimits (rejected: disabling sharing whenever a query has a target).
  • The greedy bound bounds only from a plan meeting (K) (rejected: no bound with targets; a (K)-aware greedy).
  • A readout's error is a constant (exact reformulation; the functional review's variable toggle gave identical plan_cost on 75 workloads).
  • Solver tolerance (L3, round 2): rows scaled by their limits with default tolerances was tried and dropped (RangeHeavy-8u 99 s against 26 s); a tolerance derived from the smallest limit was dropped in round 2 (unsound below 1e-7 per value, RollupHeavy-8u 2.8× slower than the port). Round 3 chose HiGHS's defaults with an exact check and one second solve at 1e-10; see above.
  • Scalar roots follow main's query_roots_with_target.
  • Cost model's DDSketch store count still reads the sources' declared signs: tracked in Cost model derives value signs separately from the memo group's column_signs #1287 (commented with the -latency and rate cases), not in this PR.

Verification

Round 3, on a 56-core host: head 9a67a5560 against main 79d99326e (the main results are round 2's, same main commit).

  • cargo fmt --all -- --check and cargo clippy -p asap-types -p asap-accuracy -p asap-stage2-physical-optimization -p asap-stage3-plan-selection -p asap-integration-tests -p asap-devtools --all-targets -- -D warnings are clean.

  • cargo test -p asap-types -p asap-accuracy -p asap-stage0-memo -p asap-stage1-pass2-reuse-candidates -p asap-stage2-physical-optimization -p asap-stage3-plan-selection -p asap-planner -p asap-integration-tests --no-fail-fast (the stage3 crate's lib, the accuracy crate, and the integration tests accuracy_at_query_roots, memo_value_bounds and the rest), compared by test name:

    passed failed ignored
    head 9a67a5560 665 7 5
    593ef4163 664 7 5
    main 642 7 5
    • The 7 failing tests are the same on all three: integration joint_search_equivalence ×2, materialized_views_equal_the_stored_outputs_main_listed, plan_sharing ×2, promql_to_post_asap::promql_quantile_of_rate_binds_kll_over_rate_accumulator, and planner summary_sharing::identical_ungrouped_queries_share_their_producers.
    • No test passes on main only. Against 593ef4163, the only differences are the two round-3 tests and the renamed one.
  • Tests that fail before their fix:

    • a_selection_missing_its_target_at_the_default_tolerance_is_solved_again_at_the_least_tolerance (round 3): Ok((1, 1, 1)) with one solve at 1e-10; it also checks that the default-tolerance solve misses (two solves).
    • a_per_value_failure_probability_below_the_solver_precision_is_an_error (round 2): without the exact check the solver selects two cheap readouts (2, 0) at 1e-10 and 1e-12 per value.
    • contexts_whose_queries_differ_only_in_values_not_split_by_k_are_copies, a_ratio_sharing_its_readouts_with_its_operands_still_meets_its_target, guarantee::tests::a_ddsketch_quantile_is_certified_only_over_values_of_one_sign, target::tests::a_whole_result_failure_probability_splits_over_the_output_values (round 1).

Exact check on the review workloads (round 2, not rerun). The 88 m/v/h/f/ma workloads of the oracle sets, planned to Stage 3 with 593ef4163 and with the round-1 rule (f3e7aab15, same main):

  • 75 plan on both, with the same plan cost (within 1e-12 relative); the exact check refuses none.
  • f14n_avg_rel_and_count_gt and v14n_near_exact_vs_rank_noraw are infeasible on both; 11 more (f01n–f10n, f15n) select no plan on both, before the MILP is solved.
  • Round 1's enumeration oracle (all selections of 75 workloads, each costed and checked against (K)) found no selection that misses (K); it was not rerun.

Planning time. asap_planner --run-until stage3 with the cost inputs and deployment capabilities of the round-2 review (release builds, one idle pinned core, the four trees alternating, median of 3, host load 3.5–26). The port is 55f0ef67d on the older main dbc57efbb, so its plan costs come from the older cost model.

Workload main port 593ef4163 (1e-10 always) head (option (c)) HiGHS solves, head plan cost per second, 593ef4163 → head
RollupHeavy-8u 13.00 s (12.95–13.71) 11.29 s (11.15–11.34) 6.60 s (6.53–6.60) 13.34 s (12.99–16.97) 1 9.86298808867367e-3 → 9.86298808872144e-3, main's plan
RangeHeavy-8u 5.71 s (5.68–5.74) 12.52 s (12.49–12.52) 76.01 s (75.84–76.13) 11.01 s (10.88–11.10) 1 9.67289259858e-5 → 9.67289257283e-5 (cheaper; main 7.2306e-5 misses its targets)
  • RangeHeavy-8u is 6.9× faster than 593ef4163 and 1.14× faster than the port.
  • RollupHeavy-8u is 2.0× slower than 593ef4163 (which gained from the 1e-10 MIP tolerance), 1.18× slower than the port, and as fast as main (1.03×), with main's plan. The 16.97 s run was the first, at host load 26.

Follow-ups

Design scope

All new types and files, by scope (query engine: frontends → IR → Stage 0 memo → Stage 1 → Stage 2 → Stage 3 selection/cost → plan):

Item Scope Why it is general
milp/accuracy.rs: AccuracyRules, RootLimits, ErrorModel, AccuracyVariables, ExecutionError, Composition, ErrorTerm Stage 3 selection, MILP formulation the accuracy extension of the selection MILP, named after the formulation's terms; AccuracyRules is what (K) reads of the memo, shared by the rows, the ceilings and the greedy check
AccuracyRules::compose, root_limits Stage 3 selection one rule each, used by every caller
ErrorSelectingPlan::AccuracyTargetBelowSolverPrecision, SelectionMilpError::TargetBelowSolverPrecision Stage 3 selection, errors any target whose limit the solver cannot resolve, named by query and per-value δ; not one workload's case
LEAST_HIGHS_FEASIBILITY_TOLERANCE (crate-private const in milp/model/mod.rs) Stage 3 selection, MILP solve the solver's precision limit, used by the one second solve; named for what it holds
Selection::milp_solve_count, SolvedSelection::solve_count Stage 3 report how many times the solver ran, beside milp_solve_ms; any selection with a target may need the second solve
guarantee/hll/ (classic_guarantee_at, ClassicHllConfidence) accuracy, per-algorithm guarantee one directory per algorithm with mod.rs and tests.rs, as the others
guarantee/ddsketch.rs guarantee(params, state_sign) accuracy, per-algorithm guarantee the sign condition is a fact of DDSketch's guarantee
tolerated_limit accuracy, requirement comparison the one widening rule within_limit and (K) share
AccuracyTarget::failure_probability_per_output_value types, accuracy target a target's δ split over the output values its scope covers
AccuracyMetric::COUNT, index types, accuracy metric arrays indexed by metric, used by the accuracy crate and Stage 3
CandidateForMemoGroup::required_accuracy types, logical candidate a readout's requirement, read by (K) and the report
ValueSign::is_one_sign types, logical group column properties "one sign" for every sign condition
GroupProperties::state_column_sign types, logical group the sign of the values a state was updated with
Memo::max_distinct_per_evaluation, workload::bounds::declared_max_distinct_per_evaluation types, memo / data-workload bounds the declared distinct bound of the sources below a group, beside the other declared-fact folds
AcceptedAccuracy::has_accuracy_target Stage 3 report whether a query had a target to accept
test_support::memo_builders::with_non_negative_values types, test support builds a memo whose sources declare non-negative values (DDSketch tests)
integration-tests/tests/accuracy_at_query_roots.rs integration tests (K) end to end through every planning stage

@zzylol zzylol added accuracy Accuracy requirements, guarantees, and propagation optimizer Workload optimization and plan selection refactor labels Nov 7, 2026
@zzylol
zzylol force-pushed the refactor/1207-contexts branch from 8a466a4 to 3b831f9 Compare November 9, 2026 09:35
@zzylol
zzylol force-pushed the refactor/1207-accuracy branch from 55a1384 to 8a376d3 Compare November 9, 2026 10:09
@zzylol zzylol changed the title refactor(stage3): constraint (K), accuracy at each query root (#1207 PR 8) refactor(stage3): constraint (K) per execution and metric; sign-conditioned accuracy rules (#1207 PR 8) Nov 9, 2026
@zzylol
zzylol force-pushed the refactor/1207-contexts branch from 3b831f9 to 6e46d1f Compare November 9, 2026 12:22
@zzylol
zzylol force-pushed the refactor/1207-accuracy branch from 8a376d3 to 6c01250 Compare November 9, 2026 12:37
@zzylol
zzylol force-pushed the refactor/1207-contexts branch from 6e46d1f to f7299f5 Compare November 9, 2026 18:08
@zzylol
zzylol force-pushed the refactor/1207-accuracy branch from 6c01250 to a95b905 Compare November 9, 2026 18:44
@zzylol
zzylol force-pushed the refactor/1207-contexts branch from f7299f5 to 4216ad0 Compare November 9, 2026 19:13
@zzylol
zzylol force-pushed the refactor/1207-accuracy branch from a95b905 to 13f5151 Compare November 9, 2026 19:13
@zzylol
zzylol force-pushed the refactor/1207-contexts branch from 4216ad0 to a1493df Compare November 9, 2026 21:36
@zzylol
zzylol force-pushed the refactor/1207-accuracy branch from 13f5151 to 2166fdf Compare November 9, 2026 21:40
@zzylol
zzylol force-pushed the refactor/1207-contexts branch from a1493df to 0816428 Compare November 9, 2026 23:55
@zzylol
zzylol force-pushed the refactor/1207-accuracy branch from 2166fdf to db34c43 Compare November 10, 2026 00:02
@zzylol
zzylol force-pushed the refactor/1207-contexts branch from 0816428 to e678f70 Compare November 10, 2026 00:19
@zzylol
zzylol force-pushed the refactor/1207-accuracy branch from db34c43 to e8a3f73 Compare November 10, 2026 00:19
@zzylol
zzylol force-pushed the refactor/1207-contexts branch from e678f70 to 2a95559 Compare November 10, 2026 00:45
@zzylol
zzylol force-pushed the refactor/1207-accuracy branch from e8a3f73 to 11e9fe2 Compare November 10, 2026 00:49
@zzylol
zzylol force-pushed the refactor/1207-contexts branch from 2a95559 to 6e7a516 Compare November 10, 2026 01:26
@zzylol
zzylol force-pushed the refactor/1207-accuracy branch from 11e9fe2 to 213dfb1 Compare November 10, 2026 01:26
@zzylol
zzylol force-pushed the refactor/1207-contexts branch from 6e7a516 to e9cc34a Compare November 10, 2026 13:37
@zzylol
zzylol force-pushed the refactor/1207-accuracy branch from 213dfb1 to f850cd1 Compare November 10, 2026 13:37
@zzylol
zzylol force-pushed the refactor/1207-contexts branch 2 times, most recently from 7784817 to 1ef0321 Compare November 13, 2026 17:47
@zzylol
zzylol force-pushed the refactor/1207-contexts branch 3 times, most recently from c746513 to 33e7d6d Compare November 14, 2026 18:34
@zzylol
zzylol force-pushed the refactor/1207-contexts branch 3 times, most recently from 796d435 to 44a223d Compare November 15, 2026 15:54
@zzylol
zzylol force-pushed the refactor/1207-accuracy branch from f850cd1 to 3916932 Compare November 16, 2026 04:08
@zzylol
zzylol changed the base branch from refactor/1207-contexts to main November 16, 2026 04:08
@zzylol
zzylol force-pushed the refactor/1207-accuracy branch from 3916932 to 55f0ef6 Compare November 16, 2026 04:46
@zzylol
zzylol force-pushed the refactor/1207-accuracy branch from 55f0ef6 to b8d6ffb Compare November 16, 2026 06:56
@zzylol zzylol changed the title refactor(stage3): constraint (K) per execution and metric; sign-conditioned accuracy rules (#1207 PR 8) refactor(stage3): constraint (K) per execution and metric; sign-conditioned DDSketch guarantee, classic HLL certificate from the source's distinct bound (#1207 PR 8) Nov 16, 2026
…tioned DDSketch guarantee, HLL distinct count from the source's distinct bound (#1207 PR 8)

Constraint (K) in Stage 3's selection MILP: per execution and error metric,
E and D composed along the selected edges through each candidate's error
contract (with its validity caps and the metric rule), and checked at each
query root in the metrics the root allows, with the target's failure
probability per value. A copy of an equivalent context has no rows of its
own; copies pair only queries whose (K) rows are equal (target, n_q and the
metrics each output allows). The greedy upper bound is used only when its
plan meets (K).

Accuracy: DDSketch's quantile guarantee holds only over values of one sign;
classic HLL's distinct count is certified over the scanned sources' declared
distinct bound. Types: AccuracyTarget::failure_probability_per_value,
GroupProperties::state_column_sign, Memo::max_distinct_per_evaluation.
Report: the accepted metric is the readout's; a query without a target
reports none.
… topological order, copies keyed on root limits, solver tolerance from the smallest limit; sign condition in the DDSketch guarantee (#1207 PR 8, review round 1)

- AccuracyRules::compose is (K)'s rule once: selection_meets_targets and the
  ceilings call it; accuracy_constraints writes it as rows.
- Ceilings in one pass in the memo's topological order (no fixpoint).
- Context copies are keyed on each output's RootLimits, exactly what (K)'s
  root rows read; queries differing only in n_q without a whole-result
  delta are now copies.
- HiGHS's feasibility tolerance is min(1e-7, 1e-3 x the smallest positive
  target limit), at least 1e-10, instead of a fixed 1e-10; scaling the
  rows by their limits kept the defaults but solved RangeHeavy-8users
  about 4x slower.
- AccuracyTarget::failure_probability_per_output_value (renamed; None
  without a delta); readout_caps::query_levels is written on it.
- CandidateForMemoGroup::required_accuracy, read by (K) and the report;
  readout_error_metric is crate-private again.
- DDSketch's sign condition is in guarantee/ddsketch.rs (unknown failure
  probability over values of unknown sign), so theoretical_guarantee
  reads it too; ValueSign::is_one_sign; Stage 2 reads state_column_sign.
- AccuracyMetric::COUNT and index; the distinct-bound fold moves to
  workload/bounds.rs.
- accuracy-models.md: constraint (K) and Sign conditions sections.
- Regression test: p90, p50 and p90 / p50 sharing readouts meet relative
  0.05 only through (K).
…ctly; a target below the solver's precision is an error (#1207 PR 8, review round 2)
@zzylol
zzylol force-pushed the refactor/1207-accuracy branch from b8d6ffb to 593ef41 Compare November 16, 2026 19:56
…nly when the exact (K) check fails (#1207 PR 8, review round 3)

The selection MILP solves at HiGHS's default feasibility tolerances. The
selection is then checked exactly against every accuracy target; one that
misses a target is solved once more at primal and MIP feasibility tolerance
1e-10 (LEAST_HIGHS_FEASIBILITY_TOLERANCE) and checked again, and a miss
after that is AccuracyTargetBelowSolverPrecision as before. Selection
reports milp_solve_count (1 or 2).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

accuracy Accuracy requirements, guarantees, and propagation optimizer Workload optimization and plan selection refactor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant