Skip to content

fix(accuracy): relative error through negation, abs, set operators and relabels; readouts compose only through error-carrying inputs; exact accumulators take non-finite values and reject stale markers; finalize sign of counters - #1309

Merged
zzylol merged 5 commits into
mainfrom
fix/1307-relative-error-through-negation
Nov 16, 2026

Conversation

@zzylol

@zzylol zzylol commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Problem: negation, abs, and/unless/or and label_replace/label_join keep a value's relative error, but had no relative contract, so a sketch readout could not feed them. sum by (job) (-quantile_over_time(0.9, data[300s])) over non-negative data had no plan without raw data. Two smaller faults came from the fix(accuracy): relative error through sum and avg of values of one sign #1302 review: an exact sum accumulator failed the stream on a NaN or ±Inf input, and finalizing a rate accumulator took the weight's sign.
  • Change:
    • Relative contracts with factor 1 for negation and abs.
    • Contracts in every metric for the PromQL set operators (left 1, right 0 for and/unless; 1, 1 for or) and for relabels whose new label reads no estimate.
    • Readout levels: a query is composed only when two or more inputs carry a readout's error (β > 0), checked at every level: q and r and q / count(r) read one readout's error. A set operator takes each output value from one input, so q or r combines none either. A query's failure probability splits over every readout value its output can depend on, summed over all readouts below it, so max by (job) (q or r) splits δ over both sides.
    • Executor: exact sum, count, min and max accumulators take non-finite values with PromQL semantics and reject a Prometheus stale marker, in both update paths through one rule (exact_accumulator_input). The stale marker and is_stale_marker live in crate::values.
    • The finalize sign follows measure_sign of ExactKind::measure.
  • Effect: the example now plans a DDSketch chain without raw data (NoValidCandidate → 1.91e-6/s). With raw data it costs 2.34e-7/s instead of 2.68e-7/s. Every changed plan that the executor can run stays within its 5% target against the exact reference, with one exception: I1's division example on near-zero data returns +Inf where the exact answer is -Inf (known limits below; follow-up A relative-error division by an estimate that is exactly zero returns the wrong sign of infinity #1322). The review's fixes also stop a sketchable right side of and/unless, a count of readouts, or or from making the query "composed", which made planning several times slower (or: MILP 1351 × 2009 → 451 × 485 rows × columns, same plan cost).

Review guide

Behavior: before → after

The user writes (the issue's example):

{"query_workload": {"language": "promql", "repeating_queries": [{
  "query": "sum by (job) (-quantile_over_time(0.9, data[300s]))",
  "demand": {"fixed_interval_at": {"interval": 1000, "evaluation_phase": 0}},
  "requirements": {"accuracy": {"max_error": 0.05, "failure_probability": 0.3, "accuracy_scope": "whole_result"}},
  "predictability": {"predictable": {"known_at": null}}}]},
 "data_workload": {"arrival": {"ingesting": {"rows_per_second": 10000.0}},
  "sources": {"data": {"series_count": 1000, "metric_type": "gauge", "sample_interval": 100,
                       "column_signs": {"value": "non_negative"}}}}}
Deployment Before (4a37273cb) After
{"raw_data_retained": false} NoValidCandidate (the selection MILP is infeasible) DDSketch tumbling-window chain, 1.90766e-6/s; worst relative error 0.035 over 6 runs (target 0.05)
default (raw retained) exact sum over raw data, 2.67674e-7/s DDSketch readouts with an exact sum accumulator, 2.33818e-7/s; worst relative error 0.035

Why: Stage 0 already gives the negation's output the sign NonPositive, so the #1302 rule accepts relative error at the sum. The edge was cut one step lower: the Project computing -q accepted only absolute error. This serves correctness, because these operators do keep relative error, and it serves the MVP need to plan without raw data.

Interfaces

None public, apart from the test-support fixture set_operator. Every new function and the one new struct are crate-private. Shown for review:

crates/accuracy/src/error_contract/mod.rs

+/// Whether `expr` is a column's negation or `abs` (`promql_abs`): it keeps the
+/// column's error both absolutely and as a share of the value.
+fn negation_or_abs<C>(expr: &ScalarExpr<C>) -> bool;
+/// Whether `expr` calls one of `functions` (or its `promql_` form) on a column
+/// and then literals only.
+fn function_of_a_column_and_literals<C>(expr: &ScalarExpr<C>, functions: &[&str]) -> bool;
+/// β of a PromQL set operator's left and right inputs: (1, 0) for and/unless, (1, 1) for or.
+/// Assumes each side's series are exact: no operator with a contract drops a row by value.
+fn set_operator_error_factors(kind: &PromQLVectorSetOpKind) -> [f64; 2];

crates/accuracy/src/readout_caps/mod.rs

-/// The queries that combine several readouts: a candidate they reach has two or more
-/// inputs below which a readout group lies.
-fn composed_queries(
-    memo: &Memo,
-    order: &[MemoGroupId],
-    readout_groups: &BTreeSet<MemoGroupId>,
-) -> BTreeSet<usize>;
+/// What each query with a target reads from the readouts below it.
+struct ReadoutsReadByQueries {
+    /// The queries that combine several readouts: a candidate they reach has two or
+    /// more inputs below which lies a readout whose error reaches its output (β > 0 in
+    /// a metric of its error contract, at every step below). `q and r`,
+    /// `q / count(r)` and `q or r` are not composed.
+    composed: BTreeSet<usize>,
+    /// Per query: the readout values per evaluation its output can depend on through
+    /// β > 0, summed over the readout groups below it and, among alternative candidates
+    /// of a group, the largest; `None` when one is unknown.
+    /// `max by (job) (q{host="h0"} or q{host="h1"})`, 3 series per side: 6.
+    readout_values: BTreeMap<usize, Option<u64>>,
+}
+/// The `ReadoutsReadByQueries` of `memo`, in one walk up its topological order.
+fn readouts_read_by_queries(
+    memo: &Memo,
+    error_contract: &[Vec<ErrorContractPerMetric>],
+    order: &[MemoGroupId],
+    readout_groups: &BTreeSet<MemoGroupId>,
+) -> ReadoutsReadByQueries;
 fn query_levels(
     target: &AccuracyTarget,
     number_of_values_per_query_evaluation: Option<u64>,
-    readout_values: Option<u64>, // the readout group's own values per evaluation
+    readout_values: Option<u64>, // ReadoutsReadByQueries::readout_values of the query
     metric: AccuracyMetric,
     cap: f64,
     composed: bool,
 ) -> Option<Vec<RequiredAccuracy>>;
+/// Whether `operator` copies each output value from exactly one of its inputs: a PromQL
+/// set operator.
+fn takes_each_value_from_one_input(operator: &Operator<MemoGroupId>) -> bool;

crates/types/src/summary_type/parameters.rs

+impl ExactKind {
+    /// The aggregate an exact accumulator of this kind computes, of input column `column`
+    /// (`None`: the sample value); read by `finalized_schema` and the finalize sign.
+    pub(crate) fn measure(self, column: Option<ColumnId>) -> AggIntent;
+}

crates/types/src/test_support/memo_fixtures.rs (test support only)

+/// `lhs and rhs`, `lhs or rhs` or `lhs unless rhs`, series by series, without vector matching.
+pub fn set_operator(kind: PromQLVectorSetOpKind, lhs: MemoGroupId, rhs: MemoGroupId) -> Operator<MemoGroupId>;

crates/executor/src/values/mod.rs

+/// The bits of the Prometheus stale marker (moved from `operators::series_window`).
+pub(crate) const STALE_MARKER: u64 = 0x7ff0_0000_0000_0002;
+/// Whether `value` is the Prometheus stale marker, not a sample.
+pub(crate) fn is_stale_marker(value: f64) -> bool;

crates/executor/src/summary_kernels/factory/mod.rs

+/// The input rule of an exact accumulator of `kind`: sum, count, min, max take any
+/// value but a stale marker (PromQL semantics); a counter needs finite samples.
+pub(crate) fn exact_accumulator_input(kind: ExactKind, value: f64) -> Result<(), String>;
+/// An error when `value` is a Prometheus stale marker, which a chain's TimeRange over
+/// its time series' Scan drops; one reaching an exact accumulator is an error.
+fn not_a_stale_marker(value: f64) -> Result<(), String>;
 impl AccumulatorUpdater for PlannerExactUpdater {
+    /// Delegates to `exact_accumulator_input` for the state's kind.
+    fn validate_single_input(&self, value: f64) -> Result<(), String>;
     fn update_single(&mut self, value: f64, timestamp: i64) -> Result<(), String>;
     fn update_keyed(&mut self, key: &[String], value: f64, timestamp: i64) -> Result<(), String>;
     fn take_accumulator(&mut self) -> Box<dyn AggregateCore>;
     fn snapshot_accumulator(&self) -> Box<dyn AggregateCore>;
     fn into_accumulator(self: Box<Self>) -> Box<dyn AggregateCore>;
     fn merge_accumulator(&mut self, other: &dyn AggregateCore) -> Result<(), String>;
     fn reset(&mut self);
     fn is_keyed(&self) -> bool;
     fn memory_usage_bytes(&self) -> usize;
 }

Usage (the example above; checked by the review harness at this commit and by a_sum_of_negated_or_absolute_readouts_keeps_the_ddsketch_edge):

query  sum by (job) (-quantile_over_time(0.9, data[300s]))      // data declared non_negative
Stage 0  G_q  = quantile_over_time(0.9, data[300s])  value sign NonNegative
         G_neg = Project(-value)(G_q)                value sign NonPositive (flipped)
         G_sum = sum by (job)(G_neg)
Pass 2   edge DDSketch readout -> G_neg: error_contract(Project, Relative)
           keeps_error(-value, Relative) = negation_or_abs -> β = 1          (was None: edge cut)
         edge G_neg -> G_sum: error_contract(Aggregate Sum, Relative)
           values_of_one_sign(G_neg, value) = NonPositive -> β = 1           (#1302)
Stage 3  selects the DDSketch chain: 1.90766e-6/s without raw data (was NoValidCandidate)

Design scope

  • ReadoutsReadByQueries (private, crates/accuracy/src/readout_caps/mod.rs): Stage 1 Pass 2, readout requirements. It holds what one walk up the memo learns about each query's readouts: whether the query combines several readout errors, and how many readout values its output can depend on. It is general: every readout requirement, for any operator, reads both facts. It replaces the BTreeSet<usize> that composed_queries returned, so no second walk is added.

Algorithm changes

Error contracts (accuracy composition)

  • Problem: avg by (job) (-quantile_over_time(0.9, data[5m])), avg by (job) (abs(quantile_over_time(0.9, mixed[5m]))), ... unless ... and label_replace(...) over a DDSketch readout had no plan without raw data, because the readout's relative-error edge was cut.
  • Before / after (error_contract):
    1. Project: before, relative error was accepted only for q * c and q / c (scaled_by_a_constant). After, also for -q and abs(q) (negation_or_abs). Clamps stay absolute-only.
    2. BinaryOp with BinaryOpKind::Set: before, no arm, so None. After, set_operator_error_factors in every metric.
    3. PromqlRelabel: before, no arm, so None. After, β = 1 in every metric when the new label's expression reads no estimate (reads_an_estimate). A relabel reading the value column has no contract.
  • Correctness:
    • -x: |(−x̂) − (−x)| = |x̂ − x| ≤ ε|x| = ε|−x|. Example: 10 → 10.5 is 5% off, and −10.5 is 5% off −10.
    • abs: ||x̂| − |x|| ≤ |x̂ − x| ≤ ε|x| = ε||x|| (the reverse triangle inequality). Example: −10 → −10.5 gives 10.5, 5% off 10.
    • and/unless: each output row is a left row with its value copied, and whether it is kept depends only on the right side's labels, so the output error is at most the left error and the right values are never read. Example: q unless r with q̂ = 10.5 outputs 10.5 or nothing, whatever r's values are.
    • or: each value is copied from one side.
    • Relabel: the value is copied, and only a label computed from labels changes.
    • and/unless/or also read which series the other side has. That is exact because a row's presence never depends on an estimate: no operator with a contract drops rows by value (a Filter on a computed value, HAVING, a ranking, count_values and absent have none), and a readout emits the series the exact evaluation does. The set_operator_error_factors doc and both doc tables state this. The review checked it with series that stop just before, at, and just after the window start: every result had the reference's series.
    • Labels never carry an estimate's error here: PromQL can derive a label from a value only through count_values, which has no contract, or a relabel reading the value, which this PR rejects.
    • Tests: key_filters_and_value_preserving_projects_have_unit_contracts (updated), set_operators_and_relabels_copy_their_inputs_error and a_sum_of_negated_or_absolute_readouts_keeps_the_ddsketch_edge.
  • Complexity: unchanged, O(1) per item or operator.
  • Assumptions and limits:
    • Only a negation or abs applied directly to a column gets the rule. -(-q) and abs(q * 2) stay absolute-only, as one_lipschitz already was.
    • The relabel's destination label gets "computed" lineage, so a later sum by (x) on the rewritten label x still has no contract. That is conservative, and this PR does not change it.

Readout levels: set operators do not compose a query

  • Problem: composed_queries marked a query as composed when a candidate had two or more inputs above a readout group, whatever their β. So q and r with a sketchable r was composed even though r's values are never read. q's readout then got the whole error × failure-probability grid (MAX_ERROR_LEVELS × failure_probability_levels) instead of the single level of its cap, and the selection MILP grew about 3.3× (review: 279 × 298 → 912 × 1261 rows × columns, 110 ms → 1291 ms for a 12-series query).

  • After: an input counts only when its β is above 0 in a metric of the candidate's error contract, and a group carries a readout's error only through such an input (carries_a_readout_error[g] = readout_groups.contains(&g) || most_inputs_reading_with_error > 0; round 5 folds this flag into readout_values_per_group[g] != Some(0), see "Failure probability" below). So the rule holds at every level: p90 / count(p50) and (q and r) / s read one readout's error per sketch below. A set operator counts at most one such input (takes_each_value_from_one_input): or has β = 1 on both sides, but each output value is copied from exactly one of them, so its error is the larger of the two inputs' errors, never their sum.

  • Correctness: with one β > 0 input, only one readout's error reaches the output, so the single cap level is the sound requirement, as for any single-readout query. For or, each readout's cap is already the full target (cap / β = cap / 1), and each output value carries one readout's error, so one level per readout is sound too. The failure probability is not per output value: see the next bullet. The right readout of and/unless gets no cap through the operator (β = 0), as before. Tests: an_and_combines_no_readout_errors (the left readout gets one level, 0.05 at 1e-4; the right gets none), an_or_takes_each_value_from_one_readout (under or each readout gets one level, 0.05, at 5e-5 since round 5; under / on the same readouts both get the composed grid; it failed before the fix with 12 levels) and an_operator_above_a_count_of_readouts_combines_one_readout_error (p90 / count(p50) on 100 series at relative 0.05: the p90 readout gets one level, 0.045 = 0.05 · (1 − 0.1) at 1e-4, where it got the 10-level composed grid before).

  • Effect (review harness, raw data retained, plan only, median of 3 runs on one machine; plans and costs unchanged):

    Workload Before this fix After
    b02_unless_subset_bare 724 ms 178 ms
    s25c_and_on_some 918 ms 585 ms
    s25e_or_max, b20_or_gappy_both (or) 4068 ms, 2408 ms 4987 ms, 2538 ms (rule unchanged in that round; machine noise; round 3 below)
    single-readout s11_neg, s13_abs_mixed, e11_example_sum_neg 544, 546, 100 ms 553, 552, 101 ms

    Without raw data every workload plans in 15–66 ms on both.

  • or (round 3, efficiency E1): each readout now gets the single level of its cap, as above. On the review workloads every or query plans at the same cost with a MILP about 3–6× smaller (round 3 plan check below), for example s25e_or_max with raw data: 1351 × 2009 → 451 × 485 rows × columns.

  • Failure probability (round 5, F1): a query's result fails when any readout value it can depend on fails, so by the union bound its δ splits over all of them. Before, each readout group got δ / max(V, its own values per evaluation), with V the values the target's scope covers (n_q for a whole result). Under an operator that reads several readout groups this undercounted: max by (job) (q{host="h0"} or q{host="h1"}) with 3 series per side gave each readout δ / 3, but the max reads all 6 readout values, so the union bound reached 2δ. Now the rule is δ / max(V, R), with R the readout values the query's output can depend on: summed over the readout groups below it through β > 0 inputs (Memo::max_values_per_evaluation each), and the largest among a group's alternative candidates; unknown when one of them is. R is computed in the same walk that finds composed queries (readouts_read_by_queries). Cases:

    • max by (job) (q or r), 3 series per side: R = 6, each readout δ / 6, as for a plain selector over the same 6 series.
    • A bare q or r: each output value reads one readout value, but a whole-result scope still fails when any output value does, and the output can take its values from any of the readout values of q and r, so the union runs over R = all of them (R ≥ n_q). A per-value scope (V = 1) also divides by R, as it already did for one readout group; that is conservative.
    • q / count(r) and q and r: R counts q's readouts only, since r's values are not read (β = 0).
    • A ratio q / r (composed): R counts both; its grid now starts at the tighter per-value δ.
    • A readout group reached on two paths (q or q) counts twice. That only tightens the split, so it stays sound.
    • Tests: a_reduction_above_an_or_splits_the_failure_probability_over_both_sides (s25e's shape: each readout gets 0.01 / 6; it failed before the fix with 0.01 / 3), and an_or_takes_each_value_from_one_readout now expects 5e-5 = 0.01 / 200 for a bare or of two 100-series readouts (1e-4 before).
    • Effect on the review's or workloads (plan only, uniform data, seed 0; same harness as round 4):
    Workload (raw data retained / not) MILP rows × columns, before = after Plan cost Smallest readout δ, before → after
    s25e_or_max 451 × 485 / 233 × 218 unchanged 0.0125 → 0.0125 (n_q = 24 already covers both sides' 12 + 12 declared series)
    s25e with max(...) instead of max by (job) (n_q = 1) 451 × 485 / 233 × 218 unchanged 0.025 → 0.0125 (0.3 / 12 → 0.3 / 24); answer within 5%
    s25d_or, a03b, a20, b20, b21, b24, d11 457 × 491, 317 × 359, 264 × 305, 432 × 466, 264 × 305, 283 × 325, 514 × 575 / 239 × 224, 185 × 188, 132 × 134, 211 × 195, 132 × 134, 151 × 154, 260 × 253 unchanged unchanged (0.0125; d11 0.00833)
    s22, a03, a03c, a18 raw only (no plan without raw data on either head) unchanged exact plans, no readout level
    ratios s24, d09, d14, g03 unchanged unchanged unchanged at the 1e-5 grid floor

    Sizes do not change: each or readout still gets one level, only a tighter one. On the review workloads the facts declare each side as the whole 12-series source, so n_q already equals the sum; the δ only tightens when n_q is below it, as with a reduction to one value.

  • Open (formulation): the error contract writes an output's error as a sum over inputs (Σ β·ε). It cannot express "each value is taken from one input", whose bound is max(ε_L, ε_R). Readout levels now treat or as the max. But a sum-based constraint (K) in Stage 3, which is not written yet, would over-constrain or. See the question under Decisions.

Stale markers at exact accumulators (executor)

  • Problem: a chain over a time series drops stale markers only when it reads the series' Scan (through shifts), because only then does it get a sample-value column for its TimeRange (node_below_shifts in physical_planner/mod.rs). Every plan today puts the chain on the Scan. If a chain ever sits above a Filter or a Project, the stale marker reaches the accumulator. Before this PR the accumulator failed; with this PR's non-finite exact sums it would add the NaN silently.
  • After: exact_accumulator_input rejects a value whose bits are the stale marker (0x7ff0000000000002) for sum, count, min and max, and both update paths (PlannerExactUpdater and StateColumnUpdater, with the kind from its family) call it. A NaN from arithmetic (0 / 0) has other bits and is still taken. The check catches a marker passed on unchanged by a Filter or a Project that copies the value; a computing Project turns it into an ordinary NaN. The executor lowering comment at the chain's TimeRange (physical_planner) points to the check.
  • Tests: exact_accumulators_take_non_finite_values_as_promql_does and an_exact_sum_column_takes_non_finite_values now also feed a stale marker and expect an error.

Finalize sign (Stage 0 GroupProperties)

  • Before: a finalize of a non-count state copied the state's recorded weight sign, so Finalize(Rate(neg)) was NonPositive.
  • After: the sign is measure_sign(kind.measure(None), state sign); the arm lists every FieldDataType variant, with no wildcard. Sum, min and max keep the sign; count is NonNegative; rate, increase and irate are NonNegative only over NonNegative values and otherwise Unknown.
  • Correctness: this is the same rule as the logical Aggregate{Rate}. Over non-positive samples, −5 then −3 is an increase of +2, while −3 then −5 reads as a reset and gives −5. Test: a_finalized_exact_accumulator_has_its_aggregates_sign.
  • Impact: no plan changes. Today no group is created from a finalize, so no planner result reads this sign.

Exact accumulator inputs (executor)

  • Before: every exact accumulator rejected a non-finite input with "accumulator input must be finite", in both PlannerExactUpdater and the state-column path.
  • After: sum, count, min and max take any Float64. IEEE arithmetic already gives PromQL's results: a sum is +Inf, or NaN after it sees −Inf or NaN; a count counts the value; f64::min/max skip NaN unless every value is NaN, as Prometheus's min/max do. Rate and increase still require finite samples.
  • Complexity: unchanged.

Decisions and open questions

Needs reviewer decision

  • I1 direction. I chose to match PromQL in the executor (exact sum/count/min/max take ±Inf and NaN) rather than stop the planner from choosing an exact accumulator over a division that can be non-finite. This follows "never restrict the planner by today's executor gaps". The other non-finite gates are unchanged: the FiniteFloat64 summary-update expression, aligned arithmetic, sketch kernels and counters. Say if counters (rate/increase over a NaN sample) should follow PromQL too.
  • or is included. The issue proposes unless and and. or copies values from both sides, so it gets (1, 1); its proof and test are in this PR.
  • How or composes in constraint (K). The sum-of-β error contract cannot say "taken from one input". When (K) is written in Stage 3, should or get a max form (each input's readouts within the full target), or should the contract gain a way to mark inputs whose errors do not add? Readout levels already use the max form here. Under the max form errors do not add, but failure probabilities still do: each readout of max by (job) (q or r) gets δ over the readout values of both sides. This is a formulation question for you.
  • label_replace execution gap. A relabel now plans with a sketch, but the executor cannot run label_replace/label_join at all, on main too (QueryRunNotExecutable: value operation has no native implementation). With raw data, main already fails to install the exact plan for the same query. The planner rule stays (executor limits do not restrict the design). Should I file an executor issue for it?

Decided (review of this PR):

  • composed_queries (now readouts_read_by_queries) counts only inputs with β > 0, and at most one for a set operator.
  • A query's δ splits over every readout value its output can depend on, summed over the readout groups below it (round 5, F1).
  • E2 (decide the stale-marker check from the loaded ScalarState instead of the family's ExactKind): not done. It would list the state variants again next to exact_accumulator_input, the duplication round 2 (A2) removed, and its gain (about 1–2 ns per row) is within the ±3 ns the benchmark shifts with code layout.
  • The β = 0 assumption (series presence is exact) is documented.
  • Exact accumulators reject a stale marker.
  • I1 is not reverted: the sum accumulator now follows PromQL, and the near-zero error comes from the division (A relative-error division by an estimate that is exactly zero returns the wrong sign of infinity #1322).
  • The DDSketch interpolation miss near zero (below) belongs to A DDSketch quantile's relative bound fails when the interpolated neighbours straddle zero #807, not a new issue.

Verification

Tests first: every new or changed test failed before its fix and passes here.

Scope Result Newly failing vs base Notes
round 5 fix: cargo test --release -p asap-accuracy -p asap-types -p asap-executor --no-fail-fast 565 passed, 1 ignored (asap-accuracy lib 37; asap-executor lib 117, 1 ignored, and integration 167; asap-types lib 156, types 79; doc tests 9) none new: a_reduction_above_an_or_splits_the_failure_probability_over_both_sides (failed before with 0.01 / 3); changed: an_or_takes_each_value_from_one_readout expects 5e-5 (failed before with 1e-4)
round 5: cargo clippy -p asap-accuracy -p asap-types -p asap-executor --all-targets -- -D warnings; cargo fmt --all --check clean
round 3 fixes: cargo test -p asap-accuracy; cargo test -p asap-executor --lib asap-accuracy lib 36 passed; asap-executor lib 117 passed, 1 ignored none new: an_or_takes_each_value_from_one_readout, which failed before the fix (12 levels for each readout of or); an_and_combines_no_readout_errors no longer checks or
round 3: cargo clippy -p asap-accuracy -p asap-executor --all-targets -- -D warnings; cargo fmt --all --check clean
rebased on 3e434c760 (head bc48168dc): cargo fmt --check; cargo nextest run -p asap-accuracy -p asap-types; cargo nextest run -p asap-executor -E 'test(summary_kernels) | test(values)' fmt clean; 278 of 278 pass; 102 of 102 pass none none
round 2 fixes, rebased on 59365ce80: cargo test -p asap-types -p asap-accuracy -p asap-executor all pass: asap-accuracy lib 35; asap-types lib 156, types 79; asap-executor lib 117 (1 ignored) and its integration tests; doc tests none new: an_operator_above_a_count_of_readouts_combines_one_readout_error, which failed before the composed_queries fix (10 levels)
round 2: cargo clippy -p asap-types -p asap-accuracy -p asap-executor --all-targets -- -D warnings; cargo fmt --all clean
round 1 fixes: cargo test -p asap-accuracy --lib; -p asap-executor --lib summary_kernels; --lib physical_planner; -p asap-types --lib, --test types 34; 68 (1 ignored); 2; 156; 78 passed none new: an_and_combines_no_readout_errors; stale-marker cases in the two exact-accumulator tests
cargo test -p asap-accuracy 33 passed none new: set_operators_and_relabels_copy_their_inputs_error and a_sum_of_negated_or_absolute_readouts_keeps_the_ddsketch_edge; changed: key_filters_and_value_preserving_projects_have_unit_contracts (abs/negation now relative)
cargo test -p asap-types (lib + types) 156 + 78 passed none new: a_finalized_exact_accumulator_has_its_aggregates_sign
cargo test -p asap-stage0-memo -p asap-stage1-pass1-per-computation-candidates -p asap-stage1-pass2-reuse-candidates -p asap-stage2-physical-optimization -p asap-stage3-plan-selection 9, 37, 28, 15, 5 + 2 passed none
cargo test -p asap-executor --lib summary_kernels 68 passed, 1 ignored none new: exact_accumulators_take_non_finite_values_as_promql_does and an_exact_sum_column_takes_non_finite_values
cargo test -p asap-planner 12 + 19 passed, 2 failed, 1 ignored none summary_sharing::identical_ungrouped_queries_share_their_producers and runtime_install::check_compiles_reports_the_executors_error also fail at 4a37273cb
cargo test -p asap-integration-tests -- pass2_promql pass1_candidates plan_sharing binary_op stage3_objective nested aggregate 92 passed, 2 failed, 3 ignored none plan_sharing::slower_repeat_reuses_faster_sum and nested_exact_sum_reuses_newest_tumbling_windows also fail at 4a37273cb
cargo fmt --check; cargo clippy -p asap-accuracy -p asap-types -p asap-executor --all-targets -- -D warnings clean

New tests:

  • set_operators_and_relabels_copy_their_inputs_error: and/unless give (1, 0), or gives (1, 1), and a relabel from job gives 1, all in every metric; a relabel from the estimate gives none.
  • a_sum_of_negated_or_absolute_readouts_keeps_the_ddsketch_edge: the signs are NonPositive for -q and NonNegative for abs(q); the DDSketch edge into the Project and the Project's edge into the sum hold, and KLL's rank edge is cut.
  • a_finalized_exact_accumulator_has_its_aggregates_sign: Finalize(Rate|Increase|IRate) over a non-positive source is Unknown; Sum, Min and Max are NonPositive; Count is NonNegative.
  • exact_accumulators_take_non_finite_values_as_promql_does, an_exact_sum_column_takes_non_finite_values: sum ±Inf/NaN, count, min/max skipping NaN, counters still rejecting NaN, and a stale marker rejected.
  • an_and_combines_no_readout_errors: under and, the left readout gets the single level of its cap and the right one none.
  • an_or_takes_each_value_from_one_readout: under or, each readout gets the single level of its cap; under / on the same readouts, both get the composed grid.
  • an_operator_above_a_count_of_readouts_combines_one_readout_error: p90 / count(p50) gives the p90 readout one level, not 10.

Round 3 plan check (same harness and 246 runs; before = 5923dcef1, after = dd1973c87; both binaries carry a local, uncommitted print of MILP rows, columns and objective): a before binary built from the round-3 tree with the two accuracy files at 5923dcef1 reproduces the round-2 results exactly (same status, plan nodes and costs). After the fix, every run has the same status and every cost agrees within a relative 5e-11. Only the 14 or runs change their MILP, each with the same objective:

Workload No raw data: rows × columns Raw data: rows × columns
s25e_or_max (max(...) or max(...)) 821 × 1118 → 233 × 218 1351 × 2009 → 451 × 485
s25d_or 827 × 1124 → 239 × 224 1357 × 2015 → 457 × 491
b20_or_gappy_both 799 × 1095 → 211 × 195 1332 × 1990 → 432 × 466
b24_sum_or_on_instance 739 × 1054 → 151 × 154 1183 × 1849 → 283 × 325
a03b_or_same_series 773 × 1088 → 185 × 188 1217 × 1883 → 317 × 359
a20_or_bare_same, b21_or_same_metric_q 720 × 1034 → 132 × 134 1164 × 1829 → 264 × 305

Two of these runs (b24_sum_or_on_instance without raw data, s25e_or_max with raw data) pick a different plan of the same cost. The top sum or max is a plain query-time Aggregate instead of a query-time exact SummaryAgg followed by FinalizeExactAccumulator. The two cost the same, and the smaller MILP breaks the tie the other way. The and, unless and single-readout workloads build the same MILPs as before.

Round 2 plan check (review harness, plan only, 123 workloads from the review sets × {raw, no raw} = 246 runs, uniform data, seed 0; before = this PR's round-1 commit rebased on 59365ce80, a2c464795; after = 5923dcef1): every run has the same status and the same plan nodes, and every cost agrees within a relative 8e-11 (floating-point noise). Planning gets faster where a count sat above a readout, since such a query is no longer composed (raw data retained, 3 sequential runs each):

Workload Before After
avg(quantile_over_time(0.5, data[300s])) (avg is sum ÷ count) 1496–1510 ms 830–850 ms
quantile_over_time(0.9, data[5m]) and on(job) avg by (job) (quantile_over_time(0.9, neg[5m])) 1268–1412 ms 193–219 ms
quantile_over_time(0.9, data[5m]) unless on(avg) avg by (job) (quantile_over_time(0.9, neg[5m])) 1235–1260 ms 190–191 ms
or workloads (a20_or_bare_same, b21_or_same_metric_q) 4.0–7.6 s 4.7–6.6 s (rule unchanged in that round; machine noise; see round 3)

After the review fixes, the 8 changed workloads (s25c, s25e, b02, b20, s11, s13, s26, e11; raw and no raw; uniform data, seed 0) plan the same plans at the same costs as before the fixes, and every executed answer is within target (worst 0.0494).

Plan changes (review harness; 61 workloads × {raw, no raw} planned on base 4a37273cb and on this PR before the rebase; only these 10 differ; cost under the aws-fargate-us-east-1 cost inputs, per second):

Workload Deployment Base This PR Executed vs exact reference (uniform, skewed, near-zero data)
sum by (job) (-quantile_over_time(0.9, data[300s])) (issue example) no raw NoValidCandidate DDSketch chain, 1.90766e-6 6 runs, worst rel. error 0.035
same raw exact sum, 2.67674e-7 DDSketch + exact sum, 2.33818e-7 6 runs, worst 0.035
avg by (job) (-quantile_over_time(0.9, data[5m])) no raw NoValidCandidate DDSketch chain, 4.55565e-10 6 runs, worst 0.035
same raw exact, 5.94358e-12 DDSketch, 5.27344e-12 6 runs, worst 0.035
avg by (job) (abs(quantile_over_time(0.9, mixed[5m]))) no raw NoValidCandidate DDSketch chain, 8.98119e-10 6 runs, worst 0.0385
same raw exact, 5.94358e-12 DDSketch, 5.27344e-12 6 runs, worst 0.0385
avg by (job) (q(data) unless q(neg)) no raw NoValidCandidate DDSketch chain, 1.07807e-9 6 runs, matches (empty result)
same raw exact, 1.18804e-11 DDSketch, 1.05401e-11 6 runs, matches (empty result)
avg by (job) (label_replace(q(data), "x", "$1", "instance", "(.*)")) no raw NoValidCandidate DDSketch chain, 4.55781e-10 not executable: the executor lacks label_replace (as on base)
same raw exact, 7.02301e-12 (also not executable) DDSketch, 6.35286e-12 not executable (as on base)

This base-vs-PR diff was not redone on the new base 59365ce80. The round-2 and round-3 plan checks above ran on the rebased commits (a2c464795, 5923dcef1, dd1973c87) and found the same plans, apart from the two equal-cost ties noted in round 3. Each change is correct because the new contract is exact (proofs above), the input sign is one-signed (-q non-positive, abs non-negative, unless keeps the left sign), and every executed plan stays within its 5% target. Extra checks (not plan diffs from the base set):

  • unless on(instance) matching one series, and on(instance) matching some, and max by (job) (q(h0) or q(h1)) each plan DDSketch with and without raw data (base: NoValidCandidate without raw data, exact with raw), 3 runs each, worst relative error 0.03, 0.0412 and 0.0429.
  • I1: sum by (job) (q(data) / on(instance, job, host, region) q(neg)) with no raw data now executes. Before, it failed with "accumulator input must be finite". On near-zero data it returns +Inf for j0, j1 and j2 where the exact answer is -Inf (seeds 0–2). An earlier version of this body said it matched the reference: that harness wrote NaN and ±Inf to JSON as null and counted null == null as a match. The review's harness compares non-finite values, and found the mismatch.

Known limits

  • I1 at a signed-zero divisor. 30% of neg's near-zero samples are -0.0, so the exact quantile_over_time(0.9, neg[5m]) is -0.0, while the DDSketch readout returns +0.0 (a relative error of 0 allows the sign change). x / ±0 then flips the infinity's sign. With raw data retained, main already returns +Inf, so the error is older than this PR; this PR extends it to the no-raw plan, where main failed instead. I1 therefore holds except where the divisor's estimate is a signed zero. The relative / contract has no nonzero-divisor guard, and no planner code sets checked_relative_division: A relative-error division by an estimate that is exactly zero returns the wrong sign of infinity #1322.
  • Quantile readouts across zero. abs(quantile_over_time(0.3, mixed[5m])) and -quantile_over_time(0.3, mixed[5m]) miss the 5% target on 1 of 12 series (0.0544: −0.1606 against −0.1523). The bare quantile misses the same way on main: PromQL interpolates between ranks that straddle zero. The new contracts carry the error through, as their proofs say. Tracked in A DDSketch quantile's relative bound fails when the interpolated neighbours straddle zero #807 (comment added with this example).

Review status

  • Round 1 (functional, efficiency, abstraction, layering): fixed in a2c464795. composed_queries counts only β > 0 inputs; the presence assumption of the set operators is documented; exact accumulators reject a stale marker; I1 is not reverted and the zero-divisor case went to A relative-error division by an estimate that is exactly zero returns the wrong sign of infinity #1322; exact_accumulator_measure gained one home; the set-operator arm moved next to the arithmetic arm; one_lipschitz's helper was renamed.
  • Round 2 (layering, abstraction), fixed in 5923dcef1:
    • L1: rebased onto 59365ce80, keeping fix(frontend): name every aggregate measure and validate names match measures #1308's output_names: vec![String::new()].
    • L2, A3: STALE_MARKER moved to crate::values with is_stale_marker; the five inline bit checks use it; comments narrowed to "a Filter or a Project that copies the value".
    • A1: composed_queries propagates "carries a readout's error" only through β > 0 inputs; any_input_reading is deleted; new test for p90 / count(p50).
    • A2: StateColumnUpdater calls exact_accumulator_input with its family's kind.
    • A4: the finalize arm lists the FieldDataType variants.
    • A5: shared set_operator fixture.
    • L3–L5: "composed query" defined by β > 0 in the three places, with the and/unless right input; the zero-divisor sentence admits the sign flip of a zero; the or row states the presence assumption.
    • L7: title and Summary cover all changes. L8: ExactKind::measure replaces exact_accumulator_measure.
    • Not done: L6 / A3's optional compile-time stale-marker check in physical_planner (the kernel guard stays).
  • Round 3 (abstraction, layering, efficiency; all approve), fixed in dd1973c87:
    • E1: a set operator counts at most one error-carrying input in composed_queries, so q or r is not composed (takes_each_value_from_one_input); new test an_or_takes_each_value_from_one_readout; docs updated in both places that define a composed query. How or composes in constraint (K) is an open question above.
    • R3-A1 / E3: the inputs filter is one iterator chain with no Vec; function_of_a_column_and_literals matches with eq_ignore_ascii_case and allocates no lowercase copy.
    • N1: reads_a_readout → carries_a_readout_error. N2: the stale-marker comment in physical_planner reworded. N3, R3-A2: long lines rewrapped. R3-A3: is_stale_marker is imported everywhere it is called. N4: plan check note above.
    • Not done: E2, see Decisions.
  • Round 4 (functional verification of rounds 2–3 at dd1973c87): approve, no blocker or high finding. All round-3 findings fixed except E2 (skipped, see Decisions). 2,256 runs per head: every executed answer within target except the known A DDSketch quantile's relative bound fails when the interpolated neighbours straddle zero #807 and A relative-error division by an estimate that is exactly zero returns the wrong sign of infinity #1322 cases; no or workload answered wrong. Tests 564 passed, 1 ignored; clippy clean. Three points:
    • F1 (medium): the failure-probability budget doubled under a reduction above or.
    • F2 (nit): the readout-levels "After" bullet still named reads_a_readout.
    • F3 (nit): this status line was a placeholder.
  • Round 5, fixed in 7c0e5089b:
    • F1: δ splits over every readout value a query's output can depend on (readouts_read_by_queries replaces composed_queries and also returns these counts in ReadoutsReadByQueries). New test a_reduction_above_an_or_splits_the_failure_probability_over_both_sides; an_or_takes_each_value_from_one_readout now expects δ over both sides; both docs state the rule. The (K) decision note now says failure probabilities still add under the max form.
    • F2: the "After" bullet names carries_a_readout_error. F3: this section.

Follow-ups

Closes #1307

@zzylol zzylol added bug Something isn't working promql PromQL front-end lowering (L1→L2) error-estimation accuracy Accuracy requirements, guarantees, and propagation labels Nov 14, 2026
@zzylol
zzylol force-pushed the fix/1307-relative-error-through-negation branch from d099b0b to 5923dce Compare November 15, 2026 06:40
@zzylol zzylol changed the title fix(accuracy): relative error through negation, abs, set operators and relabels; non-finite exact sums; finalize sign of counters fix(accuracy): relative error through negation, abs, set operators and relabels; readouts compose only through error-carrying inputs; exact accumulators take non-finite values and reject stale markers; finalize sign of counters Nov 15, 2026
…d relabels; non-finite exact sums; finalize sign of counters

- Negation and abs keep a value's relative error (|-x̂ - (-x)| = |x̂ - x|,
  ||x̂| - |x|| <= |x̂ - x|, |-x| = ||x|| = |x|): the Project contract accepts
  them in relative error with factor 1.
- PromQL and/unless copy left values and read only the right side's labels
  (factors 1, 0); or copies values from both sides (1, 1); label_replace and
  label_join copy values and write a label from labels (1). Before, all of
  these fell through to "no contract".
- Exact sum, count, min and max accumulators take +-Inf and NaN with PromQL
  semantics instead of failing the stream; counters still need finite
  samples.
- Finalizing an exact accumulator takes the sign of the aggregate it
  computes (measure_sign): a rate, increase or irate over non-positive
  values is Unknown, not the weight's sign.

Closes #1307
…ale markers at exact accumulators

- composed_queries counts only inputs whose error reaches the output
  (beta > 0 in a metric of the candidate's contract): `q and r` reads
  only which series `r` has, so `q`'s readout keeps the single level of
  its cap instead of the composed error x failure-probability grid.
- Document the assumption behind beta = 0 for and/unless: a row's
  presence never depends on an estimate.
- An exact sum, count, min or max rejects a Prometheus stale marker: one
  that reaches it means its chain read the series through an operator
  other than the Scan, whose TimeRange drops stale markers.
- Nits: the BinaryOp arms of error_contract sit together;
  exact_accumulator_measure takes ExactKind by value;
  column_function_of_literals is function_of_a_column_and_literals.
… marker in values

- composed_queries: a group reads a readout only through an input with
  beta > 0, so an operator above a count or an and/unless right input
  combines no error of it (p90 / count(p50) gives the p90 readout one
  level, not the composed grid).
- STALE_MARKER moves to crate::values with is_stale_marker; the five
  inline bit checks use it.
- StateColumnUpdater calls exact_accumulator_input for its family's kind
  instead of a copied per-variant rule.
- ExactKind::measure replaces exact_accumulator_measure.
- No wildcard in the finalize sign arm; shared set_operator test fixture.
- Docs: composed query definition, and/unless right input, the sign of a
  zero divisor, the presence assumption for or.
…sed query

A set operator copies each output value from exactly one input, so its
error is that input's, not a sum: `composed_queries` counts at most one
error-carrying input for it, and each readout of `q or r` keeps the
single level of its cap instead of the composed grid.

Also: fold the inputs filter into one iterator chain, rename
`reads_a_readout` to `carries_a_readout_error`, match function names
with `eq_ignore_ascii_case` instead of allocating a lowercase copy, call
`is_stale_marker` through one import style, and reword and rewrap
comments.
… value it reads

A query's result fails when any readout value it can depend on fails, so
the union bound splits its δ over all of them. Each readout group used
its own value count instead, so a reduction above `or` undercounted:
`max by (job) (q{host="h0"} or q{host="h1"})` with 3 series per side gave
each readout δ / 3 although the `max` reads 6 readout values (2δ in all).

`readouts_read_by_queries` replaces `composed_queries`: the same walk
also sums, per group, the readout values its output depends on through
β > 0 inputs (the largest among alternative candidates), and each query
splits δ over max(values its scope covers, that sum).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

accuracy Accuracy requirements, guarantees, and propagation bug Something isn't working error-estimation promql PromQL front-end lowering (L1→L2)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Relative error through negation/abs/unless/label_replace; non-finite exact-sum input; finalize sign of rate accumulators

1 participant