Repository navigation
fix(accuracy): relative error through negation, abs, set operators and relabels; readouts compose only through error-carrying inputs; exact accumulators take non-finite values and reject stale markers; finalize sign of counters - #1309
Merged
Merged
Conversation
This was referenced Nov 14, 2026
zzylol
force-pushed
the
fix/1307-relative-error-through-negation
branch
from
November 15, 2026 06:40
d099b0b to
5923dce
Compare
…d relabels; non-finite exact sums; finalize sign of counters - Negation and abs keep a value's relative error (|-x̂ - (-x)| = |x̂ - x|, ||x̂| - |x|| <= |x̂ - x|, |-x| = ||x|| = |x|): the Project contract accepts them in relative error with factor 1. - PromQL and/unless copy left values and read only the right side's labels (factors 1, 0); or copies values from both sides (1, 1); label_replace and label_join copy values and write a label from labels (1). Before, all of these fell through to "no contract". - Exact sum, count, min and max accumulators take +-Inf and NaN with PromQL semantics instead of failing the stream; counters still need finite samples. - Finalizing an exact accumulator takes the sign of the aggregate it computes (measure_sign): a rate, increase or irate over non-positive values is Unknown, not the weight's sign. Closes #1307
…ale markers at exact accumulators - composed_queries counts only inputs whose error reaches the output (beta > 0 in a metric of the candidate's contract): `q and r` reads only which series `r` has, so `q`'s readout keeps the single level of its cap instead of the composed error x failure-probability grid. - Document the assumption behind beta = 0 for and/unless: a row's presence never depends on an estimate. - An exact sum, count, min or max rejects a Prometheus stale marker: one that reaches it means its chain read the series through an operator other than the Scan, whose TimeRange drops stale markers. - Nits: the BinaryOp arms of error_contract sit together; exact_accumulator_measure takes ExactKind by value; column_function_of_literals is function_of_a_column_and_literals.
… marker in values - composed_queries: a group reads a readout only through an input with beta > 0, so an operator above a count or an and/unless right input combines no error of it (p90 / count(p50) gives the p90 readout one level, not the composed grid). - STALE_MARKER moves to crate::values with is_stale_marker; the five inline bit checks use it. - StateColumnUpdater calls exact_accumulator_input for its family's kind instead of a copied per-variant rule. - ExactKind::measure replaces exact_accumulator_measure. - No wildcard in the finalize sign arm; shared set_operator test fixture. - Docs: composed query definition, and/unless right input, the sign of a zero divisor, the presence assumption for or.
…sed query A set operator copies each output value from exactly one input, so its error is that input's, not a sum: `composed_queries` counts at most one error-carrying input for it, and each readout of `q or r` keeps the single level of its cap instead of the composed grid. Also: fold the inputs filter into one iterator chain, rename `reads_a_readout` to `carries_a_readout_error`, match function names with `eq_ignore_ascii_case` instead of allocating a lowercase copy, call `is_stale_marker` through one import style, and reword and rewrap comments.
… value it reads
A query's result fails when any readout value it can depend on fails, so
the union bound splits its δ over all of them. Each readout group used
its own value count instead, so a reduction above `or` undercounted:
`max by (job) (q{host="h0"} or q{host="h1"})` with 3 series per side gave
each readout δ / 3 although the `max` reads 6 readout values (2δ in all).
`readouts_read_by_queries` replaces `composed_queries`: the same walk
also sums, per group, the readout values its output depends on through
β > 0 inputs (the largest among alternative candidates), and each query
splits δ over max(values its scope covers, that sum).
zzylol
force-pushed
the
fix/1307-relative-error-through-negation
branch
from
November 16, 2026 00:57
7c0e508 to
bc48168
Compare
This was referenced Nov 16, 2026
Merged
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
abs,and/unless/orandlabel_replace/label_joinkeep a value's relative error, but had no relative contract, so a sketch readout could not feed them.sum by (job) (-quantile_over_time(0.9, data[300s]))over non-negativedatahad no plan without raw data. Two smaller faults came from the fix(accuracy): relative error through sum and avg of values of one sign #1302 review: an exact sum accumulator failed the stream on a NaN or ±Inf input, and finalizing a rate accumulator took the weight's sign.abs.and/unless; 1, 1 foror) and for relabels whose new label reads no estimate.q and randq / count(r)read one readout's error. A set operator takes each output value from one input, soq or rcombines none either. A query's failure probability splits over every readout value its output can depend on, summed over all readouts below it, somax by (job) (q or r)splits δ over both sides.exact_accumulator_input). The stale marker andis_stale_markerlive incrate::values.measure_signofExactKind::measure.NoValidCandidate→ 1.91e-6/s). With raw data it costs 2.34e-7/s instead of 2.68e-7/s. Every changed plan that the executor can run stays within its 5% target against the exact reference, with one exception: I1's division example on near-zero data returns +Inf where the exact answer is -Inf (known limits below; follow-up A relative-error division by an estimate that is exactly zero returns the wrong sign of infinity #1322). The review's fixes also stop a sketchable right side ofand/unless, a count of readouts, ororfrom making the query "composed", which made planning several times slower (or: MILP 1351 × 2009 → 451 × 485 rows × columns, same plan cost).Review guide
mainat3e434c760(refactor(stage3): select over execution contexts: x_{κ,v}, y_{κ,u,c,i}, h_v and constraint (R) (#1207 PR 7) #1237). Not stacked. Rebased from59365ce80without conflicts; an earlier rebase kept fix(frontend): name every aggregate measure and validate names match measures #1308'soutput_names: vec![String::new()]infinalized_schema.error_contractin crates/accuracy/src/error_contract/mod.rs. Check the newPromqlRelabelarm (L82) and theBinaryOpset-operator arm (L129, next to the arithmetic arm),keeps_error(L282),negation_or_abs(L313) andset_operator_error_factors(L358). Then readreadouts_read_by_queriesin readout_caps/mod.rs (L327, withReadoutsReadByQueriesat L299 and its use inquery_levelsat L185) andtakes_each_value_from_one_input(L404). Each helper's comment gives the one-line proof and an example.exact_accumulator_inputin factory/mod.rs (L94),not_a_stale_marker(L114),is_stale_markerin values/mod.rs (L25), and their use in state_column_updater/mod.rs (theGroupStates::Exactarm ofupdate_value).ExactKind::measurein parameters.rs (L35) holds theExactKind→AggIntentmatch that was inline infinalized_schema, so that the finalize sign reuses it, with no behavior change.STALE_MARKERmoves fromoperators/series_window.rstocrate::values, sosummary_kernelsno longer importsoperators. The tests share aset_operatorfixture intest_support::memo_fixtures.function_of_a_column_and_literalsholds the function-name check thatone_lipschitzalready did. The doc tables inaccuracy-models.mdandend-to-end-accuracy-guarantees.mdlist the new contracts.Behavior: before → after
The user writes (the issue's example):
{"query_workload": {"language": "promql", "repeating_queries": [{ "query": "sum by (job) (-quantile_over_time(0.9, data[300s]))", "demand": {"fixed_interval_at": {"interval": 1000, "evaluation_phase": 0}}, "requirements": {"accuracy": {"max_error": 0.05, "failure_probability": 0.3, "accuracy_scope": "whole_result"}}, "predictability": {"predictable": {"known_at": null}}}]}, "data_workload": {"arrival": {"ingesting": {"rows_per_second": 10000.0}}, "sources": {"data": {"series_count": 1000, "metric_type": "gauge", "sample_interval": 100, "column_signs": {"value": "non_negative"}}}}}4a37273cb){"raw_data_retained": false}NoValidCandidate(the selection MILP is infeasible)sumover raw data, 2.67674e-7/sWhy: Stage 0 already gives the negation's output the sign
NonPositive, so the #1302 rule accepts relative error at thesum. The edge was cut one step lower: theProjectcomputing-qaccepted only absolute error. This serves correctness, because these operators do keep relative error, and it serves the MVP need to plan without raw data.Interfaces
None public, apart from the test-support fixture
set_operator. Every new function and the one new struct are crate-private. Shown for review:crates/accuracy/src/error_contract/mod.rs
crates/accuracy/src/readout_caps/mod.rs
crates/types/src/summary_type/parameters.rs
crates/types/src/test_support/memo_fixtures.rs (test support only)
crates/executor/src/values/mod.rs
crates/executor/src/summary_kernels/factory/mod.rs
Usage (the example above; checked by the review harness at this commit and by
a_sum_of_negated_or_absolute_readouts_keeps_the_ddsketch_edge):Design scope
ReadoutsReadByQueries(private,crates/accuracy/src/readout_caps/mod.rs): Stage 1 Pass 2, readout requirements. It holds what one walk up the memo learns about each query's readouts: whether the query combines several readout errors, and how many readout values its output can depend on. It is general: every readout requirement, for any operator, reads both facts. It replaces theBTreeSet<usize>thatcomposed_queriesreturned, so no second walk is added.Algorithm changes
Error contracts (accuracy composition)
avg by (job) (-quantile_over_time(0.9, data[5m])),avg by (job) (abs(quantile_over_time(0.9, mixed[5m]))),... unless ...andlabel_replace(...)over a DDSketch readout had no plan without raw data, because the readout's relative-error edge was cut.error_contract):Project: before, relative error was accepted only forq * candq / c(scaled_by_a_constant). After, also for-qandabs(q)(negation_or_abs). Clamps stay absolute-only.BinaryOpwithBinaryOpKind::Set: before, no arm, soNone. After,set_operator_error_factorsin every metric.PromqlRelabel: before, no arm, soNone. After, β = 1 in every metric when the new label's expression reads no estimate (reads_an_estimate). A relabel reading the value column has no contract.-x:|(−x̂) − (−x)| = |x̂ − x| ≤ ε|x| = ε|−x|. Example: 10 → 10.5 is 5% off, and −10.5 is 5% off −10.abs:||x̂| − |x|| ≤ |x̂ − x| ≤ ε|x| = ε||x||(the reverse triangle inequality). Example: −10 → −10.5 gives 10.5, 5% off 10.and/unless: each output row is a left row with its value copied, and whether it is kept depends only on the right side's labels, so the output error is at most the left error and the right values are never read. Example:q unless rwith q̂ = 10.5 outputs 10.5 or nothing, whatever r's values are.or: each value is copied from one side.and/unless/oralso read which series the other side has. That is exact because a row's presence never depends on an estimate: no operator with a contract drops rows by value (aFilteron a computed value,HAVING, a ranking,count_valuesandabsenthave none), and a readout emits the series the exact evaluation does. Theset_operator_error_factorsdoc and both doc tables state this. The review checked it with series that stop just before, at, and just after the window start: every result had the reference's series.count_values, which has no contract, or a relabel reading the value, which this PR rejects.key_filters_and_value_preserving_projects_have_unit_contracts(updated),set_operators_and_relabels_copy_their_inputs_erroranda_sum_of_negated_or_absolute_readouts_keeps_the_ddsketch_edge.absapplied directly to a column gets the rule.-(-q)andabs(q * 2)stay absolute-only, asone_lipschitzalready was.sum by (x)on the rewritten labelxstill has no contract. That is conservative, and this PR does not change it.Readout levels: set operators do not compose a query
Problem:
composed_queriesmarked a query as composed when a candidate had two or more inputs above a readout group, whatever their β. Soq and rwith a sketchablerwas composed even thoughr's values are never read.q's readout then got the whole error × failure-probability grid (MAX_ERROR_LEVELS×failure_probability_levels) instead of the single level of its cap, and the selection MILP grew about 3.3× (review: 279 × 298 → 912 × 1261 rows × columns, 110 ms → 1291 ms for a 12-series query).After: an input counts only when its β is above 0 in a metric of the candidate's error contract, and a group carries a readout's error only through such an input (
carries_a_readout_error[g] = readout_groups.contains(&g) || most_inputs_reading_with_error > 0; round 5 folds this flag intoreadout_values_per_group[g] != Some(0), see "Failure probability" below). So the rule holds at every level:p90 / count(p50)and(q and r) / sread one readout's error per sketch below. A set operator counts at most one such input (takes_each_value_from_one_input):orhas β = 1 on both sides, but each output value is copied from exactly one of them, so its error is the larger of the two inputs' errors, never their sum.Correctness: with one β > 0 input, only one readout's error reaches the output, so the single cap level is the sound requirement, as for any single-readout query. For
or, each readout's cap is already the full target (cap / β = cap / 1), and each output value carries one readout's error, so one level per readout is sound too. The failure probability is not per output value: see the next bullet. The right readout ofand/unlessgets no cap through the operator (β = 0), as before. Tests:an_and_combines_no_readout_errors(the left readout gets one level, 0.05 at 1e-4; the right gets none),an_or_takes_each_value_from_one_readout(underoreach readout gets one level, 0.05, at 5e-5 since round 5; under/on the same readouts both get the composed grid; it failed before the fix with 12 levels) andan_operator_above_a_count_of_readouts_combines_one_readout_error(p90 / count(p50)on 100 series at relative 0.05: the p90 readout gets one level, 0.045 = 0.05 · (1 − 0.1) at 1e-4, where it got the 10-level composed grid before).Effect (review harness, raw data retained, plan only, median of 3 runs on one machine; plans and costs unchanged):
b02_unless_subset_bares25c_and_on_somes25e_or_max,b20_or_gappy_both(or)s11_neg,s13_abs_mixed,e11_example_sum_negWithout raw data every workload plans in 15–66 ms on both.
or(round 3, efficiency E1): each readout now gets the single level of its cap, as above. On the review workloads everyorquery plans at the same cost with a MILP about 3–6× smaller (round 3 plan check below), for examples25e_or_maxwith raw data: 1351 × 2009 → 451 × 485 rows × columns.Failure probability (round 5, F1): a query's result fails when any readout value it can depend on fails, so by the union bound its δ splits over all of them. Before, each readout group got δ / max(V, its own values per evaluation), with V the values the target's scope covers (
n_qfor a whole result). Under an operator that reads several readout groups this undercounted:max by (job) (q{host="h0"} or q{host="h1"})with 3 series per side gave each readout δ / 3, but themaxreads all 6 readout values, so the union bound reached 2δ. Now the rule is δ / max(V, R), with R the readout values the query's output can depend on: summed over the readout groups below it through β > 0 inputs (Memo::max_values_per_evaluationeach), and the largest among a group's alternative candidates; unknown when one of them is. R is computed in the same walk that finds composed queries (readouts_read_by_queries). Cases:max by (job) (q or r), 3 series per side: R = 6, each readout δ / 6, as for a plain selector over the same 6 series.q or r: each output value reads one readout value, but a whole-result scope still fails when any output value does, and the output can take its values from any of the readout values ofqandr, so the union runs over R = all of them (R ≥n_q). A per-value scope (V = 1) also divides by R, as it already did for one readout group; that is conservative.q / count(r)andq and r: R countsq's readouts only, sincer's values are not read (β = 0).q / r(composed): R counts both; its grid now starts at the tighter per-value δ.q or q) counts twice. That only tightens the split, so it stays sound.a_reduction_above_an_or_splits_the_failure_probability_over_both_sides(s25e's shape: each readout gets 0.01 / 6; it failed before the fix with 0.01 / 3), andan_or_takes_each_value_from_one_readoutnow expects 5e-5 = 0.01 / 200 for a bareorof two 100-series readouts (1e-4 before).orworkloads (plan only, uniform data, seed 0; same harness as round 4):s25e_or_maxn_q= 24 already covers both sides' 12 + 12 declared series)s25ewithmax(...)instead ofmax by (job)(n_q= 1)s25d_or,a03b,a20,b20,b21,b24,d11s22,a03,a03c,a18s24,d09,d14,g03Sizes do not change: each
orreadout still gets one level, only a tighter one. On the review workloads the facts declare each side as the whole 12-series source, son_qalready equals the sum; the δ only tightens whenn_qis below it, as with a reduction to one value.Open (formulation): the error contract writes an output's error as a sum over inputs (Σ β·ε). It cannot express "each value is taken from one input", whose bound is max(ε_L, ε_R). Readout levels now treat
oras the max. But a sum-based constraint (K) in Stage 3, which is not written yet, would over-constrainor. See the question under Decisions.Stale markers at exact accumulators (executor)
Scan(through shifts), because only then does it get a sample-value column for itsTimeRange(node_below_shiftsinphysical_planner/mod.rs). Every plan today puts the chain on theScan. If a chain ever sits above aFilteror aProject, the stale marker reaches the accumulator. Before this PR the accumulator failed; with this PR's non-finite exact sums it would add the NaN silently.exact_accumulator_inputrejects a value whose bits are the stale marker (0x7ff0000000000002) for sum, count, min and max, and both update paths (PlannerExactUpdaterandStateColumnUpdater, with the kind from its family) call it. A NaN from arithmetic (0 / 0) has other bits and is still taken. The check catches a marker passed on unchanged by aFilteror aProjectthat copies the value; a computingProjectturns it into an ordinary NaN. The executor lowering comment at the chain'sTimeRange(physical_planner) points to the check.exact_accumulators_take_non_finite_values_as_promql_doesandan_exact_sum_column_takes_non_finite_valuesnow also feed a stale marker and expect an error.Finalize sign (Stage 0
GroupProperties)Finalize(Rate(neg))wasNonPositive.measure_sign(kind.measure(None), state sign); the arm lists everyFieldDataTypevariant, with no wildcard. Sum, min and max keep the sign; count isNonNegative; rate, increase and irate areNonNegativeonly overNonNegativevalues and otherwiseUnknown.Aggregate{Rate}. Over non-positive samples, −5 then −3 is an increase of +2, while −3 then −5 reads as a reset and gives −5. Test:a_finalized_exact_accumulator_has_its_aggregates_sign.Exact accumulator inputs (executor)
PlannerExactUpdaterand the state-column path.f64::min/maxskip NaN unless every value is NaN, as Prometheus'smin/maxdo. Rate and increase still require finite samples.Decisions and open questions
Needs reviewer decision
FiniteFloat64summary-update expression, aligned arithmetic, sketch kernels and counters. Say if counters (rate/increase over a NaN sample) should follow PromQL too.oris included. The issue proposesunlessandand.orcopies values from both sides, so it gets (1, 1); its proof and test are in this PR.orcomposes in constraint (K). The sum-of-β error contract cannot say "taken from one input". When (K) is written in Stage 3, shouldorget a max form (each input's readouts within the full target), or should the contract gain a way to mark inputs whose errors do not add? Readout levels already use the max form here. Under the max form errors do not add, but failure probabilities still do: each readout ofmax by (job) (q or r)gets δ over the readout values of both sides. This is a formulation question for you.label_replace/label_joinat all, onmaintoo (QueryRunNotExecutable: value operation has no native implementation). With raw data,mainalready fails to install the exact plan for the same query. The planner rule stays (executor limits do not restrict the design). Should I file an executor issue for it?Decided (review of this PR):
composed_queries(nowreadouts_read_by_queries) counts only inputs with β > 0, and at most one for a set operator.ScalarStateinstead of the family'sExactKind): not done. It would list the state variants again next toexact_accumulator_input, the duplication round 2 (A2) removed, and its gain (about 1–2 ns per row) is within the ±3 ns the benchmark shifts with code layout.Verification
Tests first: every new or changed test failed before its fix and passes here.
cargo test --release -p asap-accuracy -p asap-types -p asap-executor --no-fail-fasttypes79; doc tests 9)a_reduction_above_an_or_splits_the_failure_probability_over_both_sides(failed before with 0.01 / 3); changed:an_or_takes_each_value_from_one_readoutexpects 5e-5 (failed before with 1e-4)cargo clippy -p asap-accuracy -p asap-types -p asap-executor --all-targets -- -D warnings;cargo fmt --all --checkcargo test -p asap-accuracy;cargo test -p asap-executor --liban_or_takes_each_value_from_one_readout, which failed before the fix (12 levels for each readout ofor);an_and_combines_no_readout_errorsno longer checksorcargo clippy -p asap-accuracy -p asap-executor --all-targets -- -D warnings;cargo fmt --all --check3e434c760(headbc48168dc):cargo fmt --check;cargo nextest run -p asap-accuracy -p asap-types;cargo nextest run -p asap-executor -E 'test(summary_kernels) | test(values)'59365ce80:cargo test -p asap-types -p asap-accuracy -p asap-executortypes79; asap-executor lib 117 (1 ignored) and its integration tests; doc testsan_operator_above_a_count_of_readouts_combines_one_readout_error, which failed before thecomposed_queriesfix (10 levels)cargo clippy -p asap-types -p asap-accuracy -p asap-executor --all-targets -- -D warnings;cargo fmt --allcargo test -p asap-accuracy --lib;-p asap-executor --lib summary_kernels;--lib physical_planner;-p asap-types --lib,--test typesan_and_combines_no_readout_errors; stale-marker cases in the two exact-accumulator testscargo test -p asap-accuracyset_operators_and_relabels_copy_their_inputs_erroranda_sum_of_negated_or_absolute_readouts_keeps_the_ddsketch_edge; changed:key_filters_and_value_preserving_projects_have_unit_contracts(abs/negation now relative)cargo test -p asap-types(lib +types)a_finalized_exact_accumulator_has_its_aggregates_signcargo test -p asap-stage0-memo -p asap-stage1-pass1-per-computation-candidates -p asap-stage1-pass2-reuse-candidates -p asap-stage2-physical-optimization -p asap-stage3-plan-selectioncargo test -p asap-executor --lib summary_kernelsexact_accumulators_take_non_finite_values_as_promql_doesandan_exact_sum_column_takes_non_finite_valuescargo test -p asap-plannersummary_sharing::identical_ungrouped_queries_share_their_producersandruntime_install::check_compiles_reports_the_executors_erroralso fail at4a37273cbcargo test -p asap-integration-tests -- pass2_promql pass1_candidates plan_sharing binary_op stage3_objective nested aggregateplan_sharing::slower_repeat_reuses_faster_sumandnested_exact_sum_reuses_newest_tumbling_windowsalso fail at4a37273cbcargo fmt --check;cargo clippy -p asap-accuracy -p asap-types -p asap-executor --all-targets -- -D warningsNew tests:
set_operators_and_relabels_copy_their_inputs_error:and/unlessgive (1, 0),orgives (1, 1), and a relabel fromjobgives 1, all in every metric; a relabel from the estimate gives none.a_sum_of_negated_or_absolute_readouts_keeps_the_ddsketch_edge: the signs areNonPositivefor-qandNonNegativeforabs(q); the DDSketch edge into theProjectand theProject's edge into thesumhold, and KLL's rank edge is cut.a_finalized_exact_accumulator_has_its_aggregates_sign:Finalize(Rate|Increase|IRate)over a non-positive source isUnknown; Sum, Min and Max areNonPositive; Count isNonNegative.exact_accumulators_take_non_finite_values_as_promql_does,an_exact_sum_column_takes_non_finite_values: sum ±Inf/NaN, count, min/max skipping NaN, counters still rejecting NaN, and a stale marker rejected.an_and_combines_no_readout_errors: underand, the left readout gets the single level of its cap and the right one none.an_or_takes_each_value_from_one_readout: underor, each readout gets the single level of its cap; under/on the same readouts, both get the composed grid.an_operator_above_a_count_of_readouts_combines_one_readout_error:p90 / count(p50)gives the p90 readout one level, not 10.Round 3 plan check (same harness and 246 runs; before =
5923dcef1, after =dd1973c87; both binaries carry a local, uncommitted print of MILP rows, columns and objective): a before binary built from the round-3 tree with the two accuracy files at5923dcef1reproduces the round-2 results exactly (same status, plan nodes and costs). After the fix, every run has the same status and every cost agrees within a relative 5e-11. Only the 14orruns change their MILP, each with the same objective:s25e_or_max(max(...) or max(...))s25d_orb20_or_gappy_bothb24_sum_or_on_instancea03b_or_same_seriesa20_or_bare_same,b21_or_same_metric_qTwo of these runs (
b24_sum_or_on_instancewithout raw data,s25e_or_maxwith raw data) pick a different plan of the same cost. The topsumormaxis a plain query-timeAggregateinstead of a query-time exactSummaryAggfollowed byFinalizeExactAccumulator. The two cost the same, and the smaller MILP breaks the tie the other way. Theand,unlessand single-readout workloads build the same MILPs as before.Round 2 plan check (review harness, plan only, 123 workloads from the review sets × {raw, no raw} = 246 runs, uniform data, seed 0; before = this PR's round-1 commit rebased on
59365ce80,a2c464795; after =5923dcef1): every run has the same status and the same plan nodes, and every cost agrees within a relative 8e-11 (floating-point noise). Planning gets faster where a count sat above a readout, since such a query is no longer composed (raw data retained, 3 sequential runs each):avg(quantile_over_time(0.5, data[300s]))(avgis sum ÷ count)quantile_over_time(0.9, data[5m]) and on(job) avg by (job) (quantile_over_time(0.9, neg[5m]))quantile_over_time(0.9, data[5m]) unless on(avg) avg by (job) (quantile_over_time(0.9, neg[5m]))orworkloads (a20_or_bare_same,b21_or_same_metric_q)After the review fixes, the 8 changed workloads (
s25c,s25e,b02,b20,s11,s13,s26,e11; raw and no raw; uniform data, seed 0) plan the same plans at the same costs as before the fixes, and every executed answer is within target (worst 0.0494).Plan changes (review harness; 61 workloads × {raw, no raw} planned on base
4a37273cband on this PR before the rebase; only these 10 differ; cost under theaws-fargate-us-east-1cost inputs, per second):sum by (job) (-quantile_over_time(0.9, data[300s]))(issue example)avg by (job) (-quantile_over_time(0.9, data[5m]))avg by (job) (abs(quantile_over_time(0.9, mixed[5m])))avg by (job) (q(data) unless q(neg))avg by (job) (label_replace(q(data), "x", "$1", "instance", "(.*)"))label_replace(as on base)This base-vs-PR diff was not redone on the new base
59365ce80. The round-2 and round-3 plan checks above ran on the rebased commits (a2c464795,5923dcef1,dd1973c87) and found the same plans, apart from the two equal-cost ties noted in round 3. Each change is correct because the new contract is exact (proofs above), the input sign is one-signed (-qnon-positive,absnon-negative,unlesskeeps the left sign), and every executed plan stays within its 5% target. Extra checks (not plan diffs from the base set):unless on(instance)matching one series,and on(instance)matching some, andmax by (job) (q(h0) or q(h1))each plan DDSketch with and without raw data (base: NoValidCandidate without raw data, exact with raw), 3 runs each, worst relative error 0.03, 0.0412 and 0.0429.sum by (job) (q(data) / on(instance, job, host, region) q(neg))with no raw data now executes. Before, it failed with "accumulator input must be finite". On near-zero data it returns +Inf for j0, j1 and j2 where the exact answer is -Inf (seeds 0–2). An earlier version of this body said it matched the reference: that harness wrote NaN and ±Inf to JSON asnulland countednull == nullas a match. The review's harness compares non-finite values, and found the mismatch.Known limits
neg's near-zero samples are-0.0, so the exactquantile_over_time(0.9, neg[5m])is-0.0, while the DDSketch readout returns+0.0(a relative error of 0 allows the sign change).x / ±0then flips the infinity's sign. With raw data retained,mainalready returns +Inf, so the error is older than this PR; this PR extends it to the no-raw plan, wheremainfailed instead. I1 therefore holds except where the divisor's estimate is a signed zero. The relative/contract has no nonzero-divisor guard, and no planner code setschecked_relative_division: A relative-error division by an estimate that is exactly zero returns the wrong sign of infinity #1322.abs(quantile_over_time(0.3, mixed[5m]))and-quantile_over_time(0.3, mixed[5m])miss the 5% target on 1 of 12 series (0.0544: −0.1606 against −0.1523). The bare quantile misses the same way onmain: PromQL interpolates between ranks that straddle zero. The new contracts carry the error through, as their proofs say. Tracked in A DDSketch quantile's relative bound fails when the interpolated neighbours straddle zero #807 (comment added with this example).Review status
a2c464795.composed_queriescounts only β > 0 inputs; the presence assumption of the set operators is documented; exact accumulators reject a stale marker; I1 is not reverted and the zero-divisor case went to A relative-error division by an estimate that is exactly zero returns the wrong sign of infinity #1322;exact_accumulator_measuregained one home; the set-operator arm moved next to the arithmetic arm;one_lipschitz's helper was renamed.5923dcef1:59365ce80, keeping fix(frontend): name every aggregate measure and validate names match measures #1308'soutput_names: vec![String::new()].STALE_MARKERmoved tocrate::valueswithis_stale_marker; the five inline bit checks use it; comments narrowed to "aFilteror aProjectthat copies the value".composed_queriespropagates "carries a readout's error" only through β > 0 inputs;any_input_readingis deleted; new test forp90 / count(p50).StateColumnUpdatercallsexact_accumulator_inputwith its family's kind.FieldDataTypevariants.set_operatorfixture.and/unlessright input; the zero-divisor sentence admits the sign flip of a zero; theorrow states the presence assumption.ExactKind::measurereplacesexact_accumulator_measure.physical_planner(the kernel guard stays).dd1973c87:composed_queries, soq or ris not composed (takes_each_value_from_one_input); new testan_or_takes_each_value_from_one_readout; docs updated in both places that define a composed query. Howorcomposes in constraint (K) is an open question above.Vec;function_of_a_column_and_literalsmatches witheq_ignore_ascii_caseand allocates no lowercase copy.reads_a_readout→carries_a_readout_error. N2: the stale-marker comment inphysical_plannerreworded. N3, R3-A2: long lines rewrapped. R3-A3:is_stale_markeris imported everywhere it is called. N4: plan check note above.dd1973c87): approve, no blocker or high finding. All round-3 findings fixed except E2 (skipped, see Decisions). 2,256 runs per head: every executed answer within target except the known A DDSketch quantile's relative bound fails when the interpolated neighbours straddle zero #807 and A relative-error division by an estimate that is exactly zero returns the wrong sign of infinity #1322 cases; noorworkload answered wrong. Tests 564 passed, 1 ignored; clippy clean. Three points:or.reads_a_readout.7c0e5089b:readouts_read_by_queriesreplacescomposed_queriesand also returns these counts inReadoutsReadByQueries). New testa_reduction_above_an_or_splits_the_failure_probability_over_both_sides;an_or_takes_each_value_from_one_readoutnow expects δ over both sides; both docs state the rule. The (K) decision note now says failure probabilities still add under the max form.carries_a_readout_error. F3: this section.Follow-ups
label_replace/label_joinbefore relabel plans can run (on base too). Not filed; see the decision above.FiniteFloat64summary updates, aligned arithmetic, counters) are unchanged: Non-finite values across the executor should follow PromQL #1311.avg_over_timewithout raw data still fails on non-finite samples, now at query time in its checked sum ÷ count division; added to Non-finite values across the executor should follow PromQL #1311.absand negation.Closes #1307