Repository navigation
refactor(stage3): constraint (K) per execution and metric; sign-conditioned DDSketch guarantee, classic HLL certificate from the source's distinct bound (#1207 PR 8) - #1238
Merged
Conversation
zzylol
force-pushed
the
refactor/1207-contexts
branch
from
November 9, 2026 09:35
8a466a4 to
3b831f9
Compare
zzylol
force-pushed
the
refactor/1207-accuracy
branch
from
November 9, 2026 10:09
55a1384 to
8a376d3
Compare
zzylol
force-pushed
the
refactor/1207-contexts
branch
from
November 9, 2026 12:22
3b831f9 to
6e46d1f
Compare
zzylol
force-pushed
the
refactor/1207-accuracy
branch
from
November 9, 2026 12:37
8a376d3 to
6c01250
Compare
zzylol
force-pushed
the
refactor/1207-contexts
branch
from
November 9, 2026 18:08
6e46d1f to
f7299f5
Compare
zzylol
force-pushed
the
refactor/1207-accuracy
branch
from
November 9, 2026 18:44
6c01250 to
a95b905
Compare
zzylol
force-pushed
the
refactor/1207-contexts
branch
from
November 9, 2026 19:13
f7299f5 to
4216ad0
Compare
zzylol
force-pushed
the
refactor/1207-accuracy
branch
from
November 9, 2026 19:13
a95b905 to
13f5151
Compare
zzylol
force-pushed
the
refactor/1207-contexts
branch
from
November 9, 2026 21:36
4216ad0 to
a1493df
Compare
zzylol
force-pushed
the
refactor/1207-accuracy
branch
from
November 9, 2026 21:40
13f5151 to
2166fdf
Compare
zzylol
force-pushed
the
refactor/1207-contexts
branch
from
November 9, 2026 23:55
a1493df to
0816428
Compare
zzylol
force-pushed
the
refactor/1207-accuracy
branch
from
November 10, 2026 00:02
2166fdf to
db34c43
Compare
zzylol
force-pushed
the
refactor/1207-contexts
branch
from
November 10, 2026 00:19
0816428 to
e678f70
Compare
zzylol
force-pushed
the
refactor/1207-accuracy
branch
from
November 10, 2026 00:19
db34c43 to
e8a3f73
Compare
zzylol
force-pushed
the
refactor/1207-contexts
branch
from
November 10, 2026 00:45
e678f70 to
2a95559
Compare
zzylol
force-pushed
the
refactor/1207-accuracy
branch
from
November 10, 2026 00:49
e8a3f73 to
11e9fe2
Compare
This was referenced Nov 10, 2026
zzylol
force-pushed
the
refactor/1207-contexts
branch
from
November 10, 2026 01:26
2a95559 to
6e7a516
Compare
zzylol
force-pushed
the
refactor/1207-accuracy
branch
from
November 10, 2026 01:26
11e9fe2 to
213dfb1
Compare
zzylol
force-pushed
the
refactor/1207-contexts
branch
from
November 10, 2026 13:37
6e7a516 to
e9cc34a
Compare
zzylol
force-pushed
the
refactor/1207-accuracy
branch
from
November 10, 2026 13:37
213dfb1 to
f850cd1
Compare
zzylol
force-pushed
the
refactor/1207-contexts
branch
2 times, most recently
from
November 13, 2026 17:47
7784817 to
1ef0321
Compare
zzylol
force-pushed
the
refactor/1207-contexts
branch
3 times, most recently
from
November 14, 2026 18:34
c746513 to
33e7d6d
Compare
This was referenced Nov 14, 2026
zzylol
force-pushed
the
refactor/1207-contexts
branch
3 times, most recently
from
November 15, 2026 15:54
796d435 to
44a223d
Compare
zzylol
force-pushed
the
refactor/1207-accuracy
branch
from
November 16, 2026 04:08
f850cd1 to
3916932
Compare
zzylol
force-pushed
the
refactor/1207-accuracy
branch
from
November 16, 2026 04:46
3916932 to
55f0ef6
Compare
zzylol
force-pushed
the
refactor/1207-accuracy
branch
from
November 16, 2026 06:56
55f0ef6 to
b8d6ffb
Compare
…tioned DDSketch guarantee, HLL distinct count from the source's distinct bound (#1207 PR 8) Constraint (K) in Stage 3's selection MILP: per execution and error metric, E and D composed along the selected edges through each candidate's error contract (with its validity caps and the metric rule), and checked at each query root in the metrics the root allows, with the target's failure probability per value. A copy of an equivalent context has no rows of its own; copies pair only queries whose (K) rows are equal (target, n_q and the metrics each output allows). The greedy upper bound is used only when its plan meets (K). Accuracy: DDSketch's quantile guarantee holds only over values of one sign; classic HLL's distinct count is certified over the scanned sources' declared distinct bound. Types: AccuracyTarget::failure_probability_per_value, GroupProperties::state_column_sign, Memo::max_distinct_per_evaluation. Report: the accepted metric is the readout's; a query without a target reports none.
… topological order, copies keyed on root limits, solver tolerance from the smallest limit; sign condition in the DDSketch guarantee (#1207 PR 8, review round 1) - AccuracyRules::compose is (K)'s rule once: selection_meets_targets and the ceilings call it; accuracy_constraints writes it as rows. - Ceilings in one pass in the memo's topological order (no fixpoint). - Context copies are keyed on each output's RootLimits, exactly what (K)'s root rows read; queries differing only in n_q without a whole-result delta are now copies. - HiGHS's feasibility tolerance is min(1e-7, 1e-3 x the smallest positive target limit), at least 1e-10, instead of a fixed 1e-10; scaling the rows by their limits kept the defaults but solved RangeHeavy-8users about 4x slower. - AccuracyTarget::failure_probability_per_output_value (renamed; None without a delta); readout_caps::query_levels is written on it. - CandidateForMemoGroup::required_accuracy, read by (K) and the report; readout_error_metric is crate-private again. - DDSketch's sign condition is in guarantee/ddsketch.rs (unknown failure probability over values of unknown sign), so theoretical_guarantee reads it too; ValueSign::is_one_sign; Stage 2 reads state_column_sign. - AccuracyMetric::COUNT and index; the distinct-bound fold moves to workload/bounds.rs. - accuracy-models.md: constraint (K) and Sign conditions sections. - Regression test: p90, p50 and p90 / p50 sharing readouts meet relative 0.05 only through (K).
…ctly; a target below the solver's precision is an error (#1207 PR 8, review round 2)
zzylol
force-pushed
the
refactor/1207-accuracy
branch
from
November 16, 2026 19:56
b8d6ffb to
593ef41
Compare
…nly when the exact (K) check fails (#1207 PR 8, review round 3) The selection MILP solves at HiGHS's default feasibility tolerances. The selection is then checked exactly against every accuracy target; one that misses a target is solved once more at primal and MIP feasibility tolerance 1e-10 (LEAST_HIGHS_FEASIBILITY_TOLERANCE) and checked again, and a miss after that is AccuracyTargetBelowSolverPrecision as before. Selection reports milp_solve_count (1 or 2).
zzylol
marked this pull request as ready for review
November 16, 2026 20:54
This was referenced Nov 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
todo!()on main: no plan checks a query's composed accuracy at its root. B = p99 / p50 at relative 0.03 may read two readouts each required at 0.02 (each meets 0.03 alone; composed 1.111 × 0.04 = 0.044). DDSketch's quantile guarantee is also taken over values of any sign, and an HLL distinct count never serves a query with a target, even when the source declares its distinct bound.AccuracyRules::compose), and checked at each query output in the metrics its root allows, with the target's δ per output value. DDSketch's quantile guarantee needs values of one sign (guarantee/ddsketch.rs); classic HLL's distinct count is certified over the scanned sources'max_distinct_per_evaluation. The report's accepted metric is the readout's requirement's.Kll { k: 269 }for A andDDSketch { alpha: 0.01 }for B (composed 0.022); with accuracy ignored one KLL (k = 547) answers all three readouts and the report says "no accuracy target (Kll)". On p90, p50 and p90 / p50 sharing readouts at relative 0.05, main reads α = 0.05 everywhere (the ratio composes 0.111); this PR adds α ≤ 0.02 readouts for the ratio.Review guide
main79d99326e. Four commits: the porte65767acb, the round-1 fixesf3e7aab15(see "Round 1 fixes"), the round-2 fixes593ef4163(see "Round 2 fixes") and the round-3 solver-tolerance change9a67a5560(see "Round 3 fixes"). refactor: tools and snapshots over the #1207 memo, edges and contexts (#1207 PR 9) #1239 (PR 9) restacks onto this head.stage3-plan-selection/src/milp/accuracy.rs:AccuracyRules(compose,selection_meets_targets,root_limits),ErrorModel(ceilings),accuracy_constraints(rows).milp/model/mod.rs(solve_selectionwiring: the solve at HiGHS's default tolerances, the exact check of the selection, and the one second solve atLEAST_HIGHS_FEASIBILITY_TOLERANCE= 1e-10),milp/upper_bound.rs(the greedy bound only from a plan meeting (K)),milp/equivalent_contexts.rs(copies keyed onRootLimits).accuracy/src/guarantee/ddsketch.rsandguarantee/mod.rs(sign condition, distinct bound),guarantee/hll/mod.rs(classic HLL certificate),logical_edge/mod.rs(tolerated_limit),readout_caps/mod.rs(query_levelson the per-output-value δ).accuracy/target/mod.rs,stages/logical/candidate/mod.rs,stages/logical/group/,workload/bounds.rs.docs/design_docs/concepts/accuracy-models.md: "Sign conditions" and "Checking each query's target in selection: constraint (K)".query_roots_with_target,tolerated_limitandErrorContract::input_error_reaches_outputmade public for Stage 3;AccuracyMetric::COUNT/indexreplacing two local copies.Round 1 fixes
Review round 1 (abstraction and layering, plus the functional/efficiency review) of
55f0ef67d, fixed inb8d6ffb04(f3e7aab15after the rebase onto79d99326e):AccuracyRules::compose(own error + Σ β·input error, metric rule, validity cap, β = 0 skipped, D = δ + Σ H);selection_meets_targetscalls it with the selected producer, the ceilings with the largest;accuracy_constraintswrites the same rule as rowsmemo.topological_order()(AccuracyRules::executions_producers_first); adebug_assertthat no selectable edge stays in one groupCandidateForMemoGroup::required_accuracy(), read by (K) andaccepted_accuracy;readout_error_metricispub(crate)againstate_column_sign();ValueSign::is_one_signused by the error contracts and DDSketch's guaranteeMETRIC_COUNT,metric_indexcopiedAccuracyMetric::COUNT,AccuracyMetric::indexfailure_probability_per_output_valueisNonewithout a δ; the.andis goneAccuracyRules::own_error_per_executionErrorModelwithOption<&AccuracyRules>solve_selectionbuilds the model only with rules;AccuracyVariables::is_emptydeletedtheoretical_guaranteeignored the signstate_signand returnsNonefor a DDSketch quantile over values of unknown signquery_roots_with_targetcalled twicecopies_in_equivalent_contexts(inputs, kept, rules)keys each output on itsRootLimits(hashable by bits): exactly what the root rows read; queries differing only incontexts_whose_queries_differ_only_in_values_not_split_by_k_are_copies, failing before)failure_probability_per_output_value;readout_caps::query_levelsismin(per_output_value, δ / R), bit-identical to main'sδ / max(V, R); a comment at the H rows on fan-in (see "With #1360")min(1e-7, 1e-3 · smallest positive limit), at least 1e-10 (HiGHS's minimum), set only when below 1e-7: 1e-9 on RangeHeavy-8u, 13.5 s against the port's 12.3 s measured the same way (round-2 review); replaced in round 2 (see "Round 2 fixes"); testk_bounds_a_per_value_failure_probability_below_the_default_solver_tolerancefails without it; comment on #1356guarantee/ddsketch.rs(unknown failure probability over values of unknown sign), applied insideguarantee_before_failure_probability_check; cost-model follow-up commented on #1287 (already open)workload::bounds::declared_max_distinct_per_evaluationbesidedeclared_value_magnitude_range, with why their lookup rules differ;Memo::max_distinct_per_evaluationdelegatesaccuracy-models.mdgains "Sign conditions" and "Checking each query's target in selection: constraint (K)" (variables, rows, constant errors, ceilings, tolerance, copies, example); comments point there; PR numbers removed from new commentsreached_by_query_with_target,selectable_edges_per_input,error_is_constant,ErrorTerm { Constant, Variable { variable, ceiling } },RootLimits::{tolerated_error_per_metric, tolerated_failure_probability_per_output_value}, roots keyed byQueryIdaccepted_metricis the first readout'sa_ratio_sharing_its_readouts_with_its_operands_still_meets_its_target(the reviewer'sk2): fails with (K) disabled ([0.05, 0.05, 0.05])Round 2 fixes
Review round 2 (verification of
b8d6ffb04), fixed in593ef4163:AccuracyRules::selection_meets_targetschecks the selection exactly in f64; a miss isErrorSelectingPlan::AccuracyTargetBelowSolverPrecision { query, failure_probability_per_output_value }, never a plan.selection_meets_targetsreturnsResult<(), QueryId>to name the query. Testa_per_value_failure_probability_below_the_solver_precision_is_an_error(1e-10 and 1e-12 per value) fails without the check: the solver selects(2, 0)feasibility_toleranceandsmallest_positive_limitdeleted. RollupHeavy-8u: 6.7 s against the round-1 rule's 11.5 s and the port's 11.5 s (see Verification). RangeHeavy-8u got slower instead: see "Decisions and open questions"RootPruneReason::NoGuaranteedoc missed two casesh01q / h03n: both have only limits of 0.05 or more, so under the round-1 rule HiGHS ran at its defaults. Their 1.4e-9 and 1.3e-11 gaps are main's behaviour, not (K): the round-2 review made both optimal at tolerance 1e-9 or 1e-10, which this head uses (#1356, likely the MIP feasibility tolerance as the margin for pruning against the best plan found). On
79d99326e, this head and the round-1 rule select the same plan for both (see Verification).Round 3 fixes
Reviewer decision on the solver tolerance (option (c), see "Decisions and open questions"), in
9a67a5560:solve_selectionsolves at HiGHS's default tolerances; the selection is checked exactly (selection_meets_targets); a selection that misses a target is solved once more, from the same HiGHS model, at primal and MIP feasibility toleranceLEAST_HIGHS_FEASIBILITY_TOLERANCE(1e-10) and checked again; a miss after that isAccuracyTargetBelowSolverPrecision, as before. The retry is a loop insidesolve_selection, no new function or typea_selection_missing_its_target_at_the_default_tolerance_is_solved_again_at_the_least_tolerance(p99 / p50, 1e-6 per value, readouts 5.4e-7 or 4e-7: the first solve selects two cheap readouts, 1.08e-6, the check fails, the second selects one of each, 9.4e-7; two solves). It failed before the change withOk((1, 1, 1)): one solve at 1e-10Selection::milp_solve_count(1 or 2), printed byasap_plannerasmilp_solve_count;milp_solve_mssums both solvesa_selection_meeting_its_target_at_the_default_tolerance_is_solved_once(readouts 4e-7 or 3e-7: two cheap meet 1e-6 at the default; one solve)a_per_value_failure_probability_below_the_solver_precision_is_an_errorunchanged in what it asserts: at 1e-10 and 1e-12 per value both solves select two cheap readouts and the error names the query and its per-value δk_bounds_a_per_value_failure_probability_below_the_default_solver_tolerancerenamed to T1's test, which also asserts the solve countThe retry is a real case, not a seam: the 1e-6-per-value ratio (δ = 0.01 over 1e4 values) misses its target at HiGHS's defaults and meets it at 1e-10.
Behavior: before → after
Integration test
accuracy_at_query_roots::each_query_meets_its_target_at_its_root: overlatency(1,000 series, 1 sample/s, ingested, values declared non-negative), both queries every 10 s, no raw data kept. A:quantile_over_time(0.95, latency[1h])at rank 0.01; B:quantile_over_time(0.99, latency[1h]) / quantile_over_time(0.5, latency[1h])at relative 0.03.DDSketch { alpha: 0.02 }builds: composed (0.02 + 0.02) / 0.9 = 0.044 > 0.03DDSketch { alpha: 0.01 }, composed (0.01 + 0.01) / 0.9 = 0.022 ≤ 0.03Kll { k: 269 }Kll { k: 269 }NoGuarantee(unknown failure probability)COUNT(DISTINCT u)from HLL p = 18, relative 0.05, δ 0.01, source bounded at 9,636NoGuaranteemilp_solve_count2AccuracyTargetBelowSolverPrecisionnaming the query and 1e-12; no planInterfaces
crates/types/src/accuracy/target/mod.rs·stages/logical/candidate/mod.rs·stages/logical/group/·stages/logical/memo/mod.rs·workload/bounds.rsimpl AccuracyMetric { + /// ALL's length: the size of an array of one value per metric. + pub const COUNT: usize; + /// The position of `self` in ALL (Relative is 3). + pub fn index(self) -> usize; } impl AccuracyTarget { + /// δ split evenly over the values its scope covers when one evaluation returns + /// `values_per_evaluation` ($n_q$); None without a δ, or for a whole-result δ with + /// $n_q$ unknown. δ = 0.01 over 100 series → 0.0001 per output value. + pub fn failure_probability_per_output_value(&self, values_per_evaluation: Option<u64>) -> Option<f64>; } impl CandidateForMemoGroup { + /// A readout's input requirement, also the error it adds; None for any other candidate. + pub fn required_accuracy(&self) -> Option<RequiredAccuracy>; } impl ValueSign { + /// NonNegative or NonPositive. + pub fn is_one_sign(self) -> bool; } impl GroupProperties { + /// The `column_signs` entry of a state group's state column; None without one. + pub fn state_column_sign(&self) -> Option<ValueSign>; } impl Memo { + /// declared_max_distinct_per_evaluation over the sources scanned below `group`. + pub fn max_distinct_per_evaluation(&self, group: MemoGroupId) -> Option<u64>; } +/// Sum of the declared `max_distinct_per_evaluation` of `scanned`, each looked up as a +/// PromQL selector reads it (a family's bound covers its series); None when one declares none. +pub(crate) fn declared_max_distinct_per_evaluation<'a>( + sources: &BTreeMap<String, SourceFacts>, + scanned: impl IntoIterator<Item = &'a str>, +) -> Option<u64>;crates/accuracy/src/lib.rs·guarantee/mod.rs·guarantee/ddsketch.rs·guarantee/hll/mod.rspub fn theoretical_guarantee( summary_type: &FieldDataType, parameters: &SketchParams, statistic: &SketchStatistic, failure_probability: f64, + state_sign: Option<ValueSign>, + max_distinct_per_evaluation: Option<u64>, ) -> Option<AccuracyGuarantee>; pub(crate) fn guarantee_before_failure_probability_check( ..., failure_probability: f64, + state_sign: Option<ValueSign>, + max_distinct_per_evaluation: Option<u64>, ) -> Option<AccuracyGuarantee>; -pub(super) fn guarantee(params: &SketchParams) -> Option<AccuracyGuarantee>; // ddsketch +/// α, failing never over one-signed values; unknown failure probability otherwise. +pub(super) fn guarantee(params: &SketchParams, state_sign: Option<ValueSign>) -> Option<AccuracyGuarantee>; +/// The largest value `within_limit` admits for `limit` (limit · (1 + 1e-9) + ε); (K)'s rows use it. +pub fn tolerated_limit(limit: f64) -> f64; -pub(crate) fn query_roots_with_target(memo: &Memo) +pub fn query_roots_with_target(memo: &Memo) -> Vec<(usize, MemoGroupId, &AccuracyTarget, Vec<AccuracyMetric>)>; impl ErrorContract { - pub(crate) fn input_error_reaches_output(&self, input: usize) -> bool; + pub fn input_error_reaches_output(&self, input: usize) -> bool; } // crate-private, in `guarantee/hll`: +fn classic_guarantee_at(precision: u8, failure_probability: f64, max_distinct: u64) -> Option<AccuracyGuarantee>; +struct ClassicHllConfidence { max_distinct: u32, relative_error: f64 }crates/stage3-plan-selection/src/milp/accuracy.rs(crate-private) ·equivalent_contexts.rs·upper_bound.rs·report.rsAlgorithm changes
(K), per execution$c$ a targeted query reaches and that is its own original (full statement in
accuracy-models.md, "Checking each query's target in selection: constraint (K)"):tolerated_limit; a constantcompose.selection_meets_targetschecks the selection exactly; a miss is solved once more at primal and MIP feasibility tolerance 1e-10, the least HiGHS accepts, and checked again; a second miss isAccuracyTargetBelowSolverPrecision, never a plan.With no target (ignore mode, most corpus workloads)
AccuracyRules::ofreturnsNoneand nothing is built.With #1360
#1360 (draft, on main) adds
AccuracyTarget::failure_probability_per_readout_value(n_q, ReadoutValueCounts)= δ / min(R, V · fan_in) and rewritesreadout_caps::query_levelson it. This PR addsfailure_probability_per_output_value(n_q)= δ / V (None without a δ) and keeps itsquery_levelschange minimal:min(per_output_value, δ / R), bit-identical to main's δ / max(V, R).How they combine, whichever merges second:
failure_probability_per_readout_valueis defined on the per-output-value one: δ / min(R, V · fan_in) = max(δ / R, per_output_value / fan_in), an unknown term dropping out, and 1 without a δ.query_levelsthen calls it alone (this PR'sminline is replaced).RootLimits::tolerated_failure_probability_per_output_value):RootLimitsand the H rows change, and the copy key followsRootLimitswith no edit.Decisions and open questions
Decided (round 3): the solver tolerance, option (c). HiGHS solves at its default tolerances; the selection is checked exactly; only a selection that misses a target is solved once more at 1e-10 and checked again; a second miss is
AccuracyTargetBelowSolverPrecision. Why: exact correctness is kept by the post-check, not by the tolerance, since no plan that fails the check is ever returned; and the stricter tolerance, which made RangeHeavy-8u 6× slower than the port, is paid for only when the check needs it. Both paper workloads meet their targets at the defaults and solve once.The measurements behind it (round 2; serial, one pinned core, one run each):
593ef4163)Rejected: (a) 1e-10 always (RangeHeavy-8u 76 s, and a dearer plan); (b) primal 1e-10 with MIP default (as fast, but still pays a stricter tolerance on every solve and needs the check anyway). Trade-off accepted: RollupHeavy-8u loses the 6.6 s it had at 1e-10 and plans in about the port's time (see Verification).
Decided (port and round 1):
RootLimits(rejected: disabling sharing whenever a query has a target).plan_coston 75 workloads).query_roots_with_target.-latencyandratecases), not in this PR.Verification
Round 3, on a 56-core host: head
9a67a5560against main79d99326e(the main results are round 2's, same main commit).cargo fmt --all -- --checkandcargo clippy -p asap-types -p asap-accuracy -p asap-stage2-physical-optimization -p asap-stage3-plan-selection -p asap-integration-tests -p asap-devtools --all-targets -- -D warningsare clean.cargo test -p asap-types -p asap-accuracy -p asap-stage0-memo -p asap-stage1-pass2-reuse-candidates -p asap-stage2-physical-optimization -p asap-stage3-plan-selection -p asap-planner -p asap-integration-tests --no-fail-fast(the stage3 crate's lib, the accuracy crate, and the integration testsaccuracy_at_query_roots,memo_value_boundsand the rest), compared by test name:9a67a5560593ef4163joint_search_equivalence×2,materialized_views_equal_the_stored_outputs_main_listed,plan_sharing×2,promql_to_post_asap::promql_quantile_of_rate_binds_kll_over_rate_accumulator, and plannersummary_sharing::identical_ungrouped_queries_share_their_producers.593ef4163, the only differences are the two round-3 tests and the renamed one.Tests that fail before their fix:
a_selection_missing_its_target_at_the_default_tolerance_is_solved_again_at_the_least_tolerance(round 3):Ok((1, 1, 1))with one solve at 1e-10; it also checks that the default-tolerance solve misses (two solves).a_per_value_failure_probability_below_the_solver_precision_is_an_error(round 2): without the exact check the solver selects two cheap readouts(2, 0)at 1e-10 and 1e-12 per value.contexts_whose_queries_differ_only_in_values_not_split_by_k_are_copies,a_ratio_sharing_its_readouts_with_its_operands_still_meets_its_target,guarantee::tests::a_ddsketch_quantile_is_certified_only_over_values_of_one_sign,target::tests::a_whole_result_failure_probability_splits_over_the_output_values(round 1).Exact check on the review workloads (round 2, not rerun). The 88 m/v/h/f/ma workloads of the oracle sets, planned to Stage 3 with
593ef4163and with the round-1 rule (f3e7aab15, same main):f14n_avg_rel_and_count_gtandv14n_near_exact_vs_rank_noraware infeasible on both; 11 more (f01n–f10n,f15n) select no plan on both, before the MILP is solved.Planning time.
asap_planner --run-until stage3with the cost inputs and deployment capabilities of the round-2 review (release builds, one idle pinned core, the four trees alternating, median of 3, host load 3.5–26). The port is55f0ef67don the older maindbc57efbb, so its plan costs come from the older cost model.593ef4163(1e-10 always)593ef4163→ head593ef4163and 1.14× faster than the port.593ef4163(which gained from the 1e-10 MIP tolerance), 1.18× slower than the port, and as fast as main (1.03×), with main's plan. The 16.97 s run was the first, at host load 26.Follow-ups
state_column_sign/is_one_signfor DDSketch's stores.Design scope
All new types and files, by scope (query engine: frontends → IR → Stage 0 memo → Stage 1 → Stage 2 → Stage 3 selection/cost → plan):
milp/accuracy.rs:AccuracyRules,RootLimits,ErrorModel,AccuracyVariables,ExecutionError,Composition,ErrorTermAccuracyRulesis what (K) reads of the memo, shared by the rows, the ceilings and the greedy checkAccuracyRules::compose,root_limitsErrorSelectingPlan::AccuracyTargetBelowSolverPrecision,SelectionMilpError::TargetBelowSolverPrecisionLEAST_HIGHS_FEASIBILITY_TOLERANCE(crate-private const inmilp/model/mod.rs)Selection::milp_solve_count,SolvedSelection::solve_countmilp_solve_ms; any selection with a target may need the second solveguarantee/hll/(classic_guarantee_at,ClassicHllConfidence)mod.rsandtests.rs, as the othersguarantee/ddsketch.rsguarantee(params, state_sign)tolerated_limitwithin_limitand (K) shareAccuracyTarget::failure_probability_per_output_valueAccuracyMetric::COUNT,indexCandidateForMemoGroup::required_accuracyValueSign::is_one_signGroupProperties::state_column_signMemo::max_distinct_per_evaluation,workload::bounds::declared_max_distinct_per_evaluationAcceptedAccuracy::has_accuracy_targettest_support::memo_builders::with_non_negative_valuesintegration-tests/tests/accuracy_at_query_roots.rs