Repository navigation
Conversation
zzylol
force-pushed
the
test/1253-t1-corpus-builders
branch
from
November 10, 2026 02:43
9ada1ac to
083e2e3
Compare
zzylol
force-pushed
the
test/1253-t2-contracts
branch
from
November 10, 2026 02:43
5752dbe to
fc6cec8
Compare
… resolve, admissible plan edges, MILP objective = plan_cost (1e-9), non-negative node costs, served query roots, (K) recomputed over the plan (#1253 T2)
zzylol
force-pushed
the
test/1253-t1-corpus-builders
branch
from
November 10, 2026 13:37
083e2e3 to
dcc5f2f
Compare
zzylol
force-pushed
the
test/1253-t2-contracts
branch
from
November 10, 2026 13:37
fc6cec8 to
1bb8557
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
plan_costonly inmilp_objective.rs(two workloads, to 1e-6), no check that the plan's edges are admissible, that every query is served by its own group, or that the selected plan meets (K) when recomputed from the plan rather than read back from the MILP.crates/integration-tests/tests/contracts.rs:contract_violations(&PlannedWorkload, &CostInputs)checks ids resolve (memo, Pass 2 edges and pruned list, Stage 2 candidates and edges, plan nodes, roots), every surviving candidate's inputs and fused groups have a surviving candidate and every Stage 2 candidate's inputs an admissible edge, every plan edge is a Stage 2 edge with α^log = α^phys = 1, MILP objective =plan_costto a relative 1e-9 (a selected plan without a cost per second is a violation), every node cost ≥ 0, every query root is a node of its query's group, and (K) recomputed over the plan as the MILP bounds it; one test per registry source, the 30 random mixed workloads in the full tier, planned in parallel.tier_markers.rschecks everyfull tier:/nightly tier:ignore sits in its tier module.Review guide
test/1253-t1-corpus-builders), stacked on docs(testing): testing strategy, tiers and a measured coverage matrix (#1253 T0) #1254 and refactor: tools and snapshots over the #1207 memo, edges and contexts (#1207 PR 9) #1239. One commit, headfc6cec804. T4 stacks on it.contracts.rs:checked_contracts, thenconstraint_k_holds(the only non-trivial check), thenplan_edges_are_admissible,physical_inputs_have_producersandquery_roots_are_served.stage3-plan-selection/src/milp/model.rs, the query-root loop); the "inputs exist" reading.milp_objective.rschecks its two workloads withcontract_violations(helper renamedplan_meeting_contracts, returns the MILP objective, so the plan is costed once);tier_markers.rs;testing.mdnames the files and describes the (K) check.Behavior: before → after
cargo test -p asap-integration-tests --test integration contracts::on the #1207 stack head:plan_costcheckedmilp_objective.rs, relative 1e-6)accuracy_at_query_roots.rschecks θ of one example<workload id>: (K) holds: query <q>'s root node <n> has error <e> in <metric>, its target allows <ε>It serves #1253's acceptance item "every corpus workload passes the contracts".
Interfaces
None outside the test binary.
contracts::contract_violations(planned: &PlannedWorkload, cost_inputs: &CostInputs) -> Vec<String>ispubinside theintegrationtest binary only, for regression tests in that binary (milp_objective.rsuses it);cost_inputsare the ones the workload was planned with (T1 droppedPlannedWorkload.cost_inputs).Algorithm changes
(K) recomputed over the selected plan (test only)
accepted_accuracy_per_query, which is read from the MILP solution: a wrong encoding would agree with itself.constraint_k_holds), concepts doc §4, "Query composition and root checks":(metric, max_error)and δ, as the MILP reads it (only a readout's input carries one); a second input with a required accuracy is a violation.LogicalRequirements::error_propagation(memo, c.operator, m);Noneis a violation (metric rule on the selected path); an error abovevalidity_cap_per_input[i]is a violation; else addfactor_per_input[i] × ein m. Failure probabilities add (union bound). A node read on two paths contributes on both, as the MILP's worst case.failure_probability_per_value(n_q)only when the target sets a δ and the root is an operator root (no limit without a δ, or for a whole-result δ with n_q unknown). Slack 1e-9 relative.error_propagation(the operators' contracts) but not the MILP's E/F/D/H variables oraccepted_accuracy. It is a consistency check (testing doc).Decisions and open questions
Needs reviewer decision
prunedreads input groups and fuses groups that each keep a candidate not inpruned, and no logical edge joins a pruned candidate; every Stage 2 candidate expands a surviving logical candidate and each of its inputs has an admissible Stage 2 edge (holds on all 68 workloads). Alternative: only that input group ids resolve (weaker).Decided
plan_costat relative 1e-9 (task brief); it holds on all 68 workloads.mod full_tier,#[ignore = "full tier: …"]), planned in parallel withstd::thread::scopeinside the test.ContractViolationenum);ComposedErrorkeeps itsVec<(AccuracyMetric, f64)>; the source prefixes come from T1'scorpus_registry::SOURCES.Independent review, round 1: findings fixed
failure_probability.and(failure_probability_per_value(n)), skipped whenNone. Regressiona_target_without_a_failure_probability_bounds_no_failure_probability(the ratio-of-quantiles sketch plan workload with δ removed composes δ = 2) fails with the old check ("fails with probability 2 per value, its target allows 1") and passes now.PhysicalASAPMemoCandidates::new). The o11y source does not assert it: its plans are exact (no query composes an error), as the snapshots doc says.plan_costhas none is a violation.plan_meeting_contractsreturns the MILP objective, which the contract checked equalsplan_cost.tier_markers.rs::every_tier_ignore_sits_in_its_tier_module.Skipped
joint_search_equivalence.rs,production_rollup_workloads.rs,rollup_controlled.rs): they are classified with the rest of the integration tests in T3, which edits files the Tracking: implement the optimization formulation (memo candidates, properties, edges y_{u,c,i}, MILP) as named code per stage #1207 stack still changes.Verification
asap-integration-tests --test integrationtier_markerstests, +1 ignored (full tier)... -- --ignored full_tiercargo fmt --check;cargo clippy -p asap-devtools -p asap-integration-tests --tests --bins -D warningsNew tests:
every_{o11y_bench_workload,sketch_plan_workload,planner_layering_example,memo_example,fuzz_seed}_meets_the_contracts,full_tier::every_random_mixed_workload_meets_the_contracts: every registry workload of that source meets every contract.every_registry_source_has_a_contracts_test: the sources tested here equalcorpus_registry::SOURCES.a_target_without_a_failure_probability_bounds_no_failure_probability: a ratio of two DDSketch quantiles with ε and no δ meets the contracts (regression of review finding 1).a_tampered_target_edge_objective_or_root_violates_the_contracts: ε tightened to 1e-9, the rank metric and δ = 1e-12 each violate (K); an inadmissible selected edge violates admissibility and "inputs exist"; an objective 0.1 % aboveplan_costviolates the equality; swapping example 1's two roots leaves its queries unserved.tier_markers::every_tier_ignore_sits_in_its_tier_module(and a unit test of its module reading): everyfull tier:/nightly tier:ignore reason incrates/sits in afull_tier/nightly_tiermodule.Follow-ups