Repository navigation
Conversation
This was referenced Nov 10, 2026
zzylol
force-pushed
the
test/1253-t0-strategy-doc
branch
from
November 10, 2026 02:43
399034d to
2eb3209
Compare
zzylol
force-pushed
the
test/1253-t1-corpus-builders
branch
from
November 10, 2026 02:43
9ada1ac to
083e2e3
Compare
…lders that plan through the production frontends and Stage 0; snapshots read the registry (#1253 T1)
zzylol
force-pushed
the
test/1253-t0-strategy-doc
branch
from
November 10, 2026 13:37
2eb3209 to
525bf2d
Compare
zzylol
force-pushed
the
test/1253-t1-corpus-builders
branch
from
November 10, 2026 13:37
083e2e3 to
dcc5f2f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
integration-tests'fixtures::lower_promqllowers without the series identity column the production path adds); and the PromQL-or-SQL lowering and the planning-model setup were copied inasap_plannerandplanner_fuzz.asap_devtools::corpus_registry: one list of 68 workloads, each with its deployment (capabilities, cost inputs, accuracy enforced or not), and builders that plan only through the production path (plan_workload,repeating_promql_workload).asap_planner,planner_fuzzand the registry now share one lowering (planning_stages::lower_workload) and one model setup (with_executor_summaries+planning_models);planner_fuzz's tests read their seeds from the registry; the snapshot tests andmilp_objective.rsread the registry.Review guide
test/1253-t0-strategy-doc), stacked on refactor: tools and snapshots over the #1207 memo, edges and contexts (#1207 PR 9) #1239. One commit, head083e2e31e. T2 stacks on it.crates/devtools/src/corpus_registry.rs(CorpusWorkload,plan_workload,corpus_workloads, one private function per source), thenplanning_stages.rs(lower_workload,with_executor_summaries,planning_models).asap_plannerandplanner_fuzz::Plannercall the shared lowering and model helpers instead of their copies;FuzzWorkload::set_arrival_from_sourcescallsarrival_from_sources;corpus_snapshots.rsandsketch_plan_snapshots.rsloop overcorpus_registry::o11y_bench()/sketch_plans();corpus.jsonloses its always-emptyasap_planner_args;planner_fuzz.rs'sseed(name)readsfuzz_seeds/<name>from the registry.plan_workloadmust plan asasap_plannerdoes. It now shares the binary's lowering and model setup (one copy), and runs Stage 3 through the facade'srun_planning(run_search_overwithinspect), asplanner_fuzzdoes. A MetricsQL workload is now an "unsupported query language" error in all three, not a SQL parse error.Behavior: before → after
A contracts test wants every in-repo workload planned for its deployment:
corpus.json,manifest.json(with its deployment argument lists),index.json(sets, cost model names), two directory scans, and the fuzz seed file parsercorpus_registry::corpus_workloads()?: 68CorpusWorkloads, ids such aso11y_bench/q21,sketch_plans/ratio-of-quantiles-1h-mvp,random_mixed/random-mixed-seed-862/workload-01asap_planner's lowering and model setup, or run the binaryworkload.plan(RunUntil::Stage3)?:PlannedWorkload { entries, data, stages }, with the binary's own lowering and models; o11y q21 plans in 0.29 s (debug) at objective 0.9020/s, asasap_plannerprintsrepeating_promql_workload(&["sum(rate(m[5m]))"], 60_000, AccuracyTarget::exact(), &[("m", SourceFacts { series_count: Some(10), sample_interval: Some(DurationMs(1000)), ..SourceFacts::default() })]), thenplan_workload(.., RunUntil::Pass1)It serves #1253's goal 2 (one corpus registry read by contracts, snapshots and fuzz) and the rule that test inputs go through the production lowering.
Interfaces
crates/devtools/src/corpus_registry.rscrates/devtools/src/planning_stages.rscrates/devtools/tests/corpus_snapshots/o11y_bench/corpus.json: the fieldasap_planner_args(always[]; the registry rejected anything else) is removed; every o11y query is planned on the default deployment.Usage (as T2 runs it, checked by
planning_until_a_stage_leaves_the_later_ones_emptyand the contracts):Algorithm changes
None.
Decisions and open questions
Needs reviewer decision
corpus_registry, inasap-devtools, which bothdevtoolsandintegration-teststests already depend on). Example:corpus_workloads()?.iter().filter(|w| !w.id.starts_with("random_mixed/"))is T2's fast-tier list. Alternative: a test-only crate; rejected for now as one more crate for one module.sharing_issue_fixtures(they declare their SQL table in the test), notasap_plannerCLI tests' inline workloads, nottools/dag-viewer/test-fixtures/(a viewer fixture, not a planning input). No ASAPWorkloads workload files exist in-repo (onlyplanner_fuzzgenerator patterns). The memo examples are planned on the default deployment (raw data retained, accuracy enforced), as the viewer opens them; two of them are also insketch_plans/under the MVP deployment.plan_workloadplans with the executor's summary families when the deployment declares none, asasap_plannerandplan_installabledo;milp_objective.rsused every family before. Both its tests still pass.Decided
PlannedWorkload.cost_inputsdropped (callers have the input); typedAccuracyTargetandSourceFactsarguments instead of JSON values; per-source functions private except the two the snapshot tests use; oneaccuracy_guarantees_ignoredparse for the manifest and the index (no new enum);plan_workloadkept besideCorpusWorkload::plan.fixtures::lower_promql/run_all_stagesare deleted in T3 (Tracking: restructure ASAPPlanner tests around layers and independent oracles #1253's T3 line says so).Independent review, round 1: findings fixed
plan_workloadwas a copy of the binary's setup: one copy now;asap_plannerand the registry callplanning_stages::lower_workload,with_executor_summariesandplanning_models.planner_fuzz::Planner::lowercallslower_workload(andplanning_models);FuzzWorkload::set_arrival_from_sourcescalls the registry'sarrival_from_sources; the registry's ownlower_workloadcopy is gone.every_source_loads_with_unique_idsasserts each source's count (o11y from its committed snapshot files, a listing the registry does not read; sketch_plans 10, planner_layering 4, memo_examples 7, random_mixed 30, fuzz_seeds 3) and five named ids.Scanis reached,selectionisErr(NotRun), andphysical_asap_candidatesisNone.corpus_snapshots.rsplans onlyo11y_bench: new fast testthe_corpus_directory_holds_only_the_o11y_benchmark.asap_planner_args: removed fromcorpus.json, the registry check andplanner-snapshots.md.plan_workload(.., Stage3)runsrun_search_over(.., PlanSearchSettings { inspect: true }), the facade'srun_planning.lower_workloadmatchesSQL(_) | DataFusionand returns "unsupported query language" otherwise (a_language_without_a_frontend_here_is_reported_as_unsupported).planner_fuzz's tests read their seeds from the registry (fuzz_seed_workloads).Skipped
fast_corpus_workloads()/ aslowfield: the tier split stays in T2 (one place, by source).lower_sql_rootsbuilding a tokio runtime per call: once per workload, negligible at 68 workloads.Verification
asap-devtoolslibasap-devtools --test devtoolsworkload_examples_select_valid_plans_with_linked_materialized_views,the_nine_query_workload_lowers_and_installs);planner_fuzztests read the registry's seedsasap-devtools --test corpus_snapshots(debug)the_corpus_directory_holds_only_the_o11y_benchmarkasap-devtools --test corpus_snapshots --release -- --include-ignoredasap-integration-tests --test integrationmilp_objectiveplans through the facadecargo fmt --check;cargo clippy -p asap-devtools -p asap-integration-tests --tests --bins -D warningsNew tests:
every_source_loads_with_unique_ids: every source loads with its expected count, ids are unique, named workloads are present.a_repeating_promql_workload_arrives_at_its_sources_rates: 1,000 series per 1 s plus 100 per 10 s arrive at 1,010 rows/s.planning_until_a_stage_leaves_the_later_ones_empty: planning until Pass 1 keeps the PromQL series identity on every scan (at least one) and leaves Pass 2 to Stage 3 empty (NotRun).a_language_without_a_frontend_here_is_reported_as_unsupported: a MetricsQL workload fails as an unsupported language.manifest_entries_load_with_their_deployments(replacesmanifest_entries_build_their_arguments): each entry loads with the capabilities and cost inputs of its deployment's files and its accuracy setting.the_corpus_directory_holds_only_the_o11y_benchmark: a second corpus directory fails until it is added to the registry and the snapshot test.Planning time per registry workload (debug, to Stage 3): 0.02–7 s; each random mixed workload about 7 s.
Follow-ups
fixtures::lower_promql, which drops the series identity, andplanning::run_all_stages), moving their callers to these builders.asap_planner --run-untilbelow Stage 3 still runsrun_stages_until, a copy ofrun_planning's stage order; Move asap_planner's stop-after-stage into run_planning #1169 moves the stop point intorun_planning.