[zzylol]

Part of #879

Why

Missed reuse and missed sharing in the planner (a query priced higher than a cheaper equivalent plan the rules never generate) are found today only by reading plans by hand. Each one shows up as a cost that should not go up, or should not go down by less, when the workload is changed in a known way. Those relations can be checked mechanically on generated workloads.

What

Adds planner_fuzz, a metamorphic fuzz tester in the asap-devtools crate. For a generated or seeded workload it builds a related workload (looser accuracy, a filter on a group key, a coarser grouping, a nested shorter window, a duplicate query, an added query, a renamed or reordered workload, a disjoint copy, and so on), plans both with the same search, and checks the relation between their costs per second. A violation is shrunk to a small workload and written as a draft finding with the exact stage_pipeline command that reproduces it. No planner behavior changes.

How

• generate.rs draws PromQL and SQL queries and workloads from a seed, or mutates a --seed-workload file.
• relations.rs defines each relation: derive builds the related workload, evaluate plans both sides and returns holds, violated, or inconclusive. Marginal relations compare the extra cost of a query with its stand-alone cost; monotonic relations compare the two totals. Tolerance is max(relative x cost, absolute).
• shrink.rs removes queries and simplifies each remaining query while the violation still holds.
• report.rs deduplicates violations into classes, writes a per-seed summary and one draft per class.
• Each case runs in a worker process (the same binary with --worker) with a timeout, so a hang or crash in the planner becomes a counted outcome.
• sql_catalog moves from stage_pipeline into planning_stages so both tools build SQL tables from --table the same way.

Key code interfaces added / modified

crates/devtools/src/planner_fuzz/mod.rs

/// Which search plans each workload; mirrors stage_pipeline's choices. New.
pub enum SearchKind {
    Legacy,          // new: the default search plan_installable runs
    MemoBruteForce,  // new: the reference search over the memo
    Exhaustive,      // new: every generated candidate, up to EXHAUSTIVE_MAX_COMBINATIONS
}

/// How the tester plans and judges. New.
pub struct FuzzSettings {
    pub search: SearchKind,                // new: search used on both sides
    pub enforce_accuracy_guarantees: bool, // new: Stage 3 rejects missed targets, default true
    pub relative_tolerance: f64,           // new: default 0.01
    pub absolute_tolerance: f64,           // new: default 1e-6
}

/// Plans and prices fuzz workloads, caching plans. New.
pub struct Planner { /* ... */ }
impl Planner {
    pub fn new(settings: FuzzSettings) -> Self;                                  // new
    pub fn plan(&mut self, workload: &FuzzWorkload) -> Result<Rc<Planned>, String>; // new
}

crates/devtools/src/planner_fuzz/relations.rs

/// One metamorphic relation between a workload and a derived one. New.
pub enum Relation { LooserAccuracy, FilterOnGroupKey, CoarserGrouping, NestedShorterWindow, OtherQuantileRank, ExactDuplicate, OuterOverResult, OtherStatistic, OtherRepeatInterval, SmallerTopK, UnionOfSelections, TighterAccuracy, AddedQuery, NoRawRetention, ResultsRetained, RelaxedLatency, Renamed, Reordered, DisjointCopy } // all new
/// Builds the related workload for one target query, or None when the relation does not apply. New.
pub fn derive(relation: Relation, workload: &FuzzWorkload, target: usize) -> Option<Derived>;
/// Plans both sides and judges the relation: holds, violated, or inconclusive. New.
pub fn evaluate(planner: &mut Planner, check: &Check) -> Outcome;

crates/devtools/src/planning_stages.rs

/// The SQL catalog the repeated --table JSON declarations describe. Was private in stage_pipeline.
pub fn sql_catalog(tables: &[String]) -> Result<SqlCatalog, String>; // moved

CLI, crates/devtools/src/bin/planner_fuzz.rs (new):
planner_fuzz --seed <u64> --cases <n> --time-budget-s <s> [--relations <name|letter>,...] [--search legacy|memo-brute-force|exhaustive] [--seed-workload <file>]... [--ignore-accuracy-guarantees] [--relative-tolerance <f64>] [--absolute-tolerance <f64>] [--max-shrink-attempts <n>] [--case-timeout-s <s>] --out <dir>

Usage

# Check relations a (looser accuracy) and b (filter on a group key) on 5 generated
# workloads from seed 1, using the default search; results go under --out.
cargo run -p asap-devtools --bin planner_fuzz -- \
    --seed 1 --cases 5 --time-budget-s 120 --relations a,b --out /tmp/pf-usage
# prints: 5 cases, 20 checks in 5 s; ...; one summary line per relation
#   a-looser-accuracy   holds 5  violated 0  inconclusive 5  crashed 0
# --out holds summary.md / summary.json and findings/<id>-<relation>.md, one draft per
# violation class, each with the stage_pipeline command and the base and related
# workload files that reproduce it.

Before this PR

Missed-reuse bugs were found by hand. There was no way to ask whether a planner change made some related workload cheaper than the one it was derived from.

After this PR

planner_fuzz runs the relations over seeded random workloads or the committed eval workloads, shrinks each violation, and writes reproducible drafts. The crate README (crates/devtools/src/planner_fuzz/README.md) holds the relation tables, how the derived cost is computed, and the coverage against the design doc. Three regression tests re-find known planner issues from committed fixtures.

Verification

On the rebased head, with asap-devtools built where HiGHS builds:

• cargo nextest run -p asap-devtools: 84 of 84 passed, including the 9 planner_fuzz integration tests, the 9 library tests, and the 46 stage_pipeline tests.
• cargo clippy -p asap-devtools --all-targets -- -D warnings and cargo fmt --check: clean.
• The usage command above was run on the head: 5 cases, 20 checks.
• Screenshots and measurements: not applicable.

Limitation: relations are checked on cost per second only; findings are drafts for a person to review and are not filed automatically.