[zzylol]

Part of #1047 (step P5c) and #1054 (Stage 2 has one output type and carries no cost).

Why

After P5b (#1090), Stage 2's output was still spread over four types: CandidateDependencyGraph, CandidateInventory, CandidateConfigurationInventory and CandidateOperatorTemplates. Its candidates carried local_cost and resource_budgets, which Stage 3 filled in. #1047 decided that each stage boundary has one named type, and that Stage 2 does not price anything.

What

The change is behavior-preserving: selected plans are unchanged (see Verification).

Today	After
build_candidate_inventory_before + enumerate_candidate_configurations_before + prune	one entry, catalog::generate_physical_asap_memo_candidates
CandidateInventory candidate + its CandidateOperatorTemplates entry	OperatorNodeOfCandidateForMemoGroup, one per operator node
CandidateConfiguration + graph candidate	PhysicalCandidate, one per timing, materialization and scope copy
source_candidate	operator_node
producer_requirements	supplier_candidates_per_input
may_keep_with_sole_reader	may_keep_across_evaluations_per_sole_reader_query
memo_candidate_roots, returned next to the inventory	candidates_for_memo_group on each operator node
group_candidates, graph.query_roots[r]	implementation_roots(group), query_output_candidates(r) (same IDs, same order)
local_cost, resource_budgets, and the cost check in Stage 2's reconstruction	Stage 3's private DagOfPhysicalCandidatesWithCosts (plan-selection), checked by Stage 3 before reconstruction

Deleted:

• the public types CandidateInventory, CandidateConfigurationInventory, CandidateConfiguration and CandidateOperatorTemplates;
• the inventory's node_local_cost callback, with its Cost error;
• ConfigurationError::UnpricedResources.

CandidateInventory remains as a type private to the physical crate.

How

Stage 2 (physical-asap-optimization/src/catalog/):

• builds the two levels;
• runs factoring without costs: its key dropped local_cost, which was always 0 in production;
• returns PhysicalASAPMemoCandidates;
• reconstructs a selection with structure checks only.

Stage 3 (plan-selection):

• builds DagOfPhysicalCandidatesWithCosts from PhysicalASAPMemoCandidates once per round, and lower bounds fill in its costs;
• the MILP, components and refinement read that DAG, so the MILP algorithm is unchanged;
• component search builds a smaller PhysicalASAPMemoCandidates per component, with new, and clones only the operator nodes that component uses.

Shared:

• asap_types::stages::output_references flattens query roots into output references, replacing five copies.
• PruningMetrics moved to asap_types::stages as pruning_metrics, which stands in until P6 adds pruned_candidates.

Key code interfaces added / modified

crates/types/src/stages/physical/memo_candidates.rs

pub struct PhysicalASAPMemoCandidates {
    pub operator_nodes_of_candidates_for_memo_groups: Vec<OperatorNodeOfCandidateForMemoGroup>,
    pub candidates: Vec<PhysicalCandidate>,           // what Stage 3 selects
    pub query_roots: Vec<QueryRoot<MemoGroupId>>,     // passed through from Pass 2
    pub reader_constraints: Vec<ReaderConstraint>,
    pub pruning_metrics: PruningMetrics,              // until P6's pruned_candidates
    implementation_roots_of_group: Vec<Vec<PhysicalCandidateId>>, // built once by new()
    output_reference_groups: Vec<MemoGroupId>,                    // built once by new()
}
impl PhysicalASAPMemoCandidates {
    pub fn new(operator_nodes, candidates, query_roots, reader_constraints) -> Self;
    pub fn implementation_roots(&self, group: MemoGroupId) -> &[PhysicalCandidateId];
    pub fn query_output_candidates(&self, output_reference: usize) -> impl Iterator<Item = PhysicalCandidateId> + '_;
    pub fn memo_group_of(&self, candidate: PhysicalCandidateId) -> MemoGroupId;
    pub fn physical_node(&self, candidate: PhysicalCandidateId) -> &Rc<OperatorNode>;
    pub fn output_reference_count(&self) -> usize;
}
pub struct OperatorNodeOfCandidateForMemoGroup {
    pub candidates_for_memo_group: Vec<CandidateForMemoGroupId>, // several when identical terminal readers are factored
    pub is_implementation_root: bool,
    pub physical_node: Rc<OperatorNode>,
}
pub struct PhysicalCandidate {
    pub operator_node: OperatorNodeOfCandidateForMemoGroupId,
    pub timing: ExecutionTiming,
    pub materialization: Materialization,
    pub supplier_candidates_per_input: Vec<SupplierCandidates>,
    pub may_produce_query_output: bool,
    pub may_keep_across_evaluations_per_sole_reader_query: Vec<bool>,
}

crates/physical-asap-optimization/src/catalog/mod.rs

pub fn generate_physical_asap_memo_candidates(
    logical: &LogicalASAPMemoCandidates, entries: &[QueryWorkloadEntry], data: &DataWorkload,
    query_time_results_retained: bool, raw_data_retained: bool, deadline: Option<Instant>,
) -> Result<PhysicalASAPMemoCandidates, ErrorGeneratingPhysicalASAPMemoCandidates>;

Usage: stage_pipeline --workload … runs each pane-width round as follows:

1. Pass 2 returns LogicalASAPMemoCandidates.
2. generate_physical_asap_memo_candidates returns PhysicalASAPMemoCandidates.
3. Stage 3 builds DagOfPhysicalCandidatesWithCosts from it, fills in costs with lower bounds, and searches with the MILP.
4. Each proposal is reconstructed by Stage 2 from (&PhysicalASAPMemoCandidates, selection).

The selected plan is the same as before.

Major algorithm changes

None. One change in what you see: on a selection that is invalid in two ways at once, Stage 3's cost check now runs before the entry-count check, so the error message can differ.

Before / After

• Before: Stage 3 read four Stage 2 types and wrote its costs into Stage 2's graph.
• After: Stage 3 reads one type, PhysicalASAPMemoCandidates, and keeps its costs in its own DAG.

Verification

At e3079a983 (rebased on main b1f001b6e):

• fmt and clippy: cargo fmt --all and clippy --workspace --all-targets -D warnings are clean.

• Tests:

  • asap-types: 250 passed.
  • asap-physical-asap-optimization: 25 passed.
  • asap-plan-selection --lib: 336 passed.
  • asap-planner: 86 passed.
  • asap-devtools: 106 passed.
  • asap-integration-tests: 374 passed. The 2 known parallel-load flakes (filtered_kll_states_cost_no_more_than_separate_plans, topk_reuse) pass alone.

• Corpus snapshots: pass. They cover only single-query exact plans, so plan equality was also checked directly.

• Selected-plan comparison, with stage_pipeline and accuracy guarantees enforced, compared strictly on all fields:

  • Workloads: 67 runs. They cover:
    • the DAG-viewer and memo examples, including a shared DDSketch pane chain and a ratio of two DDSketch results;
    • the random-mixed seed-862, x5 and long workloads (30);
    • the sharing fixtures of issues 886, 902, 922, 923, 924 and 925;
    • the planner_fuzz seeds;
    • 4 sketch workloads.
  • Results:
    • 66 runs are identical to the base.
    • The remaining one is a search that stops at its limit and is not proven optimal. On reruns, the base and this PR both pick the same plan.
    • A further 27 workloads (including 12 planner_fuzz workloads from seed 1047), base against this PR: all identical.
    • After the review fixes, 40 workloads: identical to the pre-fix outputs.

• Dependencies: cargo tree -p asap-physical-asap-optimization --edges normal --depth 1 lists asap-accuracy, asap-types, rmp-serde, serde, serde_json and thiserror.

• Independent review: no blocking findings. It verified:

  • output-candidate order;
  • factoring equivalence;
  • pairing by operator_node;
  • per-component catalogs;
  • that the cost check still runs everywhere it ran before;
  • that test-support is enabled only in dev-dependencies.

  Its nits are fixed in the second commit: output references are built once, there is one flattening helper, the Stage 3 DAG fields are renamed, memo_group_of, and stale names and comments are cleaned up.

External impact: none outside this repo.

🤖 Generated with Claude Code