Skip to content

refactor: move the Stage 2 catalog into asap-physical-asap-optimization (#1047 P5b) - #1090

Merged
zzylol merged 1 commit into
mainfrom
refactor/1047-p5b-stage2-catalog
Oct 30, 2026
Merged

zzylol merged 1 commit into
mainfrom
refactor/1047-p5b-stage2-catalog

Conversation

@zzylol

@zzylol zzylol commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

Part of #1047 (step P5b) and #1054 (the Stage 2 catalog moves to the Stage 2 crate).

Why

Stage 2's catalog was built inside the Stage 3 crate (plan-selection/src/search/candidates/). That covers implementing each memo candidate's operators, enumerating timing and materialization, and realizing a selected assignment. #1047 and #1054 decided that Stage 2's code lives in asap-physical-asap-optimization and its output types in asap_types::stages.

What

Move only. Plans are unchanged.

What From To
Catalog data types: CandidateDependencyGraph, PhysicalCandidate(Id), ProducerRequirement, QueryRootCandidates, ResourceBudget, ReaderConstraint, SelectedCandidateGraph, … ; SupplierCandidates; CandidateInventory; CandidateOperatorTemplates; CandidateConfiguration(Inventory) plan-selection/src/search/candidates/{graph,suppliers,inventory,configurations,reconstruction}.rs asap_types::stages (types/src/stages/physical/)
Catalog building: inventory (and factoring), configuration enumeration, execution scopes, structural pruning search/candidates/*, search/refinement/pruning.rs asap_physical_asap_optimization::catalog
Realization: materialize_selected_candidates, reconstruct_selected_candidates search/candidates/{physical,reconstruction}.rs asap_physical_asap_optimization::catalog

Plan-selection keeps the MILP, search and costing code. For now it calls asap_physical_asap_optimization::catalog. #1047 P7 replaces the realization call with an injected realizer and removes that crate dependency.

How

  • Files move, and visibility widens to pub where plan-selection needs it.
  • The catalog's tests stay in plan-selection, because they call the MILP. They reach catalog internals through catalog::test_support, which sits behind a test-support feature that only plan-selection's dev-dependencies enable.
  • The physical crate gains asap-accuracy (reconstruction recomposes guarantees) and rmp-serde (moved key encoding). It has no dependency on logical, plan-selection or cost.
  • One interface change: enumerate_candidate_configurations_before took &DeploymentCapabilities and read two fields. It now takes query_time_results_retained: bool, raw_data_retained: bool (Stage 2 reads raw-data retention to drop query-time configurations of scans).

Key code interfaces added / modified

crates/physical-asap-optimization/src/catalog/mod.rs

/// Build the candidate inventory from Pass 2's memo. Moved (was plan-selection, crate-private).
pub fn build_candidate_inventory_before(/* unchanged */) -> Result<(CandidateInventory, Vec<Vec<PhysicalCandidateId>>), CandidateInventoryError>;
/// Each candidate's allowed timing and materialization configurations. Moved; deployment argument split.
pub fn enumerate_candidate_configurations_before(
    source: &CandidateInventory, output_queries: &[usize], entries: &[QueryWorkloadEntry], data: &DataWorkload,
    query_time_results_retained: bool,   // was deployment: &DeploymentCapabilities
    raw_data_retained: bool,             // was deployment: &DeploymentCapabilities
    deadline: Option<Instant>,
) -> Result<CandidateConfigurationInventory, ConfigurationError>;
/// Structural pruning of the catalog. Moved.
pub fn prune(configured: &mut CandidateConfigurationInventory) -> Result<PruningMetrics, String>;
/// Realize a selected assignment through Stage 2. Moved (Stage 3 still calls it until P7 injects it).
pub fn materialize_selected_candidates(/* unchanged */) -> /* unchanged */;
/// Rebuild IR from a selection. Moved.
pub fn reconstruct_selected_candidates(/* unchanged */) -> /* unchanged */;

Usage: stage_pipeline --workload … runs each pane-width round as follows:

  1. Pass 2 produces LogicalASAPMemoCandidates.
  2. catalog::build_candidate_inventory_before and catalog::enumerate_candidate_configurations_before (Stage 2) build the catalog.
  3. catalog::prune removes structurally impossible candidates.
  4. Stage 3's MILP selects from the catalog and realizes each proposal with catalog::materialize_selected_candidates.

The selected plan is the same as before.

Major algorithm changes

None.

Before / After

Before, the Stage 3 crate built Stage 2's catalog with crate-private code. After, Stage 2's crate builds it, and its types are shared in asap_types::stages.

Verification

At 4f548e24e (on main 053850110):

  • fmt and clippy --workspace --all-targets -D warnings are clean.
  • cargo tree -p asap-physical-asap-optimization --edges normal --depth 1 lists asap-accuracy, asap-types, rmp-serde, serde, serde_json and thiserror.
  • Tests:
    • asap-types: 148 + 97 + 8 passed, including the 2 moved supplier tests.
    • asap-physical-asap-optimization: 25 passed.
    • asap-plan-selection --lib: 342 passed (344 before, minus the 2 moved).
    • asap-planner: 86 passed. asap-devtools: 106 passed.
    • asap-integration-tests: 372 passed; the 2 known time-budget flakes pass alone.
  • Corpus snapshots (release, --ignored): pass. Plans unchanged.
  • Independent review found no blocking issues:
    • every moved body is the same apart from paths and visibility;
    • the six query-time reads and the one raw-data read map to the right bools;
    • test-support is enabled only in plan-selection's dev-dependencies;
    • only items plan-selection's production code calls (and their return types) are public;
    • no Stage 3 logic moved into Stage 2.

🤖 Generated with Claude Code

…on (#1047 P5b)

Move only. The physical candidate catalog that Stage 2 builds from the
memo, and the realization of a selected assignment, leave the Stage 3
crate:

- Catalog types (CandidateDependencyGraph, PhysicalCandidate(Id),
  ProducerRequirement, QueryRootCandidates, ResourceBudget,
  ReaderConstraint, ProducerConsumerConnection, QueryRootAssignment,
  SelectedCandidateGraph, CandidateSelectionError,
  MINIMUM_RELATIVE_COEFFICIENT, validate_selected_graph, the supplier
  types, CandidateInventory, CandidateOperatorTemplates,
  CandidateConfiguration(Inventory)) move to asap_types::stages.
- Catalog functions (inventory build and terminal-reader factoring,
  configuration enumeration, execution scopes, structural pruning,
  reconstruction and realization of a selection) move to
  asap_physical_asap_optimization::catalog.
- plan-selection keeps RejectedQuerySubgraph, the solver, search and
  costing, and calls the physical crate. Unit tests that select with the
  solver stay in plan-selection and reach catalog internals through the
  physical crate's new test-support feature.

enumerate_candidate_configurations_before takes the two deployment flags
it reads (query_time_results_retained, raw_data_retained) as bools
instead of the logical crate's DeploymentCapabilities, so the physical
crate gains no edge to Stage 1. It now depends on asap-accuracy
(guarantee recomposition), rmp-serde and serde_json.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

optimizer Workload optimization and plan selection refactor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant