Skip to content

test: area B tests pass on the stage-boundaries branch; fix decoded-output validation (plan-selection, planner) - #1110

Merged
zzylol merged 3 commits into
refactor/stage-boundariesfrom
test/m1-area-b
Oct 30, 2026
Merged

zzylol merged 3 commits into
refactor/stage-boundariesfrom
test/m1-area-b

Conversation

@zzylol

@zzylol zzylol commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

Milestone test phase, area B (part of #1108). Merges into refactor/stage-boundaries.

Why

The branch's constituent PRs were checked only with fmt and clippy. This PR makes the asap-plan-selection and asap-planner tests pass, and fixes one regression that P7 introduced.

What

Regression fixed (fix(plan-selection): decoded selections assign each query output once):

  • P7 made validate_selected_graph check selected.to_selected_candidates().
  • That conversion sorts the root assignments by output index and drops the index. So a root assigned to a nonexistent output, or two roots assigned to one output, passed validation. Before P7 both were rejected.
  • The check now requires the assigned output indices to be exactly 0..n before converting. Test: decoded_graph_is_checked_independently_of_solver_values.

Stale expectations from intended changes:

Test Change Fix
kept_pane_profiles_and_scan_classes_match_the_real_evaluator, shared_pane_range_tiers_match_evaluator_for_ingestion_and_kept_refresh P7: "query-time results retained" moved into the Stage 2 realizer the test realizer is built with the deployment's setting
runtime_install an_undeclared_gauges_merged_ddsketch_panes_get_no_relative_bound_through_avg, ignoring_accuracy_guarantees_selects_a_sketch_plan_that_runs (its "without the setting" half) P6b removed the early query pruning that proved these workloads infeasible; P7 removed the status label the tests now assert that Stage 3 selected no plan (no valid candidate, or none by the deadline), and that the failure is not a solver, realizer or input error

Coverage lost: the undeclared-gauge test no longer checks why there is no plan. Today's engine reports NoValidCandidate with no rejections for this workload. #1095's region DP reports pruned candidates with reasons; that is noted on #1095.

How

  1. Each failure was rerun alone.
  2. The "regression" classification was checked by building the commit before P7.
  3. One reference-pricer test that sleeps 1.1 s against a 1 s deadline is a known flake under parallel load; it passes alone.

Key code interfaces added / modified

None (one private validation check).

Major algorithm changes

None.

Before / After

  • Before: 5 failures, including 1 regression: invalid decoded selections passed validation.
  • After: all area-B tests pass.

Verification

Target Passed
plan-selection lib 291 (also with reference-pricer)
plan_selection_settings_json 2
planner lib 11
planner tests/main 81 (3 ignored, the existing #580 ignores on main)

fmt and clippy --workspace --all-targets -D warnings are clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01K1M6bqhAk7WA4MZcfbAoMw

zzylol and others added 3 commits October 30, 2026 17:15
P7 (#1107) made validate_selected_graph check the selection through
to_selected_candidates, which sorts the roots by output index and drops
the index. A root assigned to a nonexistent output (or two roots to one
output) then passed validation. Check that the assigned output indices
are exactly 0..n before converting, as the checker did before P7.
decoded_graph_is_checked_independently_of_solver_values covers it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K1M6bqhAk7WA4MZcfbAoMw
…sults

P7 (#1107) moved the query-time-results-retained capability from the
planning models into the Stage 2 realizer. The two kept-pane pricing tests
still set it only on their deployment, so the test realizer refused
MatAtQueryKeptAcrossEvaluations(Memory). Build the realizer with the
deployment's flag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K1M6bqhAk7WA4MZcfbAoMw
…ot an infeasibility label

Both tests asserted the infeasible_under_configured_model status and the
early pruning's rejection text. P6b (#1105) deleted early query pruning,
which proved these catalogs infeasible, and P7 (#1107) removed that status
from Stage 3's report: today's engine now stops at its budget with no
validated plan. The tests now assert that plan selection selected none
(no valid candidate, or none by the deadline); the ignore-accuracy test's
selected sketch plan is still checked end to end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K1M6bqhAk7WA4MZcfbAoMw
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working testing Corpus, regression, differential, or benchmark tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant