Skip to content

test: sketch-execution tests plan with guarantees ignored after Pass 1's accuracy pruning - #1118

Merged
zzylol merged 1 commit into
refactor/stage-boundariesfrom
fix/m1-after-pass1-pruning
Oct 30, 2026
Merged

zzylol merged 1 commit into
refactor/stage-boundariesfrom
fix/m1-after-pass1-pruning

Conversation

@zzylol

@zzylol zzylol commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

Part of #1108's CI fix. Merges into refactor/stage-boundaries.

Why

After #1116, CI's integration binary failed 7 tests. Pass 1 now prunes summaries that no reading query can use for its accuracy target. These tests relied on such summaries.

What

All 7 are stale expectations; #1116 has no over-pruning bug.

  • Confirmed by bisection: at feat: Pass 1 prunes summaries no reading query can use for its accuracy target #1116's parent all 7 pass, and at bd46e688c all 7 fail.
  • Stage 3 already rejected these summaries. At the parent, Stage 3's accuracy check rejected every one of them for the reason Pass 1 now uses. The tests only passed because they read the reference enumeration, which lists memo choices whether or not Stage 3 accepts them.
Test Why the summary has no usable guarantee for its target
filtered_aggregates::hand_filtered_kll_compiles_and_executes, kll_quantile_of_all_null_group_is_null a grouped KLL's per-value guarantee under a whole-result target, with an unbounded group count
per_computation_candidates_sql::a_filter_or_group_key_over_an_estimate_has_no_guarantee the output has no guarantee, which is exactly what the test asserts
promql_to_post_asap::planner_heap_topk_reference_execution_matches_ground_truth, planner_only_e2e_temporal_topk_preserves_query_update_and_evaluation_contract a heap-backed top-k has no guarantee
sql_to_post_asap::sql_filter_keeps_read_predicate_and_summary_population_selection, sql_relational_parents_retain_summary_bound_aggregate a value filter over an estimate has no guarantee

These tests check compilation and execution of sketch plans, not accuracy. They now plan with guarantees ignored, through a new helper, planning::logical_candidates_ignoring_guarantees. The original helpers keep the accuracy check for every other test.

How

The bisection and Stage 3's rejection reasons are recorded above. No assertions were weakened, and no #[ignore] was added.

Key code interfaces added / modified

asap_integration_tests::planning::logical_candidates_ignoring_guarantees (test support).

Major algorithm changes

None.

Before / After

Verification

  • cargo test -p asap-integration-tests --test integration passes: 362 passed, 0 failed, 24 ignored.
  • fmt and clippy --workspace --all-targets -D warnings are clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01K1M6bqhAk7WA4MZcfbAoMw

…ng guarantees

#1116 makes Pass 1 prune a summary no reading query can use for its
accuracy target when guarantees are enforced. Seven integration tests
read the reference enumeration under the built-in strict model to check
the structure or execution of a summary whose output Stage 3 already
rejected at that target before #1116: a heap-backed Top-K and a value
filter or group key over an estimate have no guarantee, and a grouped KLL
quantile's per-value guarantee meets no whole-result target over groups
the data does not bound. Pass 1 now prunes those summaries, so the tests
found no candidate.

The tests now plan with every output accepted
(logical_candidates_ignoring_guarantees, ignore_accuracy_guarantees),
as they check rewrites and execution, not accuracy acceptance.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K1M6bqhAk7WA4MZcfbAoMw
@zzylol zzylol added the testing Corpus, regression, differential, or benchmark tests label Oct 30, 2026
@zzylol
zzylol merged commit 9ab7404 into refactor/stage-boundaries Oct 30, 2026
@zzylol
zzylol deleted the fix/m1-after-pass1-pruning branch November 3, 2026 02:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing Corpus, regression, differential, or benchmark tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant