Repository navigation
Conversation
…clusion with the capability table (tests only) Tests for harness PR P1, reworked so that a target's capabilities and the caller's own exclusions expand into one list of CandidateExclusions that Stage 2 applies as one filter. Adds the dimensions an external target needs: a query-time aggregate's measure, state merged across windows versus built within one evaluation, and counter reset handling and extrapolation. Stubs only, so the tests compile: every new function body is todo!(), validate does not check excluded_candidates, Stage 2 and Stage 3 do not read the new fields. OperatorKind moves unchanged from asap-cost to asap-types (re-exported by asap-cost) so exclusions can name operators.
…clusions apply as one Stage 2 filter DeploymentCapabilities::exclusions expands every capability a deployment lacks, and the caller's excluded_candidates, into CandidateExclusions with their source. Stage 2 prunes each matched alternative as ExcludedByDeployment(source); Stage 3 reports no plan left as RejectionCause::ExcludedByDeployment. Deletes PruneReason::RawDataNotRetained, QueryTimeResultsNotRetained, the Pass 1 supported-summaries filter, DeploymentCapabilities::supports and summary_supported.
…te the p90 KLL fixture
Pass 1 now offers every summary the planner builds, so the KLL-only
fixture's memo also holds the DDSketch readout and states that Pass 2
prunes; Stage 2's reasons for capabilities read as
{"excluded_by_deployment": source}. The viewer renders that reason, and
the capabilities echo lists the new defaulted fields.
zzylol
force-pushed
the
test/deployment-exclusions
branch
from
November 16, 2026 23:49
94e966e to
aa17a02
Compare
…ble workload reported as Infeasible Only exclusion sources the default deployment does not have make the rejection ExcludedByDeployment, so planning with the defaults reports what it reported before; a source that pruned one candidate reads "(1 candidate)".
zzylol
marked this pull request as ready for review
November 16, 2026 23:56
…sions - Pass 1 skips a summary whose sketch algorithm a summary_family-only exclusion from DeploymentCapabilities::exclusions() rules out outright (CandidateExclusion::family_excluded_outright), so restricted deployments no longer plan every summary; plans are unchanged. - The ExcludedByDeployment rejection keeps the costing-failure suffix, through a helper shared with infeasible. - validate refuses exclusions whose fields match no operator in common. - CandidateDescription::of_plan_node moves to the integration test. - Schema::summary_state_type replaces a duplicate lookup; DeploymentExclusion::candidates is renamed exclusion; the viewer reads summary_type; a PR number is dropped from a comment.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
rate(c[5m])was planned as storedRatewindows on a target that cannot merge them, and the benchmark harness patched an environment-variable hook into Stage 3.DeploymentCapabilitiesis the one place a target's restrictions live.DeploymentCapabilities::exclusions()expands every capability a target lacks, and each candidate the caller excludes, intoCandidateExclusions. Stage 2 applies them as one filter (PruneReason::ExcludedByDeployment(source)). Stage 3 reports no plan left asRejectionCause::ExcludedByDeployment.rate(c[5m])every 60 s over 1,000 counter series, a target without mergeable rate state gets query-timeSummaryAgg(Rate)→FinalizeExactAccumulatorat 0.367671 per second, instead of stored windows at 0.279284. A target without counter extrapolation gets no plan, and the rejection namescounter_not_extrapolated (7 candidates). The default capabilities select the same plans as before.Part of #1370. This is harness PR P1 of #1317's design, with
CandidateExclusionunified with the deployment capability table as #1370 asks. The Stage 2 placement and the tests were approved before implementation.Review guide
mainat1760a69f5; headceb78acf4. Not stacked. Five commits:InfeasibleintoExcludedByDeployment;validaterefuses self-contradicting exclusions, and smaller cleanups (see "Review fixes" below).crates/types/src/deployment_capabilities/mod.rs:exclusions()andvalidate.crates/types/src/deployment_capabilities/exclusion.rs:CandidateExclusion::matches,invalid_reason,family_excluded_outright.crates/stage1-pass1-per-computation-candidates/src/computation_targets.rs:without_excluded_families.crates/stage2-physical-optimization/src/candidate.rs: the one filter inoffered_physical_properties.crates/stage3-plan-selection/src/report.rs:excluded_by_deployment, and its two call sites inlib.rs.pass1_memo(memo, None)→pass1_memo(memo, &[])in tests; the facade and the devtools stage runner pass&capabilities.exclusions().asap-cost's privatemeasure_namebecameAggIntent::kind_nameinasap-types.tools/dag-viewer/test-fixtures/p90-kll.jsonwas regenerated with the README command (see Verification).main(refactor: tools and snapshots over the #1207 memo, edges and contexts (#1207 PR 9) #1239) removed the viewer'sREJECTION_CAUSESlist, and with it the rejection-cause half ofviewer_contract.rs(the test is nowviewer_knows_every_view_refresh). The tests commit's addition to that half is dropped, since no viewer list is left to check.assert_excluded_by_deploymentindeployment_exclusions.rsstill asserts that the cause is not a failed check.Behavior: before → after
The user's input: a workload with one counter query, planned for a target that cannot keep rate state across windows.
state_not_mergeable_across_windowsis an unknown field, so the file is refusedScan→ ingestionSummaryTumblingWindowChain(Rate)→TimeRange→SummaryMerge→FinalizeExactAccumulator, 0.279284 per secondScan→TimeRange→SummaryAgg(Rate)→FinalizeExactAccumulator, 0.367671 per secondExcludedByDeployment(StateNotMergeableAcrossWindows){"counter_semantics": {"extrapolation": false}}NoValidCandidate([Rejection { cause: ExcludedByDeployment, reason: "… counter_not_extrapolated (7 candidates) … may be infeasible without …" }]){"excluded_candidates": [{"timing": "query_time", "operator": "Scan"}, …]}(the sweep's "stored" arm)ASAP_SB_FORBIDpatched into Stage 3sum_over_time(m[5m])reads an ingestion-time chain of exact sums, at 0.113237 instead of 0.060534 per secondRejection cause under MVP capabilities (
{"raw_data_retained": false, "query_time_results_retained": false}, whichplan_installabledefaults to): 5 of the 77 sketch-heavy workloads have no plan.mainreports them asInfeasible; this PR reports them asExcludedByDeploymentwith the reasonquery 0 has no candidate left to answer it; excluded by the deployment: raw_data_not_retained (4 candidates); the workload may be infeasible without these exclusions too(forrepeat-rate-and-l2…, the MILP form:no plan is left (the selection MILP is infeasible); …). The 5 arepromql-entropy__r4-10hz-24h…,sql-entropy-client-ip…,sql-l2-client-ip…,control-l2-16-resolvers…andrepeat-rate-and-l2…. When some execution could not be costed, the reason ends with the same; executions left out because they could not be costed: …thatInfeasiblegives (none of these 5 has one).This serves the measurement harness, whose sweep arms and target profiles need typed restrictions. It also serves #1370's capability-limited targets.
Interfaces
Every declaration below is in this PR, bodies elided.
crates/types/src/deployment_capabilities/mod.rs(wasdeployment_capabilities.rs)crates/types/src/deployment_capabilities/exclusion.rs(new)crates/types/src/summary_type/family.rs(new) andparameters.rscrates/types/src/ir/operator/kind.rs(moved fromcrates/cost/src/measured.rs, which re-exports it)crates/types/src/stages/pruning.rsandcrates/stage3-plan-selection/src/report.rspub enum PruneReason { NoIngestionPipeline, - RawDataNotRetained, - QueryTimeResultsNotRetained, InputHasNoBuildableProducer, ReachesNoQueryOutput, CountMinWeightsNotProvenNonNegative, + /// Stage 2: the deployment's exclusions match it; names their source. + ExcludedByDeployment(ExclusionSource), DominatedBy(PhysicalCandidateId), UsageNotCostable, } pub enum RejectionCause { AccuracyTargetMissed, NoAccuracyGuarantee, LatencyBoundExceeded, LatencyOverRepeatInterval, OverClusterCapacity, CostNotRepresentable, Costlier, Slower, MoreMemory, TiedLostOnEnumerationOrder, Infeasible, + /// No plan is left without the deployment's exclusions. The reason names the + /// query or the infeasible MILP, then each source with its pruned count, e.g. + /// `counter_not_extrapolated (7 candidates)`, and says the workload may be + /// infeasible without them too. Not a failed check. + ExcludedByDeployment, }crates/planner/src/lib.rs: re-exportsCandidateExclusion,CounterSemantics,CounterSemanticsSupport,ExclusionSource,SketchReadout,SketchParameterRange,SummaryFamily.crates/types/src/ir/operator/agg_intent.rs,crates/types/src/ir/schema/mod.rsandcrates/stage1-pass1-per-computation-candidates/src/lib.rsimpl AggIntent<ColumnId> { + /// The measure's JSON `kind`, e.g. `"rate"`; moved from `asap-cost`'s + /// private `measure_name`, which keys measured operation costs by it. + pub fn kind_name(&self) -> String; + /// Whether `name` is some measure's kind (`"rate"` yes, `"rte"` no). + pub fn is_kind_name(name: &str) -> bool; } impl Schema { + /// The summary state type this schema carries (its first non-plain + /// field), e.g. `Sketch(Kll)`; Stage 2 reads it for a readout's state. + pub fn summary_state_type(&self) -> Option<&FieldDataType>; } -pub fn pass1_memo( - memo: Memo, - supported_summaries: Option<&[SupportedSummary]>, -) -> Result<Memo, PerComputationCandidateError>; +/// Pass 1 skips a summary whose sketch algorithm one of `exclusions` rules +/// out outright (`family_excluded_outright`); Stage 2 prunes everything else +/// the deployment cannot run. +pub fn pass1_memo( + mut memo: Memo, + exclusions: &[DeploymentExclusion], +) -> Result<Memo, PerComputationCandidateError>;Removed by this PR (the
-lines above):PruneReason::RawDataNotRetainedandPruneReason::QueryTimeResultsNotRetained. They are nowExcludedByDeployment(ExclusionSource::RawDataNotRetained)andExcludedByDeployment(ExclusionSource::QueryTimeResultsNotRetained).without_unsupported_summariesand thesupported_summariesparameter ofpass1_memoandcomputation_targets. Pass 1 now reads the sameexclusions()list (without_excluded_families), and only to skip sketch algorithms excluded outright; Stage 2 prunes unsupported readouts (ReadoutNotSupported) and everything narrower.DeploymentCapabilities::supportsand the free functionsummary_supported. Their only callers were the Pass 1 filter and one installable test, which now readssupported_summariesdirectly.asap-cost's privatemeasure_name, moved toAggIntent::kind_name.tests::targets::unsupported_summaries_are_not_offered, which tested the deleted filter.exact_only_capabilities_offer_no_summaryreplaces it for the new Pass 1 skip; Stage 2'sunsupported_summaries_and_readouts_are_prunedcovers the Stage 2 side.Wire change: a Stage 2 pruned candidate's
reasonfor these capabilities is now{"excluded_by_deployment": "raw_data_not_retained"}(or{"excluded_by_deployment": {"excluded_candidate": {"index": 0}}}) instead of"raw_data_not_retained". The viewer shows it as "excluded by the deployment: raw data not retained".Usage (what the integration tests run):
Design scope
deployment_capabilities/exclusion.rs:CandidateExclusion,CounterSemantics,SketchReadout,CandidateDescriptionasap-types), read by Stage 2DeploymentCapabilities::{counter_semantics, state_not_mergeable_across_windows, excluded_candidates},CounterSemanticsSupport,ExclusionSource,DeploymentExclusion,exclusions()summary_type/family.rs:SummaryFamily;SketchParameterRangeSketchParams. A calibration filter can reuse them.ir/operator/kind.rs:OperatorKind(moved)asap-typesneeds it.PruneReason::ExcludedByDeployment,RejectionCause::ExcludedByDeploymentAggIntent::{kind_name, is_kind_name}(moved fromasap-cost's privatemeasure_name)CandidateExclusion::family_excluded_outrightSchema::summary_state_typeWhy the names changed from the approved plan:
SelectionOptionsandselect_plan_withare gone. The exclusions are a deployment input, andPlanningModels::deployment_capabilitiesalready reaches Stage 2.PlanningModels::selection_optionsis not needed.ExcludedByDeployment, notExcludedBySelectionOptions, because one cause now covers both capabilities and caller exclusions.Algorithm changes
Problem: the restrictions were applied in three places with three mechanisms: Pass 1 for summaries, and two separate Stage 2 rules for retention. Nothing could express counter semantics, state mergeability, measures or θ ranges.
Before:
without_unsupported_summaries).Infeasible.After:
DeploymentCapabilities::exclusions()expands everything into one list, in this order:{query_time, Scan}and{Scan, kept_across_evaluations};{query_time, kept_across_evaluations};{family, merged_across_windows: true};excluded_candidates[i].without_excluded_families, run last incomputation_targets) drops eachSummary(algorithm)way, a fused computation's included, whoseSketch(algorithm)family one exclusion of the same list rules out outright (summary_familyset and nothing else). Stage 2 would prune every build of it, so this only saves work. Exact accumulators and every narrower exclusion (readouts, mergeability, θ ranges) are left to Stage 2.offered_physical_properties) describes each offered (τ, ℓ) as aCandidateDescription: its operator, θ and physical properties, and for aSummaryEstimatethe state type of the group it reads. After the Count-Min and no-ingestion rules, it records the first matching exclusion asExcludedByDeployment(source). The existing propagation (InputHasNoBuildableProducer,ReachesNoQueryOutput) removes whatever depended on it.ExcludedByDeploymentrejection when Stage 2 pruned some alternative for a source the default deployment does not have, with a count per source. Otherwise it returns today'sInfeasible.Correctness:
The expansions of the two old retention rules match them exactly:
(query_time ∨ across) ∧ Scan ∧ ¬raw_data_retained;query_time ∧ across ∧ ¬query_time_results_retained.The Stage 2 tests of these prunes pass with only the reason renamed.
The default's only exclusion is the query-time-results one, so planning with the defaults prunes what it pruned before. Stage 3 does not count that exclusion, so the default reports
Infeasibleas before (only_exclusions_beyond_the_default_deployments_make_the_rejection).an_exclusion_matching_nothing_changes_nothingand the unchanged pass/fail sets of every existing suite (see Verification) check this.A matched candidate never reaches Stage 3, so it is never selected:
some_node_matchesis false on every selected plan indeployment_exclusions.rs.The Pass 1 skip keeps the selected plans: across 101 workloads × 5 capability sets (see Verification), all 505 plans, costs, MILP objectives, MILP sizes and Stage 2 physical-candidate counts are identical to the previous head
d9ecc8bd9, which skipped nothing in Pass 1.Complexity: O(alternatives × exclusions) matches in Stage 2, each at most O(measures). The supported-summaries expansion has at most
SketchAlgorithm::ALL.len()entries plus the readouts of the listed algorithms.Limits:
SummaryAgg,SummaryTumblingWindowChain) only. Merges, estimates and finalizations over an excluded build lose their producer.Decisions and open questions
Decided (by the user, before implementation):
supported_summaries. After review, Pass 1 reads the sameexclusions()list to skip sketch algorithms excluded outright (one source of truth, no second filter), which restoresmain's planning time for restricted deployments.PlanningModels::deployment_capabilities.ExcludedByDeploymentis not a failed check.Needs reviewer decision:
aggregate_measureis a string: theAggIntentJSONkind, e.g."rate", asOperatorCost::measurealready is.validaterefuses a name that is no measure's kind;AggIntent::is_kind_nameasks serde whether the tag is a known variant. A typed enum would duplicate the 40 intent kinds.irateandresets, and extrapolation also coversdelta, following PromQL's definitions.aggregate_measurevalidation (review L3, not done):is_kind_namestill relies on serde'sunknown varianterror text. A validating newtypeAggregateMeasureKindwould still need a way to list the measure kinds, which today only that error gives, so it was not small enough for this PR.exclusions(), because the items areCandidateExclusions, and the existingNoValidCandidatewithRejectionCause::ExcludedByDeployment. Plan for a target's deployment capabilities #1370 proposesrestrictions()andNoDeployablePlan.ExcludedByDeploymentrejection and its counts, which keeps Q-P1-3's "no exclusion matched →Infeasible" for default planning. The other way is to list it too, but then every infeasible workload under the defaults would be reported as excluded by the deployment.Verification
Built with the shared build environment.
cargo fmt --all --checkandcargo clippy --all-targetson the changed crates are clean. The changed crates' suites ran at this head and were compared by name withmain1760a69f5(mainhas not moved). No test fails here that passes onmain.maincargo test --no-fail-fast -p asap-types -p asap-stage1-pass1-per-computation-candidates -p asap-stage2-physical-optimization -p asap-stage3-plan-selection -p asap-planner -p asap-integration-tests -p asap-devtoolsmain: plannersummary_sharing::identical_ungrouped_queries_share_their_producers; integrationjoint_search_equivalence×2,materialized_views,plan_sharing×2,promql_to_post_asap; devtoolsplanner_fuzz×3. 47 tests are new; the onlymaintests gone are the renamed defaults test andunsupported_summaries_are_not_offerednode --test tools/dag-viewer/withasap_plannerbuiltPlans unchanged by the Pass 1 skip, and planning time. Both release binaries (
d9ecc8bd9, which plans every summary in Pass 1, and this head) ranasap_planner --run-until stage3over 101 workloads: 24 showcase (21 single-query and joint workloads at 1 Hz/10 s and 10 Hz/60 s, plus viewer examples 1, 3a and 3b) and the 77 sketch-heavy workloads, each under 5 capability sets: CLI default{}, exact-only{"supported_summaries": []}, HLL-only, KLL-only, and MVP. All 505 plans, costs, MILP objectives, MILP sizes, physical-candidate counts and rejection texts are identical. Times are the median of 3 runs, each pinned to one core,--format text:The restricted counts are back to
main's (683 and 711 logical candidates on the sketch-heavy set under exact-only and KLL-only). No planning-time regression test was added: wall-clock tests are not reliable here, andexact_only_capabilities_offer_no_summarypins the candidate count that causes the time instead.End to end, with the CLI (
asap_planner --run-until stage3 --format json, on the rate workload of "Behavior"):main, both capability files are refused:unknown field `state_not_mergeable_across_windows`andunknown field `counter_semantics`.{"state_not_mergeable_across_windows": [...]}selects query-timeScan→TimeRange→SummaryAgg→FinalizeExactAccumulatorat 0.367671 per second.{"counter_semantics": {"extrapolation": false}}gives:no valid candidate: [Rejection { id: "workload", cause: ExcludedByDeployment, reason: "query 0 has no candidate left to answer it; excluded by the deployment: counter_not_extrapolated (7 candidates); the workload may be infeasible without these exclusions too" }].Expected values: every cost and count in the tests commit holds at this head unchanged:
Ratestates are still pruned in Stage 2.)Existing tests whose expected values change, and why:
devtools/tests/asap_planner.rs::deployment_capabilities_flag_sets_the_capabilities_and_shows_them: the echoeddeployment_capabilitiesalso lists the three new fields with their defaults. "Pass 1 offers no DDSketch candidate" under KLL-only capabilities holds again, as onmain.tools/dag-viewer/test-fixtures/p90-kll.json, regenerated with the README command: the memo andstage3_plan_selectionare as onmain; Stage 2's 36 pruned alternatives carry the new reason encoding, anddeployment_capabilitieslists the new fields.memo.test.mjs: a Stage 2 line reads "excluded by the deployment: query time results not retained". Everything else is as onmain.Review fixes (independent review of
d9ecc8bd9)Summary(algorithm)way when asummary_family-only exclusion from the sameexclusions()list matches it (numbers above). Tests written first and seen failing:exact_only_capabilities_offer_no_summary,a_partial_exclusion_keeps_the_summary,an_excluded_exact_family_is_left_to_stage_2,a_family_is_excluded_outright_only_by_a_family_only_exclusion.costing_failuresis shared byinfeasibleandexcluded_by_deployment, so the rejection keeps; executions left out because they could not be costed: …(the_exclusion_rejection_keeps_the_costing_failures). The MVP cause change is listed under "Behavior".validaterefuses an exclusion two of whose fields match no operator in common: family, θ ormerged_across_windowswith an operator other than a state build;aggregate_measurewith one other thanAggregate;readoutwith one other thanSummaryEstimateor with a build-only field;per_entity_reductionwith an operator without a reduction (self_contradicting_exclusions_are_rejected, written first).CandidateDescription::of_plan_nodemoved into the integration test asdescription_of_plan_node. TheCgeneric stays, because that test still describes plan nodes, whose operators name inputs byNodeId.ASAPOp::produced_statecallsSchema::summary_state_type;DeploymentExclusion::candidatesis nowexclusion; the viewer's deployment line readssummary_type(it showedundefined (Quantile)); the movedOperatorKindcomment no longer names a PR.New tests, by purpose:
family_excluded_outrightholds only for a family-only exclusion;aggregate_measure,merged_across_windows, counter semantics and readout;SummaryFamily::ofand θ range bounds.krange moves the state from 547 to 1,404, then 2,863;ExcludedByDeployment;Aggregate[rate], until the measure is excluded too;rateplan andirateis unchanged; without reset handling, neither is planned.plan_installableande2e_plan.maintoo.summary_type, and the new reason reads as words.Follow-ups
excluded_candidates.asap_planner --deployment-capabilitiesreads the new fields, but the stage text does not list the exclusions yet.