Repository navigation
Conversation
zzylol
force-pushed
the
test/node-cost-terms
branch
from
November 16, 2026 22:35
d6f4a92 to
0868ecb
Compare
zzylol
marked this pull request as ready for review
November 16, 2026 22:35
…unit constants times units (tests only) Adds the twelve tests of node_cost_terms and CostConstant, and the declarations they compile against. The bodies are todo!() until the tests are approved, so eleven tests fail; the JSON-name test passes on the derived serde of the declared enum.
…onstants times units, the charge computed from those terms CostConstant names each per-unit constant of operations_per_unit.rs (values unchanged). The per-unit helpers and estimate_operator now return NodeCostTerms, and node_resource_usage_and_cost computes its formula CPU as those terms at the built-in values, so the charge and its terms cannot drift. A hash aggregate's rounding of its per-row operations stays, charged without a constant. node_cost_terms returns the terms taken before a measured entry or the calibration factor applies.
zzylol
force-pushed
the
test/node-cost-terms
branch
from
November 17, 2026 00:47
0868ecb to
32aac84
Compare
…'s summation order; terms-named helpers; no test-only wrappers NodeCostTerms carries the charge at the built-in values, added in the order each formula adds its terms, so every node's CPU is the same f64 as before the terms existed and tied plans no longer flip on summation order. The exact-distinct insert terms are divided by the rows as a whole (divided_by), and the hash aggregate attributes its per-row terms to their constants (attributing_to_constants) instead of back-solving the rounding. Helpers returning terms are named *_terms*; estimate_operator, merge_operations_per_state, merge_output_operations and CostConstant::ALL are deleted (tests enumerate the constants from their JSON names). The extension guide says a new per-unit constant needs a CostConstant variant.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
NodeCost), so refitting one per-unit constant from measured node times needs a separate copy of the formulas that parsesNodeCost::detailstrings.asap_cost::node_cost_termssplits a node's formula CPU per execution into named per-unit constants (CostConstant) times units of work, plus the operations no constant charges (NodeCostTerms). The formula CPU ofnode_resource_usage_and_costis now computed from those same terms, so the charge and its terms cannot drift; the terms also carry the charge added in each formula's own order, so every node's CPU is the samef64as onmain. No constant value changes, and no plan changes:asap_planner --run-until stage3 --format textis byte-identical tomainon 47 workloads (timings masked).max_over_time(latency[5m])over 1,000 series at 1 Hz is 300,000 ×HashAggregateOperationsPerRow(16) + 300,000 ×AggregateMeasureOperationsPerRow(4) + 30,000 ×OperationsPerGroupPerRawBatch(170) + 1,000 ×OutputRowOperationsPerGroup(2,500) = 13.6 M, the formula's charge. With 140 per group visit it is 12.7 M.Review guide
mainat1760a69f5. Not stacked. Three commits: the approved tests, the implementation, then the final review fixes.crates/cost/src/node/cost_terms.rs(the types), thennode_usage_and_cost_termsincrates/cost/src/node/usage/mod.rs(each operator arm builds terms;chargedturns them into the estimate's CPU).operations_per_unit.rs: each per-unit helper now returnsNodeCostTermsinstead off64. The formulas are the same; check each arm's constant and units.NodeCostTerms::operations_at_built_in_values(private):plus,plus_terms,timesanddivided_byupdate it in the order the formula is written, sooperations()is the formula's ownf64(e.g.rows × (update + carried label) + visits × 170), whileoperations_with(..)re-adds the units per constant (equal up to summation order, ≤ 1e-12 relative).usage/mod.rs:estimate_operator_with_termscharges rows × ⌈per-row operations⌉ without a constant;attributing_to_constantsmoves the per-row terms times rows onto their constants and leaves the rounding without one, keeping the charge unchanged.hash_aggregate_terms_per_row: "at least one operation per row for the measures" became an explicit term without a constant (max(1 − measures, 0)).pub(crate)(the four exact-distinct shape parameters stay private); helpers returning terms are named*_terms*;estimate_operator,merge_operations_per_stateandmerge_output_operationsare deleted (callers useestimate_operator_with_terms(..)?.0andmerge_terms_per_state(..).operations()).Behavior: before → after
Before:
node_resource_usage_and_costreturns 13,600,000 CPU operations per execution for the aggregate above, and its detail string "20 ops per row, 30000 group visits".After: the same charge and detail. In addition,
node_cost_termson the same node returnsand
operations_with(CostConstant::operations_per_unit)is 13,600,000. This serves the measurement harness'svalidatestep, which fits constants from node profiles × terms.Plans:
asap_planner --run-until stage3 --format text(default deployment) on the 47 workloads of the final review (the planner-layering examples,random-mixed-seed-862(-long), thememo-examplesPromQL and SQL sets, and the rangeheavy / rollupheavy evaluation workloads includingrangeheavy,rangeheavy-8users,rangeheavy-8users-counterandrollupheavy) prints byte-identical text tomainonce the build and solve times are masked. The previous head summed each node's CPU term by term and flipped between tied plans (objectives equal to 7 digits) on those four workloads; the charge now keeps the formula's summation order.Interfaces
crates/cost/src/node/cost_terms.rs(new),crates/cost/src/lib.rsKept although unread in this repository:
node_cost_terms,NodeCostTermsandCostConstant's serde names have no reader in the planner itself; their consumer is the measurement harness'svalidatestep, which reads a node's terms through the engine adapter and keys refitted constants by these JSON names.Crate-internal changes (not public API):
Usage (test 9 checks these numbers):
Design scope
crates/cost/src/node/cost_terms.rsasap-cost,node), besideusageandoperations_per_unitCostConstantoperations_per_unit.rs; the key a checked-in reference calibration will use. The four exact-distinct cache and size parameters are not variants: they shape units, they do not charge themNodeCostTermsnode_cost_termsnode_resource_usage_and_cost's inputs minus the rate;NodeCostdoes not carry the termsAlgorithm changes
None. Each formula is the same expression, now written as units per named constant; the charge is those units at the built-in values, added in the formula's order (the same
f64asmain). A hash aggregate'sceilof its per-row operations stays, its rounding charged without a constant (test 6).Decisions and open questions
operations_without_constant;NodeCostdoes not carry terms (free function only).crates/cost/src/node/usage/tests.rs. This PR merges first; the other is rebased after, keeping both blocks.Verification
cargo nextestis not installed on the build machine, so the tests ran withcargo test.cargo fmt --all --checkis clean, and so iscargo clippy -p asap-cost -p asap-accuracy -p asap-stage1-pass2-reuse-candidates -p asap-devtools --all-targets.Commands, run on this branch and on
main(1760a69f5), compared by test name:maincargo test -p asap-cost(lib)the_charge_adds_terms_in_the_formula_s_order,attributing_to_constants_keeps_the_charge)cargo test -p asap-accuracy(lib)cargo test -p asap-stage1-pass2-reuse-candidates --libcargo test -p asap-devtools --lib rqe_atomic_costscargo test --release -p asap-devtools --test corpus_snapshots -- --include-ignored(includessketch_plans_match_snapshots)cargo test --release -p asap-integration-tests --test integration -- stage3_objective default_cost_plan_rankingNo plan and no snapshot changes. At the final-review head,
asap_planner --run-until stage3 --format textprints byte-identical text tomainon all 47 final-review workloads (build and solve times masked);mainitself is deterministic on them under the same masking. Thecargo test --releaserows above (corpus snapshots, integration) were run at the earlier head; the final-review commit rerancargo test -p asap-cost --lib,cargo clippy -p asap-cost --all-targetsandcargo fmt --all --check. Planning time is unchanged within noise (median of 3 runs, one core:rangeheavy67.9 ms onmain, 63.3 ms here;w01-q3-q884.8 ms and 85.8 ms).Final review fixes
NodeCostTermsnamed*_terms*(update_terms_per_row,hash_aggregate_terms_per_row,exact_distinct_value_terms_per_row,aggregate_measure_terms_per_row,estimate_terms_per_group,chain_emit_terms_per_state,pass_through_merge_terms,carried_label_terms_per_row)estimate_operator,merge_output_operations);merge_operations_per_statedeleted too,plan/cost.rsusesmerge_terms_per_state(..).operations()CostConstant::ALLdeleted; the tests enumerate the constants from their JSON-name list, still checked against the constants ofoperations_per_unit.rsattributing_to_constantsmoves the per-row terms onto their constants, leaving the rounding5f080872bextend-asap-aware-mapping.mdstep 5: a new constant needs aCostConstantvariant and its match armrangeheavy,rangeheavy-8users,rangeheavy-8users-counter,rollupheavy: main's summation order restored (the terms carry the charge in formula order); byte-identical text tomainon all 47 workloadsBTreeMapFollow-ups
CostConstant.