Repository navigation
fix(cost): charge each operation the executor's measured time, so a sketch plan is no longer predicted cheaper than a faster exact plan - #1355
Conversation
a13cdf3 to
f6ad70d
Compare
f6ad70d to
e41cc1c
Compare
e41cc1c to
3c0c826
Compare
… k, output rows and carried labels as measured Rebased onto #1355's head with `column_magnitudes`: the PR's earlier commits (regret cases as failing tests, the fix, three tests kept under the new costs, the round-1 tests and fixes) applied as one, resolved against the base's declared value magnitudes, its no-wildcard rule in `operations_per_unit` and its restructured state-bytes tests. - A DDSketch state is charged the bytes it is expected to hold: the dense store over the span of its own rows under a named exponential spread (`exponential_values_magnitude_ratio`), not the library's worst case. - A KLL merge costs per byte of its input and merged state, a KLL estimate per byte, both fitted on the bytes the planner charges; the merged state is sized for the rows its group's states fold together (`state_input_rows`, carried by Stage 3's `logical_output` with the family). - A hash aggregate, a finalize and every estimate pay for each group's output row; a merge per output group; carried labels per row and label. - A summary state's row holds its labels and its own series identity (`state_row_bytes_beside_state`); `state_bytes_per_row` is the one rule for a state's bytes on an edge, compaction included. - Tests: the regret cases, a KLL merge within 2x of its measured time per series, the smallest offered KLL k selected, structural sharing, a disk-only objective, lognormal and narrow-gauge bands. The declared `column_magnitudes` range does not bound the DDSketch span yet; the next commits add that cap test-first.
… k, output rows and carried labels as measured Rebased onto #1355's head with `column_magnitudes`: the PR's earlier commits (regret cases as failing tests, the fix, three tests kept under the new costs, the round-1 tests and fixes) applied as one, resolved against the base's declared value magnitudes, its no-wildcard rule in `operations_per_unit` and its restructured state-bytes tests. - A DDSketch state is charged the bytes it is expected to hold: the dense store over the span of its own rows under a named exponential spread (`exponential_values_magnitude_ratio`), not the library's worst case. - A KLL merge costs per byte of its input and merged state, a KLL estimate per byte, both fitted on the bytes the planner charges; the merged state is sized for the rows its group's states fold together (`state_input_rows`, carried by Stage 3's `logical_output` with the family). - A hash aggregate, a finalize and every estimate pay for each group's output row; a merge per output group; carried labels per row and label. - A summary state's row holds its labels and its own series identity (`state_row_bytes_beside_state`); `state_bytes_per_row` is the one rule for a state's bytes on an edge, compaction included. - Tests: the regret cases, a KLL merge within 2x of its measured time per series, the smallest offered KLL k selected, structural sharing, a disk-only objective, lognormal and narrow-gauge bands. The declared `column_magnitudes` range does not bound the DDSketch span yet; the next commits add that cap test-first.
… executor measures Selection tests under the default CostInputs: with raw data kept at 1 Hz every 10 s, quantile_over_time(0.99, latency[1h]) selects the exact plan and max_over_time does not build at query time; sum(rate) at 1 Hz every 300 s keeps stored Rate states; stored exact accumulators and stored DDSketch windows stay selected where they measured faster. A ranking test checks that predicted CPU orders the exact plan and each measured alternative as the executor measured them, with the measured CPU per second as fixed expectations and the measured query-time plans as fixtures. Four of the six fail on main.
The built-in formulas charged about one CPU operation (1 ns) per row for every operator, so a DDSketch built from raw rows at query time looked cheaper than the exact plan that keeps and sorts the same rows, while the executor runs it twice as slowly (217 ns against 111 ns per row). `node::operation_counts` holds the executor's measured time per unit, as CPU operations at the default 1 ns per operation: a row kept by a time range (12), a row grouped by a hash aggregate per key (15) and per measure (12, a counter-increase measure 84, an exact quantile plus 5.5 per sort level), a row folded into a state per family (DDSketch 210, KLL 95, exact 40, counter increase 84), a state merged per family (exact 230, counter increase 350, DDSketch 2,500, KLL 7,000), a group read out (KLL 35,000, other sketches 1,000, an exact finalize 40) and a state a chain emits (2,500). A merge over a time range of a chain's windows now takes the states' family from its θ or its output column, so it is charged (and looked up in a measured table) as that family. Tests that used Stage 3's selection to obtain a summary plan now cost summaries free (`summary_work_free`); pinned costs and the disk-tolerance scenario are re-derived at the new scale.
…time; Rate chain merge family The compaction test fails before the fix: the newest window's compaction of stored DDSketch windows over 100,000 series is predicted at 0.2 ms (1 operation per merge) against at least 180 ms of measured merges.
…ew cleanups - Compaction latency charges each merge its state's family merge time (the measured entry, else the merge formula at its bytes per state), not one operation; the formula-merge parameter is gone. - operation_counts is now operations_per_unit; its doc and ResourceSpeed::ms_per_cpu_op define one CPU operation as one ns of the reference executor's work and list the operators still at 1 per unit. measured.rs says the formulas are the built-in fallback a measured table overrides. - rate, irate, increase, deriv and predict_linear measures in a hash aggregate are charged the Rate accumulator's per-row work as an estimate; changes, delta, idelta and resets as plain measures. - ExactKind::tracks_counter_increase replaces the copies in the cost model and Stage 2. - The counters a sketch row updates move to operations_per_unit; summary_state_bytes / sketch_state_bytes return bytes only. - HashAggregate carries its whole per-row cost (keys and measures), computed in one function; all unit constants are f64. - summary_cpu_free (was summary_work_free) lists every sketch family, exact kind and no family, checked by exhaustive matches. - Tests: rollup_controlled compares the KMV rollup with the exact plan's cost; different_exact_percentiles_share_the_input_window checks the shared input node; grouped_filtered_quantile_plans pins today's no-reuse under the default costs (#1358).
…alls and answers Fails before the fix: the query-time Rate chain Stage 3 selects fails every evaluation (counter samples must have strictly increasing timestamps within each group).
At query time a tumbling-window chain reads the scan's lookback and keeps each row in its window's range, the work a TimeRange over the same rows does; it was charged nothing for it, so at 10 Hz with the arrival unknown or at rest Stage 3 preferred a query-time Rate chain over the Rate accumulator. Also note in operations_per_unit that sketch constants measure about twice as high under memory and cache contention.
…hain's range term The compaction test fails before the fix: max_over_time(latency[5m]) at 100 M rows per second with its 5m time range kept at ingestion time is predicted 23,000 ms of compaction, an exact accumulator's merge (230 ns) per raw row, though partial copies of rows are concatenated, not merged. The unit test pins a query-time chain's CPU at the cost level: rows x (update + 12 range) + emitted states x 2,500, so it keeps guarding the range term once the executor runs query-time Rate chains.
…2 cleanups - Compaction latency charges 0 per merge for an ingestion-time node with no state family (raw rows of a Filter or TimeRange, an exact Aggregate's values): the executor merges ingest shards only of summary state columns. The merge time is now a closure, evaluated only for the ingestion-time states a query reads. - EdgeStatistics::bytes_per_row replaces three copies of bytes / rows. - SKETCH_COUNTER_UPDATE_OPERATIONS renamed SKETCH_UPDATE_OPERATIONS_PER_COUNTER; TIME_RANGE_OPERATIONS_PER_ROW's doc names the query-time chain. - Docs: stale sketch_shape references in the extension guide and tutorial; the merge cost wording in the analytical cost design doc; the summary_cpu_free comment says what the exhaustive match enforces.
…t tests carry no chain width Main now gives ProducerOutput a chain_tumbling_window_width_ms field. A TimeRange's windows and a scan's rows are read from no chain (a TimeRange does not keep its input's window), so None.
…costs; KLL tests on a KLL-only executor Main added three stage3_objective tests whose absolute cost bounds were set under the old one-operation unit costs. The bounds are now relative, computed in the test: the shared scan and the @ end() input each cost less than the queries planned alone and have one Scan read by both queries; the one-off queries over years cost less than the same queries answered by ingestion-time views only. With main's Stage 3, summary_cpu_free leaves memory to choose between KLL and DDSketch for the global quantile, and DDSketch now wins; kll_candidates now plans on an executor that builds only KLL, through logical_candidates_ignoring_guarantees_on.
…iant under the wildcard lint Each per-unit rule lists every aggregate measure and sketch, so a new one fails to build instead of getting a default cost. Formatting of the DDSketch arm on the declared-magnitude path.
…bytes; the 10 Hz rate test uses the shared sample helper
…a at rest With data arriving, increase(requests_total[5m]) now selects windows maintained at ingestion time, and the test answers each root at query time from raw samples only, so it answered nothing. The plan and install tests keep the arriving data.
bb78057 to
91913fe
Compare
…; assert the ingestion-time chain separately The new cost model stores the two rates' shared tumbling-window chain at ingestion time when the samples arrive, so the query-time chain test now plans with the samples at rest, where nothing can be built at ingestion time. A sibling test plans the same queries with arriving samples and asserts the chain is built at ingestion time and answers the same rates.
|
Rebased onto Conflicts were only in Validation on the merged tree: |
… k, output rows and carried labels as measured Rebased onto #1355's head with `column_magnitudes`: the PR's earlier commits (regret cases as failing tests, the fix, three tests kept under the new costs, the round-1 tests and fixes) applied as one, resolved against the base's declared value magnitudes, its no-wildcard rule in `operations_per_unit` and its restructured state-bytes tests. - A DDSketch state is charged the bytes it is expected to hold: the dense store over the span of its own rows under a named exponential spread (`exponential_values_magnitude_ratio`), not the library's worst case. - A KLL merge costs per byte of its input and merged state, a KLL estimate per byte, both fitted on the bytes the planner charges; the merged state is sized for the rows its group's states fold together (`state_input_rows`, carried by Stage 3's `logical_output` with the family). - A hash aggregate, a finalize and every estimate pay for each group's output row; a merge per output group; carried labels per row and label. - A summary state's row holds its labels and its own series identity (`state_row_bytes_beside_state`); `state_bytes_per_row` is the one rule for a state's bytes on an edge, compaction included. - Tests: the regret cases, a KLL merge within 2x of its measured time per series, the smallest offered KLL k selected, structural sharing, a disk-only objective, lognormal and narrow-gauge bands. The declared `column_magnitudes` range does not bound the DDSketch span yet; the next commits add that cap test-first.
…g keys, distinct counts and KMV at their measured costs, so the measured-best plan is selected (#1363) * fix(cost): charge DDSketch states their expected bytes, KLL merges by k, output rows and carried labels as measured Rebased onto #1355's head with `column_magnitudes`: the PR's earlier commits (regret cases as failing tests, the fix, three tests kept under the new costs, the round-1 tests and fixes) applied as one, resolved against the base's declared value magnitudes, its no-wildcard rule in `operations_per_unit` and its restructured state-bytes tests. - A DDSketch state is charged the bytes it is expected to hold: the dense store over the span of its own rows under a named exponential spread (`exponential_values_magnitude_ratio`), not the library's worst case. - A KLL merge costs per byte of its input and merged state, a KLL estimate per byte, both fitted on the bytes the planner charges; the merged state is sized for the rows its group's states fold together (`state_input_rows`, carried by Stage 3's `logical_output` with the family). - A hash aggregate, a finalize and every estimate pay for each group's output row; a merge per output group; carried labels per row and label. - A summary state's row holds its labels and its own series identity (`state_row_bytes_beside_state`); `state_bytes_per_row` is the one rule for a state's bytes on an edge, compaction included. - Tests: the regret cases, a KLL merge within 2x of its measured time per series, the smallest offered KLL k selected, structural sharing, a disk-only objective, lognormal and narrow-gauge bands. The declared `column_magnitudes` range does not bound the DDSketch span yet; the next commits add that cap test-first. * test(cost): the declared range caps a DDSketch's span; Max merges and exact chain emits at their measured cost; a few-state KLL merge per group; a merge sized from all its inputs (failing) * fix(cost): cap a DDSketch's span by the declared range; Max merges, exact chain emits and few-state KLL merges at their measured cost; size a merge from all its inputs - DDSketch: the span is the smallest of the rows' exponential spread, the column's declared `column_magnitudes` and the assumed nine decades (`ValueMagnitudeRange::magnitude_ratio`). A gauge declared within [90, 110] is charged 1,248 bytes at any row count (holds 1,168). - An exact Min or Max state merges at 125 operations (measured 113–153 ns), a Sum or Count at 230. - A chain emits an exact state at 1,500 operations (measured 1.3–2.1 µs), a sketch state at 2,500 (`chain_emit_operations_per_state`). - A KLL merge pays 19,000 more per output group: the executor measures a merge of 5 states of 60 rows at 32–33 µs per series where the states' merges are charged 14 (`merge_output_operations` takes the family). - `StateInputRows::merged` sizes a merge's state from every input's rows and states per group, not the first input's. - Tests: #1313's known DDSketch gap in `priced_bytes_agree_with_the_executor` is removed (5,992 priced, 6,288 charged); a declared range never raises the bytes of rows that span less; values spread evenly in log over seven declared decades are under-charged 1.6x (pinned). * test(cost): an ingestion-time chain assigns each row its window; a KLL estimate after updates at its measured time; the rate and increase cells at 10 Hz build at query time; log-uniform DDSketch values over a declared range (failing) - An ingestion-time chain costs a time range's work per row besides its update: a Rate chain measures 87 ns per row at 10 Hz where the same Rate built at query time measures 66-75 and its time range 11. - A KLL quantile read from a state built by updates measures 13.4 us per group (300 rows, k = 547), after a merge 5.9. - `sum by (job) (rate|increase(requests_total[5m]))` at 10 Hz every 300 s with raw data builds the accumulator at query time (measured 0.86 against 0.95 per second for the stored windows). - The DDSketch under-charge is pinned on the case the planner sees: rows known, random order, log-uniform over seven declared decades, charged 0.33-0.70 of what the library holds. * fix(cost): an ingestion-time chain pays a time range per row; a KLL read after updates at its measured cost; time range refit to 11 - A tumbling-window chain keeps each row it reads in its window's range at either timing: at ingestion time it assigns each arriving row its window, charged `TIME_RANGE_OPERATIONS_PER_ROW` as at query time. A Rate chain measures 87 ns per row at 10 Hz where the query-time Rate and its time range measure 66-76 and 11. - `TIME_RANGE_OPERATIONS_PER_ROW` is refit to 11 over 54 time ranges at 1 Hz and 10 Hz (100 to 10,000 series). The counter update (45) and the carried label (37) keep their values: refit on the same points they give 83 per row together (82 charged). Their per-row time follows the sample rate, not the rows per series, so no per-group term is added; the docs say why. - A KLL estimate reads a state built by updates (no merge since) at 1.8 per byte (`KLL_ESTIMATE_OPERATIONS_PER_BYTE_OF_UPDATED_STATE`), a merged one at 0.9: fitted on 11 executor nodes at 100 to 10,000 series, 300 to 36,000 rows per state. - The merge's fallback to the largest input state and the compaction's missing per-group term are documented where they apply. * docs(cost): a declared magnitude range caps a DDSketch's span; the chain's window assignment, the KLL read after updates, Min/Max merges and exact chain emits in the design doc * test(cost): a grouping costs more per row at a lower sample rate; a KLL read after updates is nearly flat in its rows; the identical sum_over_time pair at 1 Hz keeps stored windows (failing) The per-row cost of a per-series hash aggregate, a query-time Rate build and an ingestion-time Rate chain follows the sample rate: each group's state is visited once per raw batch, 10 rows of a series at 1 Hz and 100 at 10 Hz. A KLL estimate over a state built by updates measures nearly the same at 300 and 3,000 rows per state. These replace the test of a time range charged per arriving row. * fix(cost): a grouping pays once per group and raw batch, shared by hash aggregates, summary builds and chains; per-row constants refit at 1 Hz and 10 Hz; a KLL read after updates per unsorted item A new deployment input, ResourceSpeed::raw_batch_width_ms (10 s assumed by default), sets how many raw batches a grouping's input spans. Each group's state is visited once per batch (140 operations), and a chain also splits each group's rows of a batch by window (190) and routes each arriving row at ingestion time (8), replacing the time range charged per arriving row. The per-row constants are refit with the new term over 1 Hz and 10 Hz at 100 to 10,000 series; a hash aggregate's refit part is taken from its measure, so a grouping by several keys keeps its charge per key, and an exact distinct count, top-k or frequency statistic hashes each value as a key. A KLL estimate over a state built by updates costs 40 per unsorted item, at most k, instead of 1.8 per byte; a finalize charges its output row directly. A query-time chain sizes one window's states by the sources' rate, so Stage 3's baseline and plan_cost agree on the states a merge over it folds. * docs(cost): the per group and raw batch visit, the raw batch width input and the KLL read after updates in the design doc and library API; the rate and increase cells explained by the rate-dependent per-row cost * test(cost): batches counted from the time the input spans; a one-state merge passes its state through; keys, measures, exact distinct counts and KMV builds and rollups at their measured costs; the years objective test at a pinned batch width (failing) - a_grouping_counts_raw_batches_from_the_time_its_input_spans and a_query_time_chain_sizes_its_windows_from_the_time_its_input_spans: a filter that keeps a tenth of each series' rows drops no raw batch. - a_merge_reading_one_state_per_group_passes_it_through: KLL and DDSketch merges of one state per group measure about 1 ms per 1,000 groups. - a_hash_aggregate_pays_each_further_key_and_measure_as_measured, an_exact_distinct_count_costs_more_per_row_with_more_distinct_values, a_kmv_build_costs_per_retained_hash_it_compares and a_kmv_rollup_merge_costs_per_hash_it_inserts: SQL aggregates, exact COUNT(DISTINCT) and KMV builds and rollups measured on the executor. - the_objective_is_the_plan_cost_on_one_off_queries_over_years pins raw_batch_width_ms at 30 min, plans with those cost inputs, and checks strictly that keeping raw data costs less than the views. * fix(cost): raw batches from the time the input spans; one-state merges as pass-through; keys, measures, exact distinct counts and KMV at their measured costs; per-row, visit and chain constants refit on a grid - A grouping counts the raw batches its input arrives in from the time its raw rows span (ProducerOutput::raw_rows_span_ms: a scan's lookback, narrowed by a time range, kept by the operators that keep each row's window), and a query-time chain sizes its windows the same way, so a filter between the scan and the grouping drops no batch. - A SummaryMerge that reads one state per group passes each state through and pays only each group's row. - A hash aggregate pays 16 per row for its first key, 28 per further key and 4 per measure (SQL aggregates over 1 to 3 keys, 1 to 4 measures, 100 to 10,000 groups). - An exact distinct count or frequency statistic pays per square root of the distinct values its column takes and per value and pair it inserts (45 executor aggregates up to 3 M distinct values). - A KMV update and merge pay per retained hash they compare a hash with (asap_sketchlib's duplicate check); 22 builds and one rollup. - OPERATIONS_PER_GROUP_PER_RAW_BATCH 170, EXACT 29, COUNTER 33, DDSKETCH 210, KLL 84, CHAIN_OPERATIONS_PER_ARRIVING_ROW 14, refit by squared log error over 245 workloads (1 to 10 Hz, 100 to 10,000 series, 5 min to 1 h, every 60 and 300 s); the loss and the terms chosen by leave-one-setting-out error. The chain's own per group and batch term is deleted: it did not lower that error. - raw_batch_width_ms is the data store's batch width. - The two SQL distinct rollup tests now expect the exact counts, the plan measured cheaper; the Rate chain test takes this run's measurements. * docs(cost): raw batches from the time the input spans, the data store's batch width, one-state merges, keys, exact distinct counts and KMV in the design doc and library API * test(cost): a rollup merges its states when its group estimate ties its input's; a grouped exact distinct count pays its key and visits once; a sort, limit, dedup and concat pass on their raw rows' span (failing) * fix(cost): a rollup merges its states when its group estimate ties its input's; exact distinct terms refit with the key and visits fixed; one merge row 1,100; raw rows' span through sort, limit, dedup and concat - A SummaryMerge passes one state per group through only when it keeps its input's groups (no group_by, or the input's own keys); a rollup whose estimate ties its input's is charged its merges. - The exact distinct count's own terms are refit with the grouping key, the measure, the group visits and the output row fixed at their constants (41 per row, 0.53 per square root of the distinct values, 72 per inserted entry), so a grouped count pays them once. - MERGE_OPERATIONS_PER_OUTPUT_GROUP 950 -> 1,100: lowers the error over every merge on the fitting grid and on the held-out settings; the 10,000-group range is stated in the docs. - A Sort, Limit and Dedup pass the raw rows' span on; a Concat of raw rows spans the longest of its inputs'. - Doc fixes: the validation wording, the KMV fit (28 builds, 63.8 + 0.51), and the distinct rollup test's justification at its own shape. * docs(cost): a rollup merges its states when its estimate ties its input's; the raw rows' span through sort, limit, dedup and concat; the merge paragraph rewrapped
Summary
quantile_over_time(0.99, latency[1h])(1,000 series, 1 Hz, every 10 s, raw data kept) it selects a DDSketch built from raw data at query time and predicts it 1.16× cheaper than the exact plan. The executor runs it at 0.50× of exact: 776 ms against 388 ms per evaluation.ms_per_cpu_op(1 ns): rows kept by a time range, rows grouped by a hash aggregate (with the exact quantile's sort), rows folded into a state, states merged and groups read out, each per state family (crates/cost/src/node/operations_per_unit.rs).Partially addresses #1314: built-in fallback instead of a default table. The formulas carry the reference executor's measured coefficients, and a table measured per machine (#1317) overrides them node by node. Related: #1313, #1324.
Review guide
main(863153e3c, after feat(executor): grouped stddev/variance, corr, group, count_values, exact topk and without grouping #1242, fix(accuracy): relative error through negation, abs, set operators and relabels; readouts compose only through error-carrying inputs; exact accumulators take non-finite values and reject stale markers; finalize sign of counters #1309, fix(planner): give MetricsQL roots the series identity #1339, refactor: no silent defaults for kernel memory, operator rules and aggregate names #1330, feat(types, cost): cost DDSketch state from declared column_magnitudes #1324 and fix(executor): run each node by its plan Lifetime, so roots drain in any order #1300); headbb7805794. Rebased onto863153e3c: see "Rebased onto main 863153e" below (three rebase commits on top); the earlier rebase onto3e434c760(refactor(stage3): select over execution contexts: x_{κ,v}, y_{κ,u,c,i}, h_v and constraint (R) (#1207 PR 7) #1237) is the section after it. Nine commits: the tests first (four fail on main), the fix, a one-line provenance note in the test doc, then review round 1 in two test-first pairs (the compaction test and the fixes; the 10 Hz unknown-arrivalratetest and the query-time chain's time-range term), then review round 2 as one test-first pair (no compaction for nodes holding no states, plus the chain's range term at the cost level; then the fix and cleanups).crates/cost/src/node/operations_per_unit.rs: every measured constant, its source and the functions that apply them; which operators stay at an abstract 1 operation per unit.crates/cost/src/node/usage/mod.rs: where they are applied (Aggregate,SummaryAgg,SummaryTumblingWindowChain,SummaryMerge,SummaryEstimate,FinalizeExactAccumulatorarms) andstate_family/merged_state_family.crates/cost/src/node/analytical.rs:TimeRangeandHashAggregateformulas.crates/integration-tests/tests/default_cost_plan_ranking.rs: the new tests and their measured expectations.merged_state_family(a merge over a time range of a chain's windows now knows the states' family); the hash aggregate chargingrate,increase,derivandpredict_linearmeasures as much as the Rate accumulator (a work estimate); the compaction latency's per-family merge time, 0 for a node holding no states (plan/cost.rs).accumulator_count→operations_per_rowrename inanalytical.rstests;summary_shape/sketch_shape→summary_state_bytes/sketch_state_bytes(bytes only) and their tests; unit tests re-pinned at the new scale (Verification).Rebased onto main 863153e
Base
main863153e3c(was3e434c760); headbb7805794(wase41cc1c5c, kept asbackup/fix/cost-model-measured-unit-ratios-e41cc1c5c). The eleven commits map one to one:c00d20f27→b78767ce4,9cb93dde5→2270a36a2,b4154aa92→e6357ef45,a35a6fafd→7586a8fed,ee4adf4c0→504401db4,52d3f3108→82b607091,e8174a10f→abd697e36,93d53d279→a4c857e14,42d3b7417→7efa31b74,14974f48c→ddace5cfa,e41cc1c5c→9fab257f5. Most conflicts come from #1330 (no silent defaults;clippy::wildcard_enum_match_arminnode::usage) and #1324 (column_magnitudes,StateInputRows::value_magnitude_range, DDSketch bytes from declared magnitudes).crates/integration-tests/tests/main.rs(first commit)mod exact_reduce_execution;and this PR'smod default_cost_plan_ranking;on the same linecrates/cost/src/node/usage/mod.rs(second commit)state_family: #1330 lists every operator instead of_ => StateFamily::None; this PR givesSummaryMergeits own arm (merged_state_familywhen the producer has no family)SummaryMergetaken out of theSummaryMerge | SummaryEstimate | FinalizeExactAccumulatorarm into this PR's arm.merged_state_family's_ => Nonenow lists everyFieldDataType(#1330's lint applies to the module). #1324'sfolded_values(update)(sign and declared magnitude range of the folded values) merged cleanly into theSummaryAggand chain arms next to this PR'supdate_operations_per_rowcrates/cost/src/node/state_bytes/mod.rs(fifth commit)sketch_shapeto takeinput_rows.value_magnitude_range(elseDDSKETCH_ASSUMED_MAGNITUDE_RANGE); this PR renamedsummary_shape/sketch_shapetosummary_state_bytes/sketch_state_bytes, bytes onlysketch_state_bytes's DDSketch arm is #1324's declared-range path, returning bytes only:DdSketch::estimated_memory_bytes(α, rows, min_abs, max_abs, !non_negative)crates/types/src/summary_type/parameters.rs(fifth commit)ExactKind::measureand this PR'sExactKind::tracks_counter_increaseat the same placetracks_counter_increaseis now an exhaustivematch(true forRate,Increase,IRate; false forSum,Min,Max,Count) instead ofmatches!, so a new kind fails to buildThree more commits:
aeed055e7:operations_per_unit.rsgets refactor: no silent defaults for kernel memory, operator rules and aggregate names #1330's#![warn(clippy::wildcard_enum_match_arm, clippy::match_wildcard_for_single_variants)]. Its one wildcard (aggregate_measure_operations_per_row's_ =>) lists every otherAggIntent; the sketch arms ofupdate_operations_per_row,merge_operations_per_stateandestimate_operations_per_groupthat matched any other θ (Some(parameters),StateFamily::Sketch(_)) list everySketchParamsvariant too, so a new aggregate or sketch fails to build instead of getting a default cost. No value changes. Alsocargo fmtof the merged DDSketch arm.3c0c8261a: main's DDSketch byte tests (feat(types, cost): cost DDSketch state from declared column_magnitudes #1324:ddsketch_state_bytes_follow_the_declared_value_magnitude_range,priced_bytes_agree_with_the_executor,ddsketch_costed_bytes) callsketch_state_bytes/summary_state_bytes;runtime_install::sum_rate_at_10_hz_with_unknown_arrival_installs_and_answersuses main's sharedtime_series::sample(metric, &[labels], …)helper (fix(planner): give MetricsQL roots the series identity #1339).bb7805794:metricsql_workloads::metricsql_queries_answer_as_promql_does(fix(planner): give MetricsQL roots the series identity #1339) failed on the branch: with data arriving at 2,000 rows per second,increase(requests_total[5m])now selects stored Increase windows (Scan@Ingestion → chain(Increase, 60 s)@ingestion → TimeRange → SummaryMerge → Finalize) instead of a query-time Increase accumulator, and the test answers each root at query time from raw samples only, so it answered nothing ([] vs [… 20.0, 40.0, 60.0, 80.0]). The two answer tests now plan with the data at rest (AT_REST), which selects the query-time accumulator and checks the same hand-worked answers in both languages;metricsql_queries_plan_and_installandmetricsql_queries_select_the_promql_plankeep the arriving data (INGESTING). The installed plan's answers are not checked in this file yet: see Follow-ups (after test(planner): counter rate and increase answers, installed and at query time #1343).Gate plans before and after this rebase. The 21 gate workloads × (raw data kept, not kept), planned with the plan-survey harness on
e41cc1c5cand on3c0c8261a: all 42 select the same operators with the same timing and summary parameters, all install, and the objectives agree to within 1.3e-11 relative. The only difference is one lifetime annotation inq5_jointat 10 Hz with raw data (a query-timeFinalizeExactAccumulatornow has lifetime none instead ofOneEvaluation(Memory)), from main. #1324's declared magnitudes change no gate plan: no gate workload declarescolumn_magnitudes, so every DDSketch is still sized over the assumed[1, 1e9]range. No selected plan changed, so the gate (measured onf4a2fd59d) was not re-run.Rebased onto main after #1237 (earlier)
Base
main3e434c760(was59365ce80); heade41cc1c5c(wasa13cdf3e1, kept asbackup/fix/cost-model-measured-unit-ratios-a13cdf3e1). The nine commits map one to one:dfe342844→c00d20f27,641997f80→9cb93dde5,008704da2→b4154aa92,07f5046ad→a35a6fafd,335cee809→ee4adf4c0,db31a6364→52d3f3108,f4a2fd59d→e8174a10f,94fbd181a→93d53d279,a13cdf3e1→42d3b7417. Commit references elsewhere in this description are to the old commits.crates/cost/src/node/usage/tests.rs(second commit)an_operator_keeping_its_input_window_over_a_chain_or_its_windows_selects_what_the_range_doesand this PR'sa_merge_over_a_chain_s_windows_is_costed_by_the_chain_s_familyat the same placeProducerOutputgets main's new fieldchain_tumbling_window_width_ms: None(aTimeRange's windows read from no chain)crates/cost/src/plan/cost.rs(fifth commit)reads_one_batch_of(maintained_window_msis deleted on main), this PR'ssketch_state_bytes(wassketch_shape)use crate::node::state_bytes::{sketch_state_bytes, StateInputRows}; use crate::node::usage::{cost_source, reads_one_batch_of};. The rest merged cleanly:plan_costkeeps main'sproducers_of(PhysicalASAPDAGNode::producers(), payload input order), andquery_latencies(dag, nodes, &output, &work_ms, merge_ms)withmerge_ms: impl Fn(PhysicalASAPNodeId) -> f64sits on main's version of the functionTwo more commits:
14974f48c: two later unit tests of this PR (a_merge_over_a_rate_chain_s_windows_is_costed_by_the_rate_accumulator,a_query_time_chain_pays_the_time_range_over_the_rows_it_reads) build aProducerOutput; they set main's newchain_tumbling_window_width_mstoNone.e41cc1c5c: four tests failed after the rebase that pass on main and passed before it. The plans are right; the tests' expectations were set under the old unit costs or relied on a memory tie.stage3_objective.rs: three tests main added had absolute cost bounds set under the old one-operation unit costs (0.144, 0.04, 30,000). The bounds are not re-pinned to new numbers; each is now a relative assertion computed in the test, which holds under both cost models:the_objective_is_the_plan_cost_on_a_shared_scan: the selected plan has oneScanread by both queries, and itsplan_costis below the sum of the two queries'plan_costplanned alone.an_instant_at_end_input_shares_the_evaluation_context: the same two assertions for the@ end()input and the plain query.the_objective_is_the_plan_cost_on_one_off_queries_over_years: the selectedplan_costis below that of the same queries answered by ingestion-time views only (arriving rows, no raw data kept), the kind of plan the test's doc rules out.The three tests also pass on main
3e434c760with the same assertions. New test helpers in that file:scans_and_their_reading_query_counts,assert_one_scan_read_by_every_query,costs_of_queries_planned_alone(test-file scope).filtered_aggregates.rsglobal_kll_quantile_over_no_row_is_null: with every summary's CPU free (summary_cpu_free), memory alone chooses between KLL and DDSketch for the global quantile, and under main's Stage 3 DDSketch now wins, so no KLL candidate was found.kll_candidatesnow plans on an executor that builds only KLL, through a new test helperlogical_candidates_ignoring_guarantees_on(logical_candidates_ignoring_guaranteeswithDeploymentCapabilities; the old function calls it with the defaults). Design scope: a test helper inasap-integration-tests'splanningmodule, next to the function it generalizes.Semantic check of #1237's names: this PR calls none of
least_retention_ms_per_execution,ExecutionCosts::retention_ms_for_need,RefreshWindow::computed_fromor theErrorContracthelpers. Main's Stage 3 costs each candidate throughnode_resource_usage_and_cost, which carries this PR's per-unit operations, so Stage 3 andplan_coststill charge the same; the fourstage3_objectivetests above check that the MILP's objective equalsplan_costunder these costs.Gate plans before and after the rebase. The 21 gate workloads × (raw data kept, not kept), planned with the plan-survey harness on
a13cdf3e1and onf6ad70d69: all 42 select the same operators, with the same timing and summary parameters, and all install; the objectives agree to within 3.4e-9 relative. The only difference is the lifetime annotation of some query-time nodes (OneEvaluation(Memory)against none), which comes from main: on the 14 workloads where main3e434c760selects the same operators, its node lists, lifetimes included, are identical to this head's. No selected plan changed, so the gate (measured onf4a2fd59d) was not re-run.Re-run after the rebase (release build, compared by test name with main
3e434c760):cargo check --workspace --all-targetscargo fmt --all -- --checkcargo clippy -p asap-cost -p asap-types -p asap-executor -p asap-frontend-promql -p asap-stage2-physical-optimization -p asap-integration-tests -p asap-planner --all-targets -- -D warningscargo test --no-fail-fast -p asap-cost -p asap-types -p asap-executor -p asap-frontend-promql -p asap-stage2-physical-optimization -p asap-stage3-plan-selection -p asap-planner -p asap-integration-tests -p asap-devtools --lib --testspromql_to_post_asap::planner_heap_topk_reference_execution_matches_ground_truthfails on main and passes here; the other 64 fail on main toocargo test -p asap-types -p asap-cost --docBehavior: before → after
Workload (the showcase's 1 Hz setting):
{"query_workload": {"language": "promql", "repeating_queries": [{ "query": "quantile_over_time(0.99, latency[1h])", "demand": {"fixed_interval_at": {"interval": 10000, "evaluation_phase": 0}}, "requirements": {"accuracy": {"max_error": 0.01, "failure_probability": 0.01, "accuracy_scope": "per_value", "accuracy_metric": "relative"}}, "predictability": {"predictable": {"known_at": null}}}]}, "data_workload": {"arrival": {"ingesting": {"rows_per_second": 2000.0}}, "sources": {"latency": {"series_count": 1000, "metric_type": "gauge", "column_signs": {"value": "non_negative"}, "sample_interval": 1000}, "requests_total": {"series_count": 1000, "metric_type": "counter", "sample_interval": 1000}}}}plan_installablewith default capabilities (raw data kept) and defaultCostInputs:The exact plan is now predicted at 104 ns per row (12 for the time range, 27 to group a row per series, 65 for the sort of 3,600 values per series), the DDSketch at 222 (12 + 210). The executor measures 111 and 229.
This serves the MVP's correctness need: the planner must not pick a plan that runs slower than exact while predicting it faster.
The measurement
Main
59365ce80, release build, one core of an idle 56-core server, serially. A harness (not in the repository) plans each workload, replays one synthetic data set through every arm, and runs each query-time node of each plan as the only root over the warm stores, timing the sub-plan in thread CPU (median of 5); a node's own time is its sub-plan's minus its inputs'. Data: 1,000 series of a non-negative gauge (latency:ts,value, the series identity) and of a counter (requests_total, with ajoblabel), 1 h windows at 1 Hz (3,600 rows per series, 10 s windows) and at 10 Hz (36,000 rows per series, 60 s windows).rate,irate,increase,deriv,predict_linearmeasure in a hash aggregatechanges,delta,idelta,resetsmeasure in a hash aggregateUnmeasured families (Count-Min, Count Sketch, HyperLogLog, KMV, UnivMon) take the executor's measured cost of handing a row or state to a group's state plus a kernel term scaled from
asap_sketchlib's own per-operation costs, as their docs say.Cause
CostInputs::measured_operation_costsdefaults to empty, so every node is priced from the formulas atResourceSpeed::illustrative()(1 ns per operation).depth(1 for DDSketch, KLL and every exact accumulator), a merge 1 per state, a readout 1 per group (crates/cost/src/node/usage/mod.rs,crates/cost/src/node/state_bytes/mod.rson main). An exact quantile's sort was not charged.Algorithm changes
The analytical CPU of one node (
node_resource_usage_and_cost)TimeRangeAggregate(HashAggregate)rate/irate/increase/deriv/predict_linear84, an exact quantile 12 + 5.5 × log2(rows per group)SummaryAggSummaryTumblingWindowChainTimeRangewould)SummaryMergeSummaryEstimateFinalizeExactAccumulatorThe newest window's compaction latency charges each merge its state's family merge time (the measured entry, else
merge_operations_per_stateat its bytes per state), not 1 operation.The state's family of a
SummaryMergewhose producer has none (aTimeRangeover a chain) is its θ (parameters) or the exact accumulator of its output column (merged_state_family).plan_cost(stage3_objectivetests). Absolute scale stays anchored toms_per_cpu_op: a deployment with a slower core scales every operation together. A measured table still replaces a node's formula entirely. One CPU operation is now defined as one ns of the reference executor's work (ResourceSpeed::ms_per_cpu_opdoc); operators not measured (scan, filter, project, sort, join and the other non-summary operators) still charge an abstract 1 operation per unit, listed in the module doc.863153e3c→3c0c8261a, median of 3, one pinned core, runs alternated between commits (host load 8–31):c1610.12 → 6.36 s (−37%),c09_topk_rate_noacc21.62 → 13.57 s (−37%),defcaps-0311.53 → 1.02 s (−91%), RollupHeavy-8u 16.16 → 9.86 s (−39%), but RangeHeavy-8u 6.70 → 7.73 s (+15%). Before the rebase (main59365ce80→008704da2) c09 was the one that grew (+17%); on today's main it plans faster and RangeHeavy-8u grows instead, still solve-bound (Planning is about 20x slower since the full tumbling-window-width domain (#1185) #1197).59365ce80), on synthetic per-series data; the source scan is not measured (the harness holds raw data in memory) and keeps its formula. A hash aggregate carrying an extra label per row measured 68 ns/row against 27 without; that is not modelled (Follow-ups). DDSketch state bytes are still charged at the 1e9 value-range bound ([Bug] DDSketch memory is charged at the 1e9 value-range bound, about 6.4× what the executor holds #1313).Gate: planned ≤ 1.05× exact (query CPU p50 and total CPU) and error ≤ 1%
Query CPU × is exact ÷ plan query CPU per evaluation (p50), total CPU × the same for query plus ingestion CPU per second; above 1 is faster than exact. main: its earlier run of the same harness on
59365ce80(300 s measured at 1 Hz, 900 s at 10 Hz). This PR: re-run on the review round's headf4a2fd59d(150 s at 1 Hz, 600 s at 10 Hz, one pinned core, host load 2–19). Each row's exact and planned arms run in the same process over the same data, and every exact arm is this commit's own: for the 10 Hzraterows the query-time lowering does not compile, so the exact arm is this commit's exact plan over data at rest, now the query-time Rate accumulator (on008704da2it was a query-time Rate chain that failed every evaluation, #1341, and main's exact plan stood in). No arm failed an evaluation, and no arm exceeds its error target (worst 0.99975% of 1%). Plans:qt= built from raw data at query time,stored= windows maintained at ingestion time,(Max)= an exact accumulator,exact scan= no summary.quantile_over_time(0.99, latency[W])avg by (job) (quantile_over_time(0.9, latency[W]))max_over_time(latency[W])sum by (job) (rate(requests_total[W]))sum by (job) (count_over_time(latency[W]))sum by (job) (sum_over_time(latency[W]))joint of 5 (the three quantiles, max, sum rate)quantile_over_time(0.99, latency[W])avg by (job) (quantile_over_time(0.9, latency[W]))max_over_time(latency[W])sum by (job) (rate(requests_total[W]))sum by (job) (count_over_time(latency[W]))sum by (job) (sum_over_time(latency[W]))joint of 5 (the three quantiles, max, sum rate)quantile_over_time(0.99, latency[W])avg by (job) (quantile_over_time(0.9, latency[W]))max_over_time(latency[W])sum by (job) (rate(requests_total[W]))sum by (job) (count_over_time(latency[W]))sum by (job) (sum_over_time(latency[W]))joint of 5 (the three quantiles, max, sum rate)Ranking accuracy over the 85-case grid
main's planned, no-raw-data and exact plans for the 85 grid workloads, re-costed by each commit's
plan_cost, against the same measurements. Re-run after the rebase: main863153e3c's plans re-costed by main and by3c0c8261a; the counts are unchanged from the run on008704da2:Median measured ÷ predicted total CPU by plan kind: exact 12.97 → 1.55, query-time sketch 35.07 → 1.08, stored exact accumulators 88.28 → 1.00, stored sketches 333.09 → 0.94.
The remaining misranks are exact
sum_over_timeandcount_over_timewithsum by (job)over short windows (1m, 5m), where the exact hash aggregate carries thejoblabel per row (68 ns/row measured, 27 charged), and the joint workload without raw data (predicted 0.76–0.79×, measured 1.01–1.05×).Plan changes
Selected plans, main → this PR, planned with
run_default_search_overand the executor's summaries (asplan_installabledoes); installability byinstallable_plan_of:Re-run after the rebase, main
863153e3c→3c0c8261a. Every status is unchanged: in the 197 set, 152 installable, 26 not installable for reasons this PR does not touch, 16 without a plan, 3 timeouts, the same on both. In the paper sets, 4 installable, 4 without a plan, and 4 lowering errors (the RollupHeavy entries without their table file), the same on both. Before the rebase,n18_count_distinct_groupedwas newly installable only because the new costs steered away from main's plan (#1289); on today's main it installs on both commits.The common changes:
example3b): a DDSketch or KLL built from raw rows at query time costs more per row than keeping and sorting them.sum byover finalized accumulators → plainAggregate(most of the 23 viewer changes): a hash aggregate folds a row in 27 ns, an exact accumulator in 40 plus its finalize.sum by (job) (rate(c[5m]))workloads,max_over_time,min/maxpairs): the query-time scan now costs what it measures, so maintaining windows at ingestion wins more often.Interfaces
Public (
asap-types,asap-cost):crates/types/src/summary_type/parameters.rscrates/cost/src/node/state_bytes/mod.rs(bytes only; the counters a row updates move tooperations_per_unit)Crate-internal (
asap-cost):crates/cost/src/node/operations_per_unit.rs(new,pub(crate))crates/cost/src/node/analytical.rspub(crate) enum PhysicalOperator { HashAggregate { grouping_key_count: u64, - accumulator_count: u64, + /// CPU operations one input row costs, its grouping keys and measures together. + operations_per_row: u64, },(
PhysicalOperatoris crate-private; only itsHashAggregatevariant changes, so the others are not shown.)crates/cost/src/plan/cost.rs(private)fn query_latencies( dag: &PhysicalASAPDAG, nodes: &HashMap<PhysicalASAPNodeId, &PhysicalASAPDAGNode>, output: &HashMap<PhysicalASAPNodeId, EdgeStatistics>, work_ms: &HashMap<PhysicalASAPNodeId, f64>, - merge_ms: &HashMap<PhysicalASAPNodeId, f64>, // measured entries only - formula_merge_ms: f64, // 1 operation + merge_ms: impl Fn(PhysicalASAPNodeId) -> f64, // measured, else its family's merge formula; 0 without a state family ) -> Vec<QueryLatency>;crates/cost/src/node/operator_statistics.rs(crate-internal)impl EdgeStatistics { + /// The bytes of one row, rounded up: one state's bytes on an edge of states (1,000 states of 16,000 bytes: 16). + pub(crate) fn bytes_per_row(self) -> u64; }crates/integration-tests/src/lib.rs(test support)pub mod planning { + /// Default cost inputs with a measured table pricing the CPU of every summary operator of + /// every sketch family, exact accumulator and no family at 0 ns (memory still charged): + /// for tests of a summary plan's shape, execution or reuse. + pub fn summary_cpu_free() -> asap_cost::CostInputs; }Usage: the workload above through
plan_installable: Stage 3 costs each candidate withnode_resource_usage_and_cost. For the exact candidate, theTimeRangenode keeps 3,600,000 rows (3_600_000 × TIME_RANGE_OPERATIONS_PER_ROW= 43.2 ms per evaluation) and the per-seriesAggregatechargeshash_aggregate_operations_per_row(1, [Quantile], 3_600)= 15 + 12 + 5.5 × 11.8 = 92 operations per row (331 ms). For the DDSketch candidate,SummaryAggchargesupdate_operations_per_row(DDSketch)= 210 per row (756 ms). At 0.1 evaluations per second the exact plan costs 39.1 per second against 81.5 for the sketch, andplan_installablereturns the exact plan.default_cost_plan_ranking::quantile_at_1_hz_every_10_s_with_raw_data_selects_the_exact_planruns this. A query reading stored windows then adds the newest window's compaction: with k ingest workers, (k − 1) × states × the family's merge time (default_cost_plan_ranking::compaction_latency_charges_each_merge_its_family_s_merge_time); raw rows or an exactAggregate's values kept at ingestion time are concatenated, not merged, and add none (compaction_latency_is_zero_for_ingestion_time_nodes_holding_no_states).Design scope
crates/cost/src/node/operations_per_unit.rs(new file): scope is one node's analytical CPU in Stage 3's cost model (asap-cost,node), besideanalytical(operator formulas) andstate_bytes(state sizes). It holds the CPU operations a formula charges per unit of work, the measurements behind them, and the counters a sketch row updates; general to every operator and state family, not to one query. Named for what it holds (operations per unit), not "counts". No new type.ExactKind::tracks_counter_increase(new method,asap-typessummary_type): a property of the accumulator kind itself (it keeps first/last samples and resets), read by Stage 2 (row order) and the cost model (update and merge cost); one definition instead of two copies.summary_cpu_free(test support,asap-integration-tests): scope is the planning test helpers; general to any test that needs Stage 3 to select a summary candidate regardless of the default unit costs. Exhaustive matches overSketchParamsandExactKindmake a new family fail to compile in the match; its instance must still be added to the array by hand.EdgeStatistics::bytes_per_row(new method,asap-costnode::operator_statistics): the bytes of one row of an edge, a property of the edge statistics themselves; it replaces three copies ofbytes.div_ceil(rows.max(1))(the merge formula, a merge's output bytes, the compaction merge time).selected_plan(test helper,tests/planning_common): the plan the production stages select for a fixture workload;cost_withnow reads it, and structural sharing tests assert on it.default_cost_plan_ranking.rsand its fixtures (tests): Stage 3 selection under the default cost inputs, end to end; the fixtures are plans the executor was measured on.Decisions and open questions
CostInputs::measured_operation_costs, per machine via [Design] asap_run: one binary to execute plans, calibrate measured unit costs per machine, and bench predicted vs measured #1317's calibration) overrides them node by node; with no table, the formulas carry the reference executor's coefficients. [Design] Price CPU and latency from a measured cost table per state family (scan, update, merge), not built-in unit costs #1314 is partially addressed (a comment there records the changed MVP shape).rate,irate,increase,derivandpredict_linearmeasures in a hash aggregate are charged 84 per row as a work estimate: each reads every sample's time and value in order and keeps running terms, about what the Rate accumulator does per row.changes,delta,ideltaandresetsare charged as plain measures (12) until measured. The executor's missing per-series kernels are not a reason in the cost model: a throwawayplan_installableprobe over 12 queries × (arriving, at rest) × (raw data kept or not) gives identical results before and after this change (rate/increaseinstall via the Rate/Increase accumulator;delta,idelta,deriv,predict_linear,changes,resets,iratefail to install on both), so this change does not make Stage 3 select a plan the executor refuses. The executor gap is Executor: per-series deriv, predict_linear, delta and idelta have no native implementation, so the planner plans them but none installs #1357 (and Executor: irate, resets and changes have no native implementation, so the planner plans them but none installs #1345 forirate/resets/changes).summary_cpu_freestays for tests whose subject is a Stage 1/2 rewrite or execution (filtered_aggregates×3,sql_to_post_asap×2, the heap top-k test).plan_sharing::grouped_filtered_quantile_planskeeps it for the sketch-sharing check and also pins today's behaviour under the default costs (no reuse of the exact by-city result, Pass 2: a filter that pins a grouping key does not reuse an exact grouped quantile (no same-level view match) #1358).Review round 1: fixes
MeasuredOperationCostsmeasured.rsmodule doc,operations_per_unitdoc); body says "partially addresses #1314"; comment on #1314merge_msholds every node's merge time: measured entry, elsemerge_operations_per_state(family, bytes per state)×ms_per_cpu_op×formula_scale;formula_merge_msremoved. Test first (fails on008704da2)ExactKind::tracks_counter_increase, used by the cost model and Stage 2's row-order checkdetailbut cannot be regenerated until the CLI's #1207 PR 9todo!(Follow-ups)summary_work_freemissed Hydra and no family; CPU onlysummary_cpu_free(doc: memory still charged); everySketchParamsfamily, everyExactKindand no family, with exhaustive matches (no wildcard)ResourceSpeed::ms_per_cpu_opandResourceUnitCost::illustrativedefine it as one ns of the reference executor's work; the module doc lists the operators still at an abstract 1 per unit; module renamedoperations_per_unitcounters_updated_per_row(withunivmon_expected_layers) moved tooperations_per_unit;summary_state_bytes/sketch_state_bytesreturn bytes onlyHashAggregateformula split; u64/f64 mixhash_aggregate_operations_per_rowcomputes keys and measures in one place;HashAggregate { operations_per_row }; every unit constant isf64a_merge_over_a_rate_chain_s_windows_is_costed_by_the_rate_accumulator(350 × 5,000)grouped_filtered_quantile_planshides behaviourchanges/delta/idelta/resetsat 12; probe shows no selection change toward refused plans; executor issue #1357rollup_controlled,different_exact_percentiles_share_the_input_windowruntime_install::sum_rate_at_10_hz_with_unknown_arrival_installs_and_answers, fails on335cee809with #1341's error), then12 × rows readadded to the query-time chain's CPUc09_topk_rate_noacc+17% (#1197)n18_count_distinct_groupedpresented as fixedp29_shared_mixed7% slower than main's sketch plan;c05_count_loosepredicted equal, measured 0.87×operations_per_unitdoc: about twice as high on a busy hostagg_has_exact_per_entity_kerneldocReview round 2: fixes
merge_msis 0 for a node with no state family (raw rows of aFilterorTimeRange, an exactAggregate's values). The executor merges ingest shards only of summary state columns (StateColumnUpdater::merge); itsAggregateoperator has no merge of partial results, so an ingestion-time exactAggregateis charged 0 as well, not keyed separately. Test first:compaction_latency_is_zero_for_ingestion_time_nodes_holding_no_statesfails onf4a2fd59d(23,000 ms predicted for a 5m time range kept at ingestion time at 100 M rows per second). No gate plan changes: the 21 gate workloads × (raw data kept, not kept) select identical plans with identical objectives before and aftera_query_time_chain_pays_the_time_range_over_the_rows_it_reads: 36,000 × (40 + 12) + 5 × 2,500sketch_shapereferencesextend-asap-aware-mapping.mdand tutorial_06_extending.mdnamesketch_state_bytesandoperations_per_unitEdgeStatistics::bytes_per_rowreplaces the three copies of bytes per state; the compaction merge time is a closure evaluated only for the ingestion-time states a query reads. A sharedmerge_ms_per_statewas not added: the merge node's measured branch is the generic per-node override (units × ns per unit + one invocation fee, for every operator), not a per-merge time, so only the formula (merge_operations_per_state) is common, and it already is one functionTIME_RANGE_OPERATIONS_PER_ROWdoc names the query-time chain tooSKETCH_COUNTER_UPDATE_OPERATIONS→SKETCH_UPDATE_OPERATIONS_PER_COUNTERformula_scalea13cdf3e1and on main, compared by test name (below)Verification
After the rebase onto
863153e3c(release build, compared by test name with main863153e3c):cargo check --workspace --all-targetscargo fmt --all -- --checkcargo clippy -p asap-cost -p asap-types -p asap-integration-tests -p asap-planner -p asap-stage2-physical-optimization --all-targets -- -D warnings3c0c8261acargo test --release --no-fail-fast -p asap-cost -p asap-types -p asap-stage2-physical-optimization -p asap-stage3-plan-selection -p asap-planner -p asap-integration-tests --lib --testspromql_to_post_asap::planner_heap_topk_reference_execution_matches_ground_truthfails on main and passes here;metricsql_workloads::metricsql_queries_answer_as_promql_doespassed on main and failed here, fixed inbb7805794(cargo test -p asap-planner --test planner metricsql_workloads: 4 passed). The other 7 fail on main too (joint_search_equivalence×2,materialized_views_equal_the_stored_outputs_main_listed,plan_sharing×2,promql_quantile_of_rate_binds_kll_over_rate_accumulator,summary_sharing::identical_ungrouped_queries_share_their_producers)bb7805794cargo test -p asap-cost -p asap-types --lib --testsFunctional check on the rebased head: arrival × rate × window × query
64 workloads on
3c0c8261a:quantile_over_time(0.99, latency[W]),sum by (job) (rate(requests_total[W])),sum by (job) (increase(requests_total[W]))andsum by (job) (sum_over_time(latency[W])), for W in {5m, 1h}, at 1 Hz (every 10 s) and 10 Hz (every 60 s), with arrivalat_rest,ingesting,mixedandunknown. Each ran its exact and selected plans in the same process over the same data (150 s at 1 Hz, 600 s at 10 Hz, one pinned core each, host load 22–43).increase, 1h, 1 Hz, arrivalunknown), but its selected DAG is identical to the exact one; three serial re-runs on a pinned core gave 0.99, 1.00 and 1.00 (total CPU 1.01–1.02), so it was noise.ingestingormixed, the three non-quantile queries select stored exact windows (Rate, Increase or Sum chains read through a merge): 2.1–3.6× faster than exact on total CPU at 1 Hz, and 3.4–41× at 10 Hz.at_restorunknown, they select the query-time accumulator or the exact plan (0.95–1.06×). The 10 Hzrateandincreaserows withunknownarrival run (the rate(c[5m]) / rate(c[10m]) plans but fails at execution: counter samples must have strictly increasing timestamps #1341 path stays unselected).executereject an ingestion scope, so the private harness now runs ingestion withexecute_ingestion. The first sweep's 24 ingesting and mixed failures ("executetakes a query scope") came from the old harness; all were re-run.Before the rebase:
a13cdf3e1and on main59365ce80cargo test --release --no-fail-fast -p … --lib --teststodo!s of #1207 PR 9);promql_to_post_asap::planner_heap_topk_reference_execution_matches_ground_truthfails on main and passes herecargo test --release -p asap-cost --lib;-p asap-integration-tests --test integration default_cost_plan_rankingcargo test --release -p asap-cost -p asap-types -p asap-stage2-physical-optimization --lib;-p asap-integration-tests --test integration -- default_cost_plan_ranking plan_sharing rollup_controlled filtered_aggregates sql_to_post_asap promql_to_post_asap stage3_objective;-p asap-planner --test planner runtime_installplan_sharing::slower_repeat_reuses_faster_sum,plan_sharing::nested_exact_sum_reuses_newest_tumbling_windows,promql_to_post_asap::promql_quantile_of_rate_binds_kll_over_rate_accumulator,runtime_install::check_compiles_reports_the_executors_error) fail identically on008704da2cargo clippy --release -p asap-cost -p asap-types -p asap-stage2-physical-optimization -p asap-integration-tests -p asap-planner --all-targets -- -D warnings(round 2: without stage2, unchanged)cargo fmt --all -- --checkNewly passing: the four
default_cost_plan_rankingtests that fail on main,promql_to_post_asap::planner_heap_topk_reference_execution_matches_ground_truth(failed on main: "no heap candidate"; now selected with summaries costed free), the unit testsa_merge_over_a_chain_s_windows_is_costed_by_the_chain_s_familyanda_merge_over_a_rate_chain_s_windows_is_costed_by_the_rate_accumulator, anddefault_cost_plan_ranking::compaction_latency_charges_each_merge_its_family_s_merge_time(fails on008704da2: "predicted compaction 0.2 ms, measured merges take at least 180 ms"),default_cost_plan_ranking::compaction_latency_is_zero_for_ingestion_time_nodes_holding_no_states(fails onf4a2fd59d: 23,000 ms of compaction for a time range of raw rows), the unit testa_query_time_chain_pays_the_time_range_over_the_rows_it_reads, andruntime_install::sum_rate_at_10_hz_with_unknown_arrival_installs_and_answers(fails on335cee809: "counter samples must have strictly increasing timestamps within each group").New tests (
crates/integration-tests/tests/default_cost_plan_ranking.rs), each with the measured numbers it encodes:quantile_at_1_hz_every_10_s_with_raw_data_selects_the_exact_plan(fails on main): no sketch is selected; the query-time DDSketch measured 77.8 against 39.0 CPU-ms/s.max_at_1_hz_every_10_s_with_raw_data_does_not_build_at_query_time(fails on main): the exact plan or stored Max windows, not a query-time Max (19.1 against 14.6).quantile_at_10_hz_every_60_s_without_raw_data_keeps_a_stored_sketch: still stored DDSketch windows (6.1 against 76.9).sum_rate_and_sum_over_time_keep_stored_exact_accumulators: still stored, at 1 Hz every 10 s and 10 Hz every 60 s.sum_rate_at_1_hz_every_300_s_keeps_stored_rate_states(fails on main): stored Rate windows, measured 8× less CPU and 45× lower latency.predicted_cpu_orders_exact_and_alternative_plans_as_measured(fails on main: 5 of 15 misranked): predicted CPU orders the exact plan and 15 measured alternatives (stored-state plans, and the query-time DDSketch, KLL and Max plans main selected, kept as fixtures) as measured.Re-pinned tests (all pass on main's selections under main's costs; changed because the scale or the selection changed):
runtime_install::shifted_and_unshifted_sums_over_a_shared_query_time_chain_install_and_answer_exactly: withmarriving, stored windows are now selected and its query-time-only harness answers nothing; it now plans overmat rest, where the shared query-time chain is still selected, and checks the same answers.stage3_objective::raw_data_on_disk_costing_less_than_the_solver_tolerance_still_steers_selection: at Fargate's CPU price the stored plans now win whatever disk costs; CPU is priced at half that, so disk again decides (the plan selected ignoring disk is 50% dearer).rollup_controlled::production_distinct_rollup_finds_shared_kmv: still the shared KMV rollup; instead of an absolute budget (2e-6 $/s on main), it now checks the rollup costs less than computing both distinct counts exactly at query time (the exact plan'splan_cost).plan_sharing::different_exact_percentiles_share_the_input_window: the absolute bound 0.0883 is replaced by a structural check: the selected plan has oneScan, at most oneTimeRange, and both percentiles read that one node.plan_sharing::grouped_filtered_quantile_plans: checks the sketch reuse with summaries' CPU costed free (2.1167 together against 2.1164 for the by-city query), and pins that under the default costs the filtered query does not reuse the exact by-city result (2.2742 against 2.1164; Pass 2: a filter that pins a grouping key does not reuse an exact grouped quantile (no same-level view match) #1358), so a fix to Pass 2: a filter that pins a grouping key does not reuse an exact grouped quantile (no same-level view match) #1358 turns it red.filtered_aggregates::*(3),sql_to_post_asap::*(2): usesummary_cpu_freethroughlogical_candidates_ignoring_guarantees.analytical::tests::time_shift_is_free_and_time_range_is_priced_by_the_rows_it_keeps: 12 operations per row kept.End to end: the gate and ranking tables above, measured on
f4a2fd59d(round 2 changes no selected plan in the gate set, so the gate was not re-run) (DF_MEASURE_S150 s at 1 Hz, 600 s at 10 Hz).Follow-ups
deriv,predict_linear,delta,idelta(and per-entityrate/increase), and the stale "Stage 3 rejects" claim inagg_has_exact_per_entity_kernel's doc.city = 'NYC'overGROUP BY city).column_magnitudes; this PR's DDSketch arm uses it. Workloads that declare nothing, the gate's included, are still sized over the assumed[1, 1e9]range, so at 10 Hz every 60 s with raw data kept the quantile keeps the exact plan (gate PASS at 1.0×) although stored DDSketch windows measure 12.7× less CPU.metricsql_workloads::metricsql_queries_answer_as_promql_does_when_installed. It plans each query with the data arriving (INGESTING), answers it through test(planner): counter rate and increase answers, installed and at query time #1343'sinstalled_answers(each tumbling-window chain built window by window in ingestion runs, then the query run reading the bound views), and checks the same hand-worked answers in both languages. The arriving-dataincreaseplan this rebase moved off the answer test is already covered by test(planner): counter rate and increase answers, installed and at query time #1343'scounter_rates_and_increases_answer_from_views_built_at_ingestion_time(increase(requests_total[5m])installed with data arriving at 1,000 rows per second). Merge order test(planner): counter rate and increase answers, installed and at query time #1343 → fix(executor): read an unbound ingestion-time node as at query time, keeping sample times #1344 → fix(cost): charge each operation the executor's measured time, so a sketch plan is no longer predicted cheaper than a faster exact plan #1355 lets this test land in this PR if main gets test(planner): counter rate and increase answers, installed and at query time #1343 first.008704da2,sum by (job) (rate(requests_total[5m]))at 10 Hz with the arrivalunknown(the default) or at rest selected that chain, because a query-time chain was not charged the time range over the rows it reads; the review round charges it, Stage 3 selects the Rate accumulator again, andruntime_install::sum_rate_at_10_hz_with_unknown_arrival_installs_and_answersplans and executes it. The chain stays a candidate wherever it is cheapest, so rate(c[5m]) / rate(c[10m]) plans but fails at execution: counter samples must have strictly increasing timestamps #1341 still needs its executor fix.c05_count_loose: predicted equal to exact, measured 0.87×), the hash aggregate's cost per carried label, and small query-time sketches over-charged (a mixed 5m workload's exact plan measures 7% slower than main's sketch plan, still 1.03× faster than exact).863153e3c, RangeHeavy-8u plans 15% slower (6.70 → 7.73 s; solve-bound, Planning is about 20x slower since the full tumbling-window-width domain (#1185) #1197).c09_topk_rate_noacc, which was 17% slower before the rebase, is now 37% faster.sum by (job) (sum_over_time(latency[1m]))at 1 Hz: main's query-time Sum accumulator measured 3.09× faster than exact; this PR selects the exact plan (1.00×, still PASS), because the exact hash aggregate's per-label cost is not modelled (above).crates/devtools/tests/corpus_snapshots/**,sketch_plan_snapshots/*.json, ignored tests) compare each node'sdetailand cost, so they are stale after this PR (as after any cost change), but they cannot be regenerated today:asap_planner --run-until stage3stops attodo!("#1207 PR 9 (#1205): each pruned PhysicalCandidate with its reason"). Regenerate them withUPDATE_SNAPSHOTS=1when that lands.Rebased onto main
facaec8b2Rebased onto
mainfacaec8b2(after #1343, #1344 and #1350). Conflicts were only incrates/planner/tests/runtime_install.rs: both sides added tests there, and both are kept. The 10 Hz test now builds its samples with the sharedtime_series::sample(metric, labels, ts, value)helper.Main's #1344 test
counter_rates_over_a_shared_query_time_chain_answer_as_installednow forces query time through a data fact: the samples are at rest ("arrival": "at_rest"), so nothing can be built at ingestion time. With arriving samples, this PR's cost model stores the shared rate chain at ingestion time. That variant is asserted separately incounter_rates_over_a_shared_ingestion_time_chain_answer_as_installed, which checks that the chain is built at ingestion time and that the answers are the same hand-worked rates.