Skip to content

fix(frontend): name every aggregate measure and validate names match measures - #1308

Merged
zzylol merged 2 commits into
mainfrom
fix/1303-topk-aggregate-output-names
Nov 14, 2026
Merged

zzylol merged 2 commits into
mainfrom
fix/1303-topk-aggregate-output-names

Conversation

@zzylol

@zzylol zzylol commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Problem: topk by (region) (4, increase(data_total[5m])) (and the rate form) plans, then fails to install with QueryRunNotExecutable: invalid DAG: node 5: aggregate output names differ from measures. The PromQL frontend builds the counter-ranking TopK aggregate with output_names: [], but the executor needs one name per measure.
  • Change: every frontend Aggregate builder now gives one (empty, so default) name per measure: the PromQL counter-ranking TopK, the canonicalize ranking promotion, and the SQL asap_rate/asap_increase aggregate. NonASAPOp::validate_inputs now refuses an Aggregate whose output names and measures differ. The planner checks this at resolve and at memo insertion, not the executor at install time.
  • Effect: the issue's workload now gets past the name check. It fails at the next executor gap: invalid DAG: node 5: invalid DAG: aggregate intent has no native implementation. The executor has no native TopK reduction yet. That half of [Bug] topk over rate/increase plans but fails installation: aggregate output names differ from measures #1303 waits for executor PR fix(executor): run each node by its plan Lifetime, so roots drain in any order #1300.

Review guide

  • Base: main (4a37273). Not stacked.
  • Read first: crates/types/src/ir/operator/non_asap.rs (validate_inputs, Aggregate arm), then the three builders: crates/frontend-promql/src/promql.rs (Outer::TopK arm), crates/frontend-common/src/canonicalize.rs (ranking promotion), crates/frontend-sql/src/sql/mod.rs (asap_rate/asap_increase).
  • Needs care: crates/stage1-pass1-per-computation-candidates/src/candidates/mod.rs (single_measure_part). It used to give an empty list when the source had no name for that measure. It now always gives one name, so a split measure passes the new check.
  • Mechanical: crates/types/src/ir/operator/asap.rs changes two schema-only temporary aggregates to vec![String::new()]. The rest are test fixtures that built nameless aggregates (output_names: vec![] → vec![String::new()]).
  • Schemas do not change. An empty name and a missing name both resolve to the same intent-keyed default (topk_{k}, rate, …), so no plan or snapshot changes.

Behavior: before → after

The user writes this one-query workload (from #1303):

{"query_workload": {"language": "promql", "repeating_queries": [{
  "query": "topk by (region) (4, increase(data_total[5m]))",
  "demand": {"fixed_interval_at": {"interval": 5000, "evaluation_phase": 0}},
  "requirements": {"accuracy": {"max_error": 0.05, "failure_probability": 0.3, "accuracy_scope": "whole_result"}},
  "predictability": {"predictable": {"known_at": null}}}]},
 "data_workload": {"arrival": {"ingesting": {"rows_per_second": 200000.0}},
  "sources": {"data_total": {"series_count": 10000, "metric_type": "counter", "sample_interval": 100}}}}
plan_installable with PlanningModels::builtin()
Before lowered root Aggregate { Reduce([region]), [TopK { k: 4 }], output_names: [] } → Err(Installable(QueryRunNotExecutable { reason: "invalid DAG: node 5: invalid DAG: aggregate output names differ from measures" }))
After lowered root Aggregate { Reduce([region]), [TopK { k: 4 }], output_names: [""] } → Err(Installable(QueryRunNotExecutable { reason: "invalid DAG: node 5: invalid DAG: aggregate intent has no native implementation" }))

The rate(data_total[20m]) form behaves the same way. Both results were checked with a temporary plan_installable test on this branch (not committed). This serves correctness: the planner now enforces the Aggregate contract the executor relies on. A future frontend gap now fails at lowering, and is no longer found only at install time.

Interfaces

None. No public type, function or field changes. NonASAPOp::validate_inputs is pub(crate). It now also returns the existing SchemaDerivationError::InvalidScalarSignature("aggregate output names differ from measures") for a mismatched Aggregate. That is the same message the executor already used. No new error variant.

Algorithm changes

None.

Decisions and open questions

  • Needs reviewer decision: none.
  • Decided (2026-11-14): the check goes in the planner IR validation (validate_inputs), not in the executor. The rejected alternative from the issue was to make the executor treat an empty list as "default names". That leaves two encodings of one meaning, and the executor is not the place to catch frontend bugs. The check reuses the existing InvalidScalarSignature variant, like the other validate_inputs checks.

Verification

Commands: cargo test --workspace --exclude asap-executor --no-fail-fast, cargo test -p asap-executor, cargo fmt --all -- --check, cargo clippy -D warnings --all-targets on the changed crates.

Scope Result Newly failing vs base Notes
asap-types (lib + types tests) pass 0 includes new structure_contract::aggregate_output_names_match_measures
asap-frontend-common lib pass 0
asap-frontend-promql (promql + corpus tests) pass (197 in promql) 0 includes new promql_lowering::topk_over_counter_rate_names_every_aggregate_measure; it fails on base 4a37273 with left: 0, right: 1
asap-frontend-sql, asap-frontend-metricsql pass 0
asap-stage0-memo, stage1/2/3 crates pass 0
asap-planner --test planner 19 passed, 2 failed, 1 ignored 0 the same 2 fail on base: summary_sharing::identical_ungrouped_queries_share_their_producers and runtime_install::check_compiles_reports_the_executors_error, which already fails on base with "aggregate intent has no native implementation" (counter topk, executor half)
asap-integration-tests --test integration 163 passed, 7 failed, 4 ignored 0 the same 7 fail on base, e.g. promql_to_post_asap::planner_heap_topk_reference_execution_matches_ground_truth
asap-devtools --test devtools 17 passed, 56 failed, 1 ignored 0 the same 56 fail on base, mostly todo!("#1207 PR 9 …") in asap_planner
asap-executor 283 passed, 0 failed, 1 ignored 0
fmt; clippy -D warnings (types, frontend-common, frontend-promql, frontend-sql, stage0-memo, stage1-pass1) clean

New tests:

  • topk_over_counter_rate_names_every_aggregate_measure: the increase and rate counter top-k queries lower to TopK { k: 4 } over the rate aggregate, and every Aggregate in the lowered DAG has one output name per measure.
  • aggregate_output_names_match_measures: validate_inputs accepts one empty or explicit name per measure, and rejects zero names or too many.

Plan or snapshot changes: none. Output schemas are unchanged.

Follow-ups

Part of #1303.

…sures

The PromQL counter-ranking TopK (`topk by (region) (4, increase(m[5m]))`),
the canonicalize ranking promotion, and the SQL asap_rate/asap_increase
aggregate were built with no output names, so the executor refused the
installed plan with "aggregate output names differ from measures".

Each now carries one empty name per measure (the intent-keyed default),
as the other builders do. NonASAPOp::validate_inputs checks that an
Aggregate has one output name per measure, so the planner refuses a
mismatch at resolve and at memo insertion instead of the executor at
install time. Test fixtures that built nameless aggregates are updated.

Part of #1303.
@zzylol zzylol added bug Something isn't working promql PromQL front-end lowering (L1→L2) canonicalization Canonical IR normalization and invariants ir Pre-ASAP or post-ASAP intermediate representation sql SQL front-end lowering (DataFusion → L2) labels Nov 14, 2026
… the mismatch error

The Aggregate.output_names doc now says exactly one entry per measure, with
an empty string meaning the default name (serde default kept for old JSON).
The validation error gives both counts, and the unit test accepts an
aggregate with no measures and no names.

Part of #1303.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working canonicalization Canonical IR normalization and invariants ir Pre-ASAP or post-ASAP intermediate representation promql PromQL front-end lowering (L1→L2) sql SQL front-end lowering (DataFusion → L2)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant