Skip to content

fix(executor): global aggregates follow the input's result kind, not the identity column - #1350

Merged
zzylol merged 4 commits into
mainfrom
fix/1348-empty-global-promql
Nov 16, 2026
Merged

zzylol merged 4 commits into
mainfrom
fix/1348-empty-global-promql

Conversation

@zzylol

@zzylol zzylol commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Closes #1348.

Summary

  • Problem: a PromQL aggregate with no grouping keys over another aggregate's output was run with SQL rules. count(sum by (job) (m >= 1000)) answered {} 0 over an empty vector, where Prometheus answers no sample. sum, min, max, avg and quantile over such an output declared a nullable column, so their plans failed to install with native output type differs from Planner output.
  • Change: the executor's lowering now takes the SQL-or-PromQL rule from the IR's result kind of the rows a node reads (Relation or a PromQL vector), not from whether its input still has a series identity column. For summary states it asks the IR's own evaluated_rows_kind (the kind of the rows the states were built from), so the executor has no copy of that walk. An aggregate, per-group build or merge into no groups over a Relation opts into SQL's one row over no input (with_emit_empty_global(true, …)). The identity-column test is deleted from the aggregate's nullability, its run and the per-group build.
  • Effect: count(sum by (job) (m >= 1000)) answers no sample; SELECT count(*) … WHERE false still answers one row of 0 and SELECT sum(x) … WHERE false one NULL. On main every corpus plan is unchanged. On the open without PR's head (feat(executor): grouped stddev/variance, corr, group, count_values, exact topk and without grouping #1242), 13 corpus queries that failed to install now install (details under Verification).

Review guide

  • Base: main at 863153e3c. Head: 0c877360b. Not stacked.
  • Read first:
    • crates/types/src/ir/operator/asap.rs: evaluated_rows_kind (new, public), next to selected_from; it returns the node's own kind, or for a State node the IR's existing source_kind.
    • crates/executor/src/physical_planner/mod.rs: input_rows_kind (its first input's evaluated_rows_kind), the one over_relation computed in compile_internal, compile_node (new over_relation: bool parameter), and its uses in bind_operation (Aggregate, SummaryAgg, SummaryMerge) and summary_agg_build.
    • crates/executor/src/operators/mod.rs: Operator::with_emit_empty_global(emit_empty_global, parameters), now covering Aggregate and PerGroupSummaryBuild as well as SummaryMerge.
    • crates/executor/src/operators/aggregate/{mod,columnar}.rs: nullability (Reduction::is_null_without_values) and the empty global row.
  • Mechanical: the new over_relation argument at every test call of compile_node (true for the plain test tables, false for the PromQL binary tests); four executor tests that build a global SQL aggregate or build directly now call .with_emit_empty_global(true, None).

Rebase onto main 863153e3c, and the #1242 coordination done

Rebased from 59365ce80 onto main 863153e3c (after #1242, #1309, #1339, #1330, #1324 and #1300). Earlier heads are kept as backup/fix/1348-empty-global-promql-617615169 and backup/fix/1348-empty-global-promql-3433ca055. #1242 merged first, so this PR made the changes listed under "Coordination with #1242" in two new commits.

File Conflict Resolution
crates/executor/src/physical_planner/mod.rs compile_node's doc: #1242's note on nodes that lower to several operators; this PR's over_relation paragraph both paragraphs
crates/executor/src/operators/aggregate/mod.rs #1242's measure list in with_measure_filters (with StdDev/Variance); this PR's is_null_without_values() is_null_without_values(), whose list now includes StdDev and Variance
crates/executor/tests/physical_dag.rs #1300's Lifetime::None argument; this PR's .with_emit_empty_global(...) both

2175a8683 fix(executor): global stddev and stdvar follow the input's result kind (test first). a_global_aggregate_follows_its_input_rows_kind now also covers population StdDev and Variance. Before the fix it failed with StdDev { col: Some(1), population: true }: invalid DAG: native output type differs from Planner output: the derived output was nullable over a PromQL vector without an identity column. The fix:

  • nullability is !population || nullable;
  • finish_measure loses its None if promql => NaN arm;
  • with_emit_empty_global's doc names the standard deviation and variance measures.

Also in this commit:

  • runtime_install::a_global_promql_count_over_an_empty_aggregate_answers_no_sample adds stddev(sum by (job) (m >= 1000)) and stdvar(sum by (job) (m >= 1000)), which give no sample, and stdvar(count by (x) (m)), which gives 0.
  • sql_table_scans::a_global_sql_aggregate_over_no_row_answers_one_row adds stddev_pop and var_samp over no row, which give NULL.
  • feat(executor): grouped stddev/variance, corr, group, count_values, exact topk and without grouping #1242's sql_variance_and_stddev_follow_their_definitions and correlation_follows_datafusion build their global SQL aggregate with .with_emit_empty_global(true, None).

0c877360b test(planner): #1339 deleted CountingStore, so the PromQL empty-aggregate test runs over time_series_tables(...) directly, through the shared run helper.

Checked on 0c877360b, by test name against main 863153e3c: cargo check --workspace --all-targets, cargo fmt --all -- --check and cargo clippy -p asap-executor -p asap-planner -p asap-types -p asap-cost --all-targets -- -D warnings are clean. cargo test --no-fail-fast -p asap-executor -p asap-planner -p asap-types: 645 passed, 1 failed, 2 ignored. The 1 (summary_sharing::identical_ungrouped_queries_share_their_producers) fails on main too. Nothing newly fails.

Checked against main 3e434c760

After the Stage 3 execution-contexts refactor (#1237) merged, this head (617615169) was rebased locally onto main 3e434c760 with no conflicts and built there; it needs no new push. cargo check --workspace --all-targets, cargo fmt --all -- --check and cargo clippy -p asap-executor -p asap-planner -p asap-types --all-targets -- -D warnings are clean. cargo test on those three crates: 567 passed, 2 failed, 2 ignored. The 2 (runtime_install::check_compiles_reports_the_executors_error, summary_sharing::identical_ungrouped_queries_share_their_producers) fail on main too.

Behavior: before → after

Exact, plan_installable, run with the executor at T over gauge m (4 series, every value in (0, 1000)):

Query Prometheus / SQL Before After
count(sum by (job) (m >= 1000)) empty {} 0 empty
count(count by (x) (m)) with no sample empty {} 0 empty
count(sum by (job) (m >= 0)) {} 2 {} 2 {} 2
SELECT count(*) FROM http WHERE ts >= now() - INTERVAL '5 minutes' AND false one row, 0 0 0
SELECT sum(latency) FROM http WHERE … AND false one row, NULL NULL NULL

Executor, a global aggregate over [job, v] (the output of max by (job) (m)), jobs a = 1 and b = 3:

Measure Before (no identity column → SQL) After, over a PromQL vector After, over a Relation
sum, avg, min, max, quantile(0.5) nullable output: refused against the Planner's non-null output (native output type differs) non-null; 4, 2, 1, 3, 2; no row over an empty input nullable; same values; one NULL row over no row

This serves correctness: a PromQL query's answer must not depend on whether an intermediate aggregate kept the series identity column.

Interfaces

crates/types/src/ir/operator/asap.rs

+/// The category of the rows `node` stands for once evaluated: its own result
+/// kind, or, for summary states, the kind of the rows they were built from
+/// ([`source_kind`]). Sum states built over a SQL table `t` and then
+/// subtracted or merged stand for a `Relation`, so a global rollup of them
+/// has SQL's one row over no input rows; built over a PromQL series `m` they
+/// stand for an `InstantVector`, so it has no sample over an empty vector.
+pub fn evaluated_rows_kind<I: OperatorInput>(node: &I) -> OperatorResultKind;

crates/executor/src/physical_planner/mod.rs

 /// Bind a Planner node against the schemas supplied by its deployment edges.
-pub fn compile_node(node: &PhysicalASAPDAGNode, inputs: &[SchemaRef]) -> Result<Operator, Error>;
+/// `over_relation`: whether the node reads a SQL relation's rows (for
+/// summary states, whether they were built from one). Then an aggregate,
+/// build or merge into no groups has SQL's one row over no input rows, and
+/// a quantile is SQL's percentile. Otherwise (a PromQL vector, a range
+/// vector, or states built from either) it has no row and PromQL's
+/// quantile.
+pub fn compile_node(
+    node: &PhysicalASAPDAGNode,
+    inputs: &[SchemaRef],
+    over_relation: bool,
+) -> Result<Operator, Error>;

crates/executor/src/operators/mod.rs, crates/executor/src/operators/summary/mod.rs

 impl Operator {
-    /// A merge into no groups that outputs one empty state when it has no
-    /// input state: the global SQL aggregate a rollup of a relation's
-    /// grouped states to no keys stands for, whose one row is NULL over no
-    /// input rows. ...
-    pub fn with_emit_empty_global(self, parameters: Option<SketchParams>) -> Result<Self, Error>;
+    /// An aggregate, per-group build or merge into no groups follows SQL's
+    /// rule for an empty input when `emit_empty_global`: its one global
+    /// group still has a row, as `SELECT count(*) FROM t WHERE false`
+    /// answers 0 and `SELECT sum(x) FROM t WHERE false` NULL. Otherwise (the
+    /// default; `false` leaves the operator as it is) an empty input has no
+    /// row, as a PromQL `count(v)` over an empty vector has no sample.
+    ///
+    /// An `Aggregate`'s SUM, AVG, MIN, MAX and percentile measures become
+    /// nullable (NULL over no row). A `PerGroupSummaryBuild` outputs an empty
+    /// state of its own θ, and a `SummaryMerge` one of `parameters`: the θ
+    /// of the states it merges (the exported build node's `parameters`),
+    /// `None` for an exact family and for the other two.
+    pub fn with_emit_empty_global(
+        self,
+        emit_empty_global: bool,
+        parameters: Option<SketchParams>,
+    ) -> Result<Self, Error>;
 }

The private Kind::Aggregate and Kind::PerGroupSummaryBuild gain an emit_empty_global: bool field (the one Kind::SummaryMerge already had). It is serialized only when true, so a compiled operator without it reads back with the PromQL default. Saved compiled operators must be rebuilt: a SQL global Aggregate or PerGroupSummaryBuild compiled and serialized before this PR reads back without the field and so gets the PromQL rule (no row over an empty input). Plans compiled at install, as today, are not affected.

A private Reduction::is_null_without_values() (executor, aggregate operator) holds the one list of measures that are NULL for a group with no (matching) value; with_emit_empty_global and with_measure_filters both read it.

Usage (a_global_promql_count_over_an_empty_aggregate_answers_no_sample, crates/planner/tests/runtime_install.rs):

// One repeating PromQL query, exact, every minute over the gauge `m`.
let plan = plan_every_minute(&[("count(sum by (job) (m >= 1000))", "")]).await;
// `compile` restores the IR; for the outer `Aggregate` (count, no keys)
// `input_rows_kind` reads its input, the finalized `sum by (job)`, whose
// `evaluated_rows_kind` is `InstantVector`, so `over_relation` is false and
// the aggregate is built without `emit_empty_global`.
let values = |samples: &[RawDataSample]| -> Vec<f64> { /* run the root, collect each row's value */ };
// No sample at all, or none at or above 1000: no answer row.
assert_eq!(values(&[]), Vec::<f64>::new());
assert_eq!(values(&m_samples()), Vec::<f64>::new());

Algorithm changes

SQL-or-PromQL rule for a global aggregate.

  • Problem: count(sum by (job) (m >= 1000)) answered {} 0; sum(min without (resource) (kube_resourcequota{…})) did not install.
  • Before: the operator decided at run time and in its output schema: emit_empty_global = !input.has_promql_series_identity() for PerGroupSummaryBuild; the same test in columnar::aggregate and in Operator::aggregate's SUM/AVG nullability; MIN/MAX/percentile were nullable for every global aggregate; quantile vs percentile was chosen by the same test. Only the merge used the IR (built_from_relation).
  • After:
    1. compile_internal computes over_relation once per node, and only for the operators that read it (a NonASAP Aggregate, SummaryAgg, SummaryTumblingWindowChain, SummaryMerge): input_rows_kind(ir_node) == Relation, where input_rows_kind is the IR's evaluated_rows_kind of the node's first input. For a State input the IR follows the states to the rows they were built from through every summary operator (SummaryAgg, merges, SummarySubtract, SummaryJoin, MaintainPopulation, state selections), the same walk it uses for SummaryEstimate's result kind and quantile_may_be_empty. Every other node passes false, which none of them reads.
    2. Over a Relation, a global Aggregate, per-group SummaryAgg build (and chain build) or rollup SummaryMerge calls with_emit_empty_global(true, …); otherwise it passes false. A quantile is Reduction::Percentile over a Relation, Reduction::Quantile otherwise.
    3. The operators read only their emit_empty_global field. An aggregate's measure is nullable when its input column is, or when it emits the empty global row (Reduction::is_null_without_values: SUM, AVG, MIN, MAX, percentile). finish_measure answers NULL for a group with no value in both languages (a PromQL group always has one).
  • Correctness: one rule from the IR for builds, merges, aggregates and the quantile choice. Tests: the two e2e tests, a_global_aggregate_follows_its_input_rows_kind and a_global_merge_reads_the_rows_its_states_were_built_from (below); every corpus plan on main is unchanged.
  • Complexity: the IR walk runs once for each aggregate, build or merge node (depth of the state chain, usually 1–2) and not at all for other nodes. Run time unchanged.
  • Assumptions and limits: a state node's kind is its first input's (the IR's source_kind; one memo group's states come from rows of one kind). Keyed and all-groups builds never emit an empty global state, as before. The per-measure filter check (with_measure_filters refusing filters over a series identity) is an input validation, not a semantics choice, and is kept.

Design scope

New or changed item Scope Why it is general
evaluated_rows_kind (new pub fn, types::ir::operator::asap) IR, operator derivation rules, next to selected_from The category of the rows any node stands for, including summary states; the IR already computed it privately (source_kind) for SummaryEstimate's result kind and quantile_may_be_empty. Making it public deletes the executor's copy (rows_kind), which disagreed on states built through SummarySubtract/SummaryJoin.
Reduction::is_null_without_values (private method, executor aggregate operator) Executor operators, aggregate SQL's NULL-for-no-value rule of a measure, one list for the empty global row and for per-measure filters.
emit_empty_global on Kind::Aggregate and Kind::PerGroupSummaryBuild Executor operators The field Kind::SummaryMerge already had, with the same meaning.
compile_node(…, over_relation: bool) Executor lowering One bool form throughout the lowering (compile_node, bind_operation, summary_agg_build); the plan format is unchanged.

No new file, module or type. input_rows_kind replaces built_from_relation (deleted) in the same file.

Decisions and open questions

  • Needs reviewer decision: None.
  • Decided (2026-11-15): pass the rule into compile_node rather than adding a field to PhysicalASAPDAGNode. The plan format stays unchanged, and the executor already restores the IR. Rejected: storing the kind in the plan (a plan format change for a fact the IR already gives).
  • Decided (round 1): the bool form (over_relation) rather than the kind at every lowering function, so the == Relation comparison happens once; with_emit_empty_global takes the flag as a builder setter, so the five call sites have no if … else Ok(op).

Coordination with #1242

Whichever of this PR and #1242 merges second makes these changes (from the round-1 abstraction review, against #1242 at 586df8b7c):

  1. StdDev/Variance nullability (feat(executor): grouped stddev/variance, corr, group, count_values, exact topk and without grouping #1242 aggregate/mod.rs:84-94): replace !input.has_promql_series_identity() && (!population || groups.is_empty() || nullable) with !population || nullable. PromQL stddev/stdvar always lower with population: true (frontend-promql/src/promql.rs:555-563), so a PromQL group is non-null without an identity check; SQL sample variance stays nullable.
  2. The global empty row: add Reduction::StdDev { .. } | Reduction::Variance { .. } to Reduction::is_null_without_values, which covers both with_emit_empty_global and with_measure_filters (feat(executor): grouped stddev/variance, corr, group, count_values, exact topk and without grouping #1242's own with_measure_filters list already has them).
  3. finish_measure's Moments arm (feat(executor): grouped stddev/variance, corr, group, count_values, exact topk and without grouping #1242 columnar.rs:619): delete None if promql => Value::Float64(f64::NAN) and keep None => Value::Null. With population variance a PromQL group always has at least one value, so None does not occur there; finish_measure has no promql argument after this PR.
  4. Mechanical conflicts:

Verification

Scope Result vs main Notes
cargo nextest run -p asap-executor 285 run: 284 passed, 1 skipped +2 new tests
cargo nextest run -p asap-planner 35 run: 33 passed, 2 failed, 1 skipped same 2 failures as main (summary_sharing::identical_ungrouped_queries_share_their_producers, runtime_install::check_compiles_reports_the_executors_error) +2 new tests pass
cargo nextest run -p asap-integration-tests 170 run: 163 passed, 7 failed, 4 skipped the same 7 failures by name as main
cargo clippy -p asap-executor -p asap-planner -p asap-types --all-targets -- -D warnings, cargo fmt --all --check clean

New tests (failing on main first):

  • runtime_install::a_global_promql_count_over_an_empty_aggregate_answers_no_sample: count(sum by (job) (m >= 1000)), count(count by (x) (m)) and count(sum by (job) (m >= 0)), each planned exactly and run with no sample (empty) and with samples (empty, 1, 2). On main: count(sum by (job) (m >= 1000)) with no sample: left [0.0], right [].
  • sql_table_scans::a_global_sql_aggregate_over_no_row_answers_one_row: SQL control. COUNT(*) and SUM over the last 5 minutes AND false answer one row each (0, NULL); without false, 300 and 314,850. Passes on main and here.
  • physical_semantics::a_global_aggregate_follows_its_input_rows_kind: a global sum, avg, min, max and quantile(0.5) over [job, v] (no identity column). Over a PromQL vector: non-null, no row over empty input, 4/2/1/3/2. Over a Relation: refused against a non-null output, and with a nullable one it answers one NULL row (after a serialization round trip) and the same values. On main this shape compiled only with SQL rules.
  • physical_planner::tests::a_global_merge_reads_the_rows_its_states_were_built_from: SummaryMerge(group_by: []) over SummarySubtract and over SummaryJoin of a SummaryAgg over a SQL table reads a Relation, and over a PromQL series an InstantVector. The executor does not lower SummarySubtract/SummaryJoin yet (reserved ASAP operation has no native lowering), so the test checks input_rows_kind, which decides the rule. On round 1's head (dd6117e7b, the executor's own rows_kind) it failed with SummarySubtract: left: State, right: Relation: such a merge got the PromQL rule (no row over no input rows).

sum(max by (job) (m > 1e9)) could not be an end-to-end test here: on main it fails earlier with summary update column missing or ambiguous (#720), as do sum, max, min, avg and quantile over any … by (job) output. The executor test covers its rule.

Corpus plan diff (plan_installable, one query per workload, exact and 5 % targets, install outcome and DAG hash per entry). Corpora: the four PromQL corpora (promql_corpus_docs, promql_corpus_testdata, o11y_bench_promql, awesome_prometheus_alerts) and the four SQL corpora (data-quality, NetFlow, BGP analytics, BGP Jan-2024 workload).

  • Round 2 (this head) vs main (59365ce80) and vs round 1's head: 3,140 queries, 6,280 entries; 3,514 install on both. Every install outcome and installed DAG is the same as on main and on round 1's head; the only text differences (19 vs main, 20 vs round 1) are DataFusion's random "Did you mean …" suggestions in SQL lowering errors.

  • Round 1: main vs this PR, plus the 253-query without corpus: PromQL 6,202 entries (3,298 install on both), SQL 584 entries (216 install on both). Every installed DAG is byte-identical and every error is identical. The only text differences are DataFusion's nondeterministic "Did you mean …" suggestions in SQL lowering errors. main has no native output type differs error in these corpora: the nested shapes fail earlier there (A Sum over a grouped PromQL sum fails to execute: the update reads a sample-value column the grouped output doesn't have #720, and without is not supported yet).

  • Round 1: feat(executor): grouped stddev/variance, corr, group, count_values, exact topk and without grouping #1242's head c462ccad5 vs that head plus this fix: 15 queries (30 entries) change and all other DAGs are byte-identical. 13 distinct queries now install:

    • sum(min without (resource) (kube_resourcequota{…})) and its 7 variants (with / sum(kube_node_status_allocatable{…}) > 1.5);
    • quantile(0.9, sum without (cpu) (rate(node_cpu_seconds_total{mode="system"}[5m])));
    • sum(sum without (instance)(node_cpu) / ignoring (mode) group_left sum without (instance, mode)(node_cpu));
    • sum(sum by (mode, job)(node_cpu) / on (job) group_left sum by (job)(node_cpu));
    • sum(avg(mongodb_… - mongodb_…)) - sum(avg(… group_right …)) <= 0 (alerts);
    • quantile without(point)(NaN, data), which failed with a percentile requires Float64 input and a fraction in [0, 1] because the quantile was chosen as SQL's.

    sum without (device) (max without (fstype, mountpoint) (…)) / scalar(sum(max without …)) moves to the next gap (native Planner binding currently requires a unary operation). So 10 of the 11 without-corpus native output type differs queries now install.

Review status

Round 1 (head dd6117e7b): two independent reviews, abstraction + layering and functional + efficiency. Both approve; no blocker. Addressed in this head:

Finding Severity Resolution
A1 / F1: the executor's rows_kind copied the IR's private source_kind and disagreed on states built through SummarySubtract, SummaryJoin, MaintainPopulation, Extension medium / low rows_kind deleted; the IR's evaluated_rows_kind is public and input_rows_kind calls it. Test a_global_merge_reads_the_rows_its_states_were_built_from failed on dd6117e7b (State, the PromQL rule).
A2: the list of measures that are NULL without values was written twice (and #1242 adds to one copy) low Reduction::is_null_without_values, used by with_emit_empty_global and with_measure_filters
A3: five if global && over_relation { with_emit_empty_global } else { Ok(op) } sites nit with_emit_empty_global(emit_empty_global, parameters) takes the flag; false leaves the operator as it is
A4: bool and kind both carried the same fact low one form, over_relation: bool, from compile_internal through compile_node, bind_operation and summary_agg_build
F3: the rows kind was computed for every lowered node nit computed once, only for Aggregate, SummaryAgg, SummaryTumblingWindowChain and SummaryMerge nodes (FinalizeExactAccumulator's binding does not read it)
L3: with_emit_empty_global's doc covered only the aggregate in its first sentence low the doc opens with "An aggregate, per-group build or merge into no groups…"; the Interfaces diff above is the code's text
L4: the IR declares every global Min/Max nullable regardless of result kind, while the executor now produces non-null for PromQL low, from before this PR follow-up issue #1351
F2: saved compiled SQL global aggregates and builds read back with the PromQL default nit stated under Interfaces: saved compiled operators must be rebuilt
PR body: Design scope line on rows_kind; coordination with #1242 low Design scope table rewritten; "Coordination with #1242" section added

Checked clean in round 1: ir does not import stages; no cross-stage calls; built_from_relation and the merge-only with_emit_empty_global deleted, not left as shims; every emit_empty_global field is read; the remaining has_promql_series_identity uses in the executor are structural or input validation, not a SQL/PromQL choice; compile cost about +2% (a few tenths of a millisecond per compile), run time unchanged.

Follow-ups

@zzylol zzylol added bug Something isn't working nested queries promql PromQL front-end lowering (L1→L2) sql SQL front-end lowering (DataFusion → L2) labels Nov 15, 2026
@zzylol
zzylol force-pushed the fix/1348-empty-global-promql branch 2 times, most recently from 3433ca0 to 0c87736 Compare November 16, 2026 04:28
…the identity column

A PromQL aggregate with no grouping keys over another aggregate's output
(`count(sum by (job) (m >= 1000))`) answered one row over an empty input,
and `sum`, `min`, `max`, `avg` and `quantile` over such an output declared
nullable outputs that failed to install against the Planner's non-null
schema. The executor chose SQL or PromQL rules by whether the input still
had a series identity column, which an aggregate's output never has.

The lowering now reads the IR's result kind of the rows a node reads
(`input_rows_kind`: its input's, or for summary states the rows they were
built from). Over a `Relation`, an aggregate, per-group build or merge into
no groups opts into `with_emit_empty_global`, which also makes SUM, AVG,
MIN, MAX and percentile measures nullable; over a PromQL vector none does,
and a quantile is PromQL's. The identity-column test is gone from the
aggregate's nullability, its run and the per-group build.
…and NULL list

The executor's `rows_kind` copied the IR's private `source_kind` and
stopped at states built through `SummarySubtract`, `SummaryJoin`,
`MaintainPopulation` or `Extension`, so a global SQL rollup of such states
got PromQL's rule (no row over no input rows). The IR now exposes
`evaluated_rows_kind` (a node's own kind, or for summary states the kind of
the rows they were built from) and `input_rows_kind` calls it.

`compile_internal` computes `over_relation` once, only for aggregates,
summary builds and merges, and passes that one bool form through
`compile_node`, `bind_operation` and `summary_agg_build`.
`with_emit_empty_global` takes the flag (false leaves the operator as it
is), replacing five `if … else Ok(op)` sites, and
`Reduction::is_null_without_values` is the one list of measures that are
NULL for a group with no value, read by it and by `with_measure_filters`.
Grouped stddev/stdvar still read the series identity column to choose
between SQL and PromQL behaviour over no value. They now follow the same
rule as the other global aggregates: the measure is nullable for a sample
form or a nullable column, a global SQL aggregate gets its one NULL row
from `with_emit_empty_global`, and a PromQL `stddev(...)`/`stdvar(...)`
over an empty vector answers no sample.

Tests: the stddev and stdvar cases of
a_global_aggregate_follows_its_input_rows_kind (failed before: the
derived output was nullable over a PromQL vector without an identity
column), PromQL stddev/stdvar over an empty aggregate, SQL stddev_pop and
var_samp over no row. The SQL variance and correlation tests build their
global aggregate with `with_emit_empty_global`; the PromQL install test
uses the shared `run` over a store.
Main deleted the read-counting CountingStore; the test reads the plan's
raw-data tables directly.
@zzylol
zzylol force-pushed the fix/1348-empty-global-promql branch from 0c87736 to d78f984 Compare November 16, 2026 13:46
@zzylol
zzylol merged commit facaec8 into main Nov 16, 2026
2 of 3 checks passed
@zzylol

zzylol commented Nov 16, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto main d0f7db1ee (after #1343 and #1344). Head d78f984a5 (was 0c877360b); backup branch backup/fix/1348-empty-global-promql-0c877360b.

Conflicts, all in crates/planner/tests/runtime_install.rs: main added the counter rate/increase section in the same place as this PR's empty-global-aggregate test, so both sections are kept. main's time_series::run now takes (dag, store, views, roots, scope), so the new test calls run(&plan.dag, time_series_tables(&plan.dag, samples), &[], &query_roots(plan), query_time). The import keeps {Batch, Value}.

Validation on the merged tree: cargo fmt --all -- --check, cargo check --workspace --all-targets and cargo clippy --workspace --all-targets -- -D warnings pass. cargo test -p asap-executor -p asap-types and cargo test -p asap-planner --test planner compared by name with main: no new failures, 4 new tests pass. Only pre-existing failure: summary_sharing::identical_ungrouped_queries_share_their_producers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working nested queries promql PromQL front-end lowering (L1→L2) sql SQL front-end lowering (DataFusion → L2)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A global PromQL aggregate over another aggregate answers one row for an empty input

1 participant