Skip to content

feat(executor): grouped stddev/variance, corr, group, count_values, exact topk and without grouping - #1242

Merged
zzylol merged 8 commits into
mainfrom
exec/x2-reduce-aggregates
Nov 16, 2026
Merged

zzylol merged 8 commits into
mainfrom
exec/x2-reduce-aggregates

Conversation

@zzylol

@zzylol zzylol commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Problem. The planner no longer checks executor support, so it selects plans with cross-series aggregates that the executor cannot run, and installation fails. On main, plan_installable of stddev by (job) (m) fails with invalid DAG: node 2: invalid DAG: aggregate intent has no native implementation, and sum without (instance) (m) fails with grouping without requires resolved label columns.
  • Change. The executor gets kernels for StdDev, Variance, PearsonCorr and Group, a count_values lowering, an exact Aggregate{Reduce, [TopK]} lowering onto a sort that keeps the first k rows per group (Sort { fetch }), and without grouping. The IR marks a series identity kept by without as computed, and adds the count_values label under without (unless the grouping removes it, a label the without list names or __name__, as Prometheus removes it again). Series identity realization now renumbers the column references of every operator above a node it adds the identity column to, so topk(1, sum without (instance) (m)) ranks by the sum, not by the identity text. An aggregate over a without aggregate (sum by (job) (sum without (instance) (m))), avg without two or more labels, and a grouping key a without below already removed now plan and answer as Prometheus does.
  • Effect. These queries now plan, install and answer exactly. For example, stddev by (job) (m) gives {job="a"} 1.2472 (√(14/9)) and {job="b"} 0.25. This closes the executor half of [Bug] topk over rate/increase plans but fails installation: aggregate output names differ from measures #1303: topk by (region) (4, increase(data_total[5m])) now installs, both exact and approximate. An exact top-k holds groups × k rows instead of its whole input, and at 10^6 rows runs 5–17× faster than the full sort then limit it replaces (table below).

Review guide

  • Base: main (3e434c760).

  • Read first:

    • crates/executor/src/operators/sort.rs: sort_with_fetch, the bounded per-group heap, with key_comparator and KeptKeyValues.
    • crates/executor/src/physical_planner/mod.rs: lower_exact_topk, lower_count_values, top_k_sort and resolve_without, next to each other; compile_internal only dispatches to them.
    • crates/types/src/ir/schema/aggregate_schema.rs: without_output_schema and push_count_values_label.
    • crates/executor/src/physical_planner/promql_rows.rs: series_identity_without.
    • crates/executor/src/operators/aggregate/columnar.rs: Moments and CoMoments.
    • crates/types/src/ir/schema/series_identity.rs: positions_after_realization, renumber and SeriesIdentityError, and their use in with_promql_series_identity, with NonASAPOp::map_parts (behind map_children and map_input_columns) in crates/types/src/ir/operator/non_asap.rs.
    • crates/planner/src/run.rs: promql_root, the one place a realization error falls back or fails.
    • crates/frontend-common/src/resolve.rs: resolve_group_keys_promql and labels_removed_by_without, the absent-label rules.
  • Careful review: the NULL and NaN rules in finish_measure, Operator::aggregate's nullability, non_asap.rs's SQL schema nullability; the without rewrite over a series identity and the count_values label that replaces an identity label; tie order and memory accounting in sort_with_fetch, and that key_comparator orders NULL, NaN and -0 exactly as compare_key (a map key is compared by decoded value, not by Arrow); the per-dictionary-entry LabelIdentity path in expressions/columnar.rs, whose output dictionary may repeat a text.

  • Mechanical:

    • Operator::sort gains a fetch argument. The SQL/PromQL Sort binding and the tests pass None; the two top-k lowerings pass Some(k) through top_k_sort.
    • Test helpers moved to the shared test modules (planner/tests/time_series, integration-tests/tests/physical_common).
    • topk::ranked_column and ranked_row_projection become pub, because the executor now calls them.
  • Overlap with open fix(executor): run each node by its plan Lifetime, so roots drain in any order #1300: both PRs touch crates/executor/src/physical_planner/mod.rs and crates/executor/tests/physical_semantics.rs. Whichever lands second adds the Lifetime argument to this PR's new PhysicalDAG::add calls: Lifetime::None for the helpers (the top-k Sort, the count_values Project, the without projection), and node.lifetime for the node's own id.

Rebase onto main 3e434c760

Rebased from 59365ce80 onto main at 3e434c760, after the merge of the Stage 3 execution-contexts refactor (#1237). Same 8 commits; conflicts resolved as that PR's merge notes ask:

  • crates/planner/tests/main.rs: both mod exact_only_reads; and mod reduce_aggregates;.
  • crates/types/src/ir/operator/non_asap.rs, the Filter, Join, Sort, SQLWindowFunc and PromqlRelabel arms of map_parts: f runs on the direct inputs before any scalar expression (Filter child, then pred; Join left, right, then pred; Aggregate child, filters, then having; Sort child, then keys; SQLWindowFunc child, args, then order_by; PromqlRelabel child, then value; Project child, then cols). This PR's column renumbering is kept: map_pred(…, column, f), map_keys(…, column, f), partition_by.map_columns(column), and kept for a Scan's predicates and an Aggregate's having. ScalarExpr::map_parts maps an InSubquery's subquery before its expr.
  • Docs: the "in Self::children order … q then c" sentence moved from map_children onto map_parts; map_children now says it visits in map_parts's order.
  • non_asap::tests::map_children_calls_f_in_children_order passes.

Checked on the rebased head, by test name against main 3e434c760: cargo check --workspace --all-targets, cargo fmt --all -- --check and cargo clippy -p asap-types -p asap-executor -p asap-planner -p asap-integration-tests -p asap-frontend-common -p asap-stage1-pass1-per-computation-candidates --all-targets -- -D warnings are clean. cargo test on the same six crates: 877 passed, 8 failed, 6 ignored. All 8 fail on main too; main has one more failure, runtime_install::check_compiles_reports_the_executors_error, which this PR fixes. Nothing newly fails.

Behavior: before → after

The user writes one exact repeating PromQL query, every 10 s, over a gauge m (10 series, sampled every 10 s, ingesting):

Query Before (main) After
stddev by (job) (m) install fails: aggregate intent has no native implementation a √(14/9), b 0.25
stdvar by (job) (m) same a 14/9, b 0.0625
group by (job) (m) same a 1, b 1
count_values("v", m) same v="1" 1, v="2" 2, v="2.5" 1, v="4" 1
count_values("v", m) without (instance), over {job=a} 1 ×2, {job=b} 2, {job=b,zone=z} 2 same {job=a,v=1} 2, {job=b,v=2} 1, {job=b,zone=z,v=2} 1
count_values("zone", m) without (instance), same data same {job=a,zone=1} 2, {job=b,zone=2} 2: zone is overwritten before grouping, as in Prometheus
count_values("zone", h) without (instance, zone), over h{job=a} 2, 2, 2 (zones z, none, y) and 5 (zone z) same {job=a} 4: the excluded label is set and then removed, as in Prometheus
count_values("instance", m) without (instance) same {job=a} 2, {job=b} 1, {job=b,zone=z} 1, no instance label
count_values("__name__", h) without (instance), same h same {job=a} 1, {job=a,zone=y} 1, {job=a,zone=z} 2: without always drops __name__, as in Prometheus
sum without (instance) (m) grouping without requires resolved label columns {job="a"} 7, {job="b"} 2, {job="b",zone="z"} 2.5
count without (instance) (m) same {job="a"} 3, {job="b"} 1, {job="b",zone="z"} 1 (Int64, as count by answers)
count without (instance, zone) (m) same {job="a"} 3, {job="b"} 2
group without (instance) (m) same 1 for each of the three groups
stddev without (instance) (m) same {job="a"} √(14/9), {job="b"} 0, {job="b",zone="z"} 0
topk(1, sum without (instance) (m)) grouping without requires resolved label columns (at this PR's previous head: {job="b"} 2, ranked by the identity text) {job="a"} 7
bottomk(1, sum without (instance) (m)) same {job="b"} 2
sort(sum without (instance) (m)) / sort_desc(...) same (previous head: identity-text order) {job="b"} 2, {job="b",zone="z"} 2.5, {job="a"} 7 / the reverse
topk(1, count without (instance) (m)), sort_desc(count without (instance) (m)) same {job="a"} 3 / 3, 1, 1
topk by (job) (1, sum without (instance) (m)) same {job="a"} 7, {job="b",zone="z"} 2.5
sum by (job) (sum without (instance) (m)) grouping without requires resolved label columns (previous head: summary update column missing or ambiguous) {job="a"} 7, {job="b"} 4.5
max without (zone) (sum without (instance) (m)) same {job="a"} 7, {job="b"} 2.5
avg without (instance, zone) (m) same (previous head: invalid scalar function signature: column outside scalar input scope) {job="a"} 7/3, {job="b"} 2.25
sum without (instance) (sum without (instance) (m)) same (previous head: column instance not found in schema) as sum without (instance) (m): the outer instance is already gone and removes nothing
sum by (instance, job) (sum without (instance) (m) > 1) same {job="a"} 7, {job="b"} 4.5: instance is the empty label
max without (instance) ((10 * min without (zone) (m)) unless n) not reached (fails earlier) planning error column instance not found; it was dropped as absent and answered with instance kept wherever the rest planned
limitk(2, up), info(up) plan over their label columns same: realization does not apply, the planner falls back as before
topk(3, sum by (job) (rate(c[5m]))) aggregate intent has no native implementation j2 0.5, j3 0.7, j4 0.9
topk by (job) (1, increase(c[5m])) (the #1303 shape) same each job's instance 2, the one with the larger increase, with all its labels
SQL SELECT k, STDDEV(x), VAR_SAMP(x), STDDEV_POP(x), VAR_POP(x) FROM t GROUP BY k same k1 (√(7/3), 7/3, √(14/9), 14/9); k2 (NULL, NULL, 0, 0); k3 all NULL
SQL SELECT corr(a, b) FROM t same 1

This serves MVP correctness: every shape the frontends emit and the planner selects must run.

Interfaces

crates/executor/src/operators/mod.rs and sort.rs

 enum Kind {
     Sort {
         keys: Vec<SortKey>,
         groups: Vec<usize>,
+        /// Keep only the first `fetch` rows of each group, in sort order
+        /// (`topk by (job) (3, m)` gives `Some(3)`); `None` sorts every row.
+        #[serde(default, skip_serializing_if = "Option::is_none")]
+        fetch: Option<u64>,
     },
 }

 impl Operator {
+    /// Sort `input`'s rows by `keys` within each group of the `groups`
+    /// columns, groups in key order. `fetch` keeps only the first `fetch`
+    /// rows of each group, in sort order: `topk by (job) (3, m)` is a sort
+    /// by value descending, grouped by `job`, with `fetch` `Some(3)`, which
+    /// holds 3 rows per job instead of every input row. `None` sorts every
+    /// row.
-    pub fn sort(input: SchemaRef, keys: Vec<SortKey>, groups: Vec<usize>) -> Result<Self, Error>
+    pub fn sort(
+        input: SchemaRef,
+        keys: Vec<SortKey>,
+        groups: Vec<usize>,
+        fetch: Option<u64>,
+    ) -> Result<Self, Error>
 }

crates/executor/src/operators/aggregate/mod.rs

 pub enum Reduction {
     ...
     Percentile {
         column: usize,
         /// The fraction, in [0, 1].
         q: f64,
     },
+    /// Standard deviation of a numeric column's non-NULL values: the
+    /// square root of [`Reduction::Variance`].
+    StdDev {
+        column: usize,
+        population: bool,
+    },
+    /// Variance of a numeric column's non-NULL values, by Welford's
+    /// update as Prometheus `stdvar` and DataFusion `var_pop`/`var_samp`
+    /// compute it: the squared deviations' sum over n (population) or
+    /// n - 1 (sample); NaN once a value is NaN or infinite. SQL gives NULL
+    /// for no value, or one value of a sample; PromQL NaN. The values 1, 2
+    /// and 4 have mean 7/3: population variance 14/9, sample variance 7/3.
+    Variance {
+        column: usize,
+        population: bool,
+    },
+    /// SQL `corr(left, right)`, DataFusion's: Pearson's correlation over
+    /// the rows where both are non-NULL; NULL for fewer than two rows, a
+    /// constant input or one input's mean NaN, NaN when both are.
+    PearsonCorr {
+        left: usize,
+        right: usize,
+    },
+    /// PromQL `group`: 1 for every group.
+    Group,
 }

crates/executor/src/expressions/mod.rs

 pub enum Expression {
     ...
-    /// Canonical encoding of a label map less `excluding`, identical to
-    /// `promql_rows::encode_series_identity`.
+    /// Canonical encoding of a label map, or of a series identity, less
+    /// `excluding`, identical to `promql_rows::encode_series_identity`.
     LabelIdentity {
         column: usize,
         #[serde(default, skip_serializing_if = "Vec::is_empty")]
         excluding: Vec<String>,
     },
     Literal {
         value: Value,
         dtype: DataType,
     },
+    /// A Float64 column's value as text, as Go's
+    /// `strconv.FormatFloat(v, 'f', -1, 64)` writes it (PromQL
+    /// `count_values`): the shortest decimal that reads back as `v`,
+    /// without exponent; `NaN`, `+Inf` and `-Inf`. `2.5` is `"2.5"`, `1e21`
+    /// is `"1000000000000000000000"` and `-0.0` is `"-0"`.
+    FloatText(usize),
     Negate(Box<Expression>),
     ...
 }

crates/types/src/ir/schema/aggregate_schema.rs

 /// Where one output column of an aggregate comes from: an input column it
 /// keeps, a measure's value (with the input column the schema types it by,
 /// the measure's own or the probe value column), or a label column the
-/// measure adds (`count_values`' and `absent`'s). `sum by (job) (x)` →
+/// aggregate computes (`count_values`' and `absent`'s; a series identity
+/// a `without` grouping keeps, its labels less the excluded ones and
+/// `__name__`; and a `by` or kept column that a `count_values` label of the
+/// same name replaces: `count_values by (job) ("job", v)` →
+/// `[ComputedLabel, Measure { .. }]`). `sum by (job) (x)` →
 /// `[InputColumn(job), Measure { index: 0, input_column: Some(value) }]`. The one layout the output schema
 /// ([`aggregate_output_schema`]) and the group properties read.
 pub(crate) enum OutputColumnSource {
     InputColumn(ColumnId),
     Measure {
         index: usize,
         input_column: Option<ColumnId>,
     },
-    AddedLabel,
+    ComputedLabel,
 }

+/// The columns of `in_schema` a `without(excluded)` grouping keeps, in
+/// order: every column but the time axis, the sample value and the
+/// `excluded` positions. `[t, value, job, instance, $promql_series_identity]`
+/// under `without (instance)` keeps `[job, $promql_series_identity]`; the
+/// groups are the distinct values of those columns (the identity less the
+/// excluded labels, see [`super::is_label_set_column`]).
+pub fn without_kept_columns(in_schema: &Schema, excluded: &[ColumnId]) -> Vec<ColumnId>

crates/types/src/ir/schema/mod.rs

 pub const PROMQL_SERIES_IDENTITY: &str = "$promql_series_identity";
+
+/// Whether `field` holds a whole label set: the series identity or a label
+/// map. A `without` grouping computes such a kept column, its labels less
+/// the excluded ones and `__name__`, instead of copying it: under
+/// `without (instance)` the identity `{__name__=m, job=a, instance=1}`
+/// becomes `{job=a}`, while a kept `job` column is copied.
+pub fn is_label_set_column(field: &Field) -> bool

crates/executor/src/physical_planner/promql_rows.rs

+/// The series identity `identity` less the labels `excluding`, encoded as
+/// [`encode_series_identity`] writes it: `{"__name__":"m","instance":"1",
+/// "job":"a"}` less `instance` and `__name__` is `{"job":"a"}`. The encoding
+/// is canonical (names sorted, one way to escape each character), so this
+/// copies the kept `"name":"value"` pairs' bytes in order, decoding and
+/// encoding nothing. [...] The input must be canonical, as every
+/// identity the executor writes is (through `series_row`): only its
+/// structure is checked, so input that is not a JSON object of string
+/// labels is an error, but a non-canonical one (`{"b":"1","a":"2"}`,
+/// `{"a":"\/"}`) or an invalid escape inside a string is copied as it is:
+/// checking the canonical form would add a fifth to a third to the
+/// rewrite's time, on every identity of every batch.
+pub fn series_identity_without(identity: &str, excluding: &[String]) -> Result<String, Error>

crates/executor/src/values/batch/mod.rs

 impl Batch {
+    /// The rows `indices` of `batches` (all in `schema`), each a (batch,
+    /// row) pair, in that order: one copy of each row, where [`Self::concat`]
+    /// then [`Self::take`] copy it twice. [...]
+    pub(crate) fn interleave(
+        schema: &SchemaRef,
+        batches: &[Self],
+        indices: &[(usize, usize)],
+    ) -> Result<Self, Error>
 }

without_output_schema (private) now gives a kept series identity or label map the source ComputedLabel (via is_label_set_column), and adds the count_values label column before the count, as the by path does. On both paths a count_values label that collides with a by or kept column turns that position into a computed Utf8 label (push_count_values_label); under without, a label the grouping removes (one the without list names, or __name__, private removed_by_without) is not added. Its doc carries the example {__name__=m, job=a, instance=1, zone=z} → {job=a, zone=z} under without (instance).

crates/types/src/ir/schema/topk/mod.rs

-pub(crate) fn ranked_column<I: OperatorInput>(child: &I) -> Result<ColumnId, ResolveError> {
+pub fn ranked_column<I: OperatorInput>(child: &I) -> Result<ColumnId, ResolveError> {
-pub(crate) fn ranked_row_projection<I: OperatorInput>(
+pub fn ranked_row_projection<I: OperatorInput>(
     child: &I,
     reduction: &Reduction,
     output_name: &str,

These two are pub because the executor's top-k lowering builds the same ranked rows that the IR's schema derivation describes.

crates/types/src/ir/operator/non_asap.rs, arguments.rs and agg_intent.rs

 impl<C> NonASAPOp<C> {
-    /// The direct operator inputs, in field order, followed by the operator
-    /// nodes referenced from this operator's scalar expressions.
+    /// The direct operator inputs, followed by the operator nodes referenced
+    /// from this operator's scalar expressions.
     pub(crate) fn children(&self) -> Vec<&C>
+    /// The direct operator inputs, in field order: a `Join`'s left then
+    /// right. Their output columns, in this order, are the operator's
+    /// [`input_scope`](NonASAPOp::input_scope).
+    pub(crate) fn inputs(&self) -> Vec<&C>
+    /// The operator's input scope: the output columns of its direct
+    /// [`inputs`](NonASAPOp::inputs), in order, which its column references
+    /// index. A `Join` of `[a, b]` and `[c]` reads `c` as `Column(2)`;
+    /// [`map_input_columns`](NonASAPOp::map_input_columns) renumbers these
+    /// references.
+    pub(crate) fn input_scope(&self) -> Vec<&Field>
+    where
+        C: OperatorInput;
     pub(crate) fn map_children<D>(&self, f: impl FnMut(&C) -> D) -> NonASAPOp<D>
+    /// This operator with every column reference into its input scope
+    /// renumbered by `column`: the columns of its direct inputs, in
+    /// [`inputs`](NonASAPOp::inputs) order. A `Sort` by `Column(0)` over
+    /// `[sum]` whose input becomes `[$promql_series_identity, sum]` must sort
+    /// by `Column(1)`; the renumbering `0 → 1` gives that. Two references are
+    /// in another scope and kept: a `Scan`'s predicates index its own
+    /// schema, and an `Aggregate`'s `having` indexes its output. Operator
+    /// nodes read from scalar expressions keep their own scope.
+    pub(crate) fn map_input_columns(&self, column: &mut impl FnMut(ColumnId) -> ColumnId) -> Self
+    where
+        C: Clone;
 }

 impl<C> GroupKeys<C> {
+    /// These keys with every key mapped by `f`, `by` or `without` kept; the
+    /// first error stops the walk. A front end resolves `by ("job")` to
+    /// `by (2)` with it.
+    pub fn try_map_keys<D, E>(
+        &self,
+        f: &mut impl FnMut(&C) -> Result<D, E>,
+    ) -> Result<GroupKeys<D>, E>
 }

 impl<C: Clone> AggIntent<C> {
+    /// This intent with every column it holds mapped by `f`, in argument
+    /// order; the first error stops the walk. Every variant is listed, so a
+    /// new variant that holds a column must be mapped here: a front end
+    /// resolves `Sum { col: Some("value") }` to `Sum { col: Some(1) }` with
+    /// it, and series identity realization renumbers `Some(1)` to
+    /// `Some(2)` when it inserts a column before it.
+    pub fn try_map_cols<D, E>(
+        &self,
+        f: &mut impl FnMut(&C) -> Result<D, E>,
+    ) -> Result<AggIntent<D>, E>
 }

GroupKeys::map_columns, Reduction::map_columns and AggIntent::map_columns (pub(crate), ColumnId only) are one-line wrappers of these with an infallible map. AggIntent::input_cols walks try_map_cols too (less histogram_quantile's le, a bucket bound, not a value). NonASAPOp::map_children keeps its signature and is now a wrapper of the private map_parts.

crates/types/src/ir/schema/series_identity.rs and crates/planner/src/pass.rs

 pub fn with_promql_series_identity(
     root: &Rc<OperatorNode>,
-) -> Result<Rc<OperatorNode>, String>
+) -> Result<Rc<OperatorNode>, SeriesIdentityError>

+/// Why [`with_promql_series_identity`] gives no root carrying
+/// [`PROMQL_SERIES_IDENTITY`].
+#[derive(Debug, Clone, PartialEq, thiserror::Error)]
+pub enum SeriesIdentityError {
+    /// The root holds an operator this realization does not cover, or a
+    /// source that is not an open PromQL source: `limitk(2, m)` lowers to
+    /// a `PromqlSeriesSample`, `info(m)` to a `PromqlInfoEnrich`, and a
+    /// closed scan has no dynamic labels to carry. The root is planned
+    /// over its label columns instead.
+    #[error("{0}")]
+    NotApplicable(String),
+    /// `operator` reads input column `column`, which realization does not
+    /// keep. Renumbering it would read another column and give a wrong
+    /// answer.
+    #[error("series identity realization does not keep column {column} that {operator} reads")]
+    DroppedColumn {
+        operator: &'static str,
+        column: ColumnId,
+    },
+    /// An operator rebuilt over its realized children, with its column
+    /// references renumbered, has a schema that no longer derives. Its
+    /// unrealized form derived, and renumbering moves `Column(0)` of
+    /// `[sum]` to the same `sum` at `Column(1)` of
+    /// `[$promql_series_identity, sum]`, so this is a defect in
+    /// realization, not in the query. Planning the unrealized root would
+    /// hide it.
+    #[error("series identity realization built an invalid operator: {0}")]
+    InvalidRealizedOperator(SchemaDerivationError),
+}

 pub enum OptimizeError {
     ...
+    /// Series identity realization would make an operator read a column it
+    /// does not keep, or rebuilt an operator whose schema no longer derives:
+    /// a defect in the realization, not in the query.
+    #[error("PromQL series identity realization: {0}")]
+    SeriesIdentity(SeriesIdentityError),
 }

 // crates/types/src/ir/schema/error.rs
-#[derive(Debug, Error)]
+#[derive(Debug, Clone, PartialEq, Error)]
 pub enum SchemaDerivationError {

with_promql_series_identity's doc now says it renumbers every parent's column references, and an Aggregate's having by how the aggregate's own output moves, and lists the three errors: NotApplicable (the caller plans the root over its label columns), and DroppedColumn and InvalidRealizedOperator (realization defects; the caller must not fall back). SchemaDerivationError gains Clone and PartialEq so SeriesIdentityError can carry it.

Usage

From crates/planner/tests/reduce_aggregates.rs. A runtime plans one exact repeating query with plan_installable, then runs the installable plan's DAG with the executor over raw samples.

// The gauge `m`, one sample per series 5 s before the evaluation:
// {job=a,instance=1..3} = 1, 2, 4; {job=b,instance=1} = 2; {job=b,instance=2,zone=z} = 2.5.
let samples = gauge_samples();
// Stage 0 lowers the query, Stages 1-3 select the exact plan, and plan_installable
// compiles its query run with the executor; that compile failed before this PR.
let plan = plan_promql("stddev by (job) (m)", "m").await;
// The executor runs plan.dag at query time over the samples bound at its Scan of m.
// compile lowers the Aggregate to Reduction::StdDev { population: true }.
let batches = run_query(&plan, |dag| time_series_tables(dag, &samples));
// -> {job="a"} 1.247219128924647 (sqrt(14/9)), {job="b"} 0.25

Design scope

Every new type, field and file, placed in the engine:

New Scope Why it is general
Kind::Sort.fetch: Option<u64>, Operator::sort(.., fetch) executor operators A sort that keeps the first n rows per group (DataFusion's TopK when ungrouped). Any ordered limit can use it: PromQL topk/bottomk, SQL ORDER BY .. LIMIT n, a current-series top-k.
HeldRow, KeptKeyValues, key_comparator, heap_push, heap_sift_down_top, sort_order, key_column, compare_held_key (private, sort.rs) executor operators, sort kernel A sort-with-fetch's kept row, the key values of kept rows decoded on demand, a key comparison read from Arrow arrays, a Vec max-heap ordered by a closure, the one tie-break rule, a key column decoded with the one NaN-in-collection check, and compare_key with numbers and times inline for the heap.
arrow_columns::decode_cell (pub(crate)) executor values codec One cell of a number, bool, time, date or interval column as a Value; decode and KeptKeyValues both read cells through it.
identity_without_per_entry (private, expressions/columnar.rs) executor expressions, columnar evaluation LabelIdentity over a dictionary column, evaluated once per dictionary entry as text, keys kept.
removed_by_without (private, aggregate_schema.rs) IR schema Whether a without grouping removes a label: an excluded column's name or __name__.
Batch::interleave (pub(crate)) executor values Gather rows from several batches in one copy; any operator that holds rows across batches can use it.
ir::schema::is_label_set_column IR schema The one rule for which columns hold a whole label set; read by the IR's without schema and the executor's without rewrite.
promql_rows::series_identity_without executor PromQL row codec The identity codec's "less these labels", next to encode/decode.
Reduction::{StdDev, Variance, PearsonCorr, Group} executor operators The executor's names for the IR's existing AggIntent::{StdDev, Variance, PearsonCorr, Group}.
Moments, CoMoments, MeasureStates::{Moments, CoMoments, Constant} (private) executor aggregate kernels Welford running state for any variance or correlation.
Expression::FloatText executor expressions A float as Go writes it; any label built from a value.
without_kept_columns IR schema The one definition of a without grouping's columns, read by schema derivation and the executor.
NonASAPOp::inputs, NonASAPOp::input_scope, NonASAPOp::map_input_columns (pub(crate)), NonASAPOp::map_parts (private) IR operator An operator's inputs, its input scope (also read by Join validation), and one walk over every field that renumbers column references and maps children; map_children and map_input_columns wrap it, as ScalarExpr::map_parts does for expressions (now pub(crate)).
AggIntent::try_map_cols, GroupKeys::try_map_keys (pub) IR operator arguments The one list of where an intent or grouping holds columns, with no catch-all arm: the frontend resolves names with it, realization renumbers with it (map_columns, pub(crate) wrappers), and input_cols reads it.
SeriesIdentityError (ir::schema::series_identity) IR series identity realization Why a root gets no identity: the realization does not apply, or it would drop a column a parent reads.
positions_after_realization, moves, renumber (private, series_identity.rs) IR series identity realization Where each column sits once the identity column is inserted, whether any moved, and the renumbering that refuses a dropped column.
OptimizeError::SeriesIdentity planner facade A planning error for a defective realization, next to the facade's other stage errors.
promql_root (private, run.rs) planner facade orchestration The one rule for which realization errors fall back.
labels_removed_by_without (private, resolve.rs) frontend name resolution The labels a without aggregate below removed from every series, which a grouping key may name.
top_k_sort, lower_exact_topk, lower_count_values, check_grouping_columns, WITHOUT_PROJECTION_HELPER (private) executor lowering Lowerings of IR shapes, extracted from compile_internal; the constant names the one fixed helper index.

No new IR concept: a kept series identity under without uses the existing OutputColumnSource label variant, renamed from AddedLabel to ComputedLabel (two files) since it now also covers a rewritten identity.

Algorithm changes

Execution kernels: variance and correlation

  • Problem: stddev by (job) (m) and SELECT corr(a, b) FROM t had no kernel.
  • After:
    1. Moments::add keeps the count, the mean and the sum of squared deviations: delta = v - mean; mean += delta / n; m2 += delta * (v - mean).
    2. variance(population) divides by n or n - 1, and StdDev takes the square root.
    3. CoMoments keeps one count, both means, both sums of squared deviations and the co-moment delta_a * (b - mean_b), and divides the population covariance by both population standard deviations.
  • Correctness:
    • These are the same floating-point operations as Prometheus's aggregation() STDDEV/STDVAR and DataFusion's VarianceAccumulator. A non-finite value gives NaN, as in both.
    • The SQL NULL rules follow DataFusion.
    • CoMoments was flattened from two Moments plus its own count and means; co_moments_equal_two_moments_bit_for_bit checks the old and new layouts give the same bits over 10,000 pairs from 1e-300 to 1e300 in scale.
    • Tests: physical_semantics::{sql_variance_and_stddev_follow_their_definitions, promql_stddev_of_one_series, correlation_follows_datafusion}.
  • Complexity: O(rows) time, no per-batch allocation (the kernels read the Float64 array, an Int64 column converted once per batch). Memory: 24 bytes per group for Moments, 48 for CoMoments (was 80).
  • Limits: DataFusion's grouped corr uses a sum-of-products form, so its last bits can differ. Tests compare within 1e-12.

Execution kernel: sort with fetch

  • Problem: an exact top-k sorted every input row, then dropped all but k per group: O(rows) memory, 10–17× slower than a grouped max.
  • After: Sort { fetch: Some(k) } reads its input a batch at a time. Each group keeps a max-heap (a Vec ordered by a closure) of at most k rows, ordered by the sort keys and then by input position, so ties keep input order as the stable sort does.
    • A candidate row is compared with its heap's last kept row straight from the Arrow arrays, through one DynComparator per key, wrapped to keep the operator's NULL and NaN order (NaN after every number in both directions, -0 equal to 0). A map key is chosen by its type for decoded compare_key: Arrow does compare maps, but in another order ({a: NULL} before {a: 1}, {a: -0} before {a: 0}).
    • Only a kept row has its key values decoded (KeptKeyValues, through decode_cell). A map key's column is decoded when the batch is read, through key_column, the same NaN-in-collection check the full sort runs, so a NaN inside any row's map is an error, kept or not.
    • The heap compares held keys with compare_held_key: two numbers (neither NaN), integers or times inline, everything else through compare_key. The full sort keeps compare_key: the same arms there slowed a sort of 10^6 random numbers by 16–20 %.
    • A row that displaces the last kept row overwrites it in place, reusing its key allocation, and one sift down restores the heap.
    • A held row from the batch being read points at chunk chunks.len(), the index that batch's chunk takes. Each group is listed once per batch (a per-group stamp of the batch number, no per-batch allocation), so re-pointing the batch's rows walks each touched heap once.
    • The batch's kept rows are taken into a chunk at the end of the batch. When the chunks hold more than twice the kept rows they are gathered into one with a single interleave, and the output is one interleave in group and sort order.
  • Correctness: sort_with_fetch_equals_sort_then_limit compares it with the full sort and a per-group limit, for fetch 0–8, both directions, NULLs first and last, NaN, ±0, ±Inf and ties, across batches, grouped and ungrouped. sort_with_fetch_over_ascending_input_equals_sort_then_limit does the same over ascending input (every row displaces one), with a nullable text key before the number, 18 batches. sort_with_fetch_equals_sort_then_limit_over_random_rows runs 2,400 random cases (numbers with NULL, NaN, ±Inf and −0; nullable text; maps of nullable numbers with NULL, −0 and NaN inside; 1–3 keys, 0–2 grouping columns, fetch 0–1000) and compares rows or errors; before the map fix it failed on its first map case.
  • Complexity: O(rows · log k) time; memory O(groups × k) rows, plus one input batch and an 8-byte stamp per group. A held row is 32 bytes plus its key values (was 56 plus a Vec). sort_with_fetch_holds_groups_times_fetch_rows runs 262,144 rows in a 1 MiB limit that the full sort exceeds.

Compile: exact top-k

  • Before: Aggregate{Reduce, [TopK]} was refused. Stage 2 no longer lowers it, so no path ran it.
  • After: lower_exact_topk: top_k_sort (by ranked_column, descending, NULLs and NaN last, fetch: Some(k)), then the ranked-row Project from ranked_row_projection. The current-series top-k (EvaluatePopulation{TopK}) uses the same top_k_sort instead of its Sort and Limit.
  • Correctness: the output schema is the one the IR derives from the same two functions. Tests: reduce_aggregates::{promql_exact_topk_of_grouped_rates, promql_exact_topk_by_job_of_increases}, current_series_heap.

Compile: without grouping

  • Before: groups() refused any without.
  • After: resolve_without rewrites the grouping to by(without_kept_columns). Over a series identity (is_label_set_column) it first projects the identity less the excluded labels, __name__, and a count_values label of the same name. A count_values label that the grouping removes (one the without list names, or __name__) is not in the IR's output, so lower_count_values counts per kept group with no value-text column; the executor reads that from output and does not re-derive it. A ranking (top-k, Sort, Limit) over a series identity is refused, because it keeps each row's labels.
  • IR: without_output_schema marks that kept identity as computed (ComputedLabel), not copied from the input, so Stage 1 does not carry the input identity's column properties to it.
  • Correctness: every kept label column is a function of the identity, so the grouping equals Prometheus's. Tests: reduce_aggregates::{promql_sum_without_instance, promql_count_values_without_instance, promql_count_values_without_overwrites_a_kept_label, promql_count_values_label_excluded_by_without_is_dropped, promql_count_values_without_replaces_a_kept_label_column, promql_count_values_name_label_under_without_is_dropped}, exact_reduce_execution::without_partitions_over_a_table_rank_per_kept_column, aggregate_schema::tests::{a_without_aggregate_computes_its_kept_series_identity, count_values_adds_its_label_by_and_without_keys, a_count_values_label_replaces_a_kept_column_of_its_name, a_count_values_label_excluded_by_without_is_not_added, a_count_values_name_label_under_without_is_not_added}.
  • Complexity: one extra projection, only for without over a series identity. A Utf8 identity column is dictionary-encoded, so the projection rewrites the dictionary's texts, once per entry, as &str into a new values array, and keeps the input's keys (identity_without_per_entry): no Value per identity and no re-hashing into a new dictionary. Two identities that differ only in an excluded label become one text twice in the dictionary; the group table resolves each entry by its text, so they group together (identities_equal_but_for_an_excluded_label_aggregate_in_one_group). A dictionary much larger than its batch, or a malformed text, takes the per-row path, so errors stay those of row-by-row evaluation. series_identity_without scans the canonical JSON's "name":"value" tokens, stepping over escapes, and copies the kept pairs' bytes into a String sized to the input: no map, no decoding, no re-encoding. A name token without escapes is compared with the excluded names byte for byte; one with escapes is compared with each excluded name's encoding (so no name is encoded per call in the common case). series_identity_without_equals_decode_filter_encode checks it byte for byte against decode/filter/encode over 145 identities (escaped quotes, backslashes, control characters, unicode, empty values) and 16 exclusion sets including none and every label; malformed input is an error; series_identity_without_equals_decode_on_mutated_identities fuzzes 20,000 mutated identities: no panic, and byte-equal to decode/filter/encode wherever decode accepts the input.

IR: identity realization renumbers parent column references

  • Before: the frontend binds a parent's column ids while a without aggregate's output is still [sum]: the sort key of topk(1, sum without (instance) (m)) is Column(0). with_promql_series_identity then makes that output [$promql_series_identity, sum] and leaves the key at Column(0), so the ranking read the identity text. It answered {job="b"} 2 instead of {job="a"} 7; sort, sort_desc and bottomk ordered by the identity text too, for sum and count.
  • After: at every node, with_promql_series_identity compares the input scope before and after its inputs were realized (positions_after_realization: the same columns in the same order, identity columns inserted). If any position moved, map_input_columns renumbers every column reference into that scope: sort keys and partitions, filter predicates, projections, limit partitions, aggregate keys, inputs and filters, relabel values, dedup columns, sample-by keys, window arguments, partitions and order keys, join predicates. An Aggregate's having indexes its own output, so it is renumbered by how that output moves (positions_after_realization over the node's output before and after; PromQL emits no having). A reference to a column realization does not keep is SeriesIdentityError::DroppedColumn, not a read of another column.
  • Other places a child gains columns after its parent was bound: the frontend's PromqlScalarOp and PromqlMap realize the identity before binding (which is why filters and arithmetic over the same aggregate were right); the planner, executor and devtools call this one function; canonicalize only rewrites a node's columns when no parent reads it by position (read_by_position); OperatorNode::with_new_children keeps positions and is used only with replacements of the same columns. None has this bug.
  • Correctness: tests first, all failing before the fix: reduce_aggregates::{promql_rankings_over_sum_without_rank_by_value, promql_rankings_over_count_without_rank_by_value} (topk, bottomk, sort, sort_desc, and topk with a job matcher, which binds job first), and the control promql_topk_by_job_ranks_within_each_job; unit test series_identity::tests::realization_renumbers_parent_references_into_a_without_aggregate (sort key, filter, projection).
  • Complexity: one pass over each node's input fields during realization, once per query root; no change at run time.

Planner: when realization falls back

  • Before: Stage1ToStage3Pass replaced every realization error by the unrealized root (unwrap_or_else(|_| Rc::clone(node))), so the new "a parent reads a dropped column" error would have been planned silently without labels.
  • Why the fallback exists: realization covers only some operators. Over the frontend's PromQL corpora (5,982 lowered roots), the 54 that it does not cover all fail with "no dynamic series-identity realization": limitk/limit_ratio (PromqlSeriesSample) and info() (PromqlInfoEnrich). The other refusals (a closed source, a without ranking) are not reached: the frontend rejects topk without first.
  • After: promql_root falls back only for SeriesIdentityError::NotApplicable (an operator realization does not cover, or a source that is not an open PromQL source). DroppedColumn and InvalidRealizedOperator (an operator rebuilt after renumbering whose schema no longer derives; it used to be wrapped as NotApplicable and fall back silently) are OptimizeError::SeriesIdentity. No well-formed input reaches InvalidRealizedOperator today: every renumbered reference points at the same-named, same-typed field, and the only inserted columns are Utf8 identity columns that no name or sample-value lookup picks. So it has no IR-level test; promql_root is tested with it directly. Tests: run::tests::{a_promql_root_realization_does_not_apply_to_is_planned_as_is, a_realization_that_drops_a_read_column_fails_planning, a_realization_that_builds_an_invalid_operator_fails_planning}, e2e_plan::plans_a_promql_root_series_identity_does_not_cover_over_its_labels; IR: series_identity::tests::{realization_renumbers_a_having_over_the_aggregate_output, renumbering_a_dropped_column_is_an_error}.

Nested aggregates over without, avg without several labels, absent labels

  • sum by (job) (sum without (instance) (m)) failed at install with summary update column missing or ambiguous: the outer sum's state reads the sample value (ColumnRef::SampleValue), which the planner resolves as the value column or, after an aggregate renamed it, the one numeric column (resolve_column_ref); the executor's named_column looked only for value. It now calls resolve_column_ref, so both read sum of [job, $promql_series_identity, sum]. Test first: reduce_aggregates::promql_aggregate_over_sum_without (also max without (zone) over it). 7 of the 21 corpus queries with this error now install; the other 14 stop at the next gap, native output type differs from Planner output (a global aggregate over an aggregate, A global PromQL aggregate over another aggregate answers one row for an empty input #1348) or a Float64-only binary operator over an Int64 count.
  • avg without (instance, zone) (m) failed with column outside scalar input scope: Pass 1's exact avg casts its sum and count to Float64 with a Project that kept by.len() key columns. Under without the output has the kept labels (here only the identity), not by.len() columns. last_as_float64 now keeps every column before the value. Test first: reduce_aggregates::promql_avg_without_two_labels.
  • sum without (instance) (sum without (instance) (m)) failed with column instance not found: a grouping key absent from an open schema was an error. A key that a without aggregate below removed (labels_removed_by_without, through filters and label-keeping projections) is now absent, as in Prometheus: as a without key it removes nothing, as a by key it is the empty label. Test first: reduce_aggregates::promql_without_an_absent_label_is_a_no_op; unit: resolve::tests::group_keys_promql_still_errors_on_open_schema.
  • A closed schema with a series identity no longer proves a label absent. The fuzz harness found max without (instance) ((10 * min without (zone, region) (m)) unless (sum without (instance, zone) (m))) keeping instance: the left side, realized early by the scalar operator, is the closed [$promql_series_identity, min], so the existing closed-schema rule dropped instance from the grouping although the identity has it. That query now fails to plan instead (resolve::tests::group_keys_promql_keeps_an_absent_key_of_a_closed_identity_schema). The corpus plan diff shows no query that planned before stops planning.

An Int64 count under a kept series identity. A cross-series count answers Int64, with by or without. Under without its output keeps a series identity, so the lowering adds the label rewrite that drops __name__ (series_without_name). That rewrite only moves labels, so its layout now takes the one numeric column (Float64 or Int64) as the sample value. Operators that compute on the value read it as Float64 and go through float_layout, which still refuses an Int64 value: a binary operator between vectors, histogram_quantile and label_replace. One rule: label rewrites carry any numeric sample, and computations need a Float64 sample.

Performance

Measured with the efficiency review's round-3 harness: release build, pinned to one idle core, median of 5 runs, 8192-row batches, ns per row, peak MB from a counting allocator. Round 3 is the previous head (3fa423f3c), round 4 this head, both built and run on the same host back to back. 10^6 rows unless noted; k = 5 unless noted.

Exact top-k, random values:

partitions Sort + Limit (before the PR) Sort{fetch:5} round 3 → 4 Sort{fetch:100} round 3 → 4 grouped max peak MB, fetch 5
10 1046 59 → 61 65 → 63 48 0.1
10^2 871 63 → 63 103 → 89 49 0.2
10^3 747 79 → 79 313 → 235 53 1.1
10^4 656 132 → 128 – 76 10.4

Exact top-k, ascending values (every row displaces its group's last kept row, as with monotone counters):

partitions Sort{fetch:5} round 3 → 4 Sort{fetch:100} round 3 → 4 grouped max
10 169 → 143 414 → 233 46
10^2 170 → 144 540 → 279 49
10^3 195 → 163 635 → 384 55
10^4 235 → 213 – 76
  • Listing each group once per batch (efficiency F1) and inlining number comparisons in the heap (F2) together save 9–16 % at fetch 5 and 40–48 % at fetch 100 on ascending input. At fetch 100 and 10 partitions, F1 alone gives 414 → 345 and F2 then 345 → 233.
  • The unbounded sort is unchanged (Sort then Limit, 10^6 random numbers: 1040 → 1058 at 10 partitions, 743 → 738 at 10^3).
  • Ascending input is still 3–4× a grouped max at fetch 5 and 5–7× at fetch 100 (open question below).

sum without (instance) over a 4-label identity, ns per row:

rows distinct series round 3 round 4 the projection alone, round 3 → 4 sum by identity (reference)
10^5 10^4 530 247 518 → 155 223
10^6 10 23 18 5 → 1 17
10^6 10^3 97 74 77 → 21 93
10^6 10^4 557 247 540 → 158 163
10^6 10^5 559 247 546 → 159 316

The identity rewrite alone (series_identity_without) is unchanged: 146 ns per identity with 4 labels, 350 with 12. sum without at 10^4 or more series per batch is now 1.5× sum by the identity, down from 3.4×; at 10^5 series it is faster than sum by the identity, since it groups 30 kept label sets instead of 10^5 identities.

Decisions and open questions

Coordination with #1241

Decided

  • The SQL tests declare the table "at_rest": with ingesting, Stage 3 selected an ingestion-time view over the whole table, which installation refused (An ingestion-time view with no time window below it (SQL table, PromQL offset over raw rows) is planned but refused at install #1288).

  • series_identity_without checks only structure, not canonical form (functional F3): a check that names strictly increase and escapes are the ones encode_series_identity writes made the rewrite 18 % slower with 4 labels and 34 % with 12 (146 → 172 and 349 → 469 ns per identity in the review's microbenchmark; a first version cost 28 % and 42 %). That is above the 10 % budget, so the doc says the input must be canonical instead. Every identity the executor writes is canonical.

  • The inline number comparison is on the heap only (compare_held_key), not at the top of compare_key: at the top of compare_key it made the unbounded sort of 10^6 random numbers 16–20 % slower (1040 → 1203–1236 ns per row) in four variants tried.

  • count_values formats every row's value before grouping (efficiency review E3, not done). Grouping by the float first would format once per group, but the group table normalizes -0 and 0 into one key, while Go writes them as "-0" and "0"; Prometheus keeps them apart. It is a constant factor (3.5–4×) on count_values only.

  • Wrong ranking over a without aggregate: fixed at the root, in identity realization (see Algorithm), not refused.

Open

  • Ascending-input top-k is still 3–4× a grouped max at fetch 5 and 5–7× at fetch 100. Left: a single key kept inline in HeldRow instead of a Box<[Value]> (the review measured 10–23 % more, mostly at fetch 100 with many held rows), and the heap's own cost (an ascending row sinks from top to bottom, about 3.5 comparisons, plus about 15 % for decoding its key). The efficiency review recommends stopping after F1 and F2, so neither is done here.
  • promql_rows is a runtime codec inside physical_planner (layering R3-L2); moving the identity codec to values/series_identity.rs is a follow-up, not done here.
  • Heavy-hitter promotion could renumber positional readers instead of being skipped #1347: heavy-hitter promotion in canonicalize could renumber positional readers, as realization now does, instead of being skipped (round-5 A3).
  • A global PromQL aggregate over another aggregate answers one row for an empty input #1348: a global PromQL aggregate over another aggregate answers one row for an empty input (count(sum by (job) (m >= 1000)) gives {} 0; it does on the previous head too). The executor tells SQL from PromQL by the identity column. The fuzz harness's two remaining mismatches are this bug, and it likely causes the native output type differs install errors of 12 nested-aggregate corpus queries.
  • Labels absent from a binary operator's side: a without or by key that only the series identity carries (the left side of (10 * min without (zone) (m)) unless n) is now a planning error rather than a wrong answer; resolving it from the identity is not done here.

Review status

Round 1 (abstraction, layering, functional, efficiency):

Finding Status
F1 count_values(...) without (...) did not install; a count_values label colliding with an identity label grouped wrongly Fixed: tests first (both failed with native output type differs from Planner output); without_output_schema adds the label column; the without projection drops the label from the identity
F2 topk without refused by the frontend Not this PR (frontend)
F3 grouped corr last bit vs DataFusion Documented (Limits above)
L1 kept identity under without recorded as an input copy Fixed: AddedLabel, documented with the example; series_identity doc names without
L2 physical-compile-coverage.md Updated: new "exact reduce aggregates" section; the earlier wrong without claim corrected; totals stated
L3 inline lowerings in compile_internal Fixed: lower_exact_topk, lower_count_values next to resolve_without; compile_node doc notes the multi-operator lowerings
L4 SQL nullability derived twice Comment notes the IR defines it, the executor checks it
L5 PR body interfaces paraphrased Fixed: real definitions above
L6 comments without examples Fixed: FloatText, Moments, CoMoments, without_kept_columns, Variance
L7 names sum_squared_deviations; output_has_one_label_set_per_group; "float text" in FloatText errors; numbers removed (A8)
L8 / A2 / E4 duplicated CoMoments state Fixed: six flat fields, deltas computed once, 80 → 48 bytes; bit-identity test
L9 issue number in a test comment Fixed: the bug is described
A1 two kernels with #1241 Coordination above
A3 duplicated top-k lowering Fixed: one top_k_sort for the exact and the current-series top-k
A4 / A5 / A9 copied test helpers, duplicate cases Fixed: physical_common::{execute_series, series_labels, labels, RawDataSample} and time_series::{run, sample, labels}; the integration topk, topk by and without cases dropped (the planner tests run them end to end)
A6 bounds check repeated Fixed: check_grouping_columns
A7 helper id 3 hard-coded Fixed: WITHOUT_PROJECTION_HELPER, the last helper index; helper_id's doc states the allocation
A8 / E5 two numeric readers, per-batch Vec<Option<f64>> Fixed: one float64_values reader returning a Float64 array; kernels loop over it
E1 exact top-k sorts the whole input Fixed: Sort { fetch }, measured above
E2 without identity rewrite cost Fixed: borrowed parse, no canonical check, one encode; body corrected (once per distinct identity per batch)
E3 count_values formats per row Not done: see Decided

Round 2 (abstraction, layering, functional, efficiency):

Finding Status
F1 a count_values label also in the without list split the groups and stayed in the output Fixed, tests first (promql_count_values_label_excluded_by_without_is_dropped failed with {job=a,zone=2} 3, {job=a,zone=5} 1; a_count_values_label_excluded_by_without_is_not_added): the IR adds no label, lower_count_values counts per kept group with no projection
Layering R2-L1 a count_values label colliding with a by or kept column was recorded as an input copy Fixed, test first (a_count_values_label_replaces_a_kept_column_of_its_name, by and without, failed with InputColumn(2)): that position is ComputedLabel, Utf8 non-null. Planner test promql_count_values_without_replaces_a_kept_label_column covers the executor side
R2-L2 / abstraction L1 label-set rule in two layers Fixed: ir::schema::is_label_set_column, read by without_output_schema and resolve_without
Abstraction L2 count_values label push copied Fixed: push_count_values_label for both paths
Abstraction L3 lowerings re-match their caller Fixed: compile_internal binds keys, filters, k and label and checks one input and no filter once; lower_exact_topk and lower_count_values return their operators, and the four unreachable branches are gone
L4 / R2-L3 identity encoding in two files; efficiency E3 streaming rewrite Fixed: promql_rows::series_identity_without, a streaming copy; identity_less deleted; byte-equality test first
L5 test helpers overlap across test crates Left as the review allows (no shared home today)
Efficiency E1 per-row decode and key allocation Fixed: per-key DynComparator against the heap's last row, decode only kept rows, in-place replace with one sift down. peek_mut needs Ord, which needs the sort keys in every row (E2), so the heap is a Vec with a closure and the replace is the same in-place overwrite plus sift. Key kept as Box<[Value]>, no new dependency. Ascending input 283 → 168 ns per row (10^6 rows, 10 partitions), short of the ~2× the review's model predicted
Efficiency E2 HeldRow repeats the sort keys; concat then take Fixed: HeldRow is 32 bytes; compact and the output use Batch::interleave from (chunk, row) indices. Peak at 10^5 rows × 10^4 partitions 11.9 → 9.1 MB, now below the full sort's 10.2
Efficiency E4 Int64 converted per batch Not done (optional; 2–6 %)
Abstraction N1 tie-break written twice Fixed: sort_order
N2 float64_values repeats floats Fixed: floats(column).or_else(..)
N3 TwoMoments Kept, commented as a one-time equivalence proof
Layering N1 AddedLabel Renamed ComputedLabel (two files)
Layering N2–N5 PR body Fixed: enum Kind {; Operator::sort callers; DataFusion's TopK wording; #1288
Layering N6 coverage rows Moved: PromQL aggregates under row 1, without under row 2, SQL under row 29
Layering N7 IR doc wording Fixed: "computed by the aggregate"; without_kept_columns names the identity less the excluded labels

Round 3 (abstraction and layering, functional, efficiency):

Finding Status
Abstraction A3-1 = functional F2: map sort keys took Arrow's order in Sort{fetch}, the heap took compare_key's; a NaN inside a map was checked only on kept rows Fixed, test first (sort_with_fetch_equals_sort_then_limit_over_random_rows failed on seed 9, case 2, map key, fetch 5): key_comparator picks the decoded compare_key by type for a map; a map key's column is decoded and NaN-checked when its batch is read, by the same key_column as the full sort. Comment and Algorithm text corrected
Functional F1: count_values("__name__", v) without (...) kept __name__ and split by value Fixed, tests first (promql_count_values_name_label_under_without_is_dropped gave 4 rows with __name__; a_count_values_name_label_under_without_is_not_added): removed_by_without treats __name__ as removed; the lowering reads that from output and needs no change. Now {job=a} 1, {job=a,zone=y} 1, {job=a,zone=z} 2
Functional F3: series_identity_without accepts malformed and non-canonical input Doc reworded, check not added: it cost 18–34 % (see Decided). Fuzz test against decode added
Efficiency F1: touched held one entry per kept row Fixed: a per-group stamp of the batch number; touched reused across batches
Efficiency F2: heap comparisons through Value::compare Fixed on the heap (compare_held_key), not in compare_key (see Decided). Equality tests cover NULL and NaN order
Efficiency F3: without limited by per-identity Value work Fixed: per-dictionary-entry rewrite into a new values array, keys kept; duplicate texts group correctly (test first, passed before and after). 10^6 rows, 10^4 series: 557 → 247 ns per row
Abstraction A3-2: KeptKeyValues::write copied decode's cell table and NaN check Fixed: arrow_columns::decode_cell, read by decode's number, bool, time, date and interval arms and by write; one NaN check in key_column
A3-3: count_values aggregate built twice Fixed: built once after the branch
A3-4: redundant excluded-label guard in resolve_without Dropped, comment says removing a label twice changes nothing
A3-5: READING sentinel Replaced by chunks.len(), the index the batch's chunk takes
Layering R3-L1: OutputColumnSource doc missed the replaced column Fixed in the doc and the Interfaces diff
Layering R3-L2: promql_rows placement Follow-up note (Open)

Round 4 (verification of round 3): approved. These are the final fixes:

Finding Status
Medium: count without (instance) (m) failed at install with PromQL vector requires labels, time and one value Fixed, tests first (promql_count_without_instance failed with that error; group without and stddev without passed before and after): the cause was the series_without_name rewrite's Float64-only layout (see Algorithm). It also unblocks count without (...) * 2 and count without (...) > 1. sort, topk and bottomk over it now install, but rank wrongly (see Open). count_values ... without already installed, since it lowers to an Aggregate that adds no rewrite
Low: sort_with_fetch_equals_sort_then_limit_over_random_rows took 63 s in a debug build Fixed: every seed (each key kind) still runs; 40 cases per seed in a debug build, 400 in release. 2.4 s in debug
Nit: a null value inside a LabelIdentity dictionary becomes NULL, not "" Left: the encoder never writes one
Nit: a map sort key is decoded twice per batch Left: no plan produces a map key today

Wrong-ranking fix (81d429edc):

Finding Status
High: an ungrouped ranking or sort over a without aggregate ranked by the identity text (topk(1, sum without (instance) (m)) gave {job="b"} 2) Fixed at the root, tests first (see Algorithm): identity realization renumbers every parent column reference; a reference it cannot keep is an error

Round 5 (abstraction and layering, functional, of 81d429edc):

Finding Status
A1 AggIntent column variants listed three times, catch-all arms Fixed: AggIntent::try_map_cols with every variant and no catch-all; map_columns, input_cols and the frontend's resolve_agg_intent use it (75 lines deleted there). GroupKeys::try_map_keys for resolve_group_keys and map_columns; the PromQL lenient path keeps its filter
A2 map_input_columns repeats map_children's walk Fixed: private NonASAPOp::map_parts; both are one-line wrappers; the copied pred/key helpers are gone
A3 canonicalize refuses where realization renumbers Filed #1347
A4 two impl Reduction blocks Merged
L1 having checked by width Renumbered instead, by positions_after_realization over the aggregate's output; a dropped column is an error. Documented in with_promql_series_identity's doc; test realization_renumbers_a_having_over_the_aggregate_output
L2 input scope computed in three places NonASAPOp::input_scope next to inputs, used by realization and Join validation
B1 / B2 PR body Window partitions and order keys listed; interface docs copied from the code
F1 nested aggregate over without fails to install Fixed, test first: the executor resolves the sample value as the planner does (see Algorithm); 7 of 21 corpus queries install, the rest hit #1348 or the Int64 gap
F2 avg without two or more labels fails to plan Fixed, test first: Pass 1's avg cast keeps every column before the value
F3 repeating an excluded label fails Fixed, test first: a key a without below removed is absent (no-op for without, empty label for by)
F4 realization errors swallowed Fallback only for NotApplicable (today limitk, limit_ratio, info(): 54 of 5,982 corpus roots); DroppedColumn is OptimizeError::SeriesIdentity; tests for both paths
Fuzz on this head New wrong answer found and fixed: a closed identity-carrying schema dropped a label that the identity has (see Algorithm). Two remaining mismatches are #1348, which predates this PR

Round 6 (verification of c462ccad5): approved. Final polish:

Finding Status
N1 a schema error after renumbering was wrapped as NotApplicable, so promql_root fell back silently Fixed: SeriesIdentityError::InvalidRealizedOperator(SchemaDerivationError), a planning error like DroppedColumn; NotApplicable keeps only "operator not covered" and "not an open PromQL source"; both docs updated; promql_root test. Not reachable from a well-formed input (see Algorithm), so no IR-level test. The 253-query without corpus plan diff is unchanged: all 506 entries identical to round 6
N2 an absent label is detected only one without level down Follow-up #1349
Double avg/stddev without plan-selection error Follow-up #1349
Body nits (Follow-ups, Verification crate set) Follow-ups updated; Verification row added

Ready for review.

Verification

Scope Command Result
Round-6 polish (N1) cargo nextest run -p asap-types -p asap-planner --no-fail-fast 299 passed, 1 failed: summary_sharing::identical_ungrouped_queries_share_their_producers, which fails on main too. New: run::tests::a_realization_that_builds_an_invalid_operator_fails_planning
Round-6 lint cargo clippy -p asap-types -p asap-planner --all-targets --all-features -- -D warnings, cargo fmt --all --check clean
Round-6 plan diff the 253-query without corpus × exact and 5 % targets, DAG hash per entry, vs c462ccad5 all 506 entries identical: nothing that planned before now errors
Round-5 review fixes (c462ccad5), by test name cargo nextest run --no-fail-fast -p asap-types -p asap-frontend-promql -p asap-frontend-common -p asap-planner -p asap-executor -p asap-integration-tests 1,030 passed, 8 failed, the same 8 by name that fail on the previous head and on main (joint_search_equivalence ×2, materialized_views ×1, plan_sharing ×2, promql_to_post_asap ×2, summary_sharing::identical_ungrouped_queries_share_their_producers). New: reduce_aggregates::{promql_aggregate_over_sum_without, promql_avg_without_two_labels, promql_without_an_absent_label_is_a_no_op} (all three failed on 81d429edc), e2e_plan::plans_a_promql_root_series_identity_does_not_cover_over_its_labels, two run::tests, two series_identity::tests, one resolve::tests
Round-5 lint cargo clippy --workspace --all-targets --all-features --locked -- -D warnings, cargo fmt --all --check clean
Round-5 fuzz the functional review's reference evaluator and fuzzer: seeds 1–6 × 300 expressions, the fixed cases, the without/by pairs 1,800 expressions: 810 match the reference exactly, 17 ties match by value, 2 mismatches, both #1348 (a global count over an empty aggregate gives {} 0; its by rewrite gives the same wrong row; on the previous head too). Of the 63 cases that failed to plan only in the without form on 81d429edc, 58 now plan: all 35 column outside scalar input scope (F2), all 13 summary update column (F1), 10 of 15 column not found (F3); the 5 left name a label only a binary operator's side or a label_replace carries. Fixed cases (205): 0 mismatches; without/by pairs (190): 108 same, 1 same values with labels differing, as before. The harness's values() panics on NaN (partial_cmp().unwrap()), which now stops seeds 2–5 early; those seeds ran with total_cmp there, nothing else changed
Round-5 plan diff the 253-query without corpus × exact and 5 % targets, DAG hash per entry, vs 81d429edc all 372 entries that installed before have byte-identical DAGs; 14 more install (7 of the 21 summary update column queries); none stops planning
Round 5 fix (81d429edc) vs previous head c045e05e5, by test name cargo test -p asap-types -p asap-frontend-common -p asap-frontend-promql -p asap-planner -p asap-executor -p asap-integration-tests --no-fail-fast types 161+78+8, frontend-common 50, frontend-promql 219, executor 297 passed. Planner planner 39 passed, 1 failed (summary_sharing::identical_ungrouped_queries_share_their_producers, fails on the previous head too); the two new ranking tests failed on the previous head and pass. Integration 165 passed, the same 7 failing on both
Round 5 lint cargo clippy -p asap-types -p asap-frontend-common -p asap-frontend-promql -p asap-executor -p asap-planner -p asap-integration-tests --all-features --all-targets -- -D warnings, cargo fmt --all -- --check clean
Round 5 plan diff the round-2 review's 253-workload without corpus, exact and 5 % targets, DAG hash per workload, previous head vs fix 505 of 506 plans byte-identical; the one change is topk(3, sum without (device, fstype, mountpoint) (node_filesystem_size_bytes)) (both targets), the bug's shape: same nodes, length and cost, its sort key now reads the sum
Executor and types, final head cargo nextest run -p asap-executor -p asap-types --all-features 535 passed, 1 skipped (incl. a_label_rewrite_carries_an_int64_count_and_a_computation_refuses_it; sort_with_fetch_equals_sort_then_limit_over_random_rows 2.4 s in debug)
Executor, round 3 cargo test -p asap-executor --all-features lib 119 passed, 1 ignored (incl. series_identity_without_equals_decode_filter_encode, series_identity_without_refuses_a_malformed_identity, series_identity_without_equals_decode_on_mutated_identities, co_moments_equal_two_moments_bit_for_bit); executor 171 passed (incl. sort_with_fetch_equals_sort_then_limit, sort_with_fetch_over_ascending_input_equals_sort_then_limit, sort_with_fetch_equals_sort_then_limit_over_random_rows, identities_equal_but_for_an_excluded_label_aggregate_in_one_group, sort_with_fetch_holds_groups_times_fetch_rows); memory-limit binaries pass
Integration, final head vs 1beca789d cargo nextest run -p asap-integration-tests --all-features --no-fail-fast 165 passed, 7 failed on both, by name the same tests (joint_search_equivalence ×2, materialized_views ×1, plan_sharing ×2, promql_to_post_asap ×2); none new
Lint, final head cargo clippy -p asap-executor -p asap-types -p asap-planner -p asap-integration-tests --all-features --all-targets -- -D warnings, cargo fmt --all -- --check clean
Planner e2e cargo nextest run -p asap-planner --all-features --test planner reduce_aggregates 16 passed (incl. promql_count_without_instance, promql_group_without_instance, promql_stddev_without_instance)
Frontend IR → executor cargo test -p asap-integration-tests --test integration -- exact_reduce_execution per_series_exact_execution 10 passed
Types cargo test -p asap-types --all-features lib 160 passed; integration 78 and 8 passed
Lint cargo clippy -p asap-executor -p asap-types -p asap-planner -p asap-integration-tests --all-features --all-targets -- -D warnings, cargo fmt --all -- --check clean

From the round-2 functional review (on dec40d748):

  • Sort{fetch} matched sort then limit on 400 random cases (1–30 batches including empty ones; no, one or two grouping columns; one or two keys in random directions, NULLs first or last; NaN, ±Inf, −0 and duplicates; fetch 0, 1, 2, 3, 5, 1000): identical rows in identical order. All 13 raw-series top-k cases were byte-identical to base.
  • Marking the kept identity as computed left every non-count_values without DAG byte-identical to base over 215 planned workloads (231 corpus without queries plus joint workloads, exact and 5 % targets); only the 9 count_values ... without (instance) workloads changed (their IR schema gains the label column).
  • No test that passes on base fails on head.

Follow-ups

@zzylol zzylol added enhancement New feature or request promql PromQL front-end lowering (L1→L2) sql SQL front-end lowering (DataFusion → L2) labels Nov 7, 2026
@zzylol
zzylol force-pushed the exec/x2-reduce-aggregates branch from 33293b7 to aef6562 Compare November 15, 2026 05:25
…s, exact topk and without grouping

Reduce aggregates the frontends emit that the executor refused:
Reduction::StdDev/Variance (Welford, population and sample, SQL NULL
rules), Reduction::PearsonCorr (DataFusion corr), Reduction::Group,
count_values as a value-text column plus a count, an exact
Aggregate{Reduce, [TopK]} as per-partition Sort + Limit + ranked-row
Project, and GroupKeys::without resolved to the kept columns (series
identity less the excluded labels and __name__) for Aggregate,
SummaryAgg, Sort and Limit. SQL sample STDDEV/VAR_SAMP are typed
nullable.

The exact top-k lowering is the only one now that Stage 2 no longer
lowers TopK, so the executor reads the ranked-row helpers in
types::ir::schema::topk, which become pub. The planner tests run the
installable plan's DAG directly and declare the SQL table at rest; an
ingesting whole-table aggregate hits a planner install bug (#1320).
- count_values ... without (...) installs and matches Prometheus: the IR
  adds the value-label column under without, and the without projection
  drops a same-named label from the series identity before grouping.
- The IR marks a series identity kept by without as computed
  (OutputColumnSource::AddedLabel), with the rule documented.
- Sort gains fetch: Option<u64>, a bounded per-group heap keeping the
  first n rows of each group; exact top-k and the current-series top-k
  lower to it instead of Sort then Limit.
- The without identity rewrite parses borrowed labels and encodes once.
- CoMoments is six flat fields (bit-identical, tested); the variance and
  correlation kernels read the Float64 array directly.
- lower_exact_topk / lower_count_values / top_k_sort extracted from
  compile_internal; WITHOUT_PROJECTION_HELPER names the fixed helper index;
  one grouping bounds check.
- Test helpers shared through physical_common and time_series; duplicate
  integration top-k and without cases dropped.
- physical-compile-coverage.md: reduce aggregates section, without row fixed.
- count_values: a label that collides with a by or kept column makes that
  position a computed Utf8 label; a label the without list names is
  removed again, counting per kept group (IR schema and lowering).
- One label-set predicate, ir::schema::is_label_set_column, read by the
  IR and resolve_without; one count_values label push for both paths.
- compile_internal binds keys, filters, k and label once; the top-k and
  count_values lowerings return their operators.
- promql_rows::series_identity_without: streaming rewrite of the
  canonical identity JSON, byte-equal to decode/filter/encode.
- Sort with fetch: candidates compared with the heap's last row straight
  from the Arrow arrays, keys decoded only for kept rows, replace-top on a
  Vec heap, HeldRow without the sort-key slice, chunks gathered with one
  interleave (Batch::interleave).
- OutputColumnSource::AddedLabel renamed ComputedLabel; nits.
- Sort{fetch}: a map sort key is compared by decoded value, chosen by
  type (Arrow's map order differs); a map key column is NaN-checked
  when its batch is read, as the full sort does. Randomized test of
  Sort{fetch} against sort then limit, with map keys.
- count_values("__name__", v) without (...) drops __name__, as
  Prometheus does (removed_by_without in the IR schema).
- series_identity_without: doc says the input must be canonical (a
  canonical check cost 18-34 %); fuzz test against decode.
- Each group is listed once per batch (per-group stamp); the heap
  compares numbers and times inline (compare_held_key).
- LabelIdentity over a dictionary rewrites the dictionary's texts once
  per entry and keeps the keys.
- decode_cell shared by decode and KeptKeyValues; one NaN check
  (key_column); count_values aggregate built once; redundant guard in
  resolve_without dropped; READING sentinel replaced by chunks.len().
- OutputColumnSource doc lists the replaced-column case.
`count without (instance) (m)` failed to install with `PromQL vector
requires labels, time and one value`: its output keeps a series identity,
so the lowering adds the label rewrite that drops `__name__`, and that
rewrite's layout accepted only a Float64 value, while a cross-series count
is Int64 (as `count by` answers it). The layout now takes the one numeric
column as the value; operators that compute on the value (binary
operators, `histogram_quantile`, `label_replace`) read it through
`float_layout`, which still requires Float64.

Tests: planner `count without (instance)` and `(instance, zone)`,
`group without` and `stddev without` with hand-computed answers; an
executor test that a label rewrite carries an Int64 count and a binary
operator refuses it.

`sort_with_fetch_equals_sort_then_limit_over_random_rows` runs every seed
with 40 cases per seed in a debug build (400 in release): 63 s to 2.4 s.
…tity

topk(1, sum without (instance) (m)) answered {job="b"} 2 instead of
{job="a"} 7: the frontend binds the sort key to Column(0) of the
aggregate's [sum] output, and identity realization then inserts the
identity column ahead of the measure without updating the parent.
sort, sort_desc and bottomk ordered by the identity text the same way.

with_promql_series_identity now compares each node's input scope before
and after realization and, when a column moved, renumbers every column
reference into that scope (NonASAPOp::map_input_columns, with
map_columns on GroupKeys, Reduction and AggIntent). A reference to a
column realization does not keep is an error.
- One generic column walk per IR argument type, with no catch-all arm:
  AggIntent::try_map_cols (map_columns, input_cols and the frontend's
  resolve_agg_intent use it) and GroupKeys::try_map_keys. A new
  column-carrying intent now fails to compile instead of being skipped.
- NonASAPOp::map_parts walks every variant once; map_children and
  map_input_columns are thin wrappers. NonASAPOp::input_scope sits next
  to inputs and serves realization and Join validation.
- Series identity realization renumbers an Aggregate's HAVING by how its
  output moves, and returns a typed SeriesIdentityError. The planner
  falls back to the unrealized root only when realization does not apply
  (limitk, info()); a dropped column is OptimizeError::SeriesIdentity.
- Nested aggregates over a without aggregate install: the executor
  resolves a summary update's sample value as the planner does (the one
  numeric column after an aggregate renamed it).
- avg without two or more labels plans: Pass 1's avg casts the last
  column, whatever the number of kept labels.
- A grouping key that a without aggregate below removed is absent, as in
  Prometheus; a closed schema that carries the series identity no longer
  proves a label absent (it may be in the identity).
A schema error from rebuilding an operator after series identity
renumbering was wrapped as NotApplicable, so promql_root silently
planned the unrealized root. It is now
SeriesIdentityError::InvalidRealizedOperator, a planning error like
DroppedColumn. NotApplicable keeps only operators realization does not
cover and sources that are not open PromQL sources.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request promql PromQL front-end lowering (L1→L2) sql SQL front-end lowering (DataFusion → L2)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant