Repository navigation
Description
Problem, by example
Workload h02b: two range queries, both every 120 s at phase 0, step 60 s, with spans 180 s and 420 s. Data: 1,000 series of m, 1 sample/s, raw data kept.
sum_over_time(m[1m] @ start())
At a request time T, query 0 must return, at each of its steps, sum_over_time(m[1m]) evaluated at T − 180 s. Query 1 must return the same expression evaluated at T − 420 s.
-
Today, install: the planner selects a plan, but installing it fails:
QueryRunNotExecutable { reason: "invalid DAG: per-entity summary requires a resolved raw time range" }Every
@ start()and@ end()workload in the refactor(stage3): select over execution contexts: x_{κ,v}, y_{κ,u,c,i}, h_v and constraint (R) (#1207 PR 7) #1237 review fails this way, instant and range alike (h01, h02, h02b, k04s, k08, k19, k20, …). This was observed with the round-2 review harness at refactor(stage3): select over execution contexts: x_{κ,v}, y_{κ,u,c,i}, h_v and constraint (R) (#1207 PR 7) #1237's head33e7d6d56. -
Today, evaluation: even with a plan that installs, nothing evaluates a range request. No caller sets a run's query range, and the review harness evaluates instant queries at one time t only.
-
Expected: the plan installs, and a range request at T returns each step's value with
@ start()= T − span and@ end()= T. The answers match an exact reference at two or more request times.
The planner fixes for @ (#1294, and #1237's request contexts) are checked only by objective = plan_cost today. None of them can be confirmed by execution until this works.
Why it doesn't work today
-
Install.
crates/executor/src/physical_planner/mod.rs:614-622requires a per-entitySummaryAgg(reduction: PerEntity) to read aTimeRangedirectly:let Some(NonASAPOp::TimeRange { child, .. }) = restored[input_id].non_asap() else { return Err(invalid("per-entity summary requires a resolved raw time range")); };
The PromQL front end puts the
TimeShift { at: Some(Start | End) }between them:SummaryAgg→TimeShift(@)→TimeRange→Scan. TheTimeShiftitself is planned asOperator::evaluate_ata few lines above. -
Range-request runner.
RunContext::with_query_range(crates/executor/src/runtime/context.rs:70) has no caller.anchored_context(crates/executor/src/operators/time_range.rs:98) falls back to(evaluation_time, evaluation_time)when no range is set, which is right for an instant query only. -
Reference. The review harness has no exact range-query evaluator with Prometheus
@ start()/@ end()semantics. Its reference is an instant evaluation at one t. -
Declared span. The planner costs and plans a range query for the span its workload entry declares. With Stage 3 shares nodes above an @ start() shift between queries whose anchors resolve to different times #1294's design,
@ start()is lowered as@ end()offset by that span. A request whose actual range has a different length would get T − (declared span), not its own start, and nothing checks for this.
Proposal
- Install a per-entity aggregate over an
@shift. Look through the anchoredTimeShiftwhen resolving the raw time range, asbelow_time_shiftsalready does foroffsetshifts. Evaluate the input withevaluate_at's anchored context. - A range-request entry point. Run a request [T − span, T] with
with_query_range(T − span, T)and evaluate every step t, each@input once per request.- Under Stage 3 shares nodes above an @ start() shift between queries whose anchors resolve to different times #1294's design, every anchor in a plan is
end − offset. Queries requested at the same T can therefore share one run without per-query ranges.
- Under Stage 3 shares nodes above an @ start() shift between queries whose anchors resolve to different times #1294's design, every anchor in a plan is
- A range-query reference in the review harness. Evaluate each step exactly from the raw samples, resolving
@ start()/@ end()to the request bounds. Compare at two request times. - Refuse a mismatched range. The request path checks that the range length equals the span the plan was made for, and returns an error otherwise.
Scope and acceptance
- In scope:
@ start()and@ end()in range and instant PromQL queries, including nested subqueries (k20),offsetwith@, and two queries with different spans (h02b). - Out of scope:
@ <timestamp>, which is a fixed time; planner cost changes (Stage 3 shares nodes above an @ start() shift between queries whose anchors resolve to different times #1294, Anchored inputs are over-costed: @ end() raw retention adds the span, and nested intervals on one grid add their rates #1295). - Acceptance:
- h02b, h02, k04s, k19, k20 and the two-span
@ start() offset 1mworkload install. - Their range answers match the exact reference at two request times.
- A request whose range is shorter than the planned span is refused with an error.
- The workloads come from the refactor(stage3): select over execution contexts: x_{κ,v}, y_{κ,u,c,i}, h_v and constraint (R) (#1207 PR 7) #1237 review set:
r1237/rv2/wl, generated bymk.py, plus Stage 3 shares nodes above an @ start() shift between queries whose anchors resolve to different times #1294's new cases.
- h02b, h02, k04s, k19, k20 and the two-span
Related
- Stage 3 shares nodes above an @ start() shift between queries whose anchors resolve to different times #1294:
@ start()identity; its plans can only be verified by execution after this. - refactor(stage3): select over execution contexts: x_{κ,v}, y_{κ,u,c,i}, h_v and constraint (R) (#1207 PR 7) #1237: execution contexts and
@request contexts. - Executor deadlocks when two query roots that share a producer are drained one after another #1290 and fix(executor): run each node by its plan Lifetime, so roots drain in any order #1300: executor runs whose roots share a producer. Range requests evaluate many steps over shared nodes, so they build on fix(executor): run each node by its plan Lifetime, so roots drain in any order #1300's per-
Lifetimeruntime. - Anchored inputs are over-costed: @ end() raw retention adds the span, and nested intervals on one grid add their rates #1295:
@cost overcounts.
Activity
zzylol commentedon Nov 15, 2026
The time-operators PR (#1250) is closed as superseded. Its
@part had a per-entity aggregate over an@shift and instant@ start()/@ end(). That part is re-filed here instead of in a new issue, because proposal items 1 and 2 above already cover it.What the closed PR had that is still missing on main (
59365ce80):@shift (proposal 1). It is still refused at install. Instant queries with@ <timestamp>now fail earlier, in the planner's install check:sum_over_time(c[1m] @ 1799999760)(exact, repeating every 60 s) givesinstallable plan: raw-data window of node 2 is unsupported: an @ anchor below the range(check_raw_window_below,crates/types/src/plan/materialized_view/raw_data.rs). The closed PR restricted that check to the windows a view's refresh reads. The fix here needs the same change.Do not port the closed PR's design: its
time_range(at),ExactEvaluation.atand executor walks (shifted_end,source_below,window_hull). Build on main'sOperator::evaluate_at/anchored_context, and read windows from the edgeread_windowof the approved time-walk design (#1291/#1292). The testsan_at_timestamp_pins_the_selector_windowandan_anchored_counter_window_ends_at_the_anchorincrates/planner/tests/time_operators.rs(branchexec/x4-time-operators) can be ported as instant-query acceptance tests.Related follow-ups filed from the same PR: #1318 (subqueries), #1319 (
TimeRangeover computed rows), and #748 (counter@, commented there).zzylol commentedon Nov 17, 2026
The per-series range functions added in #1241 also stop at
@. Run on #1241's branch against Prometheus 3.16.0, exact plans over raw samples, evaluated at 600 s:rate(x[1m] @ 300),increase(x[1m] @ 300),irate(x[1m] @ 300),delta(x[1m] @ 300)compile: invalid DAG: node 2: invalid DAG: value operation has no native implementation@ 330 offset 30sPrometheus answers each with the function over
(240 s, 300 s]. The per-entity range function kernel reads its window as a fixed offset back from the evaluation time (time_range_window,crates/executor/src/physical_planner/time_window.rs), which an@anchor is not; compile then fails on theTimeShiftwith an anchor. The fix belongs with this issue's@handling, so that one resolved window feeds both the summary path and the range function kernel. Subqueries over these functions are #1318 and #1248.