Skip to content

Memoize LAST_VALUE and NTH_VALUE with IGNORE NULLS so BoundedWindowAggExec can prune its buffer #26537

Open

Description

@comphead

Is your feature request related to a problem or challenge?

NthValueEvaluator::memoize returns early for LAST_VALUE and NTH_VALUE when IGNORE NULLS is set (nth_value.rs#L414-L415). For a frame with an UNBOUNDED PRECEDING start, such as the default RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, the frame start then never moves. Reading the code, this means BoundedWindowAggExec cannot prune its input buffer and keeps the whole partition in memory, while RESPECT NULLS keeps only the last row (LAST_VALUE) or the last n rows (NTH_VALUE(expr, -n)).

#26533 fixes the CPU cost of these frames (#26532), but not the buffering. I have not measured the memory use.

Describe the solution you'd like

Let memoize move the frame start under IGNORE NULLS too, keeping the rows the result can still depend on:

  • LAST_VALUE: from the last non-NULL row of the frame, or nothing before the current row if the frame has none.
  • NTH_VALUE(expr, -n): from the nth non-NULL row from the end of the frame.
  • NTH_VALUE(expr, n) with n > 0: the result is final once the frame holds n non-NULL rows, as FIRST_VALUE already does.

Describe alternatives you've considered

None.

Additional context

Raised by @andygrove in the review of #26533.

Activity

  1. self-assigned this
    on Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions