Skip to content

bench: add first_value/last_value cases for a group-by with many groups - #26554

Closed
tanmay442 wants to merge 1 commit into
apache:mainfrom
tanmay442:perf/first-last-bench-only
Closed

tanmay442 wants to merge 1 commit into
apache:mainfrom
tanmay442:perf/first-last-bench-only

Conversation

@tanmay442

@tanmay442 tanmay442 commented Oct 21, 2026 •

Copy link
Copy Markdown

Which issue does this PR close?

  • N/A

Rationale for this change

The grouped first_value and last_value accumulator allocates one slot per
group. For a group-by that touches few groups per batch, the work the
accumulator does per batch therefore grows with the number of groups in the
query rather than with the number of rows the batch carries. There is no
benchmark for that shape, so the cost cannot be measured and a change that
reduces it cannot be shown to help.

This pull request adds that benchmark. It changes nothing else.

The benchmark is a separate pull request from the change that uses it, as the
contributor guide suggests, so that the benchmark is in the merge base and the
benchmark runner has a baseline to pair the new rows against.

What changes are included in this PR?

One file, datafusion/functions-aggregate/benches/first_last.rs. The change is
purely additive, 234 inserted lines and no deleted line, so the baselines that
other pull requests compare against are unaffected.

Three cases are added, each for first_value and last_value:

  1. update_bench_many_groups groups=1000000, touched={64,8192} allocates a
    million groups for the accumulator and touches 64 or 8192 of them per batch.
    The same batch is replayed 100 times, as the other benchmarks in the file
    do. After the first iteration the accumulator already holds that batch's
    extreme for every group, so only the first iteration writes state. This
    isolates the per-batch work that scales with the number of allocated groups.
    It is not a stand-in for a query.

  2. update_bench_many_groups_distinct_batches groups=1000000, touched={64,8192} has the same shape but walks through 100 distinct
    batches. Batch b owns the groups
    [b * groups_touched, (b + 1) * groups_touched), so no two batches touch
    the same group and the accumulator writes the state of every winner, the
    same work a query does when each batch brings new rows. This is the case to
    judge a change to the accumulator by.

  3. update_bench_many_groups_dense groups=100000 is the dense counterpart:
    100,000 groups and a 100,000 row batch, so every batch touches every group.
    That is the shape where the list of touched groups is longest and the added
    per-batch cost is highest, so it is an upper bound on the price. It does not
    reproduce the shape of the eight rows of perf(aggregate): collect the first/last value winners from the touched groups #26536 that came out slower, which
    is 1,024 groups with 64 rows per group.

The two sparse cases bracket two ends rather than modelling the middle of
anything: one where the groups are all known and only the per-batch bookkeeping
runs, and one where every group is new and the state is written for every
winner.

Both sparse cases use BatchSize::PerIteration. A million-group accumulator
holds about 100 MiB, because resize_states creates one Vec<ScalarValue> per
allocated group. Criterion's SmallInput keeps ceil(iters/10) setup outputs
alive at once, so when an earlier revision of this benchmark used SmallInput,
the runner reported a peak memory of 501.9 MiB for it. PerIteration keeps one
accumulator alive. It does not change what is timed: the setup runs the same
number of times either way, and only the routine is measured.
iter_batched_ref is used so that dropping the accumulator, which is one
million small allocations, stays out of the timed region.

Observed on an x86 machine, author-local: the sparse cases take 50 ms to 96 ms
per iteration and the dense case 400 ms. The peak resident memory of a run of
these cases is about 180 MiB, of which the million-group accumulator is about
100 MiB. The runner's budget depends on that number, and it is set by the
accumulator rather than by the benchmark code.

The performance pull request that uses this benchmark is #26536. Its numbers
will come from a paired run of the benchmark runner, after this one has merged.

What is the testing strategy for this PR?

cargo check -p datafusion-functions-aggregate --benches compiles the new
benchmarks, and cargo clippy -p datafusion-functions-aggregate --all-targets -- -D warnings is clean.

The cases are new benchmarks in an existing file. They change no behaviour, so
the existing unit tests and sqllogictest files for this aggregate are
untouched.

The benchmark runner in the project's CI will exercise these cases once this
pull request is merged.

Are there any user-facing changes?

No. This changes only the benchmark file.

The grouped accumulator allocates one slot per group, so for a group-by that
touches few groups per batch the per-batch work is dominated by the work that
scales with the number of allocated groups rather than with the number of rows
the batch carries.

These cases allocate a million groups and touch 64 or 8192 of them per batch.
`update_bench_many_groups` replays one batch, which isolates that per-batch
work. `update_bench_many_groups_distinct_batches` walks through distinct
batches that own disjoint ranges of groups, so the accumulator also writes the
state of every winner -- the work a query does, and the number a change to the
accumulator has to be judged by.

Both use `BatchSize::PerIteration`: a million-group accumulator holds about
100 MiB, and holding several of them at once is what pushed the benchmark
runner's peak memory up.
@tanmay442
tanmay442 force-pushed the perf/first-last-bench-only branch from b15f964 to 71efc9e Compare October 21, 2026 08:39
@tanmay442 tanmay442 closed this Oct 21, 2026
@tanmay442
tanmay442 deleted the perf/first-last-bench-only branch October 21, 2026 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant