Skip to content

Gforce single thread - #7929

Merged
ben-schwen merged 5 commits into
masterfrom
gforce_single_thread
Oct 19, 2026
Merged

ben-schwen merged 5 commits into
masterfrom
gforce_single_thread

Conversation

@ben-schwen

@ben-schwen ben-schwen commented Oct 18, 2026 •

Copy link
Copy Markdown
Member

Closes #3330

library(data.table)
NN = 1e8
for (ngrp in c(8L, 1e6L)) {
  set.seed(304093)
  DT = data.table(grp = sample(ngrp, NN, TRUE),
                  V = rpois(NN, 10), key = 'grp')
  setDTthreads(12L)
  print(system.time(DT[ , mean(V), keyby = grp]))
  setDTthreads(6L)
  print(system.time(DT[ , mean(V), keyby = grp]))
  setDTthreads(1L)
  print(system.time(DT[ , mean(V), keyby = grp]))
}

I ran it for 10 rounds and took min / median / mean and max.

groups threads version min median mean max
8 1 1.11.6 0.85 0.86 0.87 0.89
8 1 1.12.1 0.69 0.71 0.73 0.89
8 1 master 1.82 1.92 1.99 2.41
8 1 PR 0.79 0.83 0.87 1.16
8 6 1.11.6 0.85 0.86 0.86 0.90
8 6 1.12.1 0.44 0.52 0.52 0.61
8 6 master 0.96 1.18 1.17 1.33
8 6 PR 0.83 0.91 0.92 1.04
8 12 1.11.6 0.84 0.85 0.87 1.05
8 12 1.12.1 0.50 0.56 0.57 0.64
8 12 master 0.97 1.09 1.15 1.64
8 12 PR 0.88 0.92 0.93 0.99
1e6 1 1.11.6 0.72 0.75 0.76 0.84
1e6 1 1.12.1 0.66 0.70 0.76 1.05
1e6 1 master 1.28 1.41 1.51 2.17
1e6 1 PR 0.82 0.88 0.99 1.58
1e6 6 1.11.6 0.70 0.70 0.71 0.76
1e6 6 1.12.1 0.25 0.29 0.28 0.31
1e6 6 master 0.61 0.65 0.67 0.76
1e6 6 PR 0.26 0.28 0.32 0.60
1e6 12 1.11.6 0.70 0.70 0.72 0.82
1e6 12 1.12.1 0.21 0.28 0.27 0.32
1e6 12 master 0.56 0.58 0.61 0.78
1e6 12 PR 0.24 0.27 0.27 0.33

With one thread, use a single batch and put as much of the group id into
low as fits in uint16_t. For up to 65536 groups there is then a single
high bucket, gather() returns x itself instead of copying it, and gsum and
gmean reduce to direct loops.
With one thread, use a single batch and put as much of the group id into
low as fits in uint16_t. For up to 65536 groups there is then a single
high bucket, gather() returns x itself instead of copying it, and gsum and
gmean reduce to direct loops.
@codecov

codecov Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.08%. Comparing base (10bfd70) to head (fc67973).
⚠️ Report is 3 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7929      +/-   ##
==========================================
+ Coverage   99.02%   99.08%   +0.06%     
==========================================
  Files          88       88              
  Lines       17387    17418      +31     
==========================================
+ Hits        17217    17259      +42     
+ Misses        170      159      -11     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

Copy link
Copy Markdown

No obvious timing issues in HEAD=gforce_single_thread

Comparison Plot

Generated via commit 47da9f1

Download link for the artifact containing the test results: ↓ atime-results.zip

Task Duration
R setup and installing dependencies 1 minutes and 57 seconds
Installing different package versions 8 minutes and 51 seconds
Running and plotting the test cases 3 minutes and 51 seconds

@jangorecki jangorecki left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, amazing improvement!

@ben-schwen
ben-schwen merged commit 9693c56 into master Oct 19, 2026
14 of 15 checks passed
@ben-schwen
ben-schwen deleted the gforce_single_thread branch October 19, 2026 13:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Single-threaded grouping performance lower

2 participants