Skip to content

test_runner: use AsyncLocalStorage instead of async_hooks - #66795

Open
mcollina wants to merge 1 commit into
nodejs:mainfrom
mcollina:test-runner-als
Open

mcollina wants to merge 1 commit into
nodejs:mainfrom
mcollina:test-runner-als

Conversation

@mcollina

@mcollina mcollina commented Oct 16, 2026 •

Copy link
Copy Markdown
Member

Track the test that owns the current async context with an AsyncLocalStorage instead of a createHook() init/destroy map, so node:test no longer installs async hooks (including promise hooks) for every run.

Test#runInAsyncScope() enters the test's context directly rather than wrapping AsyncResource#runInAsyncScope(), to avoid adding frames to user stack traces. The snapshot changes drop one <node-internal-frames> line, since that frame now lives in the test runner and is filtered out.

Refs: #61030


AI generated, reviewed by me

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/test_runner

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem. labels Oct 16, 2026
@mcollina
mcollina requested review from MoLow and atlowChemi and removed request for MoLow October 16, 2026 16:01
@mcollina
mcollina marked this pull request as ready for review October 16, 2026 16:01
@codecov

codecov Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.49%. Comparing base (8ae0ad0) to head (1f16984).
⚠️ Report is 8 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #66795      +/-   ##
==========================================
- Coverage   90.52%   90.49%   -0.03%     
==========================================
  Files         797      797              
  Lines      278220   278225       +5     
  Branches    53456    53458       +2     
==========================================
- Hits       251857   251793      -64     
- Misses      16751    16810      +59     
- Partials     9612     9622      +10     
Files with missing lines Coverage Δ
lib/internal/test_runner/harness.js 87.32% <100.00%> (-0.81%) ⬇️
lib/internal/test_runner/test.js 97.97% <100.00%> (+0.03%) ⬆️
lib/internal/test_runner/utils.js 66.30% <100.00%> (+0.13%) ⬆️

... and 37 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@atlowChemi atlowChemi added the needs-benchmark-ci PRs that need a benchmark CI run. label Oct 17, 2026

@atlowChemi atlowChemi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but let's also see bench affects

@mcollina mcollina added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 19, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 19, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@mcollina

Copy link
Copy Markdown
Member Author

Benchmark results for benchmark/test_runner, comparing this PR (ef8d21b) against its parent (9df97a7). I used 30 runs per binary on an 8-core Linux machine with no CPU pinning.

Every benchmark that creates many tests is faster, and the gain grows with the number of tests:

Benchmark Change
suite-tests (all 24 configs) +16% to +70% ***
test-options (all 9 options) +40% to +44% ***
hooks (all 4 hook types) +40% to +43% ***
test-only +40.5% ***
global-concurrent-tests +29% to +46% ***
run-single-test-file, isolation=none +10% / +15% *** (10 / 100 files)
run-single-test-file, isolation=process +2% to +3%
mock-timers scheduler.wait +59% ***
test-reporters, mock-fn, global-sequential-tests flat

mock-timers mode='setTimeout' shows -17% (**), but its accuracy band is ±12–22%. The PR doesn't touch mock timers, and the neighboring setImmediate/setInterval rows are flat, so I read it as noise.

Full output
                                                                                                    confidence improvement accuracy (*)    (**)   (***)
test_runner/global-concurrent-tests.js type='async' n=100                                                  ***     29.50 %       ±4.92%  ±6.55%  ±8.54%
test_runner/global-concurrent-tests.js type='async' n=1000                                                 ***     36.30 %       ±2.03%  ±2.70%  ±3.52%
test_runner/global-concurrent-tests.js type='async' n=10000                                                ***     46.40 %       ±1.09%  ±1.46%  ±1.91%
test_runner/global-concurrent-tests.js type='sync' n=100                                                   ***     29.50 %       ±4.52%  ±6.04%  ±7.92%
test_runner/global-concurrent-tests.js type='sync' n=1000                                                  ***     34.64 %       ±1.75%  ±2.33%  ±3.03%
test_runner/global-concurrent-tests.js type='sync' n=10000                                                 ***     43.89 %       ±1.21%  ±1.62%  ±2.11%
test_runner/global-sequential-tests.js type='async' n=100                                                           1.43 %       ±5.40%  ±7.19%  ±9.37%
test_runner/global-sequential-tests.js type='async' n=1000                                                        -21.64 %      ±27.16% ±36.13% ±47.03%
test_runner/global-sequential-tests.js type='async' n=10000                                                        -3.14 %       ±7.34%  ±9.80% ±12.85%
test_runner/global-sequential-tests.js type='sync' n=100                                                            4.88 %       ±5.62%  ±7.49%  ±9.78%
test_runner/global-sequential-tests.js type='sync' n=1000                                                          -4.82 %      ±26.65% ±35.46% ±46.15%
test_runner/global-sequential-tests.js type='sync' n=10000                                                          0.27 %       ±6.80%  ±9.04% ±11.77%
test_runner/hooks.js hook='after' n=1000                                                                   ***     42.29 %       ±1.89%  ±2.52%  ±3.28%
test_runner/hooks.js hook='afterEach' n=1000                                                               ***     39.96 %       ±1.57%  ±2.08%  ±2.72%
test_runner/hooks.js hook='before' n=1000                                                                  ***     40.97 %       ±1.68%  ±2.25%  ±2.95%
test_runner/hooks.js hook='beforeEach' n=1000                                                              ***     43.30 %       ±1.38%  ±1.84%  ±2.39%
test_runner/mock-fn.js mode='define' n=1000000                                                                      0.69 %       ±0.97%  ±1.29%  ±1.68%
test_runner/mock-fn.js mode='execute' n=1000000                                                            ***      2.39 %       ±0.82%  ±1.09%  ±1.43%
test_runner/mock-timers.js mode='AbortSignal.timeout' n=1000                                                        0.32 %       ±5.21%  ±6.94%  ±9.03%
test_runner/mock-timers.js mode='Date' n=1000                                                               **      8.16 %       ±5.87%  ±7.82% ±10.17%
test_runner/mock-timers.js mode='enable-AbortSignal.timeout' n=1000                                          *      7.26 %       ±5.76%  ±7.67% ±10.00%
test_runner/mock-timers.js mode='enable-all' n=1000                                                          *     -2.29 %       ±2.09%  ±2.78%  ±3.62%
test_runner/mock-timers.js mode='enable-Date' n=1000                                                                1.43 %       ±3.65%  ±4.85%  ±6.32%
test_runner/mock-timers.js mode='enable-default' n=1000                                                            -0.57 %       ±2.30%  ±3.07%  ±4.00%
test_runner/mock-timers.js mode='enable-empty-apis' n=1000                                                          0.57 %       ±7.23%  ±9.62% ±12.52%
test_runner/mock-timers.js mode='enable-scheduler.wait' n=1000                                                     -6.64 %       ±6.73%  ±8.95% ±11.65%
test_runner/mock-timers.js mode='enable-setImmediate' n=1000                                                        0.93 %       ±6.33%  ±8.42% ±10.96%
test_runner/mock-timers.js mode='enable-setInterval' n=1000                                                        -1.49 %       ±5.89%  ±7.84% ±10.21%
test_runner/mock-timers.js mode='enable-setTimeout' n=1000                                                   *     -7.29 %       ±5.92%  ±7.87% ±10.24%
test_runner/mock-timers.js mode='runAll' n=1000                                                                     1.65 %       ±7.03%  ±9.36% ±12.21%
test_runner/mock-timers.js mode='scheduler.wait' n=1000                                                    ***     59.11 %       ±9.95% ±13.30% ±17.43%
test_runner/mock-timers.js mode='setImmediate' n=1000                                                              -5.86 %      ±13.10% ±17.43% ±22.68%
test_runner/mock-timers.js mode='setInterval' n=1000                                                                1.54 %       ±4.62%  ±6.15%  ±8.00%
test_runner/mock-timers.js mode='setTime' n=1000                                                                    6.64 %      ±34.95% ±46.51% ±60.54%
test_runner/mock-timers.js mode='setTimeout' n=1000                                                         **    -17.34 %      ±12.49% ±16.64% ±21.71%
test_runner/run-single-test-file.js isolation='none' numberOfTestFiles=1                                            0.05 %       ±7.27%  ±9.68% ±12.60%
test_runner/run-single-test-file.js isolation='none' numberOfTestFiles=10                                  ***      9.90 %       ±4.54%  ±6.04%  ±7.86%
test_runner/run-single-test-file.js isolation='none' numberOfTestFiles=100                                 ***     14.95 %       ±2.11%  ±2.81%  ±3.67%
test_runner/run-single-test-file.js isolation='process' numberOfTestFiles=1                                 **      2.90 %       ±2.11%  ±2.81%  ±3.66%
test_runner/run-single-test-file.js isolation='process' numberOfTestFiles=10                               ***      2.43 %       ±0.49%  ±0.65%  ±0.84%
test_runner/run-single-test-file.js isolation='process' numberOfTestFiles=100                              ***      2.09 %       ±0.17%  ±0.23%  ±0.29%
test_runner/suite-tests.js concurrency='no' testType='async' testsPerSuite=10 numberOfSuites=10            ***     17.17 %       ±2.49%  ±3.32%  ±4.33%
test_runner/suite-tests.js concurrency='no' testType='async' testsPerSuite=10 numberOfSuites=100           ***     32.35 %       ±1.90%  ±2.53%  ±3.30%
test_runner/suite-tests.js concurrency='no' testType='async' testsPerSuite=100 numberOfSuites=10           ***     28.22 %       ±2.18%  ±2.91%  ±3.79%
test_runner/suite-tests.js concurrency='no' testType='async' testsPerSuite=100 numberOfSuites=100          ***     44.18 %       ±1.11%  ±1.47%  ±1.92%
test_runner/suite-tests.js concurrency='no' testType='async' testsPerSuite=1000 numberOfSuites=10          ***     42.72 %       ±1.11%  ±1.48%  ±1.93%
test_runner/suite-tests.js concurrency='no' testType='async' testsPerSuite=1000 numberOfSuites=100         ***     64.60 %       ±1.13%  ±1.51%  ±1.99%
test_runner/suite-tests.js concurrency='no' testType='sync' testsPerSuite=10 numberOfSuites=10             ***     16.68 %       ±1.66%  ±2.21%  ±2.88%
test_runner/suite-tests.js concurrency='no' testType='sync' testsPerSuite=10 numberOfSuites=100            ***     32.94 %       ±1.90%  ±2.53%  ±3.29%
test_runner/suite-tests.js concurrency='no' testType='sync' testsPerSuite=100 numberOfSuites=10            ***     30.97 %       ±1.97%  ±2.63%  ±3.42%
test_runner/suite-tests.js concurrency='no' testType='sync' testsPerSuite=100 numberOfSuites=100           ***     42.32 %       ±1.26%  ±1.68%  ±2.19%
test_runner/suite-tests.js concurrency='no' testType='sync' testsPerSuite=1000 numberOfSuites=10           ***     41.06 %       ±1.23%  ±1.64%  ±2.14%
test_runner/suite-tests.js concurrency='no' testType='sync' testsPerSuite=1000 numberOfSuites=100          ***     64.84 %       ±0.95%  ±1.27%  ±1.66%
test_runner/suite-tests.js concurrency='yes' testType='async' testsPerSuite=10 numberOfSuites=10           ***     18.75 %       ±3.33%  ±4.45%  ±5.83%
test_runner/suite-tests.js concurrency='yes' testType='async' testsPerSuite=10 numberOfSuites=100          ***     30.69 %       ±1.83%  ±2.44%  ±3.17%
test_runner/suite-tests.js concurrency='yes' testType='async' testsPerSuite=100 numberOfSuites=10          ***     28.47 %       ±2.10%  ±2.80%  ±3.65%
test_runner/suite-tests.js concurrency='yes' testType='async' testsPerSuite=100 numberOfSuites=100         ***     40.10 %       ±1.14%  ±1.52%  ±1.99%
test_runner/suite-tests.js concurrency='yes' testType='async' testsPerSuite=1000 numberOfSuites=10         ***     40.18 %       ±0.92%  ±1.23%  ±1.60%
test_runner/suite-tests.js concurrency='yes' testType='async' testsPerSuite=1000 numberOfSuites=100        ***     70.10 %       ±1.34%  ±1.80%  ±2.37%
test_runner/suite-tests.js concurrency='yes' testType='sync' testsPerSuite=10 numberOfSuites=10            ***     16.00 %       ±2.81%  ±3.75%  ±4.89%
test_runner/suite-tests.js concurrency='yes' testType='sync' testsPerSuite=10 numberOfSuites=100           ***     30.93 %       ±1.94%  ±2.58%  ±3.36%
test_runner/suite-tests.js concurrency='yes' testType='sync' testsPerSuite=100 numberOfSuites=10           ***     27.68 %       ±2.06%  ±2.75%  ±3.59%
test_runner/suite-tests.js concurrency='yes' testType='sync' testsPerSuite=100 numberOfSuites=100          ***     41.52 %       ±1.42%  ±1.90%  ±2.49%
test_runner/suite-tests.js concurrency='yes' testType='sync' testsPerSuite=1000 numberOfSuites=10          ***     39.17 %       ±1.06%  ±1.41%  ±1.85%
test_runner/suite-tests.js concurrency='yes' testType='sync' testsPerSuite=1000 numberOfSuites=100         ***     69.22 %       ±1.17%  ±1.57%  ±2.06%
test_runner/test-only.js selected=1 n=10000                                                                ***     40.52 %       ±1.28%  ±1.70%  ±2.23%
test_runner/test-options.js option='none' n=10000                                                          ***     43.96 %       ±1.07%  ±1.42%  ±1.86%
test_runner/test-options.js option='skip-method-with-message' n=10000                                      ***     41.00 %       ±1.34%  ±1.79%  ±2.34%
test_runner/test-options.js option='skip-method' n=10000                                                   ***     39.91 %       ±1.05%  ±1.40%  ±1.83%
test_runner/test-options.js option='skip-with-message' n=10000                                             ***     42.29 %       ±1.27%  ±1.69%  ±2.21%
test_runner/test-options.js option='skip' n=10000                                                          ***     41.89 %       ±1.06%  ±1.41%  ±1.85%
test_runner/test-options.js option='todo-method-with-message' n=10000                                      ***     39.87 %       ±1.08%  ±1.44%  ±1.88%
test_runner/test-options.js option='todo-method' n=10000                                                   ***     39.96 %       ±0.88%  ±1.17%  ±1.53%
test_runner/test-options.js option='todo-with-message' n=10000                                             ***     41.44 %       ±0.93%  ±1.24%  ±1.62%
test_runner/test-options.js option='todo' n=10000                                                          ***     41.92 %       ±1.22%  ±1.63%  ±2.14%
test_runner/test-reporters.js reporter='dot' n=10000                                                               -0.55 %       ±1.16%  ±1.54%  ±2.01%
test_runner/test-reporters.js reporter='junit' n=10000                                                       *      1.08 %       ±0.95%  ±1.27%  ±1.65%
test_runner/test-reporters.js reporter='lcov' n=10000                                                              -0.04 %       ±1.35%  ±1.80%  ±2.34%
test_runner/test-reporters.js reporter='spec' n=10000                                                        *      1.08 %       ±0.93%  ±1.23%  ±1.61%
test_runner/test-reporters.js reporter='tap' n=10000                                                                0.23 %       ±1.35%  ±1.79%  ±2.34%

Be aware that when doing many comparisons the risk of a false-positive
result increases. In this case, there are 80 comparisons, you can thus
expect the following amount of false-positive results:
  4.00 false positives, when considering a   5% risk acceptance (*, **, ***),
  0.80 false positives, when considering a   1% risk acceptance (**, ***),
  0.08 false positives, when considering a 0.1% risk acceptance (***)

Track the test that owns the current async context with an
AsyncLocalStorage instead of a createHook() init/destroy map.

Signed-off-by: Matteo Collina <hello@matteocollina.com>

@atlowChemi atlowChemi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mcollina this might be worth a notable-change PRs with changes that should be highlighted in changelogs.

@atlowChemi atlowChemi added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-benchmark-ci PRs that need a benchmark CI run. needs-ci PRs that need a full CI run. request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. test_runner Issues and PRs related to the test runner subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants