[not-matthias]

Summary

Reduce run-to-run variance from random seeds the runner doesn't control.

• ASLR off in walltime and memory. Simulation already runs under setarch --addr-no-randomize. Walltime and memory now do too, so the memory layout and address-seeded hashes (e.g. abseil's absl::Hash) stay the same across runs.
  • setarch wraps the innermost benchmark command, because the kernel clears ADDR_NO_RANDOMIZE when exec'ing setuid or file-capability binaries (sudo, memtrack).
  • setarch is probed once. If it fails (e.g. Docker's default seccomp profile), the runner warns and runs the benchmark unchanged.
• GODEBUG=randautoseed=0 in every mode. The global math/rand starts from Seed(1). The value is prepended to the user's GODEBUG, so their settings are kept and an explicit randautoseed still wins.

Testing

• cargo clippy --all-targets, executor and runner-shared tests pass.
• Local walltime run: benchmark sees personality 0x40000 and GODEBUG=randautoseed=0,<user value>.
• Memory mode not run end to end (needs eBPF privileges); the generated shell chain was checked by hand.

Open

• Run a walltime variance A/B before un-drafting: ASLR off is on by default with no opt-out.

Fixes COD-4013

[codspeed]

Merging this PR will not alter performance

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ 8 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

✅ 41 untouched benchmarks
⏩ 6 skipped benchmarks¹

---

_{Comparing cod-4013-disable-aslr-in-walltimememory-and-fix-go-mathrand-seed-in (a20700e) with main (d14d95d)}

Footnotes

1. 6 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[greptile-apps]

Confidence Score: 4/5

[Medium impact] The changes reach benchmark runs across several measurement modes.

The PR appears safe to merge, with a non-blocking gap in tests for the new ASLR behavior.

Findings

1.  ASLR behavior lacks test coverage ▶

Fix with agent prompt

### Issue 1
src/executor/helpers/aslr.rs:20-24
The new wrappers have no tests that check whether ASLR is disabled or whether a failed probe leaves the command unchanged. Existing executor tests only check that scripts run and receive their environment, so they also pass when ASLR stays enabled. This could hide a loss of benchmark repeatability.

Add tests for both probe outcomes and the memory command's quoting, plus a Linux check of the benchmark's personality flag when supported.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

This PR reduces uncontrolled randomness in benchmark runs.

• Walltime and memory benchmarks run with ASLR disabled when Linux allows it.
• Go benchmark processes use a repeatable global random seed by default.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Prepare benchmark] --> B[Add shared runtime settings]
  B --> C{Linux and setarch probe succeeds?}
  C -->|Yes| D[Wrap benchmark with setarch]
  C -->|No| E[Keep benchmark command unchanged]
  D --> F[Add environment and measurement wrappers]
  E --> F
  F --> G[Run benchmark]

Loading

_{Reviews (1) · Last reviewed commit: "feat: disable ASLR for walltime and memo..." · Reviewed by Greptile}