Skip to content

nix: apply SIMD hs_memcmp + inline MO_Memcmp patch to ghc967 on musl targets - #6772

Open
angerman wants to merge 1 commit into
masterfrom
angerman/musl-ghc967-simd-memcmp
Open

angerman wants to merge 1 commit into
masterfrom
angerman/musl-ghc967-simd-memcmp

Conversation

@angerman

@angerman angerman commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Description

musl libc implements memcmp (src/string/memcmp.c) as a portable 1-byte scalar loop:

for (; n && *l == *r; n--, l++, r++);
return n ? *l - *r : 0;

In static musl Linux builds (hydraJobs.*.musl.*, projectCross.musl64, projectCross.aarch64-multiplatform-musl), large Data.ByteString, ShortByteString, Data.Text, and ByteArray# (compareByteArrays# / MO_Memcmp) comparisons run ~21× slower than native glibc builds that use SIMD memcmp, while small fixed-size MO_Memcmp comparisons incur an out-of-line C call.

Until haskell.nix is bumped past ican-group/haskell.nix#2640 (upstream GHC MR: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/16986; see also ican-group/haskell.nix#2639), this PR applies ghc-9.6.7-simd-memcmp.patch to haskell-nix.compiler.ghc967 on musl targets in nix/pkgs.nix:

  • Inline small constant-size MO_Memcmp in X86_64 & AArch64 NCGs (0..64 bytes): Unrolls constant-size MO_Memcmp inline in GHC.CmmToAsm.X86.CodeGen (XOR for 0; MOVZxL + SUB for 1; branchless load + BSWAP + CMP + SETCC + SUB for 2, 4, 8; equality scan + shared cold mismatch block with X86 CFG edge updates for 3, 5..7, 9..64) and GHC.CmmToAsm.AArch64.{CodeGen,Instr,Ppr} (LDR/REV/CMP/CSET/SUB), gated by ncgInlineThresholdMemcpy (-fmax-inline-memcpy-insns, default 32, preserving the 9.6.7 DynFlags/NCGConfig ABI). All loads stay strictly within [0, n).
  • GHC 9.6.7 RTS (hs_memcmp + static MuslMemcmp.o): Exports hs_memcmp in libHSrts with SSE2 + runtime CPUID/XGETBV AVX2 (4×32B loop + vzeroupper) on x86_64 non-glibc/non-Bionic/non-uClibc Linux, ARMv8-A NEON (64B loop) on little-endian aarch64 non-glibc/non-Bionic/non-uClibc Linux, a bounded 64-bit word fallback on other non-glibc Linux architectures, and an n == 0-guarded delegation to platform memcmp elsewhere. All loads stay strictly within [0, n). Static RTS archives (!DYNAMIC && !SHARED) also provide MuslMemcmp.o (memcmp -> hs_memcmp) before -lc so C FFI callers resolve to hs_memcmp without extracting scalar libc.a(memcmp.lo).
  • Out-of-line MO_Memcmp & Stage 1+ bytestring: Lowers dynamic/large MO_Memcmp (compareByteArrays#) to hs_memcmp across NCG and LLVM backends (while preserving inline 0 for n == 0 and "memcmp" for constant 1..32 in LLVM so SimplifyLibCalls unrolls them), and compiles Stage 1+ libraries/bytestring with -DHS_MEMCMP_IN_RTS so Data.ByteString calls hs_memcmp directly while Stage 0 bootstraps cleanly against the unpatched host RTS.
  • Scoped to musl targets & content-addressed: Applied via lib.optionalAttrs final.stdenv.targetPlatform.isMusl on top of the existing haskell.nix 9.6.7 patch stack (12 patches on x86_64-musl, 16 patches on aarch64-musl), wrapped in builtins.path so unrelated repository commits do not invalidate the musl compiler store path. Stage 0 bootstrap compilers and all native Linux, macOS, and Windows (ucrt64) derivations and devShells remain bit-for-bit hash-identical.

Verification

  1. Derivation invariance (noGitRev.drvPath):
    • hydraJobs.x86_64-linux.native.cardano-node.passthru.noGitRev.drvPath and packages.aarch64-darwin.cardano-node.passthru.noGitRev.drvPath are unchanged before and after this commit.
    • Patch-stack application verified cleanly against ghc-9.6.7 for both pkgsCross.musl64 (13 patches total) and pkgsCross.aarch64-multiplatform-musl (17 patches total).
  2. NCG inline MO_Memcmp & end-to-end static musl build (x86_64 + aarch64):
    • Verified X86_64 (-dasm-lint enabled) and AArch64 NCG inline MO_Memcmp lowering and live execution across constant sizes 0..65, unaligned offsets, high-bit unsigned bytes, and back-to-back calls in the same Cmm basic block.
    • Built hydraJobs.x86_64-linux.musl.cardano-node (cardano-node 11.2.1 - linux-x86_64 - ghc-9.6, statically linked) and hydraJobs.x86_64-linux.musl.cardano-cli (cardano-cli 11.7.0.0), confirming hs_memcmp -> hs_memcmp_simd and memcmp -> hs_memcmp in the resulting static ELF binaries.
  3. Microbenchmark (x86_64-linux, AMD Ryzen 5 3600, single core, stock unpatched musl):
Compiler (GHC 9.6.7 -O2, 13,466 × 64 KiB comparisons) Data.ByteString ShortByteString / compareByteArrays#
Native glibc 14.061 ms (1.00×) 13.779 ms (1.00×)
Unpatched static musl 298.485 ms (21.23× slower) 299.354 ms (21.73× slower)
Patched static musl (this PR) 14.147 ms (21.10× speedup, 1.01× glibc) 13.834 ms (21.64× speedup, 1.00× glibc)

Checklist

  • Commit sequence broadly makes sense and commits have useful messages
  • New tests are added if needed and existing tests are updated
  • Any changes are noted in the CHANGELOG.md for affected package (N/A: Nix toolchain patch only)
  • The version bounds in .cabal files are updated (N/A)
  • CI passes
  • Self-reviewed the diff

@angerman
angerman requested a review from a team as a code owner November 12, 2026 22:33
…targets

musl libc implements memcmp as a portable 1-byte scalar loop, making large ByteString, ShortByteString, Data.Text, and ByteArray# (compareByteArrays# / MO_Memcmp) comparisons ~21x slower than glibc on x86_64 and AArch64 Linux, while small fixed-size MO_Memcmp calls incur an out-of-line C call.

Append ghc-9.6.7-simd-memcmp.patch (haskell.nix#2640, upstream ghc!16986) on top of the existing haskell.nix 9.6.7 patch stack for musl targets only, unrolling constant 0..64B MO_Memcmp inline in X86_64 and AArch64 NCGs and routing larger/dynamic comparisons and Stage1+ bytestring to SIMD hs_memcmp while leaving native glibc, macOS, and Windows derivations unchanged.
@angerman
angerman force-pushed the angerman/musl-ghc967-simd-memcmp branch from 70fd027 to 0eb3c85 Compare November 13, 2026 05:14
@angerman angerman changed the title nix: apply SIMD hs_memcmp RTS patch to ghc967 on musl targets nix: apply SIMD hs_memcmp + inline MO_Memcmp patch to ghc967 on musl targets Nov 13, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant