Skip to content

re: investigate avoiding data-stack allocation for the initial match context #162056

Open

Description

@nityanandmathur

Proposal

Investigate keeping the initial SRE(match_context) in a local C variable instead of allocating it in the resizable data stack. Simple matches could then avoid that allocation. Nested contexts and saved marks would remain on the existing explicit stack; this does not move unbounded backtracking onto the C stack.

A candidate uses context offset -1 to identify the local initial context, preserves nested-context restoration after reallocation, and adds a regression test covering recursive ignorecase-set counting, greedy/lazy/possessive repeats and scanner iteration.

Exploratory evidence and unresolved trade-offs

Ratios below are baseline time / candidate time (greater than 1 favors the candidate). Historical timing uses CPython ce841a9a02b9a28367b52b7b0c9606b5bdeacada, not current-main performance measurements.

  • Linux x86-64, GCC 14.2, independent full PGO builds: a five-file Pygments highlighting/HTML workload had a median ratio of 1.243× (about 19.6% less time). Same-binary A/A median was 1.017×, so controls were not perfectly centered.
  • Native macOS: five individual Pygments inputs had ratios 1.231–1.246×. The findall case regressed to 0.983969×, about 1.63% more elapsed time, with 9/12 groups slower. This warning remains unresolved.
  • Windows x64, MSVC 19.44.35230 / SDK 10.0.26100, matched PGO builds: Pygments ratios 1.2162–1.2700×; findall 0.9787× and substitution 0.9558×, both descriptive intervals crossing 1. Three SRE A/A quality flags remain. This used a custom worker, not pyperf.

These are one build pair per experiment, repeated-process observations, not independent-build replications or a general regex/Python speedup. Inputs differ between platforms and results are not pooled. Raw historical reports are retained privately, not currently available as public reproducibility artifacts.

Current-head checks

The candidate applies to 369d8bc4f6fe1f751de2512523a1acb95491a5eb. Local Linux debug baseline/candidate builds succeed, and the new regression passes. Both focused unfiltered runs encounter the same existing multiprocessing test's AF_UNIX socket permission error; neither is an all-green suite result.

Potential costs include a larger fixed C frame and an extra conditional for context restoration. Broader release-like workloads, independent builds, free-threaded coverage and investigation of the Mac findall signal are still needed. This is an exploratory proposal, not a ready-to-merge claim.

AI tools assisted implementation, tests and analysis.

Linked PRs

Activity

  1. added 2 commits that reference this issue on Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions