[rwgk]

— codex GPT-6 on behalf of Ralf —

Layer 7 of 7 of the staged "CUDA 12 bindings on main" stack. Previous: #3321. Stack map at the bottom.

What this PR does

Removes what the previous six layers made obsolete, and writes down how the two roots are maintained from now on.

• ci/versions.yml moves to schema 2: cuda.bindings.package_roots with toolkit_version and release_status; backport_branch, cuda.build, and cuda.prev_build are gone. bindings_config.py still reads the old format, because nightly may point at an older artifact source and release dry runs may point at retained CUDA 13 tags.
• New pre-commit hook check-bindings-config: each root's setuptools-scm selector must be v<major>.* for its CUDA major, the metapackage selector table must agree, and Core's cu12/cu13 bindings floors must not exceed the pinned toolkits. The old floor hook in toolshed/ is narrowed to extras and docs.
• The Pixi pin hook now checks Core's maintenance environment as well as its current environment.
• backport.yml becomes manual-only: a merged PR number plus an existing target branch, with validation. No more automatic backports on the to-be-backported label. Dependabot's 12.9.x entry is removed.
• ci-nightly.yml extracts only ci/versions.yml from the artifact-source archive: schema-2 sources take the new route, older sources the legacy route (legacy-backport-branch). The legacy artifact aliases (cuda-python-wheel, <core artifact>-tests) are no longer produced.
• Docs: CONTRIBUTING.md (selector table, transferring a fix between roots with git format-patch --relative and git am --directory, CUDA-major artifact flow, release branches), ci/README.md (registry, SCM milestones, nightly rollout, how to read CI job names, historical reruns, maintaining both roots, changing the supported lines), AGENTS.md (SYNCED blocks), cuda_bindings_12/MAINTENANCE.md (new), install page driver baseline correction (Linux 525.60.13, Windows 528.33), CI diagram.

After this merges

• The 12.9.x branch is frozen except for emergency fixes; routine CUDA 12 work happens in cuda_bindings_12/ on main.
• The retained v12.9.11a0 milestone is already on main; both alternatives are restacked onto it and contain no temporary SCM bootstrap.
• Remove the legacy nightly route after the first schema-2 main run succeeds.

Where to look

• ci/versions.yml: the final registry, 19 lines.
• ci/tools/bindings_config.py: check_package_metadata() and check_core_toolkit_floors().
• .github/workflows/backport.yml and ci-nightly.yml.
• ci/README.md, section "Maintaining Both Bindings Roots": the policy reviewers are really approving.

All changes (36 files, +977 / -377)

Config and hooks: ci/versions.yml, .pre-commit-config.yaml (new hook; floor hook narrowed), ci/tools/bindings_config.py (metadata checks), toolshed/check_cuda_core_bindings_floor.py, cuda_core/cuda/core/_bindings_floor.py (comment), cuda_core/tests/test_bindings_floor.py, ci/tools/tests/test_bindings_metadata.py (new), ci/tools/check_pixi_cuda_version.py, ci/tools/tests/test_bindings_config.py.

CI workaround: cuda_core/tests/test_memory.py runs supported pool-trim cases serially for the known driver synchronization issue #3350.

Workflows: backport.yml, .github/dependabot.yml, ci-nightly.yml, coverage.yml (registry-driven), build-wheel.yml (aliases removed), test-wheel-linux.yml, test-wheel-windows.yml (legacy-backport-branch input).

Docs: AGENTS.md, CONTRIBUTING.md, README.md, ci/README.md, ci/ci-pipeline.mmd, .github/RELEASE-bindings.md, cuda_bindings_12/{AGENTS.md,MAINTENANCE.md,README.md,docs/README.md,docs/source/contribute.md,docs/source/install.md,docs/source/release/12.X.Y-notes.rst}, cuda_bindings_13/{AGENTS.md,README.md,docs/README.md,docs/source/install.rst,docs/source/support.rst}, cuda_core/AGENTS.md.

Validation

• Not exercised: the manual backport workflow against a live branch, and the nightly workflow against a schema-2 artifact source (there is none until this stack is on main).
• 374 CI-tool tests passed locally; full pre-commit passed at 126ed74fb5bbfbc91a04e50557cf936efef4da7f.
• Full CI on earlier head 126ed74fb5bbfbc91a04e50557cf936efef4da7f passed: 105 successful jobs and one expected skip.
• The Windows 3.14t/A100-MCDM regression job passed on driver 596.36. DeviceMR and PinnedMR pool-trim cases ran serially and passed; ManagedMR had its normal unsupported-device skip.

Stack map: what main can do after each layer merges

Layer	PR	After it merges, main ...
1	#3314	behaves exactly as before; CI has one reader for the two bindings roots; CUDA 13 versions can no longer match a v12.* tag
2	#3315	lets a developer build, test, and document cuda_bindings_12/; the PR-time CPU source-build smoke covers both roots; three CUDA 12 runtime fixes are in
3	#3317	builds the CUDA 12 bindings wheel itself and runs its GPU tests; cuda.core and the metapackage still consume 12.9.x artifacts
4	#3318	uses same-run CUDA 12 wheels everywhere; PR CI no longer reads the 12.9.x branch
5	#3319	can release CUDA 12 from a v12.9.N tag on main; CUDA 13 releases are unchanged
6	#3321	builds and tests only what a change needs, per bindings root, reusing baseline artifacts
7	#3322	carries the schema-2 registry, a manual-only backport workflow, a metadata consistency hook, and the maintenance docs; 12.9.x is frozen

Every layer was designed to be mergeable on its own and had a green full CI run at its pre-rebase head on 2026-10-31. The stack is the "individually mergeable pieces" alternative to #3174-#3182; both stacks reach the same end state except that this one keeps compatibility routes for older artifact sources and retained CUDA 13 tags.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[rwgk]

/ok to test e91c0cb

[github-actions]

Doc Preview CI
🚀 View preview at https://nvidia.github.io/cuda-python/pr-preview/pr-3322/
https://nvidia.github.io/cuda-python/pr-preview/pr-3322/cuda-core/
https://nvidia.github.io/cuda-python/pr-preview/pr-3322/cuda-bindings/
https://nvidia.github.io/cuda-python/pr-preview/pr-3322/cuda-pathfinder/
Preview will be ready when the GitHub Pages deployment is complete.

[rwgk]

— codex GPT-6 on behalf of Ralf —

Completed a CodeRabbit sweep across #3317, #3318, #3319, #3321 and this PR, including collapsed review comments and the older findings here.

• Aligned the cibuildwheel pins starting in ci: build and test local CUDA 12 bindings #3317.
• Fixed source smoke for instruction-only changes and separated package-loop input from child-command stdin in ci: plan builds and artifact reuse per bindings root #3321; its workflow now explicitly requests read-only repository access.
• Restored nightly standard-mode legs for GPU, runtime and pinned-driver coverage in ci: plan builds and artifact reuse per bindings root #3321. CUDA 12 nightly artifact routing completes in ci: retire legacy branch-based bindings maintenance #3322; main retains the full PR matrix.
• Kept toolkit helpers and nested dependency actions aligned with the workflow revision when testing historical sources in ci: plan builds and artifact reuse per bindings root #3321.
• Removed unnecessary retained checkout credentials and restricted permissions in the two configuration jobs here.
• Clarified non-merge commits for cross-root transfers and the CUDA 12 samples' NVRTC requirement. The pip 25.1 prerequisite was already documented for both bindings roots.

Every finding now has a reply. The Git-error fallback and docs-helper relocation were deliberately left unchanged, with the reasons recorded on #3318 and #3319. Broader optional cache and artifact-payload validation work remains deferred: CI does not enable that cache, and baseline reuse already verifies the producer, exact commit and required artifacts.

Validation: Full pre-commit checks (including workflow linting), focused trigger/stdin/permission and matrix-selection checks, and an independent patch review passed. The release path was also exercised by the two tag-triggered producer runs and eight successful release rehearsals completed before this sweep. CI results are tracked in the PR checks.

[rwgk]

/ok to test 441bff5

[rwgk]

/ok to test 0fabbbb

[rwgk]

/ok to test 137ca2f

[rwgk]

/ok to test 916db96

[rwgk]

/ok to test ec30734

[rwgk]

— codex GPT-6 on behalf of Ralf —

Release rehearsal results — November 7

All eight release dry runs passed. Two real tag-triggered CI runs supplied the new CUDA 12/13 artifacts; both tags point to the same candidate commit, based on this stack snapshot.

Release dry run	Source tag	Result
CUDA 13 bindings	v13.5.1a0	Passed
CUDA 13 metapackage	v13.5.1a0	Passed
CUDA 12 bindings	v12.9.11a1	Passed
CUDA 12 metapackage	v12.9.11a1	Passed
Core	cuda-core-v1.2.2	Passed
Retained CUDA 13 bindings	v13.5.0	Passed
Retained CUDA 13 metapackage	v13.5.0	Passed
Pathfinder	cuda-pathfinder-v1.8.6	Passed

Artifact checks covered 112 wheels, source archives and docs bundles containing 3,926 versioned HTML pages: versions/dependencies, complete wheel sets and archive contents. Automatic run lookup selected the matching tag; a successful CUDA 13 run was rejected for CUDA 12 despite their identical source SHA.

The CUDA 13 producer passed on its first attempt. The CUDA 12 producer passed after one failed-jobs rerun for a Windows free-threaded crash and a WSL pyperf error.

These dry runs published no packages, GitHub Releases, release assets or documentation. Core and Pathfinder used retained source/artifacts; fresh Core-tag production and actual publishing authorization/deployment remain untested.

Subsequent changes leave release selection and publishing unchanged. The source/workflow revision boundary in toolkit setup was checked separately: current/historical helper selection, pinned public metadata for Linux/Windows selections, and Linux archive installation/cache restore all passed.

[rwgk]

— codex GPT-6 on behalf of Ralf —

Mike, all five active PR CIs are now green. Here is the delta since your November 6 review of #3319.

Changes since the November 6 review

• Release correctness: fixed the metapackage-major selection in ci: release CUDA 12 bindings from main #3319 itself and documented the two-tag release procedure. Both tag orders passed local builds; the end-to-end rehearsals now provide further evidence.
• Publishing permissions: wheel validation/source archiving and docs builds run in read-only jobs. Publishers and docs deployment consume prepared artifacts; release/control checkouts discard their credentials. Point-by-point response.
• Simplification: CI and release validation share the Windows ARM64/Python policy; CI tooling uses a shared Python 3.11 default, and artifact listings avoid recursive control-checkout output. The shared-theme follow-up is tracked in #3403.
• Main integration: carried forward upstream's toolkit hash checks and additive cuda-gdb/sanitizer setup.
• CodeRabbit sweep: aligned build-tool pins, fixed instruction-only source-smoke triggers and package-loop stdin, restricted configuration-job permissions, and clarified cross-root transfer/sample prerequisites. Toolkit actions now use matching helpers/dependencies when testing historical source revisions. Sweep summary; toolkit follow-up.
• Nightly coverage: kept the full PR matrix on main and restored standard suites for H200, older runtimes, pinned drivers and the remaining prior GPU/runtime cases through existing caller jobs. Coverage follow-ups.

Full CI: #3317, #3318, #3319, #3321, #3322.

Your review comments have direct replies. The Core-build extraction remains a post-migration follow-up in #3402.

[rwgk]

This is mostly about feedback on #3321, but logging the high-level assessment here:

codex GPT-6.1-Sol ultra:

Mike has approved #3318 and #3319. His #3321 review raises three major issues and 13 inline comments. There is no published #3322 feedback yet.

The nightly increase is real, and I recommend undoing the broad expansion.

For a normal scheduled run using the committed matrix:

Expanded jobs	Production main	Current stack
GPU tests	25	44
Matrix computation	11	20
Other jobs	5	5
Total	41	69

That is 68% more jobs and 76% more GPU-test jobs. These are configured job counts, not measured GPU-hours. Eight integration callers now run a second, standard-mode pass; the H200 expansion adds another GPU job and matrix-computation job. Production workflow, current workflow.

The important correction is that production main already uses the PR matrix. I compared coverage against #3319’s intermediate branch/nightly behavior, rather than production. Consequently, my Sunday follow-up added testing beyond production’s existing schedule. I should have checked that baseline and disclosed the increase. Current production policy.

My recommended next steps:

1. Restore the production nightly scope. Remove the eight doubled test-mode matrices while preserving their original integration tests, filters, permissions and source-aware routing. That leaves 43 jobs/26 GPU tests if we retain H200. For strict production parity, return to 41/25 and handle H200 separately. I favor that migration-first scope; additional standard coverage deserves its own proposal. H200 is a legitimate separate omission: production declares its nightly row but never selects it.

2. Address the repeated data handling substantively. Mike’s strongest simplification proposals are phases 1 and 3: extend the existing Python build-environment writer, and compute per-row TEST_* flags in the existing selector instead of repeating the policy in Linux/Windows expressions. Consolidate the registry’s Python/jq/yq output step too. Keep JSON across job/workflow boundaries, where it serves a purpose. This can reduce duplicated policy even if Python needs more lines.

3. Make legacy routing explicit. Accept the LEGACY_CONFIG sentinel and reject accidentally empty configuration. This protects a meaningful boundary: omission must not silently choose a different artifact algorithm. Preserve intentional old-source support until a deliberate cutover.

4. Accept the small inline cleanups. Remove the unnecessary metapackage chown and dead Windows branch; restore toolkit context in build-job names using existing metadata; clarify that the workflow applies jq filtering before calling the selector; replace the unexplained “until7” comment with the actual compatibility condition.

5. Qualify the two hardcoding concerns carefully. CUDA 12’s metapackage exception is historical: package policy requires Core for every major ≥13. Replacing 12 with “maintenance major” would incorrectly exempt CUDA 13 when CUDA 14 becomes current. Clarify that reason rather than making that substitution. The merger test’s fixed 12/13 values are independent synthetic fixtures; production version bumps do not invalidate them. Dependency policy, variant extraction.

6. Use the seven-phase plan selectively. Start with build flags, row flags and registry output consolidation, then simplify Pixi’s substantial Bash/jq selection and fake baseline ID. A wholesale Bash port or removal of every short fromJSON expression is not automatically worthwhile. Preserve bootstrap ordering, registry-before-baseline ordering, integration flags, Windows paths and .ci-control execution.

I would keep this cleanup in #3321, propagate it into #3322, and leave the approved lower layers unchanged. Validate old/new behavior locally—including actual legacy callers—before one CI refresh for the affected upper PRs. The replies and descriptions should explicitly correct our earlier nightly-coverage rationale and report the resulting job counts.