Skip to content

CI: build both bindings majors from main - #3175

Draft
rwgk wants to merge 11 commits into
rwgk/cb12-on-main-03-package-selectionfrom
rwgk/cb12-on-main-04-ci-builds
Draft

rwgk wants to merge 11 commits into
rwgk/cb12-on-main-03-package-selectionfrom
rwgk/cb12-on-main-04-ci-builds

Conversation

@rwgk

@rwgk rwgk commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

— codex GPT-6 on behalf of Ralf —

Layer 4 of 8; second of the review group #3174 + #3175 + #3176, which lands as one merge. Previous: #3174. Next: #3176. Stack map at the bottom.

What this PR does

The build side: every CI run builds both bindings majors from main, and the planner decides per bindings root what needs rebuilding.

  • build-wheel.yml: builds cuda_bindings_13 with the current mini-CTK, then cuda_bindings_12 with the maintenance mini-CTK, both cuda.core ABI halves against the same-run wheels, both metapackage wheels, and the Cython test extensions. The 12.9.x-branch artifact download is gone. Artifact names carry the toolkit: cuda-bindings-python<py>-cuda<toolkit>-<platform>-<sha>, cuda-python-wheel-cuda<toolkit>.
  • ci/tools/compute_ci_plan.py: targets become (bindings, root), (core, cuNN), (python, root), pathfinder. A CUDA 12 bindings edit rebuilds the CUDA 12 bindings and metapackage, reuses Core, Pathfinder, and CUDA 13 wheels from the baseline run at the exact merge base, and tests Core only against CUDA 12. Core edits rebuild both halves. Shared or unknown paths build everything. A bare v12.9.* or v13.4.* tag builds only its own line, with published Pathfinder.
  • ci/tools/bindings_scm_version.py (new): the 12.9.x tags are not reachable from main, so a plain checkout would derive 12.9.1.devN. In the disposable CI checkout this helper adds one local tag v12.9.11a0.dev0 at HEAD^, so builds get 12.9.11a0.devN; it never pushes, never moves a tag, and stands down once a real v12.9.* tag at or above 12.9.10 is reachable. Git failures retain their stderr diagnostics.
  • ci/tools/env-vars: the build environment is written by bindings_config write-build-env (Python) instead of shell parsing.
  • ci/tools/download-wheels: release staging selects by toolkit and excludes *-tests artifacts.
  • test-sdist-*.yml: one job per package root, taking a bindings-package record.

Why it cannot merge alone

ci.yml passes bindings-config to the test workflows, which only accept it in #3176; actionlint (pre-commit) is therefore red on this layer by design.

Where to look

All changes (16 files, +1,543 / -649)

Workflows: build-wheel.yml, ci.yml (ci-vars folded into detect-changes; registry and workplan outputs; Windows ARM64 gated by the plan), test-sdist-linux.yml, test-sdist-windows.yml.

Tools: compute_ci_plan.py, bindings_config.py (write-build-env), bindings_scm_version.py (new), download-wheels, env-vars, merge_cuda_core_wheels.py (validates one wheel per CUDA variant).

Tests: test_bindings_scm_version.py (new), test_compute_ci_plan.py (rewritten around outcomes), test_download_wheels.py (new), test_merge_cuda_core_wheels.py (new).

Docs: ci/README.md, ci/ci-pipeline.mmd.

Review history and validation
Stack map: review scopes and merge units
Layer PR Scope Merge unit
1 #3172 import cuda_bindings_12/, byte-identical to the 12.9.x branch merged
2 #3173 integrate the import with the shared build tooling merged
3 #3174 registry, packaging, version selectors, developer environment, guidance group: #3174 + #3175 + #3176 land together
4 #3175 build both majors from main; planner; version preview bootstrap group
5 #3176 test, document, nightly, coverage; first CI-complete state group
6 #3177 releases driven by the tagged source's configuration after the group
7 #3178 maintenance policy, manual backport workflow, retire backport_branch after #3177
8 #3182 three CUDA 12 runtime fixes independent of the CI layers; stacked only for ordering

Layers 3 to 5 are review boundaries, not working intermediate states: #3174 alone breaks CI (workflows still read the removed cuda.build.version), and #3175 alone fails actionlint (callers pass an input the test workflows only accept in #3176). The plan is to review them separately and land the reviewed group as one merge. #3176 is the full-CI target for the group.

The staged stack #3314-#3322 is the alternative in which every layer is mergeable on its own; both stacks reach the same end state, except that the staged stack keeps compatibility routes for older artifact sources and retained CUDA 13 tags.

@rwgk rwgk added this to the cuda.bindings next milestone Oct 21, 2026
@rwgk rwgk added enhancement Any code-related improvements CI/CD CI/CD infrastructure cuda.bindings Everything related to the cuda.bindings module labels Oct 21, 2026
@copy-pr-bot

copy-pr-bot Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@rwgk rwgk self-assigned this Oct 21, 2026
@rwgk
rwgk added this pull request to stack #3179 October 21, 2026 04:48
@rwgk
rwgk force-pushed the rwgk/cb12-on-main-04-ci-builds branch 2 times, most recently from 3feda76 to 0c4765b Compare October 21, 2026 13:57
@rwgk
rwgk force-pushed the rwgk/cb12-on-main-04-ci-builds branch from 0c4765b to 044ddc6 Compare October 21, 2026 14:09
@rwgk
rwgk force-pushed the rwgk/cb12-on-main-04-ci-builds branch from 044ddc6 to 171b820 Compare October 21, 2026 16:51
@rwgk
rwgk force-pushed the rwgk/cb12-on-main-04-ci-builds branch 2 times, most recently from 4deddeb to ccb12ed Compare October 21, 2026 18:22
@rwgk
rwgk force-pushed the rwgk/cb12-on-main-04-ci-builds branch 2 times, most recently from 820c4ca to 560ff7d Compare October 22, 2026 03:52
@mdboom
mdboom force-pushed the rwgk/cb12-on-main-04-ci-builds branch from 560ff7d to 7a269e9 Compare October 23, 2026 15:04
@rwgk
rwgk force-pushed the rwgk/cb12-on-main-04-ci-builds branch from 7a269e9 to 595c0fd Compare October 25, 2026 23:56

@mdboom mdboom left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The most important thing here is to fix the cuda-core rebinding logic.

Comment thread .github/workflows/ci.yml Resolved
Comment thread .github/workflows/ci.yml Resolved
Comment thread .github/workflows/ci.yml Resolved
Comment thread ci/tools/compute_ci_plan.py Resolved
Comment thread ci/tools/compute_ci_plan.py Resolved
Comment thread .github/workflows/ci.yml Resolved
@rwgk
rwgk force-pushed the rwgk/cb12-on-main-04-ci-builds branch from 595c0fd to ec1fc8a Compare October 26, 2026 21:35

@rwgk rwgk left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

— codex GPT-6 on behalf of Ralf —

With Codex's help, I rechecked all six of Mike's inline comments here. Each already had a substantive reply. We drafted fresh answers from the current code, verified their source references, and compared them with the earlier replies. We also compared a fresh overall summary with those answers and the earlier cross-PR response.

One additional cleanup is retained as a single inline follow-up: the remaining cosmetic workplan-label generation and its eight workflow consumers are now removed. Earlier replies had already reported removing the Markdown table and label-dependent tests; the new change removes the remaining planner-label machinery.

The cleanup follow-up is in the planner thread. The other fresh replies and the overall recap are collected here to document the comparison without repeating them in the original threads.

Six previous replies paired with fresh drafts, plus the overall summary

Each link opens the previously posted reply. The indented text is the fresh comparison draft. Only the further planner-label cleanup is a new implementation update.

  • Previous reply: Dependency graph names both bindings

    • Fresh comparison draft: Updated the dependency graph to identify both bindings versions explicitly.
  • Previous reply: Reuse unchanged Core for bindings-only changes

    • Fresh comparison draft: A bindings-only change now reuses the unchanged merged Core wheel from an exact successful baseline, while testing Core against the affected bindings major. The planner regression and source-impact rules retain full rebuilds for Core/shared changes or an unavailable baseline.
  • Previous reply: Descriptive registry step name

    • Fresh comparison draft: The renamed step now reads “Read bindings configuration and select supported build platforms,” covering both outputs.
  • Previous reply: bindings_packages variable naming

    • Fresh comparison draft: Renamed it to bindings_packages throughout the workplan function.
  • Previous reply: Replace nested comprehension

    • Fresh comparison draft: Removed the nested comprehension. The planner retains the workplan JSON needed for diagnosis; cosmetic plan-label scaffolding and label-coupled tests are removed in the refreshed version.
  • Previous reply: Windows ARM64 single-major condition

    • Fresh comparison draft: Kept the single-major condition specific to current CUDA 13: maintenance CUDA 12 is unavailable on Windows ARM64. When current becomes CUDA 14, maintenance CUDA 13 should also build; >=13 would incorrectly keep that maintenance root disabled.
  • Overall summary: compared with the earlier cross-PR response and the six individual replies linked here.

    • Fresh comparison draft:

      Unnecessary Core rebuilds are addressed in the workplan: a bindings-only change rebuilds that bindings/metapackage pair and tests the affected Core ABI, while reusing unchanged Core from an exact successful baseline. Core/shared changes and unavailable baselines retain full rebuilds.

      The dependency graph, registry-step name and bindings_packages naming are updated. Cosmetic plan scaffolding is removed while expandable workplan JSON remains. The Windows ARM64 single-major condition deliberately stays specific to current CUDA 13, so a later CUDA 14/maintenance13 pair can build both.

      Extracting the current/maintenance build blocks into a composite action remains the previously disclosed follow-up; it is not claimed as completed here.

Comment thread ci/tools/compute_ci_plan.py Resolved
rwgk added 11 commits November 1, 2026 19:38
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
codex on behalf of Ralf:

Use the verified workflow interpreter before installing CI tools, replacing the removed harden-pip bootstrap.

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
@rwgk
rwgk force-pushed the rwgk/cb12-on-main-04-ci-builds branch from d184d81 to e6e0511 Compare November 2, 2026 03:52

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI/CD CI/CD infrastructure cuda.bindings Everything related to the cuda.bindings module enhancement Any code-related improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants