Skip to content

Raise Cython enum overflows as compile-time errors - #3417

Open
mdboom wants to merge 2 commits into
NVIDIA:mainfrom
mdboom:enum-overflow
Open

mdboom wants to merge 2 commits into
NVIDIA:mainfrom
mdboom:enum-overflow

Conversation

@mdboom

@mdboom mdboom commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Make the gcc build fail when a constant overflows int, so a bug like the one fixed in #3368 is caught at compile time instead of shipping as a wrong value.

Background

In #3368, cdef enum: CU_DEV_SM_RESOURCE_GROUP_LOCALITY_DOMAIN_ID_ANY = 4294967295 produced -1 in Python. Cython emits the value as an anonymous C++ enum, then converts it to a Python object through __Pyx_PyLong_From___pyx_anon_enum(int value). The enumerator does not fit in int, so it was truncated at that call, and nothing in the build complained.

Change

_build_flags in cuda_bindings_13/_build_shared.py now adds -pedantic -Werror=overflow for the gnu toolchain.

  • gcc only reports this overflow under -pedantic, so -pedantic is needed to surface it. -Werror=overflow then promotes only that diagnostic to an error, rather than turning on a blanket -Werror.
  • _build_shared.py is shared (symlinked) by cuda_bindings_12, cuda_bindings_13, cuda_core and cuda_pathfinder, so one edit covers every package.
  • clang and MSVC are unchanged. clang does not handle the flags identically to gcc, and I could not find a combination that didn't also introduce false positive. CI builds with gcc, so gcc coverage is enough for now.

What I tried that did not work

  • Cython: -Werror, -Wextra and the warn.* directives report nothing, because Cython never flags the truncation.
  • gcc without -pedantic: silent, even with -Wall -Wextra -Wconversion -Wsign-conversion -Woverflow.
  • clang++: -pedantic is silent when compiling C++. Only -Wsign-conversion fires, and that is too broad to enable here. Not pursued.

Testing

All builds used CUDA 13.5 headers.

  • cuda_bindings_13: a forced rebuild (-std=c++14) succeeds. All 31 compile units use the new flags, and none report -Woverflow.

  • cuda_core: pip install ./cuda_core (-std=c++17) succeeds. The flags appear in the compile commands and no -Woverflow is reported.

  • Negative check: with the original construct restored in cydriver.pxd, the cuda_bindings_13 build fails:

    driver.cpp:711775:49: error: overflow in conversion from '<unnamed enum>' to 'int'
    changes value from '__pyx_e_..._CU_DEV_SM_RESOURCE_GROUP_LOCALITY_DOMAIN_ID_ANY' to '-1' [-Werror=overflow]
    

    I reverted that edit afterwards. I did not run a negative check against the cuda_core build; for C++17 I only confirmed the same failure on a small demo module.

Not verified

  • cuda_bindings_12 was not built locally (pyclibrary was missing from my environment). It uses the same shared file, so CI will be the first real check.
  • There is no automated regression test for the guard itself. It only takes effect on gcc builds.

Cross-root check

Counterparts inspected: cuda_bindings_12/build_hooks.py and cuda_core/build_hooks.py (both pass through resolve_toolchain, and neither overrides these flags), plus the _build_shared.py symlinks in cuda_bindings_12, cuda_core and cuda_pathfinder. No per-package change is needed.

🤖 Generated with Claude Code

@github-actions github-actions Bot added the cuda.bindings Everything related to the cuda.bindings module label Nov 9, 2026
@mdboom mdboom self-assigned this Nov 9, 2026
@mdboom mdboom added bug Something isn't working P0 High priority - Must do! cuda.core Everything related to the cuda.core module labels Nov 9, 2026
@mdboom mdboom added this to the CI milestone Nov 9, 2026
@mdboom mdboom added the CI/CD CI/CD infrastructure label Nov 9, 2026
@coderabbitai

coderabbitai Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Summary

Summary by CodeRabbit

  • Chores
    • GNU builds now use stricter compiler checks, including treating overflow warnings as errors. This may cause builds to fail when overflow warnings are present. For GNU builds with Python 3.13, pedantic warnings are not treated as errors. Other compiler configurations are unaffected.

The shared build helper documents GNU compile flags and adds -pedantic and -Werror=overflow. For GNU builds with Python 3.13, it also adds -Wno-error=pedantic.

Changes: GNU build flags

Layer / File(s) Summary
GNU flag configuration
cuda_bindings_13/_build_shared.py
The _build_flags documentation describes the GNU flags. GNU builds add -pedantic and -Werror=overflow; GNU builds with Python 3.13 also add -Wno-error=pedantic.

Possibly related PRs

  • NVIDIA/cuda-python#3173: Connects the CUDA 12 build helper to shared build logic, which may use the GNU flags added here.

Suggested labels: enhancement, cuda.pathfinder

Suggested reviewers: rwgk

Change: Feature

Merge Risk: Moderate · up to deda6

CUDA 12 GNU source builds will require the enum fix before they can succeed. The maintainers have acknowledged this limitation for the current merge.

Architecture risk: Low · up to deda6

The change affects 1 system.

Changed systems: cuda_bindings_13

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

  • observed — cuda_bindings_13 (service) was modified; 1 changed file maps to changed impact.

Before / after behavior

  • observed — Modified behavior in cuda_bindings_13/_build_shared.py: The _build_flags documentation now describes the GNU compile flags that make constant overflows errors.
  • observed — Modified behavior in cuda_bindings_13/_build_shared.py: GNU builds now compile with -pedantic and -Werror=overflow; on Python 3.13, they additionally use -Wno-error=pedantic.
- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 04fcc67 and deda680.

Files selected for processing: 1

  • cuda_bindings_13/_build_shared.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between e96903e and 04fcc67.

Files selected for processing: 1

  • cuda_bindings_13/_build_shared.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

extra_compile_args += [f"-std=c++{cxx_std}"]
# Compiler-specific flags.
if name == "gnu":
extra_compile_args += ["-pedantic", "-Werror=overflow"]

@coderabbitai coderabbitai Bot Nov 9, 2026 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major · Functional Correctness · Quick win

Supported by static analysis

Script executed:

rg -n 'CUDA_EGL_INFINITE_TIMEOUT|4294967295' cuda_bindings_12/cuda/bindings/cydriver.pxd cuda_bindings_13/cuda/bindings/cydriver.pxd
ls -la cuda_bindings_12/_build_shared.py cuda_bindings_12/build_hooks.py
rg -n '_build_shared|_build_flags' cuda_bindings_12/build_hooks.py
git log --oneline -5 -- cuda_bindings_13/cuda/bindings/cydriver.pxd

Repository: NVIDIA/cuda-python

Length of output: 1087


Script executed:

git diff e96903e7058d1e26fb056b1a9e4f6e98d67c967e 04fcc67decf7cf8a654eca8322dd1ce1ed86ac15 -- cuda_bindings_13/_build_shared.py cuda_bindings_12/cuda/bindings/cydriver.pxd cuda_bindings_13/cuda/bindings/cydriver.pxd
printf '\n--- declarations and exports ---\n'
sed -n '3472,3492p' cuda_bindings_12/cuda/bindings/cydriver.pxd | nl -ba -v3472
sed -n '3998,4017p' cuda_bindings_13/cuda/bindings/cydriver.pxd | nl -ba -v3998
rg -n -F 'CUDA_EGL_INFINITE_TIMEOUT' cuda_bindings_12/cuda/bindings cuda_bindings_13/cuda/bindings
printf '\n--- build hook bindings ---\n'
rg -n 'build_flags|resolve_toolchain|extra_compile_args|language=|cythonize|driver.pyx|cydriver' cuda_bindings_12/build_hooks.py cuda_bindings_13/_build_shared.py
sed -n '205,260p' cuda_bindings_13/_build_shared.py | nl -ba -v205
printf '\n--- large enum declarations ---\n'
python3 - <<'PY'
import pathlib,re
for p in [pathlib.Path('cuda_bindings_12/cuda/bindings/cydriver.pxd'),pathlib.Path('cuda_bindings_13/cuda/bindings/cydriver.pxd')]:
    print('\n', p)
    for i,line in enumerate(p.read_text().splitlines(),1):
        for v in re.findall(r'(?<![\w])(?:0x[0-9a-fA-F]+|[0-9]{10,})(?![\w])',line):
            try: n=int(v,0)
            except ValueError: continue
            if n>2147483647: print(i,line.strip())
PY
printf '\n--- generated artifacts and cython availability ---\n'
rg --files cuda_bindings_12/cuda/bindings -g '*driver*.cpp' -g '*driver*.c'
python3 - <<'PY'
import importlib.util
s=importlib.util.find_spec('Cython')
print('Cython:', s.origin if s else 'unavailable')
PY

Repository: NVIDIA/cuda-python

Length of output: 11804


Script executed:

printf '%s\n' '--- CUDA 12 build call chain ---'
sed -n '20,42p;151,180p;432,456p;464,500p;548,590p;600,628p' cuda_bindings_12/build_hooks.py | nl -ba
sed -n '280,308p' cuda_bindings_13/_build_shared.py | nl -ba -v280
printf '%s\n' '--- Cython version requirements ---'
rg -n -i 'cython[<=>~]|cython[[:space:]]*[>=<~]' cuda_bindings_12/pyproject.toml cuda_bindings_13/pyproject.toml
printf '%s\n' '--- large numeric enum declarations in both binding trees ---'
rg -n 'cdef enum:.*(4294967295|0[xX][fF]{8}|[0-9]{10,})' cuda_bindings_12/cuda/bindings cuda_bindings_13/cuda/bindings
printf '%s\n' '--- potential generated code ---'
rg --files -g '*.cpp' -g '*.c' cuda_bindings_12/cuda/bindings | grep -E '(driver|cydriver)' || :
printf '%s\n' '--- compiler executables ---'
command -v g++ || :
command -v cython || :

Repository: NVIDIA/cuda-python

Length of output: 11122


Web query:

Cython anonymous cdef enum large value generated C++ __Pyx_PyLong_From_int 4294967295 overflow GCC Cython 3

Result:

Sources consulted:
- Cython Changelog — Cython 3.3.0a2.dev0 documentation — https://cython.readthedocs.io/en/latest/src/changes.html?highlight=bool
- [BUG] Duplicate `__Pyx_PyInt_As_int` functions · Issue #5623 · cython/cython · GitHub — GitHub issue 5623 in cython/cython (link omitted to avoid creating a cross-reference)
- Cython Changelog — Cython 3.4.0a0 documentation — https://cython.readthedocs.io/en/latest/src/changes.html?highlight=bool
- cython: Cython/Compiler/PyrexTypes.py Source File - doxygen documentation | Fossies Dox — https://fossies.org/dox/cython-3.3.0/PyrexTypes_8py_source.html
- Language Basics — Cython 3.1.8 documentation — https://cython.readthedocs.io/en/3.1.x/src/userguide/language_basics.html
- …and 26 more

Script executed:

python3 - <<'PY'
from urllib.request import urlopen
from urllib.error import URLError
base = 'https://raw.githubusercontent.com/cython/cython/3.2.5/Cython/Compiler/'
for filename, terms in [
    ('PyrexTypes.py', ['class CAnonEnumType', 'c_anon_enum_type', 'class CIntType']),
    ('Nodes.py', ['class CEnumDefNode', 'class CEnumDefItemNode', 'generate_enum_definition']),
    ('Symtab.py', ['declare_const', 'CAnonEnumType']),
    ('ExprNodes.py', ['class CoerceToPyTypeNode', 'to_py_function']),
]:
    url=base+filename
    print('\n---',url,'---')
    try:
        text=urlopen(url,timeout=12).read().decode().splitlines()
    except Exception as e:
        print('unavailable:',repr(e)); continue
    hits=[i for i,s in enumerate(text) if any(t in s for t in terms)]
    for i in hits[:12]:
        print('\nlines', max(1,i-4), '-', min(len(text),i+15))
        for j in range(max(0,i-5), min(len(text),i+15)):
            print(f'{j+1}: {text[j]}')
PY
printf '\n--- exact build and imports ---\n'
nl -ba cuda_bindings_12/build_hooks.py | sed -n '151,174p;432,451p;487,496p;561,585p;605,615p'
nl -ba cuda_bindings_12/cuda/bindings/driver.pyx | sed -n '15,55p;345,356p'
nl -ba cuda_bindings_12/cuda/bindings/cydriver.pxd | sed -n '8,40p'

Repository: NVIDIA/cuda-python

Length of output: 14892


Web query:

Cython 3.2.5 CAnonEnumType CEnumDefNode generate_execution_code anonymous enum to_py_function generated enum

Result:

Sources consulted:
- https://raw.githubusercontent.com/cython/cython/3.2.5/Cython/Compiler/PyrexTypes.py
- https://raw.githubusercontent.com/cython/cython/3.2.5/Cython/Compiler/Nodes.py
- …and 17 more

Script executed:

python3 - <<'PY'
from urllib.request import urlopen
base='https://raw.githubusercontent.com/cython/cython/3.2.5/Cython/Compiler/'
checks={
 'Nodes.py':['class CEnumDefNode','class CEnumDefItemNode'],
 'Symtab.py':['def declare_enum('],
 'ModuleNode.py':['def generate_enum_definition','enum_values','enum {'],
 'PyrexTypes.py':['def create_to_py_utility_code','class CAnonEnumType','def get_to_py_type_conversion'],
 'ExprNodes.py':['class CoerceToPyTypeNode'],
}
for name,terms in checks.items():
    print('\n---',base+name,'---')
    try: lines=urlopen(base+name,timeout=12).read().decode().splitlines()
    except Exception as e: print('unavailable:',repr(e));continue
    for term in terms:
        hits=[i for i,l in enumerate(lines) if term in l]
        print('term',repr(term),'lines',[i+1 for i in hits])
        for i in hits[:2]:
            before=2; after=85 if term in ('def generate_enum_definition','class CoerceToPyTypeNode','def declare_enum(') else 25
            for j in range(max(0,i-before),min(len(lines),i+after)):
                print(f'{j+1}: {lines[j]}')
PY
printf '\n--- GCC standalone C++ enum conversion probe ---\n'
printf '%s\n' 'enum { TIMEOUT = 4294967295 };' 'long from_int(int v) { return v; }' 'long f() { return from_int(TIMEOUT); }' | g++ -std=c++14 -pedantic -Werror=overflow -x c++ -fsyntax-only - 2>&1
printf 'g++ exit: %s\n' "$?"

Repository: NVIDIA/cuda-python

Length of output: 23224


Script executed:

python3 - <<'PY'
from urllib.request import urlopen
base='https://raw.githubusercontent.com/cython/cython/3.2.5/Cython/'
for path, ranges in {
 'Compiler/PyrexTypes.py':[(1950,1985),(2070,2170),(2268,2278),(5070,5080)],
 'Compiler/ExprNodes.py':[(2100,2200),(14652,14668),(14714,14725)],
 'Utility/TypeConversion.c':[(1,1)],
}.items():
 print('\n---',base+path,'---')
 try: lines=urlopen(base+path,timeout=12).read().decode().splitlines()
 except Exception as e: print('unavailable:',repr(e));continue
 if path.endswith('TypeConversion.c'):
  hits=[i for i,l in enumerate(lines) if 'CIntToPy' in l]
  ranges=[(i+1,i+35) for i in hits[:1]]
 for lo,hi in ranges:
  for j in range(lo-1,min(len(lines),hi)):
   print(f'{j+1}: {lines[j]}')
 print('--- relevant identifiers ---')
 if path.endswith('ExprNodes.py'):
  for i,l in enumerate(lines):
   if 'enum_int_value' in l: print(f'{i+1}: {l}')
PY

Repository: NVIDIA/cuda-python

Length of output: 14704


Script executed:

nl -ba cuda_bindings_12/cuda/bindings/driver.pyx | sed -n '1,18p;348,353p'
rg -n -F 'cimport cydriver' cuda_bindings_12/cuda/bindings/driver.pyx cuda_bindings_12/cuda/bindings/driver.pxd
python3 - <<'PY'
from pathlib import Path
import re
for root in ('cuda_bindings_12/cuda/bindings','cuda_bindings_13/cuda/bindings'):
 print('\n--- large numeric literals inside Cython enum definitions:',root,'---')
 for p in Path(root).rglob('*'):
  if p.suffix not in ('.pxd','.pyx','.pxi'): continue
  enum_indent=None
  for n,line in enumerate(p.read_text(errors='replace').splitlines(),1):
   body=line.split('#',1)[0]
   m=re.match(r'^(\s*)(?:cdef|cpdef|ctypedef)\s+enum(?:\s+class)?(?:\s+\w+)?(?:\s*\([^)]*\))?\s*:(.*)$',body)
   if m:
    enum_indent=len(m.group(1)); text=m.group(2)
   elif enum_indent is not None and body.strip():
    if len(body)-len(body.lstrip())<=enum_indent: enum_indent=None; continue
    text=body
   else: continue
   for v in re.findall(r'(?<![\w])(?:0[xX][0-9a-fA-F]+|\d+)(?![\w])',text):
    try: val=int(v,0)
    except ValueError: continue
    if val>2147483647: print(f'{p}:{n}: {line.strip()}')
PY

Repository: NVIDIA/cuda-python

Length of output: 1779


Script executed:

nl -ba cuda_bindings_12/cuda/bindings/cydriver.pxd | sed -n '1,20p'
nl -ba cuda_bindings_12/cuda/bindings/driver.pxd | sed -n '1,45p'
rg -n -F 'cydriver' cuda_bindings_12/cuda/bindings/driver.pxd cuda_bindings_12/cuda/bindings/driver.pyx | sed -n '1,15p'
rg -n -F 'CUDA_EGL_INFINITE_TIMEOUT' cuda_bindings_12 --glob '!cuda/bindings/cydriver.pxd' --glob '!cuda/bindings/driver.pyx'

Repository: NVIDIA/cuda-python

Length of output: 3841


important: Fix the CUDA 12 timeout before enabling this flag.

A clean GNU build can fail when Cython converts CUDA_EGL_INFINITE_TIMEOUT to Python: the conversion passes the enum value to an int parameter, triggering -Werror=overflow.

Generate the CUDA 12 declaration as an unsigned constant, as in CUDA 13.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This finding is correct, but we can not easily fix it right now. The bugfix doesn't exist in the cuda_bindings_12 directory, so the new flags will cause that build to fail. However, that doesn't currently work for other reasons (the cuda_bindings_12 merge work is still in progress), and updating the generated files to include the bugfix right now will create merge conflicts for all of that work.

I think it acceptable to merge this as-is for now and regenerate the 12.x bindings files when we have that CI all merged.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mdboom suggestion: Understood. The CUDA 12 GNU build is expected to fail until the bindings include the unsigned-constant fix.

After the CI merge and regeneration, please verify a clean GNU build and the Python value of CUDA_EGL_INFINITE_TIMEOUT. Would you like me to open a follow-up issue?

You are interacting with an AI system.

@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

@mdboom
mdboom marked this pull request as draft November 10, 2026 13:50
@copy-pr-bot

copy-pr-bot Bot commented Nov 10, 2026

Copy link
Copy Markdown
Contributor

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@mdboom

mdboom commented Nov 10, 2026

Copy link
Copy Markdown
Contributor Author

I'm marking this as a draft. If we merge now, and then #3322 merges, the 12.9.x cuda_bindings will fail because it does yet have the fix for the bug this is meant to catch. Regenerating that now will cause major merge conflicts with #3322.

Since this is a nice-to-have for catching future bugs, and #3322 is big, complex and in progress and we don't want to delay it further, let's get #3322 in, regenerate all of the 12.9.x bindings, then come back to this.

@mdboom
mdboom marked this pull request as ready for review November 10, 2026 22:39

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working CI/CD CI/CD infrastructure cuda.bindings Everything related to the cuda.bindings module cuda.core Everything related to the cuda.core module P0 High priority - Must do!

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant