You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Raise Cython enum overflows as compile-time errors - #3417
Make the gcc build fail when a constant overflows int, so a bug like the one fixed in #3368 is caught at compile time instead of shipping as a wrong value.
Background
In #3368, cdef enum: CU_DEV_SM_RESOURCE_GROUP_LOCALITY_DOMAIN_ID_ANY = 4294967295 produced -1 in Python. Cython emits the value as an anonymous C++ enum, then converts it to a Python object through __Pyx_PyLong_From___pyx_anon_enum(int value). The enumerator does not fit in int, so it was truncated at that call, and nothing in the build complained.
Change
_build_flags in cuda_bindings_13/_build_shared.py now adds -pedantic -Werror=overflow for the gnu toolchain.
gcc only reports this overflow under -pedantic, so -pedantic is needed to surface it. -Werror=overflow then promotes only that diagnostic to an error, rather than turning on a blanket -Werror.
_build_shared.py is shared (symlinked) by cuda_bindings_12, cuda_bindings_13, cuda_core and cuda_pathfinder, so one edit covers every package.
clang and MSVC are unchanged. clang does not handle the flags identically to gcc, and I could not find a combination that didn't also introduce false positive. CI builds with gcc, so gcc coverage is enough for now.
What I tried that did not work
Cython:-Werror, -Wextra and the warn.* directives report nothing, because Cython never flags the truncation.
gcc without -pedantic: silent, even with -Wall -Wextra -Wconversion -Wsign-conversion -Woverflow.
clang++:-pedantic is silent when compiling C++. Only -Wsign-conversion fires, and that is too broad to enable here. Not pursued.
Testing
All builds used CUDA 13.5 headers.
cuda_bindings_13: a forced rebuild (-std=c++14) succeeds. All 31 compile units use the new flags, and none report -Woverflow.
cuda_core:pip install ./cuda_core (-std=c++17) succeeds. The flags appear in the compile commands and no -Woverflow is reported.
Negative check: with the original construct restored in cydriver.pxd, the cuda_bindings_13 build fails:
driver.cpp:711775:49: error: overflow in conversion from '<unnamed enum>' to 'int'
changes value from '__pyx_e_..._CU_DEV_SM_RESOURCE_GROUP_LOCALITY_DOMAIN_ID_ANY' to '-1' [-Werror=overflow]
I reverted that edit afterwards. I did not run a negative check against the cuda_core build; for C++17 I only confirmed the same failure on a small demo module.
Not verified
cuda_bindings_12 was not built locally (pyclibrary was missing from my environment). It uses the same shared file, so CI will be the first real check.
There is no automated regression test for the guard itself. It only takes effect on gcc builds.
Cross-root check
Counterparts inspected: cuda_bindings_12/build_hooks.py and cuda_core/build_hooks.py (both pass through resolve_toolchain, and neither overrides these flags), plus the _build_shared.py symlinks in cuda_bindings_12, cuda_core and cuda_pathfinder. No per-package change is needed.
No actionable comments were generated in the recent review.
Summary
Summary by CodeRabbit
Chores
GNU builds now use stricter compiler checks, including treating overflow warnings as errors. This may cause builds to fail when overflow warnings are present. For GNU builds with Python 3.13, pedantic warnings are not treated as errors. Other compiler configurations are unaffected.
The shared build helper documents GNU compile flags and adds -pedantic and -Werror=overflow. For GNU builds with Python 3.13, it also adds -Wno-error=pedantic.
Changes: GNU build flags
Layer / File(s)
Summary
GNU flag configuration cuda_bindings_13/_build_shared.py
The _build_flags documentation describes the GNU flags. GNU builds add -pedantic and -Werror=overflow; GNU builds with Python 3.13 also add -Wno-error=pedantic.
Possibly related PRs
NVIDIA/cuda-python#3173: Connects the CUDA 12 build helper to shared build logic, which may use the GNU flags added here.
Suggested labels:enhancement, cuda.pathfinder
Suggested reviewers:rwgk
Change: Feature
Merge Risk: Moderate · up to deda6
CUDA 12 GNU source builds will require the enum fix before they can succeed. The maintainers have acknowledged this limitation for the current merge.
Architecture risk: Low · up to deda6
The change affects 1 system.
Changed systems:cuda_bindings_13
Architecture concerns
No architecture-level concerns identified.
Review details
Systems and components
observed — cuda_bindings_13 (service) was modified; 1 changed file maps to changed impact.
Before / after behavior
observed — Modified behavior in cuda_bindings_13/_build_shared.py: The _build_flags documentation now describes the GNU compile flags that make constant overflows errors.
observed — Modified behavior in cuda_bindings_13/_build_shared.py: GNU builds now compile with -pedantic and -Werror=overflow; on Python 3.13, they additionally use -Wno-error=pedantic.
- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Reviewing files that changed from the base of the PR and between 04fcc67 and deda680.
Files selected for processing: 1
cuda_bindings_13/_build_shared.py
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.
Reviewing files that changed from the base of the PR and between e96903e and 04fcc67.
Files selected for processing: 1
cuda_bindings_13/_build_shared.py
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.
Sources consulted:
- https://raw.githubusercontent.com/cython/cython/3.2.5/Cython/Compiler/PyrexTypes.py
- https://raw.githubusercontent.com/cython/cython/3.2.5/Cython/Compiler/Nodes.py
- …and 17 more
Script executed:
python3 - <<'PY'from urllib.request import urlopenbase='https://raw.githubusercontent.com/cython/cython/3.2.5/Cython/Compiler/'checks={ 'Nodes.py':['class CEnumDefNode','class CEnumDefItemNode'], 'Symtab.py':['def declare_enum('], 'ModuleNode.py':['def generate_enum_definition','enum_values','enum {'], 'PyrexTypes.py':['def create_to_py_utility_code','class CAnonEnumType','def get_to_py_type_conversion'], 'ExprNodes.py':['class CoerceToPyTypeNode'],}for name,terms in checks.items(): print('\n---',base+name,'---') try: lines=urlopen(base+name,timeout=12).read().decode().splitlines() except Exception as e: print('unavailable:',repr(e));continue for term in terms: hits=[i for i,l in enumerate(lines) if term in l] print('term',repr(term),'lines',[i+1 for i in hits]) for i in hits[:2]: before=2; after=85 if term in ('def generate_enum_definition','class CoerceToPyTypeNode','def declare_enum(') else 25 for j in range(max(0,i-before),min(len(lines),i+after)): print(f'{j+1}: {lines[j]}')PYprintf'\n--- GCC standalone C++ enum conversion probe ---\n'printf'%s\n''enum { TIMEOUT = 4294967295 };''long from_int(int v) { return v; }''long f() { return from_int(TIMEOUT); }'| g++ -std=c++14 -pedantic -Werror=overflow -x c++ -fsyntax-only - 2>&1printf'g++ exit: %s\n'"$?"
Repository: NVIDIA/cuda-python
Length of output: 23224
Script executed:
python3 - <<'PY'from urllib.request import urlopenbase='https://raw.githubusercontent.com/cython/cython/3.2.5/Cython/'for path, ranges in { 'Compiler/PyrexTypes.py':[(1950,1985),(2070,2170),(2268,2278),(5070,5080)], 'Compiler/ExprNodes.py':[(2100,2200),(14652,14668),(14714,14725)], 'Utility/TypeConversion.c':[(1,1)],}.items(): print('\n---',base+path,'---') try: lines=urlopen(base+path,timeout=12).read().decode().splitlines() except Exception as e: print('unavailable:',repr(e));continue if path.endswith('TypeConversion.c'): hits=[i for i,l in enumerate(lines) if 'CIntToPy' in l] ranges=[(i+1,i+35) for i in hits[:1]] for lo,hi in ranges: for j in range(lo-1,min(len(lines),hi)): print(f'{j+1}: {lines[j]}') print('--- relevant identifiers ---') if path.endswith('ExprNodes.py'): for i,l in enumerate(lines): if 'enum_int_value' in l: print(f'{i+1}: {l}')PY
Repository: NVIDIA/cuda-python
Length of output: 14704
Script executed:
nl -ba cuda_bindings_12/cuda/bindings/driver.pyx | sed -n '1,18p;348,353p'
rg -n -F 'cimport cydriver' cuda_bindings_12/cuda/bindings/driver.pyx cuda_bindings_12/cuda/bindings/driver.pxd
python3 - <<'PY'from pathlib import Pathimport refor root in ('cuda_bindings_12/cuda/bindings','cuda_bindings_13/cuda/bindings'): print('\n--- large numeric literals inside Cython enum definitions:',root,'---') for p in Path(root).rglob('*'): if p.suffix not in ('.pxd','.pyx','.pxi'): continue enum_indent=None for n,line in enumerate(p.read_text(errors='replace').splitlines(),1): body=line.split('#',1)[0] m=re.match(r'^(\s*)(?:cdef|cpdef|ctypedef)\s+enum(?:\s+class)?(?:\s+\w+)?(?:\s*\([^)]*\))?\s*:(.*)$',body) if m: enum_indent=len(m.group(1)); text=m.group(2) elif enum_indent is not None and body.strip(): if len(body)-len(body.lstrip())<=enum_indent: enum_indent=None; continue text=body else: continue for v in re.findall(r'(?<![\w])(?:0[xX][0-9a-fA-F]+|\d+)(?![\w])',text): try: val=int(v,0) except ValueError: continue if val>2147483647: print(f'{p}:{n}: {line.strip()}')PY
important: Fix the CUDA 12 timeout before enabling this flag.
A clean GNU build can fail when Cython converts CUDA_EGL_INFINITE_TIMEOUT to Python: the conversion passes the enum value to an int parameter, triggering -Werror=overflow.
Generate the CUDA 12 declaration as an unsigned constant, as in CUDA 13.
The reason will be displayed to describe this comment to others. Learn more.
This finding is correct, but we can not easily fix it right now. The bugfix doesn't exist in the cuda_bindings_12 directory, so the new flags will cause that build to fail. However, that doesn't currently work for other reasons (the cuda_bindings_12 merge work is still in progress), and updating the generated files to include the bugfix right now will create merge conflicts for all of that work.
I think it acceptable to merge this as-is for now and regenerate the 12.x bindings files when we have that CI all merged.
The reason will be displayed to describe this comment to others. Learn more.
@mdboom suggestion: Understood. The CUDA 12 GNU build is expected to fail until the bindings include the unsigned-constant fix.
After the CI merge and regeneration, please verify a clean GNU build and the Python value of CUDA_EGL_INFINITE_TIMEOUT. Would you like me to open a follow-up issue?
I'm marking this as a draft. If we merge now, and then #3322 merges, the 12.9.x cuda_bindings will fail because it does yet have the fix for the bug this is meant to catch. Regenerating that now will cause major merge conflicts with #3322.
Since this is a nice-to-have for catching future bugs, and #3322 is big, complex and in progress and we don't want to delay it further, let's get #3322 in, regenerate all of the 12.9.x bindings, then come back to this.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bugSomething isn't workingCI/CDCI/CD infrastructurecuda.bindingsEverything related to the cuda.bindings modulecuda.coreEverything related to the cuda.core moduleP0High priority - Must do!
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Make the gcc build fail when a constant overflows
int, so a bug like the one fixed in #3368 is caught at compile time instead of shipping as a wrong value.Background
In #3368,
cdef enum: CU_DEV_SM_RESOURCE_GROUP_LOCALITY_DOMAIN_ID_ANY = 4294967295produced-1in Python. Cython emits the value as an anonymous C++enum, then converts it to a Python object through__Pyx_PyLong_From___pyx_anon_enum(int value). The enumerator does not fit inint, so it was truncated at that call, and nothing in the build complained.Change
_build_flagsincuda_bindings_13/_build_shared.pynow adds-pedantic -Werror=overflowfor thegnutoolchain.-pedantic, so-pedanticis needed to surface it.-Werror=overflowthen promotes only that diagnostic to an error, rather than turning on a blanket-Werror._build_shared.pyis shared (symlinked) bycuda_bindings_12,cuda_bindings_13,cuda_coreandcuda_pathfinder, so one edit covers every package.clangdoes not handle the flags identically togcc, and I could not find a combination that didn't also introduce false positive. CI builds with gcc, so gcc coverage is enough for now.What I tried that did not work
-Werror,-Wextraand thewarn.*directives report nothing, because Cython never flags the truncation.-pedantic: silent, even with-Wall -Wextra -Wconversion -Wsign-conversion -Woverflow.-pedanticis silent when compiling C++. Only-Wsign-conversionfires, and that is too broad to enable here. Not pursued.Testing
All builds used CUDA 13.5 headers.
cuda_bindings_13: a forced rebuild (-std=c++14) succeeds. All 31 compile units use the new flags, and none report-Woverflow.cuda_core:pip install ./cuda_core(-std=c++17) succeeds. The flags appear in the compile commands and no-Woverflowis reported.Negative check: with the original construct restored in
cydriver.pxd, thecuda_bindings_13build fails:I reverted that edit afterwards. I did not run a negative check against the
cuda_corebuild; for C++17 I only confirmed the same failure on a small demo module.Not verified
cuda_bindings_12was not built locally (pyclibrarywas missing from my environment). It uses the same shared file, so CI will be the first real check.Cross-root check
Counterparts inspected:
cuda_bindings_12/build_hooks.pyandcuda_core/build_hooks.py(both pass throughresolve_toolchain, and neither overrides these flags), plus the_build_shared.pysymlinks incuda_bindings_12,cuda_coreandcuda_pathfinder. No per-package change is needed.🤖 Generated with Claude Code