Repository navigation
gh-161185: Fix strict aliasing issue in Py_SETREF() macros family - #161186
Conversation
Py_SETREF(), Py_XSETREF() and Py_CLEAR() now use memcpy() to set _tmp_old_dst/_tmp_old_op to fix a strict aliasing issue. Py_SETREF(), Py_XSETREF() and Py_CLEAR() now also cast _tmp_old_dst/_tmp_old_op to PyObject* when calling Py_DECREF()/Py_XDECREF() to avoid compiler warnings with the limited C API. Co-Authored-by: Nathan Goldbaum <nathan.goldbaum@gmail.com>
|
To reproduce #161185 issue:
=> The test does crash (segfault) or fails with a C assertion error. Patch to force the memcpy() implementation and reintroduce the strict aliasing bug: Details |
ngoldbaum
left a comment
There was a problem hiding this comment.
Looks good besides the one test nit. I'm hitting approve.
My clanker says the fallback path isn't exercised in any extant test configuration. Does it maybe make sense to force it? like this:
Details
--- a/Include/pyport.h
+++ b/Include/pyport.h
@@ -550,6 +550,11 @@
# define _Py_TYPEOF(expr) __typeof__(expr)
#endif
+#if defined(_Py_IS_TESTCEXT) && defined(_Py_TESTCEXT_NO_TYPEOF)
+ // Exercise fallback implementations in test_cext.
+# undef _Py_TYPEOF
+#endif
+
/* A convenient way for code to know if sanitizers are enabled. */
#if defined(__has_feature)
--- a/Lib/test/test_cext/__init__.py
+++ b/Lib/test/test_cext/__init__.py
@@ -146,6 +146,18 @@
# Please ask the C API WG before adding a new C++11-only feature.
self.check_build('_test_cppext_cpp03', std='c++03')
+ @unittest.skipIf(MSVC, "MSVC doesn't support /std:c++03")
+ def test_build_cpp03_memcpy(self):
+ # Exercise the memcpy fallback with strict aliasing enabled.
+ self.check_build(
+ '_test_cppext_cpp03_memcpy',
+ std='c++03',
+ extra_cflags=(
+ '-D_Py_IS_TESTCEXT -D_Py_TESTCEXT_NO_TYPEOF '
+ '-O2 -fstrict-aliasing'
+ ),
+ )
+
Move variable declaration before code.
There was a problem hiding this comment.
@vstinner Thank you for the PR, I was about to comment on the test typo. This looks good to me
Edit: Oh I see Nathan has already pointed that out!
Add the _Py_TESTCEXT_NO_TYPEOF macro to undefine the _Py_TYPEOF macro. Co-Authored-by: Nathan Goldbaum <nathan.goldbaum@gmail.com>
Yes, it makes sense. It should help to catch regressions earlier, rather than waiting until someone hit a bug in a specific operating system with a specific C or C++ compiler. I applied your patch (with some minor changes). |
|
I reintroduced the bug to make sure that test_cext catches it. And it does catch it! TestPublicC.test_build_no_typeof() and TestPublicCpp.test_build_cpp03_no_typeof() does fail with a crash. I ran test_cext on Fedora 45 with GCC 16.2.1. I also tested clang 23.1.4 using Patch to reintroduce the bug on top of this PR: |
The Py_tp_basicsize slot is not required to define a type.
|
I modified the test so it can also be run on Free Threading. I tested manually that test_cext also catch the regresssion if I apply #161186 (comment) patch. |
|
🤖 New build scheduled with the buildbot fleet by @vstinner for commit 4d3f620 🤖 Results will be shown at: https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F161186%2Fmerge If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again. |
|
Ok, let me run the test on buildbots to double check that I didn't miss anything. If buildbots are happy, I will merge the change ;-) |
|
Note to myself: once this PR is merged, I can also python/pythoncapi-compat#187 (Add Py_SETREF/Py_XSETREF to old limited C API versions) with the updated Py_CLEAR/Py_SETREF macro implementation. |
Unrelated failures:
Unrelated failure |
Unrelated failure: test_tcl failed. |
|
Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14, 3.15. |
|
Sorry, @vstinner, I could not cleanly backport this to Please backport manually with cherry_picker, see the devguide for more information. |
|
Sorry, @vstinner, I could not cleanly backport this to Please backport manually with cherry_picker, see the devguide for more information. |
…ly (python#161186) Py_SETREF(), Py_XSETREF() and Py_CLEAR() now use memcpy() to set _tmp_old_dst/_tmp_old_op to fix a strict aliasing issue. Py_SETREF(), Py_XSETREF() and Py_CLEAR() now also cast _tmp_old_dst/_tmp_old_op to PyObject* when calling Py_DECREF()/Py_XDECREF() to avoid compiler warnings with the limited C API. Add the _Py_TESTCEXT_NO_TYPEOF macro to undefine the _Py_TYPEOF macro. Co-authored-by: Nathan Goldbaum <nathan.goldbaum@gmail.com> (cherry picked from commit f8e65ee)
|
GH-161360 is a backport of this pull request to the 3.15 branch. |
|
Thanks for your reviews! I merged my change and backport it to 3.15 (the 3.14 backport will follow). |
…ily (#161186) (#161360) gh-161185: Fix strict aliasing issue in Py_SETREF() macros family (#161186) Py_SETREF(), Py_XSETREF() and Py_CLEAR() now use memcpy() to set _tmp_old_dst/_tmp_old_op to fix a strict aliasing issue. Py_SETREF(), Py_XSETREF() and Py_CLEAR() now also cast _tmp_old_dst/_tmp_old_op to PyObject* when calling Py_DECREF()/Py_XDECREF() to avoid compiler warnings with the limited C API. Add the _Py_TESTCEXT_NO_TYPEOF macro to undefine the _Py_TYPEOF macro. (cherry picked from commit f8e65ee) Co-authored-by: Nathan Goldbaum <nathan.goldbaum@gmail.com>
…ily (GH-161186) (GH-161360) (#161362) [3.15] gh-161185: Fix strict aliasing issue in Py_SETREF() macros family (GH-161186) (GH-161360) gh-161185: Fix strict aliasing issue in Py_SETREF() macros family (GH-161186) Py_SETREF(), Py_XSETREF() and Py_CLEAR() now use memcpy() to set _tmp_old_dst/_tmp_old_op to fix a strict aliasing issue. Py_SETREF(), Py_XSETREF() and Py_CLEAR() now also cast _tmp_old_dst/_tmp_old_op to PyObject* when calling Py_DECREF()/Py_XDECREF() to avoid compiler warnings with the limited C API. Add the _Py_TESTCEXT_NO_TYPEOF macro to undefine the _Py_TYPEOF macro. (cherry picked from commit f8e65ee) (cherry picked from commit 0dc1a96) Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Nathan Goldbaum <nathan.goldbaum@gmail.com>
|
|
|
|
|
|
|
|
Ah. I didn't test test_cext locally before merging the 3.14 backport, and obviously it fails on all buildbots :-) I wrote #161373 to fix test_cext. |
|
Py_SETREF(), Py_XSETREF() and Py_CLEAR() now use memcpy() to set _tmp_old_dst/_tmp_old_op to fix a strict aliasing issue.
Py_SETREF(), Py_XSETREF() and Py_CLEAR() now also cast _tmp_old_dst/_tmp_old_op to PyObject* when calling Py_DECREF()/Py_XDECREF() to avoid compiler warnings with the limited C API.