Skip to content

gh-158841: _PyBytes_Resize: for bytes subclasses, return new bytes - #159070

Open
encukou wants to merge 3 commits into
python:mainfrom
encukou:resize-exact-bytes
Open

encukou wants to merge 3 commits into
python:mainfrom
encukou:resize-exact-bytes

Conversation

@encukou

@encukou encukou commented Oct 9, 2026 •

Copy link
Copy Markdown
Member

This reuses the test from #158846, but makes the change _PyBytes_Resize by avoiding the optimization for subclasses, and always returning exact bytes.

drakeo338 and others added 3 commits October 9, 2026 14:04
…es subclass

_Unpickler_ReadFromFile() resized the object returned by read() in place, which is invalid for a bytes subclass. Copy it into an exact bytes object first.

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that it's correct to convert a bytes subclass to bytes. I would prefer to raise TypeError and return NULL (set *bytes to NULL). It sounds weird to that that in the caller, _PyBytes_Resize() changes the argument type.

The issue has been seen in pickle.load() with special code designed to trigger the code. IMO pickle should be fixed to never pass bytes to _PyBytes_Resize(). Did the bug have been seen elsewhere so far?

In the current code, _PyBytes_Resize() calls PyObject_Realloc(v, PyBytesObject_SIZE + newsize) to resize the bytes subclass in-place. The allocated size is wrong, a subclass needs more bytes (see bytes_subtype_new()). So the current code does fail / crash. It confirms that no code can rely on passing bytes subclass to _PyBytes_Resize() since it doesn't work (fail / crash).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants