You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
tempfile: speed up the prefix/suffix validation added in gh-79459 #159029
The performance regression suite I run against CPython builds picked up a small slowdown in tempfile after #150477, which added the ValueError for a prefix or suffix with a directory component (gh-79459).
The check calls os.path.dirname() twice in _sanitize_params(), which runs on every mkstemp(), mkdtemp(), NamedTemporaryFile() and so on. A typical case is a function that processes a batch of uploaded documents and writes each one to a temporary file, because the library it hands them to only accepts a path:
On Lambda x86_64 (PGO+LTO), _sanitize_params() got about 56% slower, and creating 512 real files in /tmp about 2% slower. Locally it's roughly 375 ns more per call. Not a big deal next to the file system calls, but it's paid by every caller, including the ones that use the defaults.
I think it can be avoided without changing the behavior at all:
Only validate values passed by the caller. The defaults (template and an empty suffix) never contain a directory, so mkstemp() with no arguments doesn't need the check.
For exact str and bytes, check for the separator characters first, and only call os.path.dirname() when one is present. On POSIX that check is exact. On Windows, any value with \, / or : still goes through dirname(). Subclasses and path-like objects always go through dirname() as today.
With both, the no-argument case goes back to the cost from before #150477, and calls with an explicit prefix/suffix drop from about +380 ns to about +100 ns. I compared the fast path against posixpath.dirname() and ntpath.dirname() for every combination of the relevant characters up to 5 characters, as str and bytes, without any mismatch, and test_tempfile passes, including the refleak run.
Since this is a security fix, the goal is to keep exactly the same validation, just cheaper. Happy to send a PR. cc @vstinner
Fair enough, I get the preference for keeping a security check as simple as possible. My thinking was mostly about high-volume workloads, like code that creates thousands of temporary files in a tight loop, where it adds up a bit. But you're right that it's small compared to the actual file creation, so I'm fine closing this. Thanks for taking the time to look at it!
Makes sense, thanks! That's useful input for my tooling too. I'll raise the threshold it uses to flag a change as a regression, so smaller cases like this one get filtered out before they turn into a report.
Activity
vstinner commentedon Oct 8, 2026
I don't think that it's worth it to micro-optimize
tempfile._sanitize_params(). I prefer to keep the code simple and safe: always calldirname().I wouldn't call "2% slower" a "performance regression".
I suggest closing this issue.
leandrodamascena commentedon Oct 8, 2026
Fair enough, I get the preference for keeping a security check as simple as possible. My thinking was mostly about high-volume workloads, like code that creates thousands of temporary files in a tight loop, where it adds up a bit. But you're right that it's small compared to the actual file creation, so I'm fine closing this. Thanks for taking the time to look at it!
vstinner commentedon Oct 8, 2026
I don't think that it's a realistic use case.
leandrodamascena commentedon Oct 8, 2026
Makes sense, thanks! That's useful input for my tooling too. I'll raise the threshold it uses to flag a change as a regression, so smaller cases like this one get filtered out before they turn into a report.