Skip to content

gh-157384: Document and test PyImport_CreateModuleFromInitfunc() semantics - #157389

Merged
encukou merged 7 commits into
python:mainfrom
itamaro:gh-116146-initfunc-docs-tests
Oct 19, 2026
Merged

encukou merged 7 commits into
python:mainfrom
itamaro:gh-116146-initfunc-docs-tests

Conversation

@itamaro

@itamaro itamaro commented Sep 13, 2026 •

Copy link
Copy Markdown
Contributor

Document that single-phase init modules are added to sys.modules by the call itself (unlike multi-phase init modules), and that the spec name identifies the module, so a later call with the same name and a different init function reuses the cached module without calling the new init function. Add regression tests for both behaviors.

AI assisted with Claude Fable 5.1

…) semantics

Document that single-phase init modules are added to sys.modules by
the call itself (unlike multi-phase init modules), and that the spec
name identifies the module, so a later call with the same name and a
different init function reuses the cached module without calling the
new init function.  Add regression tests for both behaviors.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HoZE3WdRWQshvinfBZ137N
@read-the-docs-community

read-the-docs-community Bot commented Sep 13, 2026 •

Copy link
Copy Markdown

@encukou

encukou commented Sep 14, 2026

Copy link
Copy Markdown
Member

Could you turn “legacy single-phase initialization” into a link? (And remove the parenthesis -- PyModule_Create is just an example.)

Also, please link the “as is done for”/“as for” parts -- the docs for the mentioned feature should explain what happens in more detail; if they don't then the mention isn't very useful.

The existing docs already point out that the init function is not called twice. It would be nice to clarify there what the “lookup key“ is when determining subsequent imports of the “same” module.

Comment thread Doc/c-api/import.rst Outdated
@itamaro

itamaro commented Sep 18, 2026

Copy link
Copy Markdown
Contributor Author

Done, thanks for the suggestions @encukou !

@encukou

encukou commented Oct 14, 2026

Copy link
Copy Markdown
Member

I'll rebase, and reword a bit.

@encukou

encukou commented Oct 14, 2026

Copy link
Copy Markdown
Member

@itamaro, how's this look?

@itamaro

itamaro commented Oct 14, 2026

Copy link
Copy Markdown
Contributor Author

@itamaro, how's this look?

Thanks @encukou , this reads much better than what I had! the merge looks good too.

should we make the collision case more accurate? in two places (extension-modules.rst from my commits and import.rst in your commit) we're explaining how initfunc is not called again on name collision, but this is technically correct only for builtins and inittab extensions (and modules created through PyImport_CreateModuleFromInitfunc) - those can't collide with .so-file-based extensions (the path is part of the cache key).

possible changes:

  • If a single-phase module was previously created under the same name by this function or as a built-in module (e.g. one registered with PyImport_AppendInittab), including in another interpreter, Python does not call initfunc again...

and for extension-modules.rst: "Modules are matched to the saved contents by the module's full dotted name (and, for extensions loaded from a file, the file's path)".

another nit: "reuses the previous module's contents" is only accurate for m_size == -1. For m_size >= 0, the behavior depends on which interpreter loaded the module first (it may re-run the first cached init function, or call the new one). not sure how deeply we want to document that. maybe a short "for modules with m_size of -1" qualifier, or just rely on the "implementation detail" framing. wdyt?

happy to push the changes if you'd like.

@encukou

encukou commented Oct 15, 2026

Copy link
Copy Markdown
Member

Hm, right. I guess it makes sense to document this, especially since it's soft-deprecated and thus should not change.
We get a wall of text full of weird edge cases; rather than “how to use this” it tells you “why you shouldn't use single-phase init”.

I still left out the subinterpreter-specific behaviour, saying it could go either way.

Does this look good?

Please feel free to push directly, it's your PR :)

@itamaro

itamaro commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

thanks @encukou, I merged main and pushed a couple of tiny nits, let me know if you have any final feedback before merging! (or just approve if it looks good to you :) )

@encukou encukou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Thank you!

@encukou
encukou merged commit 0c4ca17 into python:main Oct 19, 2026
106 of 108 checks passed
@miss-islington-app

Copy link
Copy Markdown

Thanks @itamaro for the PR, and @encukou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

@bedevere-app

bedevere-app Bot commented Oct 19, 2026

Copy link
Copy Markdown

GH-159791 is a backport of this pull request to the 3.15 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 bugs and security fixes label Oct 19, 2026
itamaro added a commit that referenced this pull request Oct 19, 2026
…() semantics (GH-157389) (#159791)

gh-157384: Document and test PyImport_CreateModuleFromInitfunc() semantics (GH-157389)

Document that single-phase init modules are added to sys.modules by
the call itself (unlike multi-phase init modules), and that the spec
name identifies the module, so a later call with the same name and a
different init function reuses the cached module without calling the
new init function.  Add regression tests for both behaviors.

(cherry picked from commit 0c4ca17)

Co-authored-by: Itamar Oren <itamarost@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip news tests Tests in the Lib/test dir

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants