Repository navigation
Conversation
Resolve a frame or traceback from its registered globals owner only when its canonical filename matches. Reuse the existing filename cache and retain the fallback for other objects and namespaces. Developed and tested with Codex (AI assistance).
The fixture left a stale __file__ value beneath a module subclass property. LOAD_ATTR_MODULE specialization can read that value, making the assertion depend on warmup state in the existing filename fallback, including main. Remove the shadow value and exercise warmed getfile access, two changing origins, and independent cold frame/traceback lookups. Keep production code unchanged. Verified the corrected test 50 times on each of main and the candidate; all six focused suites pass (1,135 tests reported, four skipped).
|
@eendebakpt, would you be willing to review this follow-up to #92042? It implements the frame/traceback split you suggested, without the registry watchers or C changes, and is independent of and complementary to #156685. CI is green; regression coverage and reproducible benchmarks are linked in the description. I’d particularly appreciate your view on verified frame globals taking precedence for an uncached filename collision. Thanks for your earlier feedback. |
| except (TypeError, FileNotFoundError): | ||
| pass |
There was a problem hiding this comment.
| except (TypeError, FileNotFoundError): | |
| pass | |
| except FileNotFoundError, TypeError: | |
| pass | |
| return None |
| if (type(module) is not types.ModuleType or | ||
| module.__dict__ is not namespace or '__getattr__' in namespace or | ||
| type(namespace.get('__file__')) is not str): |
There was a problem hiding this comment.
| if (type(module) is not types.ModuleType or | |
| module.__dict__ is not namespace or '__getattr__' in namespace or | |
| type(namespace.get('__file__')) is not str): | |
| if (type(module) is not types.ModuleType | |
| or module.__dict__ is not namespace | |
| or '__getattr__' in namespace | |
| or type(namespace.get('__file__')) is not str): |
| frame = object if isframe(object) else ( | ||
| object.tb_frame if istraceback(object) else None) |
There was a problem hiding this comment.
| frame = object if isframe(object) else ( | |
| object.tb_frame if istraceback(object) else None) | |
| if isframe(object): | |
| frame = object | |
| elif istraceback(object): | |
| frame = object.tb_frame | |
| else: | |
| frame = None |
| module = sys.modules.get(name) | ||
| # Leave dynamic module metadata to the existing filename fallback. | ||
| if (type(module) is not types.ModuleType or | ||
| module.__dict__ is not namespace or '__getattr__' in namespace or |
There was a problem hiding this comment.
Is '__getattr__' in namespace ever needed?
| @property | ||
| def __file__(self): | ||
| return self.source | ||
| # Do not leave a shadow value that LOAD_ATTR_MODULE can read instead |
| @@ -0,0 +1,3 @@ | |||
| Speed up :func:`inspect.getmodule` for frames and tracebacks whose globals | |||
| belong to a registered module with a matching filename. The globals owner | |||
There was a problem hiding this comment.
This promises a bit more than the code does. For example:
import inspect, os, sys, tempfile, types
fn = os.path.join(tempfile.mkdtemp(), 'owner.py')
open(fn, 'w').close()
owner = types.ModuleType('owner_mod'); owner.__file__ = fn
other = types.ModuleType('other_mod'); other.__file__ = fn
sys.modules['owner_mod'] = owner
sys.modules['other_mod'] = other # inserted later -> wins a registry scan
exec(compile("import sys\nframe = sys._getframe()\n", fn, 'exec'), owner.__dict__)
frame = owner.__dict__['frame']
print("1 frame, cold :", inspect.getmodule(frame).__name__)
inspect.getmodule(compile("pass", "<nowhere>", "exec")) # unrelated miss
print("2 frame, after a miss :", inspect.getmodule(frame).__name__)
inspect.modulesbyfile.clear(); inspect._filesbymodname.clear()
print("3 frame, cold again :", inspect.getmodule(frame).__name__)
inspect.modulesbyfile.clear(); inspect._filesbymodname.clear()
print("4 code object, cold :", inspect.getmodule(frame.f_code).__name__)
inspect.modulesbyfile.clear(); inspect._filesbymodname.clear()
inspect.getmodule(frame)
print("5 code obj after frame :", inspect.getmodule(frame.f_code).__name__)
This is a pathological case (intentionally giving two modules the same filename), so maybe fine to keep the code as is. But rewrite the news entry.
There was a problem hiding this comment.
Also please indicate this change in the docs of inspect directly. This may be breaking for some people.
Documentation build overview
29 files changed ·
|
…hits Preflight the ownership-only revision before updating python#160254. Keep source retrieval unchanged and preserve linecache.getlines overrides. Developed and tested with Codex (AI assistance).
|
Updated in 6a91668 to give verified frame/traceback globals ownership consistent precedence over conflicting filename-cache entries. Successful ownership matches no longer populate the shared filename map; bare code-object lookup remains filename based. The documentation and NEWS now state this behavior directly. A narrow cached-module/globals identity check retains the cheap warm path. Source retrieval is unchanged, including linecache.getlines overrides and their module-globals argument. The revision remains independent of #156685 and composes cleanly in either order. All six relevant suites pass standalone and composed (1,149 reported, 4 skipped each); configured lint and the documentation warning gate pass. Updated source-pinned measurements and fallback costs are in the description: direct registered-frame lookup is approximately 5× faster than the pinned mainline, while the warm four-frame source-context stack has no statistically resolved regression (time ratio 1.002, 95% interval 0.992–1.012). |
|
I measured the current revision’s cold-cache behavior, including with/without #156685. The main benefit is avoiding registry scans on first inspection and when encountering a newly registered module. Four-frame
At 1,125 entries, this saves about 9 ms on first inspection and 0.23 ms when encountering the new module. These are savings on particular inspection calls, not a sustained application-wide speedup. One private-namespace frame still triggers filename fallback and largely removes the cold-cache benefit. #156685 does not eliminate that initial scan in this workload. Measurements use pinned “Cold” means empty A longer warm-only run shows no resolved difference: paired time ratio 0.996 (95% interval 0.974–1.021) versus mainline and 1.008 (0.994–1.025) versus #156685 alone. Identical-source controls are consistent with no change. Source-context checks agree across variants, and separate instrumentation confirms that verified registered-frame lookup avoids the registry scan. |
Resolve registered frames and tracebacks directly from their globals owner when the registered module identity and normalized filename match. This avoids scanning
sys.modulesand gives a verified owner consistent precedence over a conflicting filename-cache entry.Successful ownership resolution does not populate the shared filename map, so bare code-object lookup remains filename based and is not changed by this fast path. A cached exact module with the frame's identical globals can be returned immediately: it cannot conceal a different globals owner, and the existing filename fallback would return that same module. Other cached results still require ownership verification.
Private execution namespaces, mismatched origins, module subclasses, dynamic filenames, fileless modules and unrelated filename overrides retain the filename fallback. Source/bytecode normalization is preserved. Source retrieval is unchanged, including
linecache.getlines()overrides and the module globals passed to them. Documentation and NEWS describe the ownership precedence directly.This implements the frame/traceback split discussed in #92042. It remains independent of and complementary to #156685; the changes compose cleanly in either order. No registry watchers, C changes or negative caching are included.
Performance
Measurements use source-pinned variants in one CPython 3.16
-O3Linux build without PGO/LTO. Eight fresh processes use counterbalanced ABBA/BAAB batches; intervals are 95% process-level bootstrap intervals, with all samples retained and identical-code controls.These are microbenchmark results for the specified workloads, not an application-wide speedup. Earlier cold-registry measurements linked in the discussion concern earlier source snapshots and are not presented as measurements of this revision.
Validation
--fail-env-changed.linecache.getlines()hooks receiving the frame owner's globals.git diff --checkpass.Developed and tested with Codex (AI assistance).