You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A plain-script run reports passed whatever happened. _send_default_suite is
called with a hardcoded state at packages/selenium-devtools-py/src/selenium_devtools/instrumentation.py:356:
_send_default_suite(capturer, "passed") # the whole script run is done
and the adapter has no failure path at all. The dashboard shows a green 1/1 passed for a run that raised, which is the one signal a user reads at a glance.
Reproduced on examples/selenium/python-test/login.py with a deliberately
broken assertion: the tree shows 1/1 passed while the Actions list stops dead
at the getElementText that preceded the failing line, with the logout click
and final getCurrentUrl absent. The command stream tells the truth; the tree
contradicts it.
pytest runs are unaffected._external_suites suppresses the synthetic
suite entirely, and pytest_plugin derives each state from report.passed/failed/skipped. This is the plain-script path only.
Why it is not a one-line change
The "passed" is sent from driver.quit(), which runs inside the user's finallybefore the exception reaches top level — so at that moment the
adapter genuinely does not know the outcome yet. The final state has to move out
of the quit path and into teardown:
install a sys.excepthook wrapper in enable() that records the failure, and
send the final suite state from disable() / the atexit handler, which run
after the hook
sys.exc_info() inside the quit path is the tempting shortcut — it does
return the in-flight exception during a finally — but it also returns one
when the caller is merely inside an unrelated except block, so it would
report false failures
Also worth deciding here: a plain script has no test boundaries, so "the run
failed" is the most the adapter can honestly say.
Acceptance criteria
A plain script that raises reports failed for its synthetic suite and test; a
clean run still reports passed; an unrelated except block in the user's code
does not make a passing run report failed. pytest runs are unchanged. Unit tests
cover all three.
Follow-on found while verifying this: commands carry no test identity
Chasing why the screencast picker offers "Recording 1/2/3" with no way to know
which belongs to the selected test turned up a larger gap underneath it.
frames.command_log emits command, args, result, timestamp, startTime, callSource, id — and nothing else. grep -rn "testUid" packages/selenium-devtools-py/src/ returns nothing. Shared's CommandLog
has testUid?: string, and the Nightwatch and WDIO adapters stamp it; the
Python adapter never has.
Consequence, visible in the dashboard today: selecting a test does not narrow
the Actions list. It cannot — every command in the run is one undifferentiated
stream, so the middle panel shows all of them whatever you click. On the pytest
example that is three tests' commands under every selection.
So there are two missing links, not one:
link
state
enables
command → test (testUid)
absent
Actions scoped to the selected test
test → session
absent
the right recording selected automatically
The second is cheap once the first exists: the app's #videos list in components/browser/snapshot.ts is already keyed by sessionId, and send_screencast already files each video under the session it came from. What
is missing is knowing which session a given test ran in.
One wrinkle specific to a function-scoped pytest fixture (the shape the
example uses, and the common one): the driver is created during test setup, so
at pytest_runtest_logstart no session exists yet. Attribution has to happen
when the session starts — instrumentation knows the driver and its session id
— against the test currently running, which only the plugin knows. That means
the two modules need a shared notion of "current test", which is exactly what
the JS adapters carry.
Not this issue's outcome fix, and bigger than it. Recording here because it was
found on this branch and the two are adjacent in the same surface.
Activity
vishnuv688 commentedon Aug 17, 2026
Follow-on found while verifying this: commands carry no test identity
Chasing why the screencast picker offers "Recording 1/2/3" with no way to know
which belongs to the selected test turned up a larger gap underneath it.
frames.command_logemitscommand,args,result,timestamp,startTime,callSource,id— and nothing else.grep -rn "testUid" packages/selenium-devtools-py/src/returns nothing. Shared'sCommandLoghas
testUid?: string, and the Nightwatch and WDIO adapters stamp it; thePython adapter never has.
Consequence, visible in the dashboard today: selecting a test does not narrow
the Actions list. It cannot — every command in the run is one undifferentiated
stream, so the middle panel shows all of them whatever you click. On the pytest
example that is three tests' commands under every selection.
So there are two missing links, not one:
testUid)The second is cheap once the first exists: the app's
#videoslist incomponents/browser/snapshot.tsis already keyed bysessionId, andsend_screencastalready files each video under the session it came from. Whatis missing is knowing which session a given test ran in.
One wrinkle specific to a function-scoped pytest fixture (the shape the
example uses, and the common one): the driver is created during test setup, so
at
pytest_runtest_logstartno session exists yet. Attribution has to happenwhen the session starts —
instrumentationknows the driver and its session id— against the test currently running, which only the plugin knows. That means
the two modules need a shared notion of "current test", which is exactly what
the JS adapters carry.
Not this issue's outcome fix, and bigger than it. Recording here because it was
found on this branch and the two are adjacent in the same surface.