test(conformance): run each adapter's scenario through its own test runner

.github/workflows/ci.yml

@@ -45,13 +45,12 @@ jobs:
       # because it is fast and its failures are cheaper to read.
       - name: 🔎 Typecheck
         run: pnpm typecheck
-      # The conformance suite runs the Python adapter beside the JS ones, so a
-      # capture fix missing from one language fails this job.
+      # The conformance suite runs the Python adapter beside the JS ones; the `test` extra brings pytest, its runner.
       - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
         with:
           python-version: '3.13'
       - name: Install the Python adapter
-        run: pip install -e packages/selenium-devtools-py
+        run: pip install -e 'packages/selenium-devtools-py[test]'
       - name: 🧪 Run Tests
         run: pnpm test
       - name: Upload conformance report

ARCHITECTURE.md

@@ -190,15 +190,15 @@ The DOM-walking scripts run in the page via `browser.execute`, so — like `scri
 
 ### `packages/conformance`: cross-adapter conformance suite
 
-Test-only and never published or bundled. Each adapter runs one scenario (open `/login`, type, submit to `/secure`, log out, then a navigation that fails) as its own process, against a fake W3C WebDriver server that models two pages and records what really happened in the browser: which documents were born and when, and which navigations failed. The adapter reports into a recording backend that stands in for the dashboard, reached through the same reuse handshake a rerun uses, so no adapter gains a code path for being tested. `src/checker.ts` then judges the recorded frame stream against behavioural rules, for example "a row replays the page the browser was on when its action completed" and "every document gets exactly one anchor at its birth time". Every adapter is checked by the same rules, so a capture fix that lands in one language and not another fails CI.
+Test-only and never published or bundled. Each adapter runs one scenario (open `/login`, type, submit to `/secure`, wait for a late element and assert on its text, wait on a condition reading two elements and on a helper repeating the test's own read, log out, then an assertion that fails, an element wait that times out and a navigation that fails) as its own process, against a fake W3C WebDriver server that models two pages and records what really happened in the browser: which documents were born and when, and which navigations failed. The adapter reports into a recording backend that stands in for the dashboard, reached through the same reuse handshake a rerun uses, so no adapter gains a code path for being tested. `src/checker.ts` then judges the recorded frame stream against behavioural rules, for example "a row replays the page the browser was on when its action completed" and "every document gets exactly one anchor at its birth time". Every adapter is checked by the same rules, so a capture fix that lands in one language and not another fails CI.
 
 Each scenario runs twice, with and without BiDi. The fake answers the BiDi commands the adapters send, including the navigation, context, element lookup and script calls WebdriverIO routes over BiDi once a session offers it, and a registered preload puts the collector into every later document from birth, as in Chrome. A difference an adapter is allowed to show lives in `src/allowlist.ts` with a mandatory reason, and an entry that stops matching fails the run, so a closed gap forces its own removal.
 
 Every run, passing or failing, dumps its frames, ground truth, violations and scenario output to `packages/conformance/out/<adapter>-bidi-<mode>.json` and rebuilds `out/index.html`, a self-contained page laying each run out as a timeline of documents, command rows and DOM anchors with every violation beside the event it concerns. CI uploads `out/` as the `conformance-report` artifact when the `build` job fails.
 
 Beside the rules, the adapters are compared with each other: each run is reduced to what a dashboard user sees (steps, outcomes, the page each step replays, snapshots, run controls) and any difference between adapters in the same mode fails unless a declared gap explains it. The rules catch every adapter being wrong in the same way; the comparison catches a difference no rule anticipated. Adapters are registered once in `src/adapters.ts`.
 
-Covers all four adapters: `devtools-service`, `selenium-devtools`, `selenium-devtools-py` and `nightwatch-devtools`. The WDIO service and Nightwatch run through their own CLIs against a config under `scenarios/wdio/` and `scenarios/nightwatch/`, which is the shape a runner-based adapter takes. The fake emits no BiDi events, so console and network capture are not checked.
+Covers all four adapters: `devtools-service`, `selenium-devtools`, `selenium-devtools-py` and `nightwatch-devtools`. Each runs its scenario through its framework's own runner (WebdriverIO's, the Nightwatch CLI, mocha with `--require`, pytest), from a folder per adapter under `scenarios/`: five tests, two of which fail by design, one on an assertion and one on an element wait that times out. The fake emits no BiDi events, so console and network capture are not checked.
 
 ### `examples/`
 

CLAUDE.md

@@ -364,6 +364,13 @@ Documented divergences from the conventions above. They exist today as debt to b
 - **A Nightwatch NATIVE assert's target comes from the CALL, not from value provenance.** Its element assertions take the element definition as their first argument (`textContains(definition, expected)`) while the page-level ones take the expected value there (`urlContains(expected)`), so `nightwatch-devtools/src/helpers/assertTarget.ts` reads the locator straight off the recorded args. It is an **allowlist transcribed from Nightwatch's own assertion signatures**, never a "first string arg" heuristic — the same namespace also carries `title*`/`url*` and node:assert's value mirrors (`ok`, `equal`, `strictEqual`), whose first argument is a plain value. A `{selector, locateStrategy}` bag and a Nightwatch `Element` are duck-typed on `.selector`; a definition built from a resolved handle yields none, and `not.*` reads through the negated namespace. Measured: 2 of 4 assert rows carry `#flash` and both `urlContains` rows carry none, on both the BDD and cucumber interfaces, with action counts, DOM anchors and row→document mapping identical to a same-build baseline.
 - **Two mechanisms resolve a command's target selector, and the WDIO one is wrong for interleaved handles.** Selenium keys a `WeakMap` on handle identity (`WebElement.id_` is a promise, so no id is readable when a command is invoked); the service uses a mutable last-selector in `service/src/command-selectors.ts`, which stamps the wrong locator for `const a = await $('#a'); const b = await $('#b'); await a.click()` (the Selenium side is covered in `selenium-devtools/tests/element-locators.test.ts`; the WDIO failure itself has **no** test — it is an unverified reading of `command-selectors.ts`). They cannot share a registry — WDIO's hook sees a *serialized* handle carrying an id string and never the live object, Selenium has the object and no readable id. Unifying would need the policy parameterized rather than the storage shared; until then the WDIO path is knowingly wrong in that case. Nightwatch is a third mechanism: it reads arg 0 through a per-kind allowlist (`assertTarget.ts`) rather than tracking handles at all, because its classic API takes selector strings. The allowlists are deliberately **not** derived from shared's `ACTION_MAP` — that table says how a command *renders* (its `Element` entries include WDIO commands called on a handle, with no selector argument), not whether arg 0 is an element definition.
 - Service renders expect-webdriverio matchers as single `expect.<matcher>` rows by **folding**, not stack/depth suppression (the old `#assertionDepth`/`#matcherStarted`/self-heal machinery is gone). The matcher's value-read (`toHaveText`→`getText`, `toExist`→`isExisting`, …) is captured as a normal command; `afterAssertion` then coalesces the synthesized `expect.*` row into that read in place — inheriting its callSource, screenshot, and timeline position — and the fold replaces **by timestamp, never a public `id`**: `id` is the per-worker `commandCounter`, which resets per spec, so stamping one lets the app's id-first `replaceCommand` swap a same-id row from another spec (duplicate rows + a fold from another spec vanishing, in multi-spec live mode). `beforeAssertion` arms the pending matcher (depth-counted so aliases like `toBeChecked`→`toBeSelected` fold once); a matcher that **hard-throws** — element never resolves, so expect-webdriverio's `waitUntil` rethrows and `afterAssertion` never fires — is synthesized at `afterTest`/`afterStep` from the throwing read, so a failing assertion renders as `expect.<matcher>` whether or not the element existed. Two limits: its error is then the read's (`Can't call getText on … element wasn't found`), not an assertion-phrased message; and `MATCHER_READ_COMMANDS` is a hand-maintained allowlist, so a matcher whose read isn't listed leaves its raw read visible alongside the `expect.*` row. Plain-value jest matchers (`expect(x).toBe(y)`) don't fire the ewdio hooks, so they aren't captured as rows.
+- **What the conformance scenario shows the adapters doing differently.** Each entry is a `KNOWN_GAPS` reason in `packages/conformance/src/allowlist.ts`, scoped to the differences it explains (one hunk, one moved step, or one test's reported state), and reported stale once they stop occurring, except an entry marked `timingDependent`, which is never reported stale and has to be removed by hand.
+  - **A same-millisecond replaceCommand overwrites the Nightwatch row after it** (#458). A read's late screenshot (`browserProxy.ts` `attachScreenshot`) re-sends its row with `sendReplaceCommand(timestamp, ...)`; Nightwatch rows carry no numeric `id`, so shared `replaceCommandIndex` falls back to the last row with that timestamp, and a node:assert row stamped in the same millisecond is the one replaced. The stream then lists the read twice and loses the assertion (3 of 9 runs).
+  - **Nightwatch reports a passing test skipped** (#459). The scenario's two passing tests that call no assertion (the condition waits, and the caught refused navigation) run, their rows are in the stream, and the runner does not count them failed, but the suite tree reports both `skipped` in every run; the passing test that asserts is reported `passed`.
+  - **A Selenium wait has no row when it succeeds, and a condition row when it fails** (#464). A successful `driver.wait(until.elementLocated(...))` is a folded `findElements` row, which ACTION_MAP leaves unmapped; a timed-out one is a `wait` row named after the condition and mapped to `Browser.waitForFunction` without a target, where the WDIO service and Nightwatch show `Element.waitForExist` on the selector.
+  - **A condition wait shows its reads in some adapters and one wait row in others** (#464). A passing wait on a condition that reads the page (`waitUntil(() => getText ...)`, `driver.wait(async () => ...)`, `WebDriverWait(...).until(lambda d: ...)`) is one `Browser.waitForFunction` row in the WebdriverIO service, which logs only the command on top of its stack and so hides the reads inside it; its reads and no wait row in Selenium and Python; and both in Nightwatch. The reads are what show a fold bug (one read folded over another), so the scenario keeps them visible where the adapter shows them.
+  - **A Python WebDriverWait leaves no row** (#461). `WebDriverWait(...).until(...)` is a client-side loop: the adapter patches the protocol chokepoint, where its polls arrive as unmapped `findElement` commands, so a wait has no row and a timed-out one leaves its failed test with no error row.
+  - **Python assertion rows carry no target** (#462). The pytest plugin reads a comparison's operands, and the adapter keeps no record of which element produced a value; the JS adapters' value provenance (`core/read-value-locators.ts`) has no Python counterpart.
 
 ### What a per-action trace capture costs