[vishnuv688]

What & why

Closes #454.

The conformance suite checked how the event stream is built, but not three things a user sees in the dashboard:

• a failed test with nothing red in its actions;
• a call site that points somewhere other than the spec;
• a test with no rows at all.

All three broke in the Selenium adapter (fixed in #453) without any check noticing. This PR adds a rule for each, judged per adapter on the scenario tests and attribution from #465:

Rule	Checks
every-test-has-rows	Every test the dashboard reports owns at least one row. A test never reported is left to tests-end-as-designed.
failed-test-has-error-row	Every test reported failed, or written to fail, has a failed action by shared's commandFailure. An unmapped poll's error does not count, since a passing wait's polls error too.
call-site-names-the-scenario	Every row with a callSource names a line inside the spec the runner executed. DevTools' own packages/, any other file, unknown:0 and an out-of-range line all fail.

Each adapter's launch now declares its scenarioFile, and the harness reads it before cleanup, so the Python copy is still readable.

Proof that the rules guard #453. On this branch I reverted every source file #453 changed (core, shared, trace, selenium-devtools), rebuilt, and ran the conformance suite. Selenium failed in all three BiDi modes, and the cross-adapter agreement check failed too. WebdriverIO, Nightwatch and Python stayed green. The three new rules caught all three of #453's bugs:

• failed-test-has-error-row: the timed-out wait left its failed test with no failed action (once per mode).
• call-site-names-the-scenario: a poll row named unknown:0 (once per mode).
• tests-end-as-designed and one-row-per-browser-action: the first test lost its rows.

The run also flagged every-document-anchored, row-replays-its-page, every-browser-request-listed and adapters-agree-on-actions / -snapshots / -requests, all from the lost first test. With #453 restored, the suite passed 15/15 in two runs.

Type of change

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Polish (an improvement to an existing feature)
• Breaking change (existing behavior changes for users)
• Documentation
• Internal (build, CI, dependencies, tooling)

Packages touched

• shared (types and contracts)
• core (framework-agnostic capture/reporting)
• elements (published element/snapshot API — @wdio/elements)
• service (WebdriverIO adapter)
• nightwatch-devtools (Nightwatch adapter)
• selenium-devtools (Selenium adapter)
• selenium-devtools-py (Selenium Python adapter)
• backend (server)
• app (UI)
• script (page-injected runtime)
• trace (Trace mode)

Capability matrix

• packages/conformance/src/features.ts updated, and tables regenerated with pnpm docs:capabilities, if this adds or changes user-facing behaviour

The Source tab row is now "✅ tested" for all four adapters, since call-site-names-the-scenario holds with no gap. failed-test-has-error-row also backs the Errors row for WebdriverIO and Selenium. every-test-has-rows is listed in INTERNAL_RULES, because it is a precondition of every per-test view rather than a feature.

Notes for reviewers

• Stacked on test(conformance): run each adapter's scenario through its own test runner #465, which is stacked on fix(selenium-devtools): give a failed driver.wait() a row, and fix the call site and first-test rows #453. Retarget to main once those merge.
• Gaps: two known adapter bugs are scoped gaps on failed-test-has-error-row, each matched to its one test by a new failureUnexplained matcher:
  • Python's timed-out WebDriverWait leaves no row (Python: a WebDriverWait leaves no row, so a timed-out wait shows nothing red #461). It fires every run, so the gap is never stale.
  • Nightwatch's same-millisecond replaceCommand overwrites the failed assertion row (Nightwatch: a late screenshot replace can overwrite an assertion row stamped in the same millisecond #458). This is a race, so the gap is marked timingDependent.
• Shared parser: the dashboard's parseCallSource moved from app to shared, and the app re-exports it. The rule therefore reads a call source exactly the way the Source tab does, rather than through a second parser that had already drifted (it converted file:// URLs and the app does not). The conversion to a path stays in the rule, where the comparison is against files on disk.
• No changeset: the only published package touched is app, and its change is a move with no behaviour change.

🤖 Generated with Claude Code

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge; no actionable new defects were found.

Summary

Adds three conformance checks for missing test rows, failed tests without a failed action, and call sites outside the scenario file.

• Reported tests need rows, and failed tests need a failed action.
• Command source links must point into the scenario that ran.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Launch[Adapter launch] --> Spec[Read scenario file]
  Launch --> Run[Run scenario]
  Run --> Frames[Recorded rows and tests]
  Frames --> Checks[Check test rows and failures]
  Frames --> Sites[Check call sites]
  Spec --> Sites
  Checks --> Judge[Apply scoped known gaps]
  Sites --> Judge
  Judge --> Report[Run report]

Loading

_{Reviews (2) · Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." · Reviewed by Greptile}