You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Conformance: realistic scenarios run through each adapter's real test runner #455
The conformance scenario is one short login flow, about five commands, in a single test, and for Selenium and Python it runs as a plain script rather than through a test runner. Real suites look different, and the bugs fixed in #453 lived exactly in that difference:
No element waits. No scenario calls driver.wait(until.elementLocated(...)) or an equivalent, so the wait-poll folding and a timed-out wait were never exercised.
No failing test and no second test. Failure reporting and per-test boundaries are only exercised by unit tests.
No test runner for Selenium or Python. A plain script skips hook registration entirely, which is where the first-test race was: under mocha --require the plugin loads before mocha publishes its globals.
What would you like DevTools to do?
Extend every adapter's scenario, keeping them equivalent so the cross-adapter agreement checks still apply:
Run each through its real runner: WDIO's runner (already), the Nightwatch CLI, mocha with --require @wdio/selenium-devtools for Selenium, and pytest for Python.
Several tests, including one that fails on an assertion and one whose element wait times out.
An element wait that succeeds, element reads (getText) and assertions on them.
A fast fake browser on purpose, so timing races that a real browser hides show up.
The fake WebDriver (packages/conformance/src/fake-webdriver.ts) needs whatever endpoints the new steps use (for example findElements returning nothing until a delay, so a wait polls).
Which part of DevTools would this touch?
Conformance suite (packages/conformance/scenarios, the fake WebDriver).
Which frameworks should this apply to?
All four adapters.
Anything you've tried or considered?
Adding the steps to one adapter's scenario only makes the "adapters agree" checks fail on the difference, so all four have to grow together.
Activity