Skip to content

fix: read navigation performance timings without breaking Nightwatch commands - #467

Merged
vishnuv688 merged 5 commits into
mainfrom
fix/performance-read
Oct 27, 2026
Merged

vishnuv688 merged 5 commits into
mainfrom
fix/performance-read

Conversation

@vishnuv688

@vishnuv688 vishnuv688 commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

What & why

Two bugs in the same code path: how the JS adapters read a page's performance timings after a navigation.

1. Nightwatch: the read could cut the test's own command short. Half a second after a navigation, the plugin called browser.execute(CAPTURE_PERFORMANCE_SCRIPT). That is a queued Nightwatch command. Issued from a timer, it becomes a child of whatever command the test is running at that moment, and when it finishes, Nightwatch's async tree resolves that parent command's promise early. Found by the examples smoke job (#463) in CI:

  • The login test's await browser.setValue('#password', ...) returned early, so the next click submitted the form while the password was still being typed.
  • The run log shows the click before the password setValue finished, with a stale element reference in between, and the click's screenshot shows "Your password is invalid!".
  • Reproduced against a local copy of the login page: forcing the read into that window failed the login every time on the old build, and 0 of 69 times with the fix.

The read now goes over the raw WebDriver transport (webdriverExecute), the same path the plugin's collector drains already use for this reason.

2. The capture script never returned its result. CAPTURE_PERFORMANCE_SCRIPT in core was (function() { ... })() with no outer return. WebDriver's execute runs a string script as a function body, so the driver answered null. As a result, the WebdriverIO service, Selenium and Nightwatch attached no load timings, cookies or document info to any navigation row. The Python adapter has its own copy, which already starts with return. The script now returns the payload.

3. Once the timings arrived, two adapters handled them wrongly. This only showed up after fix 2:

  • WebdriverIO service: sent the row again as a new command, so every navigation was listed twice. It now replaces the row (sendReplaceCommand), as Selenium already did.
  • Nightwatch: applied the timings to its stored row but never told the dashboard, so live mode never showed them. It now sends the same replace.

Type of change

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Polish (an improvement to an existing feature)
  • Breaking change (existing behavior changes for users)
  • Documentation
  • Internal (build, CI, dependencies, tooling)

Packages touched

  • shared (types and contracts)
  • core (framework-agnostic capture/reporting)
  • elements (published element/snapshot API — @wdio/elements)
  • service (WebdriverIO adapter)
  • nightwatch-devtools (Nightwatch adapter)
  • selenium-devtools (Selenium adapter)
  • selenium-devtools-py (Selenium Python adapter)
  • backend (server)
  • app (UI)
  • script (page-injected runtime)
  • trace (Trace mode)

Capability matrix

  • packages/conformance/src/features.ts updated, and tables regenerated with pnpm docs:capabilities, if this adds or changes user-facing behaviour

The "Performance timings on navigation rows" cells for the three JS adapters stay "checked by hand": no adapter test asserts the timings reach a row yet.

Notes for reviewers

  • Tests:
    • core/tests/performance-capture.test.ts runs the script as a function body, as WebDriver does, and asserts that the payload comes back. It fails without the return.
    • nightwatch-devtools/tests/session.test.ts asserts that the read reaches the driver over HTTP, that browser.execute is never called, and that the timings reach the dashboard as a replaceCommand. It fails without either fix.
    • service/tests/session.test.ts asserts that a navigation is sent once and its timings arrive as a replaceCommand. It fails with the old resend.
  • Checks: pnpm test 2872 passed; lint, typecheck and docs:capabilities --check clean; the conformance suite passed 15/15 twice.
  • Verified in a real browser: with pnpm demo:wdio, pnpm demo:selenium and pnpm demo:nightwatch, each navigation is listed once and its detail shows the load timings and document info.
  • Why not one fix in core: the Nightwatch change is about how that adapter reaches its driver (its queue), not about the capture logic, which already lives in core.
  • ci: smoke-run each adapter's desktop example against a local fixture site #463 (examples smoke job) needs the Nightwatch fix to go green, so this PR should merge first.

🤖 Generated with Claude Code

Half a second after a navigation the plugin read the page's performance
timings with browser.execute, a queued Nightwatch command. Issued from a
timer, it became a child of whichever command the test was running, and
finishing it resolved that command's await early: in CI a login clicked
submit while setValue was still typing the password. The read now goes
over the raw WebDriver transport, like the drains.
WebDriver's execute runs a string script as a function body, and the
script called its wrapper without returning it, so every adapter that
sends core's script got null back and no navigation row carried load
timings, cookies or document info. Python's copy already returned.
@greptile-apps

greptile-apps Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; both numbered previous findings are fixed and no new actionable issues remain.

Summary

Restores navigation timings without sending Nightwatch reads through its command queue.

  • Navigation rows now receive the page timings and details.
  • Nightwatch reads page timings without interrupting test commands.
  • WebdriverIO updates navigation rows instead of adding copies.

Diagram

sequenceDiagram
    participant N as Nightwatch adapter
    participant D as WebDriver
    participant U as Dashboard
    N->>U: Send navigation row
    Note over N: Wait 500 ms
    N->>D: Execute capture script at configured path
    D-->>N: Return timings
    alt Original row remains in commandsLog
        N->>U: Replace navigation row with timings
    else Retry removed original row
        Note over N: Skip old row update
    end
Loading

Reviews (3) · Last reviewed commit: "fix(nightwatch-devtools): send no timing..." · Reviewed by Greptile

With the capture script returning its payload, the WebdriverIO service
sent the timed row again as a new command, so every navigation was
listed twice. It now replaces the row, as Selenium already did.
Nightwatch applied the timings to its stored row but never told the
dashboard, so live mode never showed them; it now sends the same
replace.
The probes, drains and performance read reach the driver outside
Nightwatch's queue through one address, which dropped the configured
default_path_prefix; only the Appium executor carried it. A driver
served under a prefix such as /wd/hub rejected every one of them.
A retried command's row is removed and reissued under a new timestamp.
A performance read still pending for the old row then replaced a row
the dashboard no longer had, and the dashboard listed it again. The
timings are sent only while their row is still in the log.
@vishnuv688
vishnuv688 merged commit 1daf52c into main Oct 27, 2026
11 checks passed
vishnuv688 added a commit that referenced this pull request Oct 27, 2026
An awaited Nightwatch wait ran after the commands that followed it
because the plugin's performance read, a queued browser.execute fired
from a timer, became a child of the running command and resolved its
await early. With that read off the queue the wait keeps its place in
every run and the login test passes, so both gaps went stale. The
tests that used the gap to exercise a move scoped to one place now
build that gap themselves.
Winify pushed a commit to Winify/devtools that referenced this pull request Oct 30, 2026
…ce rules

Two bugs fixed this week (webdriverio#467, webdriverio#476) were caught by review, not by the
conformance suite, because no rule covered them.

navigation-row-carries-timings: the fake browser now runs a script that
reads getEntriesByType('navigation') against the page's own timings, title
and url, as a WebDriver body, a BiDi function declaration or a BiDi
expression, so it answers only what the script returns. A capture script
that builds its payload and never returns it is caught the way a real
browser exposes it. The rule judges every successful navigation row after
its replaceCommand updates are applied, as the dashboard shows it. The
script is matched by content, not by import path, so it survives moving the
script out of core.

page-level-assertion-names-no-element: a new scenario test reads the
heading, reads the page title (which now reads the same), and asserts the
title is not 'Secure Area', a value an element read earlier. Taking the
target from either the colliding read or the expected value boxes the wrong
element. The WebdriverIO service fails this rule on main, because it stamps
node:assert rows with the last element it resolved and hides the user's own
getTitle; that is declared as a scoped gap against webdriverio#481 rather than fixed
here.

The service's rows carry a testUid the suite tree never reports, so the
assertion that ends a test was placed by time and, sharing a millisecond
with the next test's start, landed in the wrong test. Rows of a uid the
tree does not report are now placed together by the first of them.

The performance timings cells are now backed by the rule for all four
adapters; the page-level rule is listed as internal, being one property of
assertion capture.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant