Skip to content

feat(limrun): capture session traffic into a live HAR and report tunnel failures - #4043

Merged
thymikee merged 10 commits into
mainfrom
feat/limrun-network-har
Oct 29, 2026
Merged

thymikee merged 10 commits into
mainfrom
feat/limrun-network-har

Conversation

@thymikee

@thymikee thymikee commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

Summary

connect limrun --provider-network-logs writes a live HAR 1.2 file of device traffic through Limrun's tunnel inspection. --provider-network-hosts narrows the capture to named hosts. The SDK logged tunnel failures only to the console. The capture now records typed failure kinds (request-failed with host:port, inspection-error, inspection-gap, inspection-stream-error, har-write-failed, har-truncated) plus the tunnel's typed lastDialFailure.reason. artifacts and close print a summary, and artifacts --json carries the report. A capture that can't start is an unavailable artifact with its reason.

The capture is bounded: bodies are cut to 64 KiB, the file to 8 MiB, and writes are async with one in flight. The HAR is owner-only, named per lease, and deleted when its record is evicted. The docs cover the egress, sensitivity, pinning, DoH and HTTP/2-only limits. 36 files, 10 commits. Follow-up: #4064. Upstream evidence is in a comment.

agent-device connect limrun --platform ios --provider-network-logs --provider-network-hosts "api.example.com,localhost:8081"
agent-device artifacts

Validation

  • check:affected --run at a597bf4 failed only on ios-record-trace timeouts (they also fail on base; host load average about 70). 14 static gates pass via pnpm gate. CI was all green at e812372.
  • Live Limrun iOS at a597bf4 with --provider-network-hosts httpbin.org,agent-device-missing-q7z.com. jq read mid-session:
    200 GET https://httpbin.org/get
    0 GET https://agent-device-missing-q7z.com/ …client_dial_failed
    Network capture failures: 2; request-failed 2, latest agent-device-missing-q7z.com:443 (…); last dial failure dns_not_found
    
    close printed the same summary, and HAR mode is -rw-------. An invalid host showed unavailable. No instances were left.

🤖 Generated with Claude Code

@thymikee

Copy link
Copy Markdown
Member Author

Evidence for an upstream Limrun report (@limrun/api 0.64.1; 0.66.0 has the same dialer surface)

  1. The TLS dial offers ALPN h2 only. For every intercepted HTTPS connection, the server's open transport request asks the host dialer for tls.connect({ ALPNProtocols: ["h2"] }). A probe that wrapped tls.connect logged httpbin.org ["h2"], sequoia.cdn-apple.com ["h2"] and configuration.ls.apple.com ["h2"]. Hosts without HTTP/2 answer with TLS alert 120: openssl s_client -connect sequoia.cdn-apple.com:443 -alpn h2 gives tlsv1 alert no application protocol, and -alpn h2,http/1.1 negotiates http/1.1. The device then sees client_dial_failed, and getTunnelStatus().lastDialFailure.reason is tls_handshake_failed. This reproduces in every iOS session (sequoia.cdn-apple.com/sequoia-prod/mapping-info-plist). Expected: offer the device's own ALPN list, or ["h2","http/1.1"].
  2. Dial failures have no callback. startDestinationTcpTunnel reports a per-connection dial failure (host, port, classifyOpenFailure reason) only through its internal console.warn logger. getTunnelStatus().lastDialFailure has the typed reason but no host, and only the latest failure. Request: an onOpenFailure({ connId, host, port, reason, osCode }) option on DestinationTunnelStartOptions.
  3. First-session stall: not reproduced. It did not recur in 4 iOS sessions and 1 Android session on 2026-10-29. The original trace was a host-side Failed to connect to configuration.ls.apple.com:443: timed out, which is the SDK's 10 s connectTimeoutMs against an h2-only dial. agent-device now reports that case as request-failed plus last dial failure client_timeout.
  4. Requests can bypass capture, from the first request up to a whole session. In 9 sessions on 2026-10-29, one missed only its first request and one (ios_euna_01m63m9cdvfd2sa7av435y4nym) captured nothing: neither a request to a resolvable host nor one to an NXDOMAIN host produced an inspection event or a dial failure. First-request case: On 2026-10-29, connect --provider-network-hosts httpbin.org,… followed by open https://httpbin.org/get immediately after allocation produced no HAR entry, no inspection event and no dial failure. An identical rerun captured it. The tunnel was ready (startTunnel resolved) before openUrl. This fits the SDK's "relaunch apps that connected before it" log line: the device's resolver or connection state from before the tunnel was ready seems to win briefly.

@vercel

vercel Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
agent-device-website Ready Ready Preview Oct 29, 2026 2:06pm UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 35 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread website/docs/docs/limrun.md Resolved
Comment thread src/builtin-provider-plugins.ts Resolved
Comment thread packages/provider-limrun/src/device.ts Resolved
Comment thread skills/agent-device/SKILL.md Resolved
Comment thread website/docs/docs/limrun.md Resolved
Comment thread website/docs/docs/limrun.md Resolved
thymikee and others added 6 commits October 29, 2026 12:21
…-network-logs

The opt-in opens the instance's one destination tunnel with Limrun's own
inspection and rewrites a HAR 1.2 file under the daemon state dir on every
completed request, so the artifact stays valid while the session runs and
after it. The Metro/companion localhost route rides the same tunnel. The HAR
path is reported at allocate, listed by artifacts, and printed at disconnect.
… close

The SDK logs per-connection dial failures only to the console, so a hung
page under --provider-network-logs left no trace an agent could read. The
capture now records the typed inspection failures (failed requests with
host and port, inspection errors, stream gaps) per kind, reads the tunnel's
last typed dial failure, and reports them in the HAR artifact metadata and
as the artifacts/close summary. A tunnel conflict is now classified by the
SDK's typed session error code instead of its message text.

Docs state that captured traffic exits through the agent-device host, that
the HAR is sensitive, that hosts without HTTP/2 fail under capture, and
that agents should pass --provider-network-hosts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he capture

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…HAR file

A capture that failed to start was dropped with the allocate result, so
artifacts reported no provider support at all. Each lease now keeps one
capture state (capturing, stopped, unavailable): artifacts and close
report an unavailable capture with its typed reason, and a release retry
reports the stopped capture again. Completed requests share one coalesced
HAR rewrite, a write failure is recorded as har-write-failed, the file is
owner-only, and it is named per lease so attached instances do not
overwrite each other. The tmpdir fallback is gone; without a capture
directory the opt-in refuses with capture-dir-unset.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thymikee
thymikee force-pushed the feat/limrun-network-har branch from f1ca675 to 7de97ba Compare October 29, 2026 11:36
@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 5.58 MB 5.60 MB +14.0 kB
Package (unpacked) 5.58 MB 5.60 MB +14.0 kB
Package (download) 1.70 MB 1.70 MB +6.1 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 40.9 ms 41.6 ms +0.7 ms
CLI --help 84.4 ms 85.2 ms +0.8 ms

@thymikee

Copy link
Copy Markdown
Member Author

Thanks for this. I reviewed f1ca675 and found one problem that must be fixed before merge: the HAR capture has no bound. The 8 checks reported at f1ca675 are green.

network-har.ts:301 keeps every completed entry for the whole session (L145, L169). Bodies are captured, and the SDK assembler allows 10 MiB per body by default. Each 250 ms window then runs JSON.stringify over all entries and calls writeFileSync on the daemon event loop (L152-160). The default selectors include iOS system and CDN traffic, so a long or media-heavy session makes each rewrite slower, which can stall commands on every session the daemon owns. Past V8's string limit, JSON.stringify throws on every flush, the HAR stops updating, and memory keeps growing. The rule: capture memory and per-write cost must stay bounded no matter how long the session runs. Pass a small bodyLimit to DestinationTunnelHARAssembler, cap entries or bytes, record a typed har-truncated failure when older entries are dropped, and write with fs.promises with one write in flight. Or build the HAR only when artifacts or close asks for it. I confirmed the mechanism in the code and the 0.64.1 SDK source. I did not measure a long session.

Could a smaller design work here? Keep capture in provider-limrun, but hold a bounded in-memory ring of assembled entries (small bodyLimit, at most N entries, a typed truncation count). Limrun's networkDump operation could return that ring while a capture is live, so network dump --include headers|body shows tunnel traffic with the daemon's existing payload caps and redactNetworkLogText, and the CLI, MCP and Node surfaces get it with no new output code. At artifacts or close, write one HAR snapshot and register it with trackDownloadableArtifact, so remote and proxy clients can download it and the tracker's expiry owns retention. That removes the rewrite loop, the daemon-host path in CloudArtifact.url, and the unbounded files in the state dir. The size is under the thresholds (618 net production lines, 1066 gross). Mapping DestinationTunnelHAREntry to the NetworkDumpResult shape comes first. If the HAR must stay readable as a live file during the session, please say so explicitly; then the bounds above are the minimum.

Not blocking, and fine to take or leave: nothing deletes the network-har/*.har files when forgetStoppedNetworkCaptures evicts a record (runtime.ts:452), and they hold raw Authorization headers, cookies and bodies, so a file should live exactly as long as its capture record and limrun.md should state the retention; the capture sits beside the existing networkDump route and the artifact tracker, and network dump on a capture-enabled Limrun session still never sees tunnel traffic (runtime.ts:735); and the new paragraph in SKILL.md:24 could be a one-line pointer to help or the Limrun docs, since skills are routing cards, and the PR body says 4 commits while the range has 6.

The live Limrun run is reported in the PR body only. I could not reproduce it, so I have not seen artifacts list the HAR after close or the unavailable artifact appear. I also could not read the check:affected receipt.

The newer head 7de97ba rebases onto main and drops the src/cli/connection/profile-fields.ts hunk that #3865 superseded, which resolves the conflict; the other five commits are unchanged, so the problem above still applies there. Before merge, bound the capture and rerun check:affected at the new head.

Of the open inline threads from other reviewers, these still stand: the close-before-flush ordering, the dial-only failure summary, the repeat allocation early return, and the state-dir owner (narrower than described: the default CLI route works, and the owner is resolveDaemonPaths(...).baseDir). Also still standing: the artifacts without the flag doc claim, the boolean evidence filter, the status-0 test, and the un-aborted status read. Four lower-priority threads also still apply. These do not apply, and you can resolve them: the temp file mode (the only writer creates it with 0o600 inside a 0o700 directory named per lease), the 'this host' wording (the daemon writes the HAR itself, so the claim is right in substance), and the jq pipe (SKILL.md reads the HAR at the printed path, matching limrun.md).

- Bound memory and per-write cost: bodies are cut to 64 KiB by the tunnel and
  the assembler, each entry is serialized once into a ring capped at 8 MiB,
  the oldest entries are dropped with a typed har-truncated failure, and
  writes are async with one in flight through exclusive temp files.
- close() is async: it writes the HAR even when the tunnel refuses to close,
  and a later call retries the tunnel.
- A repeat allocation starts the capture it newly asks for; a dial-only
  failure reaches the summary; a stopped capture's HAR is deleted when its
  record is evicted.
- The capture directory comes from the daemon's resolved state dir, and
  doctor counts an enabled opt-in boolean as provider scope.
- Docs: opt-in artifacts wording, allocation timing, retention, size bounds,
  the data wrapper, the daemon host in flag help, help remote, and the skill
  as a one-line pointer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thymikee

Copy link
Copy Markdown
Member Author

Thanks. 31cf8cd bounds the capture. The live file is an explicit requirement for this change: an agent reads the HAR with jq while the session runs. So I took the bounds you listed as the minimum, not the ring plus network dump design:

  • Memory and per-write cost: bodies are cut to 64 KiB, both server-side (inspection.maxBodyBytes) and in DestinationTunnelHARAssembler(bodyLimit). Each entry is serialized once into a ring capped at 8 MiB. A write joins the ring and never re-stringifies the session.
  • Truncation: the oldest entries are dropped, and the drops are reported as a typed har-truncated failure whose count is the number of entries dropped.
  • Writes: async through fs.promises, one in flight, coalesced over 250 ms, through exclusive wx temp files with mode 0600.
  • Retention (your non-blocking note): a stopped capture's HAR is deleted when its record is evicted, after 32 settled captures. limrun.md states the retention, and that files from a stopped daemon stay until you delete them.
  • SKILL.md: now a one-line pointer to help remote and the Limrun docs. help remote gained the capture line.
  • network dump: not taken here. Routing tunnel traffic through networkDump and trackDownloadableArtifact changes how this capture is delivered, so it fits better as a follow-up. I can open an issue if you want it.

All inline threads are fixed or answered. The status read can't be aborted because the SDK exposes no signal; that is on the upstream list.

Live Limrun iOS at 31cf8cd (rebuilt). The cases you couldn't reproduce:

$ agent-device connect limrun --platform ios --provider-network-logs --provider-network-hosts "not a host!!"
$ agent-device open https://example.com/ && agent-device artifacts
network: Network traffic (HAR, live file on the daemon host) unavailable
Network capture did not start (tunnel-start-failed): invalid tunnel domain not a host!!
$ agent-device close            # prints the same line
$ agent-device artifacts        # after close: same unavailable artifact

# a capturing session, after close:
$ agent-device artifacts
network: Network traffic (HAR, live file on the daemon host) ready /tmp/ad-har-final/network-har/ios_euna_01m63r1748fd2v8qtdtm74pjpn-5e14ef619b323c14434cbd5db951f790.har

After each run, no Limrun instances were left running and the trial daemons were stopped.

Validation: check:affected --run at 31cf8cd failed only at vitest-related: test/integration/provider-scenarios/ios-record-trace.test.ts hits 5 s timeouts. The same test also fails on clean origin/main 4025dfb on this host, so it is base-blocked locally and does not come from this branch. Because the run stops at the first failure, I ran the remaining static gates one by one with pnpm gate <id>, and all 14 pass: integration-progress, daemon-wire-compat, replay-compat, gate-manifest, depgraph, tmpdir-leaks, production-exports, bundle-owner-files, command-docs, capabilities-matrix, docs-links, docs-prose, agent-guidance and xctest-selection. GitHub CI is authoritative for the unit, coverage and provider-integration lanes.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 12 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread src/commands/schema/cli-help.ts Resolved
…e file in UTF-8 bytes

The first publication of the HAR now has to succeed before the tunnel opens,
so an unwritable capture path is an unavailable capture instead of a ready
artifact pointing at a missing file. The size cap counts UTF-8 bytes plus the
document wrapper and commas, and an entry that cannot fit alone is dropped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thymikee

Copy link
Copy Markdown
Member Author

At 31cf8cd I found no new code problems, but two open review threads still need a fix or an answer before merge: the first HAR write failure still opens the tunnel and the size cap counts UTF-16 units. Both hold. In the first, a failed first write still reports a ready artifact that points to a missing file. In the second, non-ASCII bodies can push the file well past the documented 8 MiB.

The third thread, on the help example, does not apply: the example matches limrun.md, and the flag help says the flag is optional. Please resolve it.

Smoke Tests is still running. The diff touches no local platform dispatch, and its shared-route changes only add lines when the provider returns cloud artifacts, so a failure there is unlikely to relate to this PR. There are no conflicts. I could not reproduce the live Limrun run, so the mid-session read, the close summary, the 0600 mode and the artifact states are as the author reported them for 31cf8cd. I did not measure a long or media-heavy session against the new bounds.

Not blocking, and you can take or leave these: CloudArtifact.url carries an absolute daemon-host path that clients print as the link, and tunnel traffic stays outside network dump, so the follow-up issue you offered is a good place for that. No test covers the discard in forgetStoppedNetworkCaptures, so one test that releases 33 capturing leases and checks the first HAR is gone would pin it. A repeat allocation retries only when no capture record exists, so an unavailable record never retries. Retrying on that phase would fix it, or the docs can say reconnecting is the only way. mkdir with mode: 0o700 does not tighten an existing wider directory, so either chmod it or drop the claim.

…e dir, and pin HAR retention

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thymikee

Copy link
Copy Markdown
Member Author

Thanks. Both blocking threads were fixed at 44b0e2d:

  • The first HAR write has to succeed, or the capture is refused before the tunnel opens.
  • The cap counts UTF-8 bytes plus the document wrapper.

The help-example thread is answered and resolved.

e812372 takes the four take-or-leave notes:

Live: Limrun iOS at e812372 (rebuilt). The jq read mid-session showed 200 GET https://httpbin.org/get and 0 GET https://agent-device-missing-q7z.com/ …client_dial_failed. close printed Network capture failures: 2; request-failed 2, …; last dial failure dns_not_found. The directory is drwx------ and the file -rw-------. No Limrun instances were left running.

Validation at e812372: check:affected --run failed only in vitest-related, on ios-record-trace (it also fails on base) and remote-test-artifacts-materialization (it passes when run alone). The host load average is about 95. All 14 static gates pass via pnpm gate. CI was all green at 44b0e2d.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 4 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thymikee

Copy link
Copy Markdown
Member Author

The code looks good at a597bf4: the issues from the earlier review (#4043 (comment)) are fixed at this commit, and nothing new blocks it. The 20 checks are green, and there are no conflicts.

Not blocking, take or leave: in stopNetworkCapture (packages/provider-limrun/src/runtime.ts:460), networkCaptures.set on an existing key keeps the capture's start position in the Map, so the retention cleanup evicts by start order instead of stop order. A long-running lease that stops last can lose its record and HAR file once 32 other captures settle. Calling delete before set (also on the unavailable re-set in startNetworkCapture) keeps the order by settle time, and the retention test could release lease 0 last to cover it.

All three earlier cubic-dev-ai threads, already resolved, are fixed at this commit: the flush(true) rethrow before startTunnel (#4043 (comment)), the byte cap counting entries and oversize entries (#4043 (comment)), and the non-absolute harPath refusal (#4043 (comment)).

I did not reproduce the live Limrun iOS run, so the mid-session HAR, the close summary, the 0600 mode and the unavailable state are as the author reported them. I did not run the provider-limrun tests locally, and I did not measure a long or media-heavy session against the 8 MiB cap.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Oct 29, 2026
@thymikee
thymikee merged commit 231949d into main Oct 29, 2026
20 checks passed
@thymikee
thymikee deleted the feat/limrun-network-har branch October 29, 2026 16:06
@thymikee

Copy link
Copy Markdown
Member Author

This merged with the live-file design. #4080 replaces it with the simplest version that meets the goal: completed requests stay in memory (500 requests, 64 KiB bodies), and artifacts and close write the HAR (0600) on demand. The failure signal is a count plus the last typed kind and Limrun's last dial-failure reason. #4080 removes the rewrite loop, the 8 MiB ring, the write queue, the retention with deletion and the path checks, so the review threads about them no longer apply. The un-abortable status read still applies: the SDK takes no signal, and the 2 s budget stays. Limrun's own inspection.persist HAR uploads only after the tunnel stops, so it can't serve a mid-session artifacts. #4080 is +282/−1011 against main, with a live iOS run in its description.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant