Skip to content

refactor(limrun): write the network HAR on artifacts and close instead of live - #4080

Merged
thymikee merged 3 commits into
mainfrom
feat/limrun-network-har-simplify
Oct 30, 2026
Merged

thymikee merged 3 commits into
mainfrom
feat/limrun-network-har-simplify

Conversation

@thymikee

@thymikee thymikee commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

Summary

Follow-up to #4043. It keeps the same flags and replaces the live HAR file with a simpler design. --provider-network-logs keeps completed requests in memory. artifacts and close write them to <state-dir>/network-har/<lease-id>.har (mode 0600) and print the path.

  • Bounds: bodies are cut to 64 KiB (inspection.maxBodyBytes and the SDK assembler). The capture keeps the latest 500 requests and counts each dropped request as har-truncated.
  • Failure signal: metadata.failures = { count, last: { kind, host, port }, lastDialFailure }. kind is typed. lastDialFailure is Limrun's typed reason, such as dns_not_found. No error text is matched.
  • Removed: the per-request rewrite loop, the 8 MiB UTF-8 ring, the async write queue, the temp-file renames, the 32-capture retention with deletion, the relative-path and chmod checks, the per-kind failure lists, providerSessionId, and their tests (network-har.test.ts and four runtime tests). Two runtime tests remain: a capture writes a private HAR with its failures on artifacts and at release, and a capture that can't start is an unavailable artifact.
  • Docs: limrun.md says traffic exits through the machine that runs agent-device, the HAR holds secrets, and pinned, DNS-over-HTTPS and HTTP/1.1-only TLS hosts can't be captured. It also tells users to pass --provider-network-hosts.

Why not Limrun's persisted HAR: inspection.persist uploads a networkLog session artifact (listSessionArtifacts(id, 'networkLog') → downloadUrl) only after the tunnel stops. artifacts must work mid-session, so the in-memory entries are needed anyway, and they cover close too.

Diff vs main: 9 files, +291/−1009. #4043 by itself was +1377/−31; the two together are +647/−30.

Validation

  • pnpm check:affected --run at fae6cb3 and 4c7714c (head): format, lint, typecheck, layering, fallow and build pass. vitest-related fails only on two Apple timing tests under host load average ~240. ios-record-trace also fails on clean main f1a17d9. apple-platform-output-guard passes alone on this branch (2 of 2 runs). docs-links and docs-prose pass via pnpm gate. At d252150, which has the same feature files, all 25 remaining static gates passed via pnpm gate.
  • Review fixes in 4c7714c: chmod an existing HAR to 0600, keep at most 32 released capture records, and correct the host-limit help. check:affected there fails only on ios-record-trace, which also fails on main. docs-links and docs-prose pass.
  • Live Limrun iOS at d252150 (same feature files): connect limrun --platform ios --provider-network-logs --provider-network-hosts httpbin.org,agent-device-missing-q7z.com, open https://httpbin.org/get, then open https://agent-device-missing-q7z.com/.
    $ agent-device artifacts            # mid-session
    network: Network traffic (HAR on the daemon host) ready /tmp/ad-har-live/state/network-har/<lease>.har
    Network capture failures: 2; last request-failed agent-device-missing-q7z.com:443; last dial failure dns_not_found
    $ jq -r '.log.entries[] | "\(.response.status) \(.request.method) \(.request.url)"' <har>
    200 GET https://httpbin.org/get
    404 GET https://httpbin.org/favicon.ico
    0 GET https://agent-device-missing-q7z.com/
    0 GET https://agent-device-missing-q7z.com/favicon.ico
    $ agent-device close                # prints the same path and summary
    $ agent-device artifacts            # after close: same file, same summary
    
    The file is -rw------- and the directory is drwx------. After disconnect and daemon stop, the Limrun API listed no non-terminated iOS or Android instances.

🤖 Generated with Claude Code

…d of live

Keep completed requests in memory, capped at 500 entries with 64 KiB bodies,
and write the HAR file when artifacts or close asks for it. This drops the
live rewrite loop, the 8 MiB UTF-8 ring, the async write queue, the 32-capture
retention with deletion, and the path checks. Failures become a count plus the
last typed kind and Limrun's last dial failure reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
agent-device-website Ready Ready Preview Oct 30, 2026 9:29am UTC

Request Review

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 5.60 MB 5.60 MB -2.7 kB
Package (unpacked) 5.60 MB 5.60 MB -2.7 kB
Package (download) 1.70 MB 1.70 MB -841 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 42.4 ms 41.5 ms -0.9 ms
CLI --help 86.6 ms 86.2 ms -0.4 ms

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 9 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread website/docs/docs/limrun.md Resolved
Comment thread packages/provider-limrun/src/runtime.ts Resolved
Comment thread packages/provider-limrun/src/runtime.ts Resolved
Comment thread packages/provider-limrun/src/runtime.ts Resolved
Comment thread packages/provider-limrun/src/runtime.ts Resolved
… and correct the host-limit help

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thymikee

Copy link
Copy Markdown
Member Author

Thanks for the change. I found one problem at 4c7714c that should be fixed before merge. CI is green: 20 checks, none failing. There are no conflicts.

Truncation is counted as a failure, and it hides real failures. In network-har.ts:110, har-truncated goes through fail(). After a capture passes 500 entries, every completed request adds to failures.count and sets failures.last to har-truncated. An earlier request-failed or inspection-error then disappears from metadata.failures.last and from the summary printed at runtime.ts:674-676, for example "Network capture failures: 1342; last har-truncated". limrun.md tells users to read this summary when a page hangs. On a busy session, or with the default *.com/*.org selectors, the summary no longer names the failing host. The regression test does not catch this. In runtime.test.ts:201 the failing request is the last event, so request-failed wins only because of ordering. The rule should be: failures.count and failures.last describe only requests that failed or could not be inspected, and truncation is a buffer statistic. Could you add a separate truncated counter to LimrunNetworkFailures, increment it at network-har.ts:110 without calling fail(), print it as its own clause in networkArtifacts, and drop har-truncated from LimrunNetworkFailureKind and from the count sentence in limrun.md? In the test, please send one successful completion after the failing one, and assert that last is still request-failed and truncated is 1.

Not blocking, and you can take or leave these: the eviction loop at runtime.ts:430 deletes in capture-start order, so it can evict the record just written for the current lease and any live lease whose capture is unavailable (it should evict only records of leases whose session is gone, least recently settled first, with delete-then-set at :427 and a test that releases 33 leases and lists the last one); writeNetworkCapture at runtime.ts:330 drops the error message from the har-write-failed record, though networkArtifacts already prints message; the chmod fix at network-har.ts:160 has no regression because runtime.test.ts:210 reads a file that writeFile just created at 0o600 (pre-create the HAR at 0o644 and assert 0o600 afterwards, and cover the 32-record bound too); nothing deletes HAR files on disk now that #4043's retention is gone, so either accept that as documented at limrun.md:105 or delete the file when a record is evicted; and the per-lease memory worst case at network-har.ts:34 is tens of MB, with a synchronous JSON.stringify per artifacts call, so no change is needed but one line in the docs would help.

On the earlier inline threads, all now resolved: the in-place rewrite of the advertised path still applies (#4080 (comment)) and so does the settle-order point on the 32-record bound (#4080 (comment)). The other eight are fixed or benign at this commit: #4080 (comment) (chmod after write, fixed), #4080 (comment) (failing hosts vs bypass, fixed), #4080 (comment) (docs wording, fixed), #4080 (comment) (retry path already covered), #4080 (comment) (close routed and asserted, fixed), #4080 (comment) (release writes before shutdown), #4080 (comment) (providerSessionId restores the old contract), #4080 (comment) (no drain on tunnel close, docs no longer claim a final HAR).

I did not reproduce the live Limrun run. It was taken before this commit added the chmod and the eviction loop, and neither touches device dispatch, so it is not required. I did not run check:affected locally, and I did not check whether inspection.persist uploads only after the tunnel stops. I also did not verify that the daemon serializes artifacts and close for one lease. Once the truncated counter and its test are in, this is ready for another look.

Truncation no longer goes through the failure count, so an earlier
request-failed or inspection-error stays in failures.last. Released
capture records are evicted in settle order and only for leases whose
session is gone. A failed HAR write keeps its error message.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thymikee

Copy link
Copy Markdown
Member Author

Thanks, addressed in 2ba9ed2.

Blocking: truncation hid real failures. LimrunNetworkFailures has a separate truncated counter now, and the truncation path increments it without calling fail(). har-truncated is gone from LimrunNetworkFailureKind and from the count sentence in limrun.md. networkArtifacts prints truncation as its own clause ("N oldest requests left the HAR"). In the test, a successful completion now follows the failing one. It asserts count: 1, last is still request-failed, and truncated: 1.

Non-blocking notes:

  • Eviction (taken): release deletes the record and sets it again, so the map stays in settle order. The loop evicts only records whose lease has no session, which spares the current lease and any live lease whose capture is unavailable. A new test keeps one live lease unavailable, releases 33 leases, and asserts that the two oldest are forgotten, the last one is ready, and the live one is still unavailable.
  • har-write-failed message (taken): the record keeps the error message.
  • chmod regression (taken): the test creates the HAR at 0o644 first, then asserts 0o600 after artifacts.
  • HAR files on disk (left): kept as documented: "It stays until you delete it."
  • Memory (taken): limrun.md now says a capture keeps the latest 500 requests, each body cut to 64 KiB, so it can hold tens of MB in the daemon.

Checks: check:affected passes format, lint, typecheck, layering, fallow, and build. Vitest's only failure is ios-record-trace, which also fails on main. The docs-prose and docs-links gates pass.

@thymikee

Copy link
Copy Markdown
Member Author

The code looks good at 2ba9ed2. The earlier findings from the first review are fixed, and the two new tests now fail on the previous commit.

Not blocking: the 33-lease eviction test allocates and releases each lease back to back, so deleting the delete-then-set at runtime.ts:432-434 keeps both new tests green and the settle-order comment is not pinned. You can take or leave a fix: allocate one extra capture before the loop, release it inside the loop before the last two releases, then assert it is still ready while released-0 and released-1 are gone.

On the open inline threads: the eviction thread (#4080 (comment)) is fixed at this head and covered by the 33-lease test, and it is already resolved. The in-place rewrite thread (#4080 (comment)) does not apply: the path rewrite at network-har.ts:158 is unchanged, and artifacts and close are the only writers, which the earlier review accepted. It is already resolved too.

CI is green: 20 checks reported and none failing at 2ba9ed2. There are no conflicts. Cubic has not re-reviewed this head.

I did not run check:affected or the provider-limrun tests locally, and I did not do a new live Limrun run, since this change only touches failure accounting and record eviction. I did not verify that the daemon serializes artifacts and close for one lease. Events that arrive between the release write at runtime.ts:430 and capture.close() at :440 are not in the HAR, which the earlier review accepted.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Oct 30, 2026
@thymikee
thymikee merged commit ade7f17 into main Oct 30, 2026
20 checks passed
@thymikee
thymikee deleted the feat/limrun-network-har-simplify branch October 30, 2026 11:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant