Repository navigation
docs(performance): several requests at once through one command server - #343
Merged
Merged
Conversation
7 reads through one commands --serve take about 0.1 s in any order, against 0.7-1 s with a process each; running them at once saves another 6-54 %. Feeds WireCatLabs/cli-core#2812 and the WireCat Desktop change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
leemour
added a commit
to WireCatLabs/cli-core
that referenced
this pull request
Nov 9, 2026
## Task Slava asked why the command server gives no pool of access, and to allow parallel calls through it. Today `commands --serve` answers one request at a time. ## Why The Memory page in WireCat Desktop sends 7 reads together. Desktop keeps a pool of 2 servers that answer one request at a time, so 5 of the 7 reads each start a fresh `memo` process. A start takes about 0.4 to 0.9 s and about 165 MB. ## What - **Up to 8 requests at once.** `serveCommands` answers each request by `id` when it ends. The ready line now says `"concurrent":8`. The protocol stays 1, so a client that sends one request at a time works as before. Desktop accepts only `protocol == 1`, so a bump would have made it refuse every server. - **Each request keeps its own state**, in an `AsyncLocalStorage` scope (`src/cli/command-server-request.ts`). These parts of the process used to be shared: - `process.stdout.write` and `process.stderr.write`: one `current` buffer. - `process.stdin`: replaced for each request. - `process.exitCode`. - The process run, `processScope` in `src/runs/scope.ts`. What each request now has: - Its own output buffers. - Its own closed stdin. `process.stdin` is now a getter, so the input policy that commands keep for each stream stays per request. - Its own exit code, through `setExitCode` and the new `EntryContext.setExitCode`. - Its own process run, through `asOwnProcess`. - **`process.exitCode` cannot be scoped.** Node defines it as not configurable. A direct write to it goes to the request that ends next, and the docs say so. Memo writes it directly in 4 places: `run/import.ts:361`, `digest/backfill.ts:221`, `doctor/command.ts:253` and `digest/digest-commands.ts:345`. Those should move to `entry.setExitCode` in a follow-up. - **The store.** Each request opens its own connection, and the store already uses WAL and `busy_timeout=5000`. - SQLite keeps the locks of connections in one process apart when they all use one copy of the library ([How To Corrupt, §2.2–2.3](https://www.sqlite.org/howtocorrupt.html)). - BUG-557 came from a raw fs open of the `.db` file. `createWritable` already avoids that. - The Node driver is synchronous, so two transactions in the process never interleave, and a write next to reads never waits inside the process. - **A `store` command runs alone.** It waits until the requests under way end, and new requests wait until it ends. `store restore`, `reset` and `recover run` swap the file, and `quiesce` does not count this process's own connections. A request that ends while it still waits for its turn (its `--timeout`, a signal) gives the turn up. - The server raises the listener limit on `process`, `stdout` and `stderr` by the cap, because each run adds its own signal and `error` listeners. - `bench/command-server/measure.mjs` measures it. Docs are updated: REFERENCE ("The command server") and ARCHITECTURE. ## Measurements The full page is in cli-testing: [performance/command-server/concurrent](https://github.com/WireCatLabs/cli-testing/blob/main/performance/command-server/concurrent/README.md) ([cli-testing #343](WireCatLabs/cli-testing#343)). Headline numbers: Command: `node bench/command-server/measure.mjs 7 5 [argv]`. It uses a throwaway store with 30 saved searches. Machine: AMD Ryzen AI 9 HX 470, Node 24.19. The table gives medians over 5 rounds, from 2 runs each. | 7 reads | `searches list --json` | `config show --json` | |---|---|---| | one server, one at a time | 95–157 ms | 153–164 ms | | one server, all at once | 76–148 ms | 70–97 ms | | a process each, all at once (about what Desktop pays today for the 5 that miss the pool) | 671–1040 ms | 755–830 ms | | a process each, one at a time | 2985–5271 ms | 3271–3992 ms | Most of the gain is from not starting a process. Running the reads at once saves another 6 to 54 %, because the work is mostly synchronous JavaScript and SQLite on one thread. Not measured: a real memo store, the Memory page's actual 7 commands, memory use, and a server under a write lock held by another process. That lock stops the whole server for up to 5 s, not only the request that waits; REFERENCE says so. ## Testing - `src/cli/command-server.test.ts` (18 tests): - Answers come back as requests end. - The cap holds. - A request waits until it is alone, and the ones after it wait for it. - A request that ends while it waits gives up its turn. - In a real process, 6 requests with interleaved stdout and stderr each get their own output and their own exit code, and they are answered in the order they end. - 10 reads, 2 store writes and a `store` command sent together all succeed, each with its own run id and no `MaxListeners` warning. - A direct `process.exitCode` write still works for a request that runs alone. - Also run: `entry.test.ts`, `scope.test.ts`, `pnpm ci:quick`, the typecheck of the tests, `pnpm docs:check` and `pnpm lint`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Task
Record the measurement behind cli-core #2812: one
commands --serverunning several requests at once.Why
WireCat Desktop's Memory page sends 7 reads together. A pool of 2 servers that answer one at a time starts a fresh
memoprocess for each of the other 5. The repository rule puts every measurement here, with its script.What
performance/command-server/concurrent/README.mdwith these sections: why it was run, setup, the exact commands, results, what it showed, and what was not measured.measure.mjs, a copy of cli-core'sbench/command-server/measure.mjs, run from a cli-core checkout.results.txt, the raw output.performance/command-server/README.md, linked fromperformance/README.md.Headline: 7 reads through one server take 70–157 ms, against 671–1040 ms with a process each. Running them at once saves another 6–54 %.
Testing
Ran
biome check,markdownlint-cli2,cspellandpnpm docs:checkon the new pages.🤖 Generated with Claude Code