Repository navigation
perf(memo): people list with each person's kind kept in the store - #346
Merged
Merged
Conversation
cli-core #2817 keeps each person's kind and sort keys on persons (store 62). On a synthetic store of 113,000 people a first page drops from 0.7 s to about 40 ms, a kind tab from about 1 s to 45 ms, newest-contact order from 2.4 s to about 50 ms, and the kind counts take 5 ms. Answers are byte-identical. Seed, bench, after-sync and write-cost scripts included. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…esults review.mjs times people.linkProposals for 500 of 25,000 synthetic link proposals: 190 ms before, 12 ms after cli-core #2817. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
leemour
added a commit
to WireCatLabs/cli-core
that referenced
this pull request
Nov 10, 2026
…and pages in SQL (store 62) (#2817) ## Task Desktop page speed plan, item 1: make `memo people list` cost what the page asks for. On the owner's store (113,705 people) it took 3.7 to 4.2 s on a warm server at any limit, kind or order, and the People page's five kind tabs took about 11 s. ## Why The list worked out every person's kind, counts and name order on every call (about 1.8 s of SQLite and 1.5 s of name checks on the owner's store) and only then applied the limit. ## What - **Store version 62** (reserved in #2814): `persons.kind`, `name_order`, `messages_from`, `task_count`, `last_contact`, an index on `kind` and a partial index on the rows still to work out. Columns, indexes and triggers only. `minCompatible` stays 56. - **Triggers** on every table a kind, order or count is read from: persons, links, identities, chats, members, member counts, mail, recipients, mailboxes, meetings, calendar guests, account identities, organizations, CRM records, tasks, assignments and messages. Each sets the touched people's `kind` to NULL, and an update does so only when a value it reads changes. Older tools that write the file fire them too. - **`listPeople`** works out the cleared people, keeps them, then filters, counts, sorts and pages in SQL. It works out ties, signals and contacts only for the returned rows. Up to 5,000 cleared people are worked out under the write lock (about 0.3 s). More, or everyone after the upgrade, are worked out outside the lock and kept only if `PRAGMA data_version` shows no other writer meanwhile. After three tries, or on a read-only open, it answers from memory as before. `KIND_RULES` is bumped when the rules change in code. A person a merge emptied is stored as `merged` and left out. - **`people.counts(query?)`**: every kind's count from one `GROUP BY`. memo's `people counts` will use it (branch `feat/people-counts`, after this is released). - **`shownReasons(db, people?)`** can narrow the reasons to some people. - **Saving chat members** keeps the rows of members who stayed (`replaceMembers`), so a sync that writes the same members clears no kind. A group's size clears its members' kinds only when it crosses `SMALL_GROUP_MEMBERS`. A test checks that the trigger holds the same number. - **`people.linkProposals`** (memo's `people review`) pages in SQL when it lists everyone's proposals. Before, it read, parsed and resolved every proposal of every status first. For 500 of 25,000 proposals: 213 to 244 ms before, 18 ms after. **Rollout:** tg-cli and max-cli pin exact cli-core versions, and their member saves still delete and rewrite every row. The triggers live in the store file, so those writes clear every member of each synced chat, until tg and max bump to this release. Bump memo, tg and max in one batch. Smaller: a re-save of outgoing mail rewrites its recipients and clears their kinds. ## Testing - Synthetic store of 113,000 people, 2,000,000 messages and 64,000 emails. The results and scripts are in cli-testing `performance/memo/people-list` ([cli-testing #346](WireCatLabs/cli-testing#346)). Medians: | Call | Before | After | | --- | ---: | ---: | | `kind:person`, limit 50 | 704 ms | 42 ms | | one kind tab, limit 1 | 945 to 1,192 ms | 44 to 68 ms | | by newest contact, limit 50 | 2,439 ms | 51 ms | | `kind:all`, limit 5000 | 1,433 ms | 162 ms | | `people.counts` | 5 lists | 5 ms | | next list after a sync of 10,000 messages from 2,000 senders | | about 145 ms | | first list after the upgrade | | 2.8 s, once | | `people.linkProposals` 500 of 25,000 | 213 to 244 ms | 18 ms | - **Same answers:** 72 pages (6 kinds × 4 orders × 3 offsets), a name search and `similar` are byte-identical before and after. - **New tests** in `people-kinds-stored.test.ts`: - after each of about 30 kinds of write, the stored list equals one worked out from nothing; - stored paging equals the in-memory full order for every sort; - `counts` is right; - writing the same chats, members, people and address book again, or moving a big group's or channel's size by one, clears no stored kind. - `identity-links.test.ts`: everyone's proposals page newest first, by status. - Dropping each trigger in turn, the "in step" test fails for 27 of 43. The rest are update variants, link insert and delete, and calendar guests, which the scenario does not reach. - `version-62.test.ts` opens a version 61 file. - Run locally: typecheck, lint on the changed files, `docs:check`, `schema:check`, and 89 affected test files (583 tests: store versions, manifest, parity, people, members, roster, contacts). The full suite runs in CI. - **Not identical in one case:** the name order compares UTF-8 bytes in SQLite, not UTF-16 code units as in JavaScript. They differ only when an emoji meets a character from U+E000 to U+FFFF at the same place after folding. - **Write cost:** 50,000 message inserts from 2,000 senders took 1.7 to 2.3 s before and 1.9 to 2.1 s after. From 110,000 different senders they took up to about a third longer. The machine was loaded. Kind `stranger` failing in memo 0.131.75 is not a cli-core bug: memo ran cli-core 0.624.0, and `stranger` came in 0.625.0 (#2805). memo main is on 0.627.0. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Task
Desktop page speed plan, item 1: measure cli-core's
people listbefore and after it keeps each person's kind onpersons(cli-core #2817, store version 62).Why
memo people listtook 3.7 to 4.2 s on the owner's 113,705 people at any limit, and the People page's kind tabs took about 11 s. This page shows what a list costs after the change, on a synthetic store of the same size, and checks that the answers do not change.What
performance/memo/people-list/:seed.mjs: 113,000 people, 2,000,000 messages, 64,000 emails, every kind present;bench.mjs: list timings, plus the answers written to a file forcmp;after-sync.mjs: the next list after a 10,000-message sync;write-cost.mjs: what the triggers add to message inserts;results.txt.Also links from
performance/memo/README.mdandperformance/README.md.Headline: a first page drops from 704 ms to 42 ms, a kind tab from about 1 s to 45 ms, and the order by newest contact from 2.4 s to 51 ms.
people.countstakes 5 ms. A list after a sync takes about 145 ms. The answers are byte-identical.Testing
I ran the scripts as the README shows. Biome passes on the new files, and
pnpm docs:checkpasses.🤖 Generated with Claude Code