[jgrodziski]

Third step of the word cloud / glossary (#6), milestone M19a in PLAN.md §20. Stacked on #27 (itself on #26): review those first; this PR's diff is the five commits after #27.

It counts the words of a whole model: buildVocabulary(graph) → the codegraph.vocabulary/1 artifact, with committed snapshots for every fixture. Nothing in the CLI calls it yet; PR 4 adds the codegraph vocabulary command and the drift report.

A qualified name's words weigh 1

Measured before this PR, with full names counted at their carrier's weight: Ruby's acme weighed 468 and Elixir's 392, against about 50 for the next domain word; TypeScript's packages weighed 168. A container's words are spoken again by every name it qualifies, so they drowned the words that name things.

• Core, Lexicon.qualified: the kinds a language names by a path, each with the separator that ends the qualifier. . for Java packages, C#/Clojure namespaces, Python modules, Elixir modules and protocols; :: for Ruby classes, modules and constants; \ for PHP namespaces; / for every file kind. Go's package and Rust's module are one identifier, so neither is listed. validateProfile gains lexicon-not-a-separator (empty, or holding a letter, mark or digit) and reports an unknown kind as lexicon-unknown-kind. codegraph profiles, METAMODEL §5.3, docs/cli.md and both website pages follow.
• The splitter returns the same tokens as before (a property pins it) plus qualifier: how many of them precede the last separator. AcmeOrder::Order::Line → acme, order, order, line with qualifier 3. C#'s interface prefix now reads the own name, so Acme.IOrder drops its I.
• The weights gain a qualifier key, 1 by default.

This field was offered as a commit on #26; it lives here instead so #27 needs no rebase or force-push.

After the change, the corpus's top domain words: Ruby order 147, acme 96; Elixir order 99, acme 49; TypeScript order 79, acme 30, packages 21; Java order 40.

Aggregation

buildVocabulary(graph, {view?, weights?, technical?, name?}), pure over the analyzer's graph:

• Kept is the view's verdict on a non-stub entity. A stub never speaks, whatever the view: String is a word another codebase chose.
• Scopes come from traits: every kept TModule and TType. Each entity counts in the corpus, its containingModule and its containingType (the fold's nearest self-or-ancestor). An entity no module contains counts at the corpus alone and is listed in diagnostics.unplaced.
• Language is the entity's model's, read from the union, never from its id. A word two languages class apart (type is a Go keyword, a Java word) is two terms.
• Each scope has [term, weight, occurrences, carriers] rows by weight then term. An entity saying a word twice is one carrier. It also keeps its entities by role, what it skipped (unnamed, conventional, numeric), distinct and distinctDomain.
• Each term carries its spread: how many module and type scopes speak it, and the one module when exactly one does (exclusive).

The artifact

codegraph.vocabulary/1 records its view, weights, classes, and the technical list's size and sha256, since a count means nothing without them. Terms are sorted by code unit then class; scopes are the corpus, then modules, then types, each in id order; rows address terms and parents by index. vocabularyToJsonString is compact by default; {pretty: true} writes one term and one scope per line.

Tests

• A hand-counted toy corpus (two packages, three types, a nested one, members, an unnamed constructor, a conventional toString, two stubs), plus option, unplaced, no-lexicon (cobol), Ruby word-said-twice and multi-language cases.
• Properties over generated corpora: closure (every index in range, sums equal rows, spread recomputed, every kept entity in one module or unplaced) and the same bytes whatever order the entities arrive in.
• Six mutations of the aggregator, one at a time: each caught.
• Fixture snapshots: fixtures/{java,csharp,typescript,elixir,ruby,unicode}/expected/vocabulary.json, pretty, rebuilt byte for byte, closed, and identical over a fresh graph. CODEGRAPH_UPDATE_SNAPSHOTS=1 rewrites them.
• Hand counts: each fixture README counts one word name by name, and a test pins it, so a snapshot rewritten in one command still has to agree with a count a person made:

Fixture	Word	Weight	Occurrences	Carriers
java	order	40	12	12
csharp	order	42	14	14
csharp	priceable	9	2	2
typescript	legacy	11	4	4
typescript	packages	21	21	21
elixir	pricing	28	9	9
ruby	pricing	8	5	5

The C# fixture had no README; it gains a short one that points to docs/csharp-extractor.md and holds the count.

Worth a look in review: a few classifications follow from the profiles and read oddly in the snapshots. Java's abstract (from AbstractOrder) and Ruby's class (from ClassMethods) tag as keywords, because those profiles reserve them.

Also

• @codegraph/vocabulary now depends on @codegraph/analyzer, as PLAN §20 planned.
• PLAN §20.1 and §20.4 describe what was built. drift.ts moves to the command's item, for PR 4.

Verification

• pnpm -r build and pnpm -r typecheck green; eslint clean on every changed .ts/.mjs/.js; pnpm run gen:schemas leaves schemas/ unchanged; the website profiles page regenerates with no diff.
• vocabulary 243 tests, core 544, cli 908; every other package passes except the Node 26 node:sqlite warning test noted on feat(core): language lexicons on profiles (M19a) #26.

Part of #6.

🤖 Generated with Claude Code

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	codegraph	2b1bdf3	Commit Preview URL Branch Preview URL	Oct 26 2026, 07:04 AM