Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions docs/design/decisions/vocabulary-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ The vocabulary was a set of base sets plus marker subsets (`particles` and `part
8. A patronymic field belongs in 3.0.
9. Gender is recorded only where a word gives a definitive clue.
10. Title-phrase modifiers count as titles only next to a head title (parked, not filed).
11. Name readings are hand-declared, and only for words some language file claims as vocabulary, so nobody maintains surname lists.
11. Name readings are hand-declared, and only for words some language file claims as vocabulary, so nobody maintains surname lists. (Widened 2026-10-19, below: or refuses.)
12. Bearer counts are structured provenance (source + count), never used to compute a reading: two data sets deciding one reading would make the decision unclear. A new census flags changed names for review.
13. `Dean Smith` reads given `Dean`, as today.
14. Order and shape are separate: a language supplies only a default order; the shapes (where titles, suffixes, particles and commas sit) belong to each order and live in the core.
Expand All @@ -37,5 +37,7 @@ The vocabulary was a set of base sets plus marker subsets (`particles` and `part
- **The comments moved with their words.** The suffix words' per-word facts became fields (`note`, `issue`, `same_as`, `expands`, `excluded`); the class-level parts of the vetting notes were checked against the decisions entries that already hold them, and what had no home went to decisions/W2's entry of this date. In the mechanically converted categories the block comments stand as TOML comments above the entry they stood above, in source order, and inline comments became reading `note`s; classifying those categories is the later work that turns them into fields.
- **Measurement (2026-10-18, Python 3.11.17).** `Lexicon.default()` and all thirteen raw constants equal the pre-change tree's, compared against values pickled from master at 6803a261; `tools/differential/snapshot.py` reports "matches the snapshot". Negative control: dropping `may_be_name` from `ma` moves 120 snapshot entries and makes `Lexicon.default()` unequal, and editing the TOML without rebuilding fails the staleness test with the refresh command. `import nameparser` measured 48.5–48.8 ms median against master's 49.4 ms (25 fresh processes each, two runs), so the generated module costs nothing visible; the spike measured parsing the TOML at import at about +18 ms on a 48 ms import, and the TOML grows with descriptive data the compiled module never carries.
- **Declined, with evidence.** PARSING THE TOML AT IMPORT: +18 ms net on every import (spike, 2026-10-18), and it could not be made lazy on 2.x without making the config modules and the shim's `CONSTANTS` lazy too. DERIVING THE MARKERS NOW: see the flags bullet. A `peelable` BOOLEAN: `attached` is the general property from the start, a table so its other side and separators can be added. A `language` KEY in the file: a second source of truth for the file name.
- **Phases 2–4 are rejected by the compiler, not accepted as inert data**: `addresses = "full"`/`"any"`, `attached` on a side other than after, and the names later phases will use (`default`, `bare_only`, `leading_only`, `trailing_only`, `attaches`, `separators`) are build errors naming their phase, so nothing reads as working that does not. Locale-pack vocabulary and the Korean surname list stay in Python: surnames are a segmentation list, outside the per-word model, and packs move in phase 3.
- **Phases 2–4 are rejected by the compiler, not accepted as inert data**: `addresses = "full"`/`"any"`, `attached` on a side other than after, and the names later phases will use (`default`, `prefer`, `strict`, `bare_only`, `leading_only`, `trailing_only`, `attaches`, `separators`) are build errors naming their phase, so nothing reads as working that does not. Locale-pack vocabulary and the Korean surname list stay in Python: surnames are a segmentation list, outside the per-word model, and packs move in phase 3.
- **Recompute the composition** rather than quoting it: `uv run python tools/vocab/query.py --kind generation --count` (readings, one per file a word is in), `--kind suffix`, `--excluded generation`, `--issue 661`.

- **2026-10-19 — decision 11 widened to words a file refuses (Derek).** A name reading may also sit on a word whose vocabulary reading some file refuses: the reading is the evidence the refusal rests on, as `fils`'s `family` reading in fr.toml backs its excluded `generation`. The build already allowed this (its soft warning fires on a name reading only where the word has neither a vocabulary reading nor an exclusion), so the decision now matches the shipped data; #758's review found the mismatch.
Loading
Loading