Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,12 @@
- `initials()` reads the base, so the anchoring moves R3's output: `Anh Do` initials 'A. D.' and `Juan van der` 'J. v. d.' — 2026-08-16 (pre-merge coherence pass); 2026-08-18 #404/#385.
- A particle run that joins nothing still chains whole into one part, so `Juan Smith van der` reads family 'van der' and middle 'Smith'; splitting it was declined, roles reassembling adjacent same-role pieces into one part and the base losing its full extent — 2026-08-18.
- The base is the family without its working particles, wherever in the family they stand (`da Silva do Amaral de Souza, Miguel` → family_base 'Silva Amaral Souza', family_particles 'da do de'), a view the same under every convention and no culture's sort key — 2026-10-28 #754.
- A connective that is not also particle vocabulary, standing between two words of a part, is set aside when the part is asked whether every word is particle vocabulary, so `Fritz Freiherr und von Bar` is all particles, base 'Freiherr und von Bar' and initials 'F. F. v. B.', and `FRITZ FREIHERR UND VON BAR` repairs to 'Fritz Freiherr und Von Bar', where the parser reads base 'und', a tracked deviation; a connective ending the part is a name word, so `Juan de y` keeps base 'y' — 2026-10-28 #754 (Derek), A CONNECTIVE AMONG PARTICLES.
- The unjoined mark is a statement about a whole part, and `initials()` honors it for every word in it, a connective included: under `Lexicon.default().add(particles={'y'})`, `Juan de y` reads family_base 'de y' and initials 'J. d. y.'; narrowing the readmission to the `particle` tag alone was tried and backed out. Such a word is listed as a particle too, so the line above's set-aside does not reach it, and it keeps this reading wherever it stands in the part (`Juan de y la` → 'J. d. y. l.') — 2026-08-29 #461; 2026-09-20 #461; 2026-10-28 #754 (Derek), A CONNECTIVE AMONG PARTICLES.
- A connective that is not also particle vocabulary, standing between two words of a part, is set aside when the part is asked whether every word is particle vocabulary, so `Fritz Freiherr und von Bar` is all particles, base 'Freiherr und von Bar' and initials 'F. F. v. B.', and `FRITZ FREIHERR UND VON BAR` repairs to 'Fritz Freiherr und Von Bar'; a connective ending the part is a name word, so `Juan de y` keeps base 'y' — 2026-10-28 #754 (Derek), A CONNECTIVE AMONG PARTICLES; 2026-11-03 #939.
- The unjoined mark is a statement about a whole part, and `initials()` honors it for every word in it, a connective included: under `Lexicon.default().add('y', kind='particle')`, `Juan de y` reads family_base 'de y' and initials 'J. d. y.'; narrowing the readmission to the `particle` tag alone was tried and backed out. Such a word is listed as a particle too, so the line above's set-aside does not reach it, and it keeps this reading wherever it stands in the part (`Juan de y la` → 'J. d. y. l.') — 2026-08-29 #461; 2026-09-20 #461; 2026-10-28 #754 (Derek), A CONNECTIVE AMONG PARTICLES.

Elsewhere: where a connective initials, a part holding nothing else for it to join (`Jon Dough and` → 'J. D.', `Juan de y` → 'J. y.') — decisions/R3's 2026-09-20 #461 entry; case repair keeping that connective lowercase, its carve-out not gated on the mark (under `Lexicon.default().add(particles={'y'})`, `Juan de y` forced → family 'De y') — decisions/R4's 2026-08-29, the CONJUNCTION CARVE-OUT entry; case repair reading the unjoined mark for the whole part (`ANH DO` → 'Anh Do') — decisions/R4's 2026-08-29 #407 entry; the snapshot recording `initials()`, so a change to the base is visible there — decisions/differential-ledger's 2026-10-13 #664, a committed snapshot replaces the classified gate entry.
Elsewhere: where a connective initials, a part holding nothing else for it to join (`Jon Dough and` → 'J. D.', `Juan de y` → 'J. y.') — decisions/R3's 2026-09-20 #461 entry; case repair keeping that connective lowercase, its carve-out not gated on the mark (under `Lexicon.default().add('y', kind='particle')`, `Juan de y` forced → family 'De y') — decisions/R4's 2026-08-29, the CONJUNCTION CARVE-OUT entry; case repair reading the unjoined mark for the whole part (`ANH DO` → 'Anh Do') — decisions/R4's 2026-08-29 #407 entry; the snapshot recording `initials()`, so a change to the base is visible there — decisions/differential-ledger's 2026-10-13 #664, a committed snapshot replaces the classified gate entry.

Open: #754 (whether a language pack supplies a convention's sort key or entry element from Phase 4's per-surname units, the RFC's Phase 4 Left open); #939 (a connective between particles reads as the base, the deviation rules/R2, rules/R3 and rules/R4 mark).
Open: #754 (whether a language pack supplies a convention's sort key or entry element from Phase 4's per-surname units, the RFC's Phase 4 Left open).

## Log

Expand Down Expand Up @@ -47,3 +47,4 @@ Declined:

- 2026-08-18 — the GROUPING half of #404: a particle run that joins nothing does not chain, so "Jong van der" would split into middle 'van' plus family 'der'. Measured and rejected, though NOT for the reason first recorded here. The first draft said the split makes the family "no longer all-particle so the base fix stops firing" — false, and `der` and `la` are both shipped particles, so a family of either IS all-particle and the rule fires on it ("Juan Smith der" gives base 'der'). What the split actually costs is the SCOPE of the base and a stray particle relocated: grouping can decline to merge but cannot keep the words apart, because roles re-assemble them and two adjacent same-role pieces are one part at the field level. "Juan Smith van der" becomes middle 'Smith van', family 'der' — a base of 'der' rather than 'van der', and a middle name nobody wrote. Keeping the run whole in one part is what gives the base its full extent. The split reading needs the leftover distribution to know these are separate units, which is mechanisms.md#UNIT-PARTITION's problem.

- 2026-11-03 #939 — THE CONNECTIVE SET-ASIDE SHIPS. The 2026-10-28 #754 (Derek), A CONNECTIVE AMONG PARTICLES entry's decision is implemented and the deviation markers on rules/R2, rules/R3 and rules/R4 are removed: `Fritz Freiherr und von Bar` reads family_base 'Freiherr und von Bar', family_particles '', initials 'F. F. v. B.', where 2.8.0 read 'und', 'Freiherr von Bar' and 'F. u.'. The set-aside connective is left without either unjoined mark, so the base keeps it (it carries no `particle` tag), `initials()` skips it as a joining connective and case repair keeps it lowercase, while the particles it joins take UNJOINED_TAG. 'Between' is the part's own first and last word excluded, in the pipeline's walk and in `_types._remarked`, its twin that `Parser.revise` runs, each pinned by a test in tests/v2/test_render.py (the revise one fails with the twin reverted). The part is any of the three, not only the family: `Bar, Fritz Freiherr und von` initials 'F. F. v. B.' through its middle, and under a family-first order the given part `Freiherr und von Bar` of `Fritz Freiherr und von Bar` initials 'F. v. B. F.', the given group's connective joining. The words' `Token.reading` moves with the mark, which the snapshot does not record: `Freiherr`, `von` and `Bar` read a name the vocabulary declares, where they read a particle, and `und` reads a conjunction, where it read a name by position (tests/v2/pipeline/test_assemble.py's rows). The snapshot moved the two committed spellings of the name, its initials and both case views, and nothing else (2026-11-03, on 39839b91). Not reached, because the part holds a name word: a connective joining two working particles before it stays in the base while they leave it, `Alois von und zu Liechtenstein` reading family_base 'und Liechtenstein', family_particles 'von zu', initials 'A. L.', as on 39839b91 and in 2.8.0 -- R2's Then, the family without its particles, keeps a connective, which is no particle; whether it should leave the base with the particles it joins is not decided here.
1 change: 1 addition & 0 deletions docs/design/decisions/R3-initials-honor-the-fold.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,3 +54,4 @@ R3's earlier history is under `decisions/R2`, which this entry does not repeat:
- 2026-10-28 #357 — THE FACADE'S INITIALS VIEW GOES WITH THE FACADE (decisions/3-0-reevaluations' 2026-10-28 #357, REMOVAL entry), so the parsed name's `initials()` is the one view of this rule. What the 2026-09-01 #462 and 2026-09-13 #528 entries decided for `HumanName.initials()` — its tag reading and vocabulary fallback, the private-`Constants` and pickle, copy and keyword-constructor costs, the `_process_initial` hook break, the `太` fallback and the `Ph. D., John` grouping difference ('J. P D.' there, 'J. P. D.' here) — has no view left to bind, and rules/R3 drops its facade bullet. The facade's fallback was the one place a spliced field initialed by the vocabulary: measured 2026-10-28, `john smith` with middle `e` spliced in gave 'j. s.' on the facade, and `parse('john smith').replace(middle='e').initials()` gives 'j. e. s.', the core view initialing every word of a spliced field (rules/R3's Accepted bullet opening "This rule reads a part the parser read"). The snapshot's three `facade` rows were those `Ph. D.` initials. Open #464, a `Parser.initials` crossing, stands as the route to answering a spliced field under the parse's vocabulary. SUPERSEDES: the 2026-09-01 #462 and 2026-09-13 #528 entries, which decided the facade's view, and the 2026-09-20 #461 entry's facade half, the facade reading the mark and its no-parse paths keeping the older reading.
- 2026-10-28 #939 — A CONNECTIVE BETWEEN PARTICLES: rules/R3's all-particles Except sets aside a connective that is not also particle vocabulary, standing between the part's words, as rules/R2 does, so `Fritz Freiherr und von Bar` should initial 'F. F. v. B.' where it gives 'F. u.', a deviation #939 tracks; decided and measured in decisions/R2's 2026-10-28 #754 (Derek), A CONNECTIVE AMONG PARTICLES entry.
- 2026-10-29 #754 — THE #585 KANA EXAMPLE LEAVES rules/R3. `マイケル シ\u3099ャクソン` → 'マ. シ\u3099.' (the 2026-10-02 #585 entry) gives '' since rules/R7 (a word in a script that writes no initials gives none, decisions/R7), and hangul, the other script whose syllables are written as letters, writes no initials either. The composed-spelling bullet keeps its separate-parts clause, which a Tamil vowel letter written with its length mark still reaches (`\u0b92\u0bd7வையார் கண்ணன்` → '\u0b92\u0bd7. க.', measured 2026-10-29), and takes that and a Latin letter with two accents (`E\u0302\u0301 Nguyen` → 'E\u0302\u0301. N.') as its examples.
- 2026-11-03 #939 — the connective set-aside ships and the deviation marker on rules/R3 is removed: `Fritz Freiherr und von Bar` gives initials 'F. F. v. B.'; decisions/R2's 2026-11-03 #939 entry.
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
- Case repair consults the unjoined mark for the whole part: a part whose every word is particle vocabulary repairs as ordinary name words, as R2 reads it (`ANH DO` → 'Anh Do', `anh van do` → 'Anh Van Do') — 2026-08-29 #407.
- That is a deliberate deviation from 1.4.0, which lowercased on vocabulary alone, and its cost is accepted: a degenerate all-particle family capitalizes too (`juan van der` → 'Juan Van Der', 1.4.0 'Juan van der') — 2026-08-29, this is a DELIBERATE DEVIATION; 2026-08-29, the ACCEPTED COST.
- `capitalized()` is a compared surface: the snapshot records both case views, `capitalized()` where it changes the parsed text and `capitalized(force=True)` where it changes what `capitalized()` gave, so a change that moves case lists every corpus name it moved; the rules/R4 and rules/R5 example lines and the unit and property tests still pin the behavior itself — 2026-10-22 #826.
- The conjunction carve-out is not gated on the unjoined mark: a connective inside an all-particle part keeps its lowercase (under `Lexicon.default().add(particles={'y'})`, `anh y van` → 'Anh y Van'), a case no default-vocabulary example can witness — 2026-08-29, the CONJUNCTION CARVE-OUT.
- The conjunction carve-out is not gated on the unjoined mark: a connective inside an all-particle part keeps its lowercase (under `Lexicon.default().add('y', kind='particle')`, `anh y van` → 'Anh y Van'), a case no default-vocabulary example can witness — 2026-08-29, the CONJUNCTION CARVE-OUT.
- A field spliced in with `replace()` carries no reading, so the all-particle clause does not reach it (`replace(family='de la')` → 'de la' forced, where a parse gives 'De La'); `Parser.revise()` is the supported crossing and repairs as the parse does. That is a documented boundary, not a defect — 2026-08-29, the REPLACE/REVISE BOUNDARY.
- Case repair's conjunction test reads the parse's `conjunction` tag rather than re-deriving the word's class (`juan e-f smith` → 'Juan E-F Smith'); the particle test still keys on lexicon membership, by decision, so `replace()` and a parse part there — 2026-08-29 #458.
- A token carrying `UNCLASSIFIED_TAG` (stamped by `replace()` only) falls back to the vocabulary for the per-word conjunction-or-initial question and never for the per-part particle question: `replace(family='velasquez y garcia')` → 'Velasquez y Garcia' — 2026-08-29 (#458 review).
Expand Down Expand Up @@ -115,3 +115,4 @@ Accepted costs, deferred to the rescoped #459 rather than relitigated here: the
- 2026-10-28 #357 — THE SHIM'S HALVES OF THIS RULE'S ENTRIES GO WITH THE SHIM (decisions/3-0-reevaluations' 2026-10-28 #357, REMOVAL entry): the v1 pickle load as a second producer of `UNCLASSIFIED_TAG`, leaving `replace()` the one; the first-parse raise for a v1 `Constants` carrying a mismatched mask (2026-09-23 #459, A DECIDED EXCEPTION), the core `Lexicon` raising on it as before; and the filter of v1 `Constants` entries with its folded cross-set checks (2026-10-02 #541, 2026-10-02 #582). A punctuated mask such as 1.4.0's `M.D.` stays legal in a `Lexicon`. SUPERSEDES: the 2026-09-23 #459, A DECIDED EXCEPTION, 2026-10-02 #541 and 2026-10-02 #582 entries, all about translating a v1 `Constants`, and the 2026-08-29 (#458 review) entry's second producer, the v1 pickle load.
- 2026-10-28 #939 — A CONNECTIVE BETWEEN PARTICLES: rules/R4's all-particles Except sets aside a connective that is not also particle vocabulary, standing between the part's words, as rules/R2 does, so `FRITZ FREIHERR UND VON BAR` should repair to 'Fritz Freiherr und Von Bar' where it gives 'Fritz freiherr und von bar', a deviation #939 tracks; decided and measured in decisions/R2's 2026-10-28 #754 (Derek), A CONNECTIVE AMONG PARTICLES entry.
- 2026-11-02 #1014 (Derek), DECIDED — A NAME OPENING MAC TAKES ITS INNER CAPITAL FROM A MASK, NOT FROM ITS SPELLING. The fall-through had capitalized after any `ma?c` prefix with two or more letters behind it since 0.2.3, so a single-case `MACHADO` repaired to 'MacHado', `MACIAS` to 'MacIas', `MACIEJEWSKI` to 'MacIejewski'; #56 asked in 2015 for an exceptions list beside the regex, and only the length check shipped. Measured over the mixed-case writings of every word the regex matched, each counted as its writer typed it, in the ORCID family-name box and the names of the UK Electoral Commission donors, GOV.UK people and ClinicalTrials.gov officials local corpora (2026-11-02): 4,616 writings of 970 words, the old convention matching the writer's spelling in 16.3% and plain title case in 83.7%; over the same four sources the Mc convention matched 96.8% of 6,312 Mc writings (94.3% of ORCID's 2,764), the rest mostly names Mc opens that are not Gaelic, `Mchunu` 24 the commonest, then `Mchedlishvili` and `Mchome` 5 each. The Gaelic Mac names split by family, so no shape decides them: `MacDonald` 143 against `Macdonald` 77, `Mackenzie` 59 against `MacKenzie` 44, `Mackay` 73 against `MacKay` 26, `MacLeod` 39 against `Macleod` 18. The mask records the spelling where its bearers lean one way: a word written MacX in more than half of at least three writings carries one, 42 words, which brings the measured match to 91.2% (in-sample, since the masks were chosen from the writings they are scored on; at least two writings gave 60 masks and 92.0%, at least five 25 and 90.2%). Recipe: for each source, read the field named (`family` for ORCID's `raw.jsonl`, `name` for the other three) and take every match of `(?<![^\W\d_])(Mac[^\W\d_]{2,})(?![^\W\d_])` whose letters after `Mac` are not all capitals; key it by its lowercase and count it MacX where its fourth letter is a capital; the comparators are the old rule (every word MacX), plain title case (none), and title case with a mask on each word that is MacX in more than half of at least three writings; the Mc figures are the same with `Mc` for `Mac` and the third letter. The masks stand on `family` readings, the first `cased` a name reading carries: an entry holding only such a reading compiles, so the build does not warn it inert. They are in en.toml, the anglicized spellings being English ones (the Gaelic writes `MacDhòmhnaill`, the Irish `Mac Cárthaigh`), and again in gd.toml, 37 of them, and ga.toml, the Irish `maccarthy`, `macmahon`, `macmanus` and `macdermott`, so a strict choice of a Gaelic pack, whose bearers are likeliest to be written so, keeps them: `parser_for(locales.GD, strict=True)` repairs `RONALD MACDONALD` to 'Ronald MacDonald' (decided, Derek, 2026-11-02). `mackrill`, an English name, is in en.toml alone. Every plain choice keeps all 42, no file holding another reading of them; a strict choice holding none of the three files keeps none, and a strict GA choice keeps only the Irish four (`RONALD MACDONALD` → 'Ronald Macdonald'). A chunk of a word that a full stop divides takes its own mask, as a hyphen part does, so the masks reach `J.MACDONALD SMITH` → 'J.MacDonald Smith', which the old convention gave and #838's parity between a chunk and a hyphen part kept; credential chunks take theirs too (`Phd.Md` forced → 'PhD.Md', `Msc.Ed.` forced → 'MSc.Ed.'): of the readings of ClinicalTrials.gov's officials that move, 125 forced and 1 default are of names holding no `mac` in any case, all credential chunks, against 611 forced and 8 default of names holding one (`snapshot.py --local ctgov-officials --base` the parent commit, 2026-11-02). A caller's lower-case mask on a one-letter or particle word reaches a chunk too (`{'y': 'y'}` gives `J.Y. SMITH` → 'J.y. Smith', where it gave 'J.Y. Smith'), as it reaches the spaced spelling ('J. y. Smith'); no shipped mask does. The cost is a Gaelic name written MacX that no mask records, `SHIRLEY MACLAINE` → 'Shirley Maclaine', and a family that writes Macdonald, which a forced repair recases to the mask. A caller's own spelling goes in `Lexicon.capitalization_exceptions`.
- 2026-11-03 #939 — the connective set-aside ships and the deviation marker on rules/R4 is removed: `FRITZ FREIHERR UND VON BAR` repairs to 'Fritz Freiherr und Von Bar'; decisions/R2's 2026-11-03 #939 entry.
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
```
- **A connective that is not also particle vocabulary, standing between two words of the part, is set aside when the part is asked whether every word is particle vocabulary** (P3): it joins the words around it and anchors no base on its own, so it stays in the base view with them but does not stop the part from being all particles, and particles joined by a connective anchor the base. A word listed as both a connective and a particle is asked the question as a particle, wherever it stands. A connective ending the part joins nothing and is a name word (R3), so the particle before it still works as one.
```
"Fritz Freiherr und von Bar" → family_base="Freiherr und von Bar" deviates: #939 (today: family_base="und")
"Fritz Freiherr und von Bar" → family_base="Freiherr und von Bar"
"Juan de y" → family_base="y" · boundary
```

Expand Down
Loading
Loading