Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions docs/design/decisions/H6-title-before-a-family-comma.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,9 @@
- The run admits a word by vocabulary only, never by H2's shape, since the slot is a surname's slot too and a surname can open with an abbreviated word: `Xyz. Smith, John` → family 'Xyz. Smith', `Sta. Maria, Juan` → family 'Sta. Maria' — 2026-10-17 #701; 2026-10-18 #710.
- The run leaves the family its last piece (`Dr., John` → family 'Dr.'), stops at a word that is a particle too (`St Pierre, Ed` and `Freiherr von Berg, Ed` keep their families, while `Dr. Freiherr von Berg, Ed` peels 'Dr.'), and stops where nothing but post-nominals stands behind a title (`RA PhD Jr., John` → family 'RA'), so a title/suffix dual with a name word behind it peels (`Rev. Sr. Smith, Mary` → title 'Rev. Sr.') — 2026-10-17 #701.
- A surname-like title opening a two-word surname is the title (`Pope Smith, John` → title 'Pope'), an accepted cost — 2026-10-17 #701.
- `gov` is a title (`Lt. Gov. Smith, John` → title 'Lt. Gov.'), the first piece the run leaves is the family whatever it is (`Dr. PhD Smith, John` → family 'PhD Smith'), and the particle stop is all that keeps a rank in the family, so `Graf Lambsdorff, Otto` reads title 'Graf' — 2026-10-17 #701.
- `gov` is a title (`Lt. Gov. Smith, John` → title 'Lt. Gov.'), the first piece the run leaves is the family whatever it is (`Dr. PhD Smith, John` → family 'PhD Smith'), and the particle stop is all that keeps a rank in the family, so `Herzog Lambsdorff, Otto` reads title 'Herzog' (`Graf Lambsdorff, Otto`, a rank particle since decisions/vocabulary-collisions' 2026-11-04 (Derek), #709 entry, keeps family 'Graf Lambsdorff') — 2026-10-17 #701.
- The comma-less writing of an abbreviated surname is not this rule's; `sta` and `sto` are saint particles, so `Juan Sta. Maria` reads family 'Sta. Maria' and `Juan Sta. Maria Reyes` family 'Sta. Maria Reyes' — 2026-10-18 #710; 2026-11-04 #725.

Open: [#709](https://github.com/derek73/python-nameparser/issues/709) (whether `graf`, `fürst`, `prinz` and the other ranks read as `freiherr` does).

## Log

- 2026-10-17 (Derek), #701 — A TITLE OPENING THE PART BEFORE A FAMILY COMMA IS THE TITLE. `Dr. Smith, John` read family 'Dr. Smith', and every release did the same: 1.4.0 from PyPI gives last 'Dr. Smith' and 'Prof. Brühl' (suffix 'Dr.') for `Prof. Dr. Brühl, Ada Cornelia` (measured 2026-10-17, `PYTHONSAFEPATH=1`, wheel path asserted). The reading was written down on purpose as Accepted clauses of rules/H2 and #H3 ("before a family comma the pre-comma text is wholly the family name") and assumed by C1's Freiherr clause ('Prof. Cruz, Ed', rules/C3's Accepted clause since #728); all three are rewritten here. The listing form moves the surname to the front and leaves the title where it was written, and CLDR's index renderings write exactly that in several locales (`prof. dr. von Selimović Hodžić, Izeta Amra`, `г-н Вернер, Иван Асен`).
Expand Down
11 changes: 9 additions & 2 deletions docs/design/decisions/vocabulary-collisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,9 @@ Recomputed 2026-09-07 with that recipe, after this section's own #342 decision l
- C-ii: where two sets claim a word and neither reading is a name, precedence is a frequency judgement recorded per word; `vd`, a never-given particle and a credential acronym, takes the Dutch van der reading where a family is named (`Berg, Jan vd` → family 'vd Berg') — 2026-08-16 #348/#360/#342/#385.
- `mc`, the other word of both the particle and the unambiguous suffix vocabulary, inherits `vd`'s answer through rules/P6's statement for the shape without being weighed itself (`Berg, Jan mc` → family 'mc Berg'), a cost #454 accepted; `do`, in the ambiguous suffix half, is outside that precedence and joins the family by P6's ordinary attachment (`Berg, Jan do` → family 'do Berg'), except in capitals, where S6 reads it as the credential (`Berg, Jan DO` → suffix 'DO') — 2026-08-16 #348/#360/#342/#385.
- `ó` and `ní` are never-given particles (`Liam Ó Murchú` and `Ó Murchú` → family 'Ó Murchú'); `ua`, `binti` and `binte` are ambiguous, `ua` for want of a thorough search and `binti` and `binte` because Binti is a Swahili given name (`Binti Jua` → given 'Binti', reported) — 2026-10-04 #604.
- Graf, Gräfin, Fürst and Prinz carry a rank reading, as Freiherr does, so a rank written inside the surname joins it (`Otto Graf Lambsdorff` → family 'Graf Lambsdorff'), each staying a title opening a name; Herzog and Ritter, borne as plain surnames far more often, are refused the reading — 2026-11-04 (Derek), #709.

Elsewhere: #454's acceptance of `mc`'s inherited answer is decisions/suffix-acronym-collisions' 2026-09-07 #342/#454 bundle; for P3's count, a word placed to join counts as a connective whatever other vocabulary holds it, decisions/P3's 2026-09-20 #397 entry answering a C-ii question in its THE COUNT REACHES THE CLASS paragraph, scoped to that count; `mc` and `ste` left the ambiguous half of the particles under #360 (decisions/P1); `do` left TITLES in #296's audit, so `Do Quang Minh` reads given 'Do' and reports the fork, and the overlap of TITLES with the ambiguous particles, `freiherr` and `st` since, is kept load-bearing by the particle-line note in decisions/vocabulary-exclusions; `rai` left the suffix acronyms rather than earning the marking, and the relative-frequency criterion that chooses between removal and the marking is decisions/suffix-acronym-collisions'.
Elsewhere: #454's acceptance of `mc`'s inherited answer is decisions/suffix-acronym-collisions' 2026-09-07 #342/#454 bundle; for P3's count, a word placed to join counts as a connective whatever other vocabulary holds it, decisions/P3's 2026-09-20 #397 entry answering a C-ii question in its THE COUNT REACHES THE CLASS paragraph, scoped to that count; `mc` and `ste` left the ambiguous half of the particles under #360 (decisions/P1); `do` left TITLES in #296's audit, so `Do Quang Minh` reads given 'Do' and reports the fork, and the overlap of TITLES with the ambiguous particles, `freiherr` and `st` since and the ranks `graf`, `gräfin`, `fürst` and `prinz` beside them since 2026-11-04 (#709), is kept load-bearing by the particle-line note in decisions/vocabulary-exclusions; `rai` left the suffix acronyms rather than earning the marking, and the relative-frequency criterion that chooses between removal and the marking is decisions/suffix-acronym-collisions'.

Not standing: the C-i correction's call to re-judge `lo` and `el` as candidate particles, which #360 took up, judging `lo` ambiguous and leaving `el` needing judgement; neither was added to the particles — 2026-08-17.

Expand Down Expand Up @@ -59,5 +60,11 @@ Elsewhere: C-i reaches the titles through a `given` reading, which makes a title
- Applications, each still its own work: #360 (mc, ste — neither is a borne given name, so both leave the ambiguous half); #342 (rai — Rai IS a borne surname, so it earns the marking rather than moving); #385 (do — resolved at decisions/R2). (#342's clause — "rai — Rai IS a borne surname, so it earns the marking rather than moving" — is SUPERSEDED 2026-09-07: Rai is borne, and what that earned was removal from the vocabulary rather than the marking. See #suffix-acronym-collisions for the relative-frequency criterion that decides between the two.)
- C-ii's per-word framing versus a rule stated for a SHAPE: (Corrected 2026-10-21: `do` was already in the ambiguous suffix half when this was written, so the precedence decides nothing for it, as rules/P6 says; only `mc` inherits `vd`'s answer, which #454 accepted.) measured 2026-08-16, the words that are both particle and suffix vocabulary are vd, do and mc — three, not the one this criterion adjudicated. rules/P6 states its precedence for the shape, so do and mc inherit vd's answer without being weighed. Recorded rather than papered over: stating a per-word judgement as a general clause is how an unexamined word acquires a decision, and the two are named here so the next reader knows which one was actually argued.
- 2026-10-04 #604 — five particles, judged by C-i one at a time. NEVER-GIVEN: `ó` (Irish "descendant of") and `ní` ("daughter of") — grammatical patronymic particles, no given-name use in the position P1 acts on, and the measured gain is the leading and mid-name surname both (`Liam Ó Murchú` and `Ó Murchú` read family `Ó Murchú`, where today Ó was a middle initial or the given name). AMBIGUOUS: `ua`, the older spelling of `ó` — no given-name use was found, but the search was thinner than for `ó`/`ní`, and C-i says default to ambiguous under uncertainty. Measured 2026-10-04, the two markings agree mid-name (`Seán Ua Buachalla` reads family `Ua Buachalla` either way) and differ in three places. Where `Ua` opens the name: the leading `Ua Buachalla` keeps given `Ua` and reports particle-or-given rather than becoming surname-only, and `Ua Buachalla Seán` under FAMILY_FIRST keeps family `Ua`, given `Buachalla` and reports. Where it opens the part after a family comma (P1's after-comma clause): `Seán, Ua Buachalla` reads given `Ua`, middle `Buachalla`, family `Seán`, where never-given would give family `Seán Ua Buachalla`. And where it ends the name (P6): `Seán, Seán Ua`, and `Seán Seán Ua` under FAMILY_FIRST, report particle-or-given only while `ua` is ambiguous, the fields being the same. Recompute: parse each with `Lexicon.default()` against `Lexicon.default().remove(particles_ambiguous={'ua'})`. The cost is small because `Ua` is the rarer spelling, but it is not leading-only, which matters to anyone re-pricing the marking under #360. `binti` and its Singapore spelling `binte` (Malay "daughter of") — ambiguous like `bin`, Binti being a Swahili given name ("daughter"); `Ina binti Navalamar` reads family `binti Navalamar`, `Binti Jua` keeps given `Binti` and reports. The one-letter `ó` needed a rule of its own against the initial it spells: rules/P7, decisions/P7.
- Caution when applying C-i to the particle set: TITLES ∩ ambiguous — {do, freiherr, st} until #296's audit took 'do' out of TITLES (2026-08-23; `Do Quang Minh` reads given 'Do' with the fork now), {freiherr, st} since — is load-bearing, per the particle-line Excluded note in decisions/vocabulary-exclusions. Emptying it makes the particle-or-given emitter dead code.
- Caution when applying C-i to the particle set: TITLES ∩ ambiguous — {do, freiherr, st} until #296's audit took 'do' out of TITLES (2026-08-23; `Do Quang Minh` reads given 'Do' with the fork now), {freiherr, st} since, and graf, gräfin, fürst and prinz since 2026-11-04 (#709) — is load-bearing, per the particle-line Excluded note in decisions/vocabulary-exclusions. Emptying it makes the particle-or-given emitter dead code.
- 2026-10-30 #965 — C-i READS INITIALS AS IT READS A NAME. `pc` (Privy Counsellor) and `dl` (Deputy Lieutenant) were gated as credential words beside a `family` reading (decisions/S11's entry of this date), though neither is borne as an ordinary name: each is written as a person's initials where its claim acts, ending a name (ORCID's boxes: `PC` given 149, family 112; `DL` 54 and 21; psc-individuals `Dl` 7, `Pc` 1). A wrong unambiguous claim there misparses the person the initials belong to exactly as it would misparse a bearer (`Ravi PC` would lose its family name), so C-i's ground is read to include it. The bar is the one a name meets: a per-word count of the initials in that position, recorded on the word's name reading, not the shape alone, so a two-letter credential is not marked because it could be initials, only where they are measured. rules/S11's When and Because say the same.
- 2026-11-04 (Derek), #709 — GRAF, GRÄFIN, FÜRST AND PRINZ ARE RANK PARTICLES; HERZOG AND RITTER ARE NOT. Since 1919 a German noble rank is part of the legal surname, written before the particle, and `freiherr` was the one rank the vocabulary read so: `Otto Graf Lambsdorff` read middle 'Graf', family 'Lambsdorff', where `Manfred Freiherr von Richthofen` reads family 'Freiherr von Richthofen'. Each of the four now carries freiherr's shape, a `rank` particle reading beside its `nobility` title reading and the particle's `given` reading, so it chains into the family behind a given name and stays a title opening the name (`Graf Zeppelin` → title 'Graf'). C-i decides the `given` reading: none is borne as a given name where a name opens (US SSA prinz 5, the others none; Wikidata prinz 1), so each is marked ambiguous as `freiherr` is, on the conservative default.
- **Measured** (2026-11-04, the tree against `origin/master` `c6207549`, py3.11). Every committed and local corpus name holding one of the four words (738): 66 move, read under the default policy, nearly all a German noble name now holding its rank in the family (`Claus Schenk Graf von Stauffenberg` → family 'Graf von Stauffenberg'; `Ernst August Prinz von Hannover`; `Marion Gräfin Dönhoff`), and a Spanish or Portuguese double surname led by Graf reading both surnames (`Rodrigo Graf Fernandez` → family 'Graf Fernandez'); two fragments of news text read worse (`Enkel Prinz William` → family 'Prinz William'). Fielded corpora, `local_agreement.py --base`: psc-individuals 8 progress, 1 regression; orcid-researchers 9 and 1; patentsview-inventors 5 and 3, the regressions mostly Austrian and Swiss filers who typed the word as a given name. The committed snapshot moves one name, `Graf Lambsdorff, Otto` → family 'Graf Lambsdorff', which rules/H6's particle stop now keeps. Recompute: `config_diff.py diff default "expr:Parser()" --committed --local all --match '(?i)(^|[\s,])(graf|gr[äa]fin|f[üu]rst|prinz)([\s,.]|$)' --readings <file>` in each tree, comparing the role fields line by line.
- **Accepted:** a family comma's trailing particle attachment (rules/P6) takes one of the four ending the given part into the family, as it does `freiherr`: `Smith, Steffi Graf` → family 'Graf Smith', where it read middle 'Graf', and with its period `Smith, Steffi Graf.` → family 'Graf. Smith', where it read title 'Graf.', reporting particle-or-given. The one corpus name that moved that way reads better for it, the rank written after the given name as German catalogs sort it (`Hardenberg, Wilko Graf von` → family 'Graf von Hardenberg', where it read middle 'Graf').
- **Refused: `herzog` and `ritter`.** Both are common plain surnames (US Census 11,146 and 33,252; Werner Herzog, John Ritter), where the attachment above takes a trailing one after a family comma into the family: given the rank and given readings the other four carry, 63 of the 879 corpus names holding either word move, the word changing role in 13 holding `herzog` and 43 holding `ritter`. Five are that attachment, a middle name taken into the family (`BABINGTON, BETHANY HERZOG` → family 'HERZOG BABINGTON', `KING, SONDRA RITTER` → family 'RITTER KING'); five a word read as a title before a family comma kept in the family (`HERZOG DIAL, MICHELE` → family 'HERZOG DIAL'); and 53 a word inside the name joining the word after it, nearly all Brazilian double surnames in tse-donors (`ANA PAULA RITTER ASCENCO` → family 'RITTER ASCENCO', where it read middle 'PAULA RITTER'). Each carries `excluded = { rank = … }`; `Herzog Lambsdorff, Otto` still reads title 'Herzog' (rules/H6). Recompute: `config_diff.py diff default "expr:Parser(lexicon=Lexicon.default().add('herzog', kind='rank').add('herzog', kind='given').add('ritter', kind='rank').add('ritter', kind='given'))" --committed --local all --match '(?i)(^|[\s,])(herzog|ritter)([\s,.]|$)'` in the tree (2026-11-04). The five-word prototype's figures this entry first gave (13 and 49 names, `dos Santos Ritter` cut to family 'Ritter') do not reproduce under that recipe and are withdrawn; the refusal was confirmed on these figures, the 53 joins reading a double surname right only where its first word is one of the two, as `ANA PAULA SILVA ASCENCO` keeps middle 'PAULA SILVA'.
- **Not added: `fürstin` and `prinzessin`**, in no vocabulary file; the feminines join with their own vetting.
- The Austrian question (#709's fourth): Austria abolished noble names in 1919 rather than folding them into the surname, so an Austrian rank is no legal part of the name. Reading it differently would be language detection; it reads as the German one does.
2 changes: 1 addition & 1 deletion docs/design/decisions/vocabulary-exclusions.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Excluded (the never-given / ambiguous particle line, a particle's `given` readin
- The audit's scope, as the module comment above the ambiguous particles recorded it until #749 turned the comments into fields: #360 examined "the 12 that appear leading in tools/differential's corpora plus a few more", and left the rest bare rather than record an absence of knowledge as a judgment. Examined is wider than the 9 justified above.
- The per-word reasons are data since #749 (2026-10-19, decisions/vocabulary-model): each member's `note` on its particle reading, in the file of each language whose names it occurs in, and each word kept out (`ben`, `nic`, unaccented `o`, `bt`, `bte`, and the native-script `ال` and `בר`) an `excluded` entry with its reason. `uv run python tools/vocab/query.py --kind particle` lists the members (each word once per file it is in), `--issue 360 --kind particle` the five never-given words #360 added or moved (das, las, los, mc, ste), `--issue 604 --kind particle` #604's five (the never-given ó and ní beside the three above), and `--excluded particle` the refusals; the nine ambiguous members #360 justified carry a `note` but no `issue`, so no query isolates them; a contributor who re-adds a refused word meets the recorded reason as a build error.
- Encoding rationale (#293, predating #360's membership questions): the data layer stores the NEVER-GIVEN set and derives the ambiguous one, because that is safe-by-default for new particles — a one-place addition — and the v1 shim translates by one-directional complement. And the constants are FROZEN specifically to kill the cached-Lexicon.default()-vs-fresh- Constants desync that runtime module-constant mutation caused. The data-layer half is superseded since 2026-10-18 (decisions/vocabulary-model): from then every particle reading stated `may_be_given`, true or false, so neither side was a default, and since 2026-10-25 a particle may be given exactly where its word has a `given` reading, every other particle reading stating `may_be_given = false`, which still leaves neither side a default; the freeze half stands.
- Load-bearing dependency: TITLES ∩ ambiguous — {do, freiherr, st} until #296's audit, {freiherr, st} since — is what keeps the particle-or-given ambiguity emitter reachable at all; moving them all would leave the shipped vocabulary unable to reach it, which is why test_the_chained_emitter_is_reachable_by_construction (tests/v2/test_parser.py) pins reachability by construction rather than by shipped vocabulary — an empty intersection fails no test and does not mean the emitter is unreachable.
- Load-bearing dependency: TITLES ∩ ambiguous — {do, freiherr, st} until #296's audit, {freiherr, st} since, and the ranks graf, gräfin, fürst and prinz beside them since 2026-11-04 (#709) — is what keeps the particle-or-given ambiguity emitter reachable at all; moving them all would leave the shipped vocabulary unable to reach it, which is why test_the_chained_emitter_is_reachable_by_construction (tests/v2/test_parser.py) pins reachability by construction rather than by shipped vocabulary — an empty intersection fails no test and does not mean the emitter is unreachable.

Open (contested vocabulary memberships — the rule is right, the word's set is questioned; the issue is canonical): none. The block held three entries and all three were answered inside a fortnight — #346, #343 and #344 together on 2026-09-06 (see #indic-honorifics) and #342 on 2026-09-07 (see #suffix-acronym-collisions). The heading stays with nothing under it on purpose, the same reason an empty ledger roster section is a statement and a missing one is nobody having looked. sa, se and om (all three since decided, sa on 2026-10-25, #857) are named as the next candidates by #342's own comment and carry no issue, so they are not entries here.

Expand Down
Loading
Loading