Skip to content

Migrate the H rules to the skeleton format; split out H7 (#761, part 5) - #779

Merged
derek73 merged 12 commits into
masterfrom
docs/761-skeleton-h
Oct 20, 2026
Merged

derek73 merged 12 commits into
masterfrom
docs/761-skeleton-h

Conversation

@derek73

@derek73 derek73 commented Oct 20, 2026 •

Copy link
Copy Markdown
Owner

Part of #761.

This PR moves H1–H6 (the titles and honorifics section) from the block format to the skeleton format, one commit per rule. Later commits fix review findings and carry out Derek's two decisions (below): H2 states that C13 wins, and H4's join clause becomes H7. No parse changes, and the migration commits change no rule's meaning:

  • Every example line is kept verbatim. Each rule's example lines match master as a multiset, and H2's two NFD-typed lines were copied byte for byte from git show.
  • The migration commits move neither the corpus nor the snapshot. The decision commits only ADD names, for their new examples.
  • All 18 code-comment quotes of H1–H6 still match verbatim, so no code comment needed retargeting.

tests/v2/rules_budget.json is re-recorded. Each H row changes format, showing as its old row beside its new one. The word counts grow because bullet leads were added; no new clause reads a new signal.

Where the old clauses went

H1

H2

H3

  • Then: the chain rule.
  • Then bullet: the floor (giving the last word back).
  • Accepted: the three Accepted clauses, unchanged.

H4

  • Then: the two halves. A single title word is a Then bullet, as is the old "ONE site … declared order" clause (a reports clause).
  • Except: "a title in front takes the input out of the suffix half" (MD DDS).
  • Precedence: H2 over a dotted lone credential (Jr.).
  • Accepted: the CJK-honorific scope and the join reach.

H5

  • Then bullets: transparency, input order, a title among post-nominals, and what reports. These were Accepted clauses that state results.
  • Precedence: a particle chain before a trailing title, the family-comma particle chain, a maiden clause, and the bound given-name word (P5). These were the two long Accepted clauses, split.
  • Accepted: the ordinary-surname reach (Mary Jane King.).
  • "The chain" named explicitly: "the title chain" where the old paragraph meant H5's own chain. Where it meant the given part's particle chain (the rules/H5 comment in _group.py), it now says "particle chain".

H6

  • Then: the run and its order-independence.
  • Then bullet: vocabulary only, never H2's shape (Sta. Maria).
  • Except: the particle stop, the post-nominal stop (H3), and leaving the family a word.
  • Accepted: ordinary-surname titles, and the German ranks. "The other German ranks" became "German ranks other than 'Freiherr'", since the sentence lost its referent when the particle-stop sentences moved out. Titles ∩ particles is exactly {freiherr, st}, so the meaning is the same.

Old interacts: IDs

H1 (old: C3 C8 O5 H3 H5 H6 P2 P3 P5 M2 S1 S2 N1 N3)

  • Kept: C3, O5, H5, H6.
  • Added: P1. The old text already named "P1's fold".
  • Dropped as noise:
    • S1, S2, N1, N3, M2: the clause "whatever suffix, nickname or maiden name stands beside it" names these by meaning.
    • P2, P3, P5: what counts as one name word. The examples van der Berg, Sir and Sheik Abu Bakar carry it.
    • H3: the run is "a run of titles".
    • C8: the comma bullet states the no-given-part case in words.

H2 (old: C5 C6 C8 S3 P4 H5 H6 W3 W4 S9 T5)

  • Kept: all except P4, H5 and W4.
  • Dropped:
    • P4: no H2 clause reads the particle-opening position.
    • H5: the front/back contrast lives in the H Background.
    • W4: the CJK veto is a script-table question, and its reading is W3's, which stays named.

H3 (old: C1 H1 H4 H5 H6 S2)

  • Kept: H1, H4, H6.
  • Dropped:
    • C1: "before a family comma" says it.
    • H5: no H3 clause reads the trailing run.
    • S2: "suffix vocabulary"/"post-nominal" say it.

H4 (old: H1 H2 H3 H5 S2 O5)

  • Kept: H1, H2, H3.
  • Added: W2, P3, P4. The old text already named them.
  • Dropped:
    • H5: H5 points at H4's report, not the reverse.
    • S2: post-nominal vocabulary is named in words.
    • O5: the order-independence bullet says it in words.

H5 (old: H1 H2 H3 H4 M2 P2 P5 P6 S2 C1)

  • Kept: all except H3 and C1.
  • Added: P3. The old text already named it.
  • Dropped:
    • H3: "any leading run" says it.
    • C1: "after a family comma" says it.

H6 (old: C1 C3 H1 H2 H3 M2 P1 P2 P4 S2 W3)

  • Kept: C3, H1, H2, H3.
  • Dropped:
    • C1: "family comma".
    • P1, P2, P4: the particle stop says these in words, with examples Dr. de la Vega, Juan and Freiherr von Berg, Ed.
    • S2: the post-nominal stop goes through H3.
    • M2: no H6 clause reads a maiden name.
    • W3: no H6 clause reads CJK.

Decided (Derek, 2026-10-20): the two gaps this migration found

His words: "Yes to both: C13 wins over H2 for a listed suffix word (Precedence bullet on H2, no behavior change), and H4's join clause moves to its own new H rule. Do it in PR #779."

  1. C13 wins over H2 (cf93669). H2 gains a Precedence bullet, and its Then names the exception. New examples: Smith, Esq. John → suffix "Esq. John", Smith, Insp. John → title "Insp.", Smith, Ma. John → title "Ma.". decisions/H2 has the dated entry.
    • Scope confirmed by Derek 2026-10-20: credentials that start S8's run, no behavior change. "A listed suffix word" and "no behavior change" cannot both hold. I measured every alphabetic suffix word of two or more letters in Smith, <Word>. John:
      • 666 read as the postnominal part.
      • 22 still read as a title by H2's shape: the 12 title/suffix duals (Cpl., Ms., Sr.…), 7 ambiguous acronyms (Ma., Do.…), and Abd., Mc. and Vd..
      • S8 bars all 22 from starting its run.
    • To keep the parser unchanged, I scoped the bullet to C13's own words, "a suffix word that starts S8's run", and pinned Smith, Ma. John beside it. Extending the win to all 22 would be a behavior change, and is not proposed.
  2. H7 — a joined name unit carrying title vocabulary (4be2b4a), split out of H4's join clause.
    • Text: verbatim save two edits, which decisions/H7 records: "as well" dropped, and the corpus count dated ("reached on 2026-09-08").
    • New: a When, a Because, and a bullet pointing to H4 for a unit of one word. H4 gains a Then bullet pointing to H7.
    • Pointers: O5's pointer and its interacts: name H7. _assign.py's emitter comment quotes H7, so H7's implemented: holds. Two cases.py notes now name H7.
    • Examples: the old clause had none (its inputs were pinned in cases.py), so H7's are new. Prince of Wales is the boundary.

Corpus and snapshot: corpus_rules.jsonl and snapshot.jsonl gain the new examples' names (3 for H2, and 6 more for H7, with Attorney General of Minnesota already in another corpus file). No existing reading moves (snapshot.py --record added names only).

Review fixes

  • a21523a (second review round):
    • H1's Except holds only the rows where the family is kept, and the ambiguous/bare sentence returns to its Accepted bullet.
    • H6's Sta. Maria row (outside When) moves to the main fence, and the German-ranks clause now reads "German ranks the title vocabulary lists". The old wording overclaimed: Markgraf, Landgraf and Freiin are unlisted and do not peel.
    • H2 now says "single-letter initial", so its T5 pointer reads in T5's terms.
    • H5's bound-word lead no longer contradicts its body.
  • a839d5a: O5's interacts: gains H7, and decisions/H7's Elsewhere: line is set apart from the bullet above it.

Still for Derek: found in review, already on master, not fixed here

  • H1 cannot explain the row St Sir de Mesnil (family-first) → given "de Mesnil". H1's Except says a never-given particle with a word behind it keeps the family (P1's fold), and de is one. The Accepted sentence the row stands under covers only an ambiguous or bare particle. The real reason is that a title-particle (St) opening the run stops P1's fold; decisions/H1 records it and no rule states it. Master's block text had the same mismatch.
  • Smith, Dr King Jr reads title "Dr King", given "Smith", suffix "Jr". H3's give-back does not fire, because H1's comma-with-only-titles reading takes the input first. Whether H3's "part carrying the given name" should cover this shape is open, and I have not verified that it is a defect.

Verification

  • uv run pytest: 22,154 passed, 123 skipped, 3 xfailed (at 4be2b4a; doc tests rerun at a839d5a).
  • uv run mypy and uv run ruff check: clean.
  • The design-docs-reviewer agent ran on the migration, on db4a4f7 and on the decision commits; code-reviewer and comment-analyzer reviewed the PR. Every migration finding is fixed above.

🤖 Generated with Claude Code

derek73 and others added 7 commits October 20, 2026 01:19
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73 derek73 added docs Documentation fixes and updates design labels Oct 20, 2026
@derek73 derek73 self-assigned this Oct 20, 2026
H1: the Except holds only the rows where a particle surname keeps the
family; the ambiguous-or-bare sentence and its given-name rows go back
to the listing-record Accepted bullet. H6: the outside-When boundary
row moves to the main fence, and the German-ranks clause says it means
the ranks the title vocabulary lists. H2: 'single-letter initial', so
the T5 pointer reads in T5's terms. H5: the bound-word lead no longer
points the other way from its body.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.05%. Comparing base (fb0b5a2) to head (37f3a27).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #779   +/-   ##
=======================================
  Coverage   99.05%   99.05%           
=======================================
  Files          47       47           
  Lines        4642     4642           
=======================================
  Hits         4598     4598           
  Misses         44       44           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

derek73 and others added 3 commits October 20, 2026 01:35
Derek's decision on the gap the migration found: 'Smith, Esq. John'
reads suffix 'Esq. John', as C13 says and as the parser already does.
H2 gains the Precedence bullet, its Then names the exception, and
decisions/H2 records it with the measured scope: of the suffix words,
the 22 that start no S8 run (duals, ambiguous acronyms, abd/mc/vd)
still read as a title there. Three new examples, so corpus_rules.jsonl
and snapshot.jsonl each gain three names; no reading moves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Derek's decision on the gap the migration found: H4 is about an input
that holds no name word but vocabulary, while its join clause reported
title-or-name for 'Smith and King' and 'Attorney General of Minnesota'.
rules/H7 carries the clause verbatim save 'as well' and a dated
corpus count; H4 and H7 point at each other, O5's pointer names both,
decisions/H7 opens with Standing and records the split, and assign's
emitter comment quotes H7 so its implemented: line holds. H7 gains
examples where the clause had none, so corpus_rules.jsonl and
snapshot.jsonl gain their names; no reading moves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73 derek73 changed the title Migrate the H rules to the skeleton format (#761, part 5) Migrate the H rules to the skeleton format; split out H7 (#761, part 5) Oct 20, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73
derek73 merged commit d4fa368 into master Oct 20, 2026
12 checks passed
@derek73
derek73 deleted the docs/761-skeleton-h branch October 20, 2026 08:56
derek73 added a commit that referenced this pull request Oct 21, 2026
…retired-ID guard (#761 follow-up) (#788)

* Quote the bullet a citation comment names (#761 follow-up)

Fourteen comments named a rule's Except or Accepted bullet by
description and quoted nothing, so test_doc_citations could not check
them. Each now quotes the bullet (an Except bullet by its whole bold
lead, which bullet_lead_findings holds it to) beside its rule ID, says
which clause the code implements or pins, and points at the decisions
entry. Two named the wrong bullet: test_parser.py's 'M2 Accepted row'
is M2's Except bullet on post-comma text, and test_pieces.py's
'no-comma peel's Accepted limit' is S7's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add the Except-fence check to the design review's axis 3 (#761 follow-up)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Refuse a rule file that reuses a retired rule's ID (#761 follow-up)

PR #786 replaced the tombstone with 'A retired rule's decisions entry
records the retirement', which left 'never reused' unenforced. A
retired rule's decisions file now carries a line opening 'Retired:',
and test_no_rule_reuses_a_retired_id refuses a rule file with that ID.
The reader lives in tests/v2/design_docs.py. A misspelled Retired line
and one in a file keyed by no rule ID are findings too, since either
would switch the refusal off silently.

No rule is retired yet, so the real-tree test refuses nothing;
test_a_rule_file_reusing_a_retired_id_is_refused builds a synthetic
decisions file and a rule file reusing its ID and is the negative
control (it fails with the ID check disabled, measured).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Address the design-docs review (#761 follow-up)

- test_properties: drop the two maiden-clause pairs said to differ;
  measured, both spellings of each read alike since #601, and M2's
  bullet the new citation quoted accepts no such split.
- _extract: the M3 contrast is a suffix word ('née PhD'), not V, which
  M2 also reads as the maiden name.
- Retired: a numbered '1. Retired:' line is a misspelling too (the new
  row failed against the old pattern first).
- docs/design/AGENTS.md axis 3: two incidents on the record (H1 in
  #779, S3's Msc.Ed. row in #783), not three; a row Then decides may
  belong under Accepted. The skip list names the new guard.
- cases.py: shape 7's U+00B7 half is what the W4 Except decides.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

design docs Documentation fixes and updates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant