Skip to content

Refs #311, Fixes #325, Fixes #416: seam check follow-ups - #506

Merged
scuttletoby merged 3 commits into
mainfrom
wseam-followups
Nov 18, 2026
Merged

scuttletoby merged 3 commits into
mainfrom
wseam-followups

Conversation

@scuttletoby

Copy link
Copy Markdown
Owner

Refs #311, Fixes #325, Fixes #416

Follow-ups to the seam check (fork-seam-check.cjs, fork-follow's seam_check).

Item by item

#325

  • Item 1, done: after a rebase, fork-follow now commits the lowered list itself, in a commit of its own on the rebased series: fork seams: lower counts after the rebase onto <tag>. It lowers every "pending" ceiling and "lost" count that fell.
    • Each change is named in the commit body and in the run's step summary.
    • The lowered list is then checked again; if it would lower again, fork-follow fails.
    • It never raises a count: a rise is still refused with the operator's next step.
    • fork-seam-check.cjs --lowered-list <path> writes the lowered list. The flag is only accepted with --from-upstream --after-rebase; otherwise it is a usage error, exit 2.
    • lowerList only lowers. It keeps // comment keys and key order, and checks its output with parseConfig.
  • Items 2 and 3: already done on main (ceilings kept as their own refusal kind; notes go to the step summary).

#311

#416

  • Item 1 (the Math.min bound) and item 4 ("remove it from lost"): already covered by tests on main. Removing either fix turns its test red.
  • Item 3, fixed: a file named like an Object property (constructor, toString, __proto__) read lost[file] from Object's prototype. Its counts were undefined, so every lost line passed. Lookups now use Object.hasOwn, also in the announcement lookup.
  • Item 2, documented in docs/fork/updates.md: dropped seam lines are counted by text. Where upstream also has a listed line (}), a PR may drop upstream's copy instead of the fork's, and the PR's run passes it. fork-follow, counting from upstream, refuses it unless the lines left read the same as with the fork's copy dropped.
  • Item 5: nothing to change.

Skipped (and why)

How the check behaves (it is a merge gate)

Missing, empty or unreadable list. All fail closed:

  • A PR against a base without the list is refused ("open it against main").
  • A branch whose merge-base has no list is refused ("update the branch").
  • Empty text, bad JSON, {} (no files) or an unknown key (such as a newer check's) stops the run with a crash, exit 3. fork-follow reports that as a failed check and pushes nothing.
  • A PR whose own list is malformed is refused.
  • I checked each of these by hand against the new script.

Old list. A key it lacks (pending, lost, forkFiles) is empty, which is stricter: every fork line it does not list is refused.

Renamed or deleted upstream file.

  • A PR that deletes or renames an upstream file is refused, except the listed removed workflows.
  • A listed file that upstream renamed or deleted, which neither the base nor the series still has, is a note: list both names until the old one is gone.
  • Under its new name the file is an unlisted upstream file, held to no fork line.
  • A pending file the base lacks is refused.

After a rebase.

  • A ceiling or count that fell is lowered in a commit on the series, and the lowered list is checked again.
  • One that rose is refused, naming the operator's step.
  • The next run judges the lowered list from the new base.
  • One case cannot be lowered: a pending file with no fork line left that a lost entry still names. A ceiling must be at least 1, and a lost entry needs its file listed. That stays a note, the next run refuses it, and the doc says to move the file to files with "added": [].
  • A re-run of the same rebase builds the same files, so it advances to a release an earlier run published.

Tests

  • node --test .github/scripts/fork-seam-check.test.cjs: 74 pass (67 before).
    • New tests cover: an Object-property file name; a shrink without lowering (unit and command line); tip copies (unit and command line); lowerList; and --lowered-list writing, writing nothing on a refused run, and its usage errors.
    • Each fix was reverted on its own and its tests went red: own-key lost, shrink rule, tip in main(), tip in findSeamBreaches, the write, raise-guard, per-count min.
  • bash .github/scripts/fork-follow.test.sh: all pass.
    • The case where upstream adopts a pending line now expects the lowering commit (subject, body, list), the summary section, and a passing next run.
    • Dropping the candidate=HEAD refresh turns it red, and so does not passing --lowered-list.
  • python3 .github/scripts/fork-contracts.test.py: pass. node --test .github/scripts/fork-ci-tests.test.cjs: pass. It caught a | grep -q in the new shell test, now fixed.
  • I formatted with oxfmt 0.74.0, the version vp pins. oxlint shows no new warnings.

General vs project

All general: the seam check, the list format and fork-follow's lowering work for any fork that keeps a .github/fork-seams.json. There is nothing project-specific. Paths come from the list and the rebase target.

🤖 Generated with Claude Code

…ip copies spent

- fork-follow commits the list with each "pending" ceiling and "lost" count
  that fell after a rebase lowered (#325 item 1), checks it again, and names
  each change in the commit body and the run summary. It never raises one.
- fork-seam-check --lowered-list (only with --from-upstream --after-rebase)
  writes that list; lowerList keeps comments and order and only lowers.
- A pull request that removes lines from a "pending" file lowers its ceiling
  in its own list (#311 comment); its own list is read only for that.
- A listed seam copy the base branch's tip has since the merge-base is spent
  (#311 item 3, as far as code goes).
- A file named like an Object property is held to its "lost" count (#416
  item 3); the announcement lookup reads own keys too.
- Docs: the new rules, #416 item 2, and how the check behaves with a
  missing, empty or old list, renamed or deleted files, and after a rebase.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens a (correctness + state), head f952c43

Verdict: SAFE WITH NITS. No blocking finding. The lowering only tightens the list, is re-checked, never raises, and fails closed. A run that stops half-way or runs again cannot publish a wrong commit.

Ran on the head: node --test .github/scripts/fork-seam-check.test.cjs (74 pass), bash .github/scripts/fork-follow.test.sh (all passed). I also probed lowerList and findUnlistedChanges by hand: __proto__/constructor keys read from JSON text, a pending entry kept while lost still names it, and own ceilings below the target.

Findings

  1. Low: the shrink rule only refuses an own ceiling that is too high, not one that is too low or removed. .github/scripts/fork-seam-check.cjs:359-360
    Example: main has "pending": {"R.md": 3}. A PR removes 2 fork lines (want = 1) and deletes the R.md entry from its own list, or sets it below 1 fork line left. The PR run passes (checked: no ceiling, no reason). After the merge, the next fork-follow run on the same base refuses main: an unlisted upstream file with 1 fork line, or "more than its ceiling". The nightly stops until a list PR fixes it.
    The PR now reads its own pending ceilings anyway. So refusing own !== Math.max(want, 0) (exact match, or the entry removed only when want <= 0) would close this cheaply. A lower own value still never widens the gate. Before this PR the hole was there too, behind the owner's review of list edits, so it is not a regression.

  2. Low: the pending + lost case keeps the old ceiling, and the next same-base run then blocks. .github/scripts/fork-seam-check.cjs:812
    When a pending file has no fork line left but lost still names it, lowerList keeps the original ceiling (for example 4), not 1. This is documented as a note, and the next run refuses it either way, so nothing ships wrongly.
    In practice the nightly after that rebase releases, and then the next same-base release run refuses main until a PR moves the file to files with "added": []. Optional: lowerList could do that move itself (it would need a why). Or the follow-up could be named in the step summary as "the next run will refuse this", which the summary text already roughly says.

  3. Nit: the "update the branch from its base" hint is misleading in some cases. .github/scripts/fork-seam-check.cjs:196-199
    The hint is added whenever the tip has more copies of the line than the merge-base, even when the line would be refused without the tip (for example the branch adds 2 copies past the allowance). The text may then point the author to an update that does not help.

  4. Nit, test gap: some described behaviour has no end-to-end test.

    • Docs and PR body say "a re-run of the same rebase ... advances to a release an earlier run published". There is no shell test of that with a lowering commit in the candidate.
    • There is no fork-follow test of a lost lowering or of a partial pending lowering (n -> m > 0).
    • The "still lowers again" fail branch (.github/scripts/fork-follow.sh:225) has no test.

    My reasoning says each holds: lowerList is deterministic, and advance compares trees (fork-follow.sh, the released_commit^{tree} check). Only unit tests pin these today.

Checked and fine (state question)

  • Runs again or stops half-way:
    • seam_run deletes seam-lowered.json before every run, so a stale file is never committed.
    • The commit is made only on exit 0 with a written file.
    • A crash after the commit and before the candidate push pushes nothing, and the next run redoes it.
    • Candidate pushed and released, but the series move failed: the next run rebuilds the same tree and moves to that release (advance).
    • A release made before this PR (list not lowered) gives a different tree, so at most one extra release.
  • Placement:
    • The commit sits last on the rebased series, after the "delete upstream workflows" commit.
    • candidate is refreshed (fork-follow.sh:586), so the outside-workflow check, the tree compare, the candidate push and the series move all use the commit that holds the lowered list.
    • dropped is not affected (comm -23 only lists lost subjects).
    • as_fork is defined before the only call that can commit (after the rebase).
  • Never raises or drops a guard:
    • Per-count Math.min, and to >= ceiling keeps the entry.
    • Any rise or other refusal means nothing is written.
    • A removed pending entry turns the file into an unlisted upstream file, which is stricter. With lines == 0 and added == 0, sameLines holds, so the re-check passes.
    • A removed lost entry means 0 allowed.
    • The output goes through parseConfig before it is written.
  • Old, empty or missing state:
    • No pending or lost key: none is added.
    • Comment keys and key order are kept. The real list is byte-identical to JSON.stringify(…, null, 2), so the commit diff stays minimal.
    • A listed file that upstream deleted or renamed is never lowered.
    • A list that cannot be read crashes before any write (exit 3), and fork-follow fails.
    • --lowered-list without --after-rebase, a missing value, or the flag given twice gives exit 2.
    • A write failure gives exit 3, so it fails closed.
  • PR side:
    • The own list is read only to refuse, so it never widens the gate.
    • A malformed own list skips the rule, but listProblem refuses the PR anyway.
    • fork-tests.yml's HEAD^1 HEAD run (merge commit) agrees with fork-checks.
    • The tip copy is spent with max(mergeBase, tip), which is conservative and not used from upstream.
  • #380 minors: seam-check test gaps (min bound, lost message), own-key lookup #416: the remaining [file] lookups are all behind Object.hasOwn. __proto__ and constructor keys round-trip through lowerList as own properties.

@scuttletoby

Copy link
Copy Markdown
Owner Author

Lens b

Verdict: SAFE WITH NITS

Reviewed head f952c437 for correctness and state. Ran node --test .github/scripts/fork-seam-check.test.cjs (74 pass) and bash .github/scripts/fork-follow.test.sh (all pass). I also probed lowerList and findUnlistedChanges directly.

Checked and sound

  • Never raises. lowerList keeps every to >= ceiling and takes Math.min per lost key. It is called only when nothing was refused, so a run with any rise writes no file. A lowered lost entry stays valid: gone <= lines always holds, because the LCS keeps at most the head's copies of a line.
  • The re-check does not report "lowers again" by mistake. A pending 0 kept for a file that is still in lost produces no change, so the second run writes no file. A pending entry is removed only when added = 0 and lines = 0, which implies sameLines. So on the re-check the file does not turn into an unlisted-file refusal.
  • Runs again or stops half-way. The lowering commit exists only in the plan's clone. Every failure after it (fail, or set -e through on_error) pushes nothing. A re-run of the same rebase rebuilds the same tree (a new SHA, but the same tree). The advance test compares trees, so the re-run moves to the earlier release. A release that the old code published with the unlowered list has a different tree, so a new release is built rather than trusted. seam_run removes a stale seam-lowered.json first, and the file is used only after exit 0.
  • Old, empty or odd state.
    • When pending or lost is missing, it stays missing.
    • // comment keys survive lowerList.
    • File names constructor and __proto__ also survive. I probed __proto__ as an own key from JSON.parse: it was lowered and removed correctly.
    • The current .github/fork-seams.json round-trips byte for byte through JSON.stringify(…, null, 2), so the lowering commit touches only the changed lines.
  • The tip rule only adds to spent (Math.max), so it can refuse more but never pass more. --lowered-list outside --from-upstream --after-rebase, a missing path, or a repeated flag is exit 2.
  • candidate is refreshed after seam_check and before refuse_outside_workflows, the tree comparison and the outputs.

Findings

  1. Low: the shrink rule bounds the PR's own ceiling only from above. .github/scripts/fork-seam-check.cjs:360
    • own > Math.max(want, 0) refuses a ceiling left too high. An own ceiling below want passes, and so does an entry removed while fork lines remain.
    • Failure case: main has "pending": {"p.ts": 3}. A PR removes 1 fork line (2 left) and sets its own ceiling to 1, or deletes the entry. The PR is green (probed). After it merges, the next fork-follow run refuses main before its rebase ("more than its ceiling of 1", or "upstream file with 2 fork lines but no entry"), so the nightly stops until a fix PR.
    • Fix: main is held at its ceiling, so the head keeps at least want fork lines in that file. Refusing own !== want when want > 0 is therefore safe and catches this at the PR.
    • It fails closed and belongs to a class that existed before (own-list edits are the owner's review). But this branch already reads the own ceiling, so the check is one line.
    • Related, very rare: when the PR does not edit the list and a newer parseConfig rejects the merge-base's list, ownPending becomes undefined (:917-921). The shrink rule is then skipped silently, and the nightly catches the case later.
  2. Low: the step summary claims a commit the run may never push. .github/scripts/fork-follow.sh:216-221
    • "### Seam list lowered … Committed into the series" is written before the re-check and before refuse_outside_workflows. If either fails, the summary says the series got a commit, while the report says "The series was left as it is".
    • The failure report after a refused re-check also does not say that the list was lowered first. The quoted log is the lowered list's.
    • Fix: write the summary after the re-check passes, or say "the candidate" instead.
  3. Nit: the tip refusal is only as fresh as the last pull request event.
    • fork-checks reads pull_request.base.sha (pull_request_target), and fork-tests' seam step runs only on pull_request. Neither runs on merge_group.
    • So "the merge queue may already cover it" (PR body, Skipped) holds only if the queue updates the branch before it merges, which fires a new synchronize event. A GitHub merge queue alone would not run this check again.
    • Worth one sentence in the doc's "update the branch" paragraph.
  4. Nit: test gaps for the claims. The fork-follow test covers the lowering commit and the next run passing. These are not covered:
    • a re-run of the same rebase that advances to the earlier release (claimed in docs/fork/updates.md and the PR body);
    • the "still lowers again" refusal;
    • a run that lowers one count and sees a rise in another, which pushes nothing. This is covered only at unit level (fork-seam-check.test.cjs:2183).
  5. Nit: every rebase that lowers something adds a permanent fork seams: lower counts after the rebase onto … commit to the replayed series. This is harmless, but the docs could say that these commits may be squashed together.

Nothing here can raise a count, drop a guard or produce a commit the release then trusts wrongly.

@scuttletoby
scuttletoby merged commit 217569a into main Nov 18, 2026
37 checks passed
scuttletoby added a commit that referenced this pull request Nov 18, 2026
* fork seams: lower counts after a rebase, shrink lowers its ceiling, tip copies spent

- fork-follow commits the list with each "pending" ceiling and "lost" count
  that fell after a rebase lowered (#325 item 1), checks it again, and names
  each change in the commit body and the run summary. It never raises one.
- fork-seam-check --lowered-list (only with --from-upstream --after-rebase)
  writes that list; lowerList keeps comments and order and only lowers.
- A pull request that removes lines from a "pending" file lowers its ceiling
  in its own list (#311 comment); its own list is read only for that.
- A listed seam copy the base branch's tip has since the merge-base is spent
  (#311 item 3, as far as code goes).
- A file named like an Object property is held to its "lost" count (#416
  item 3); the announcement lookup reads own keys too.
- Docs: the new rules, #416 item 2, and how the check behaves with a
  missing, empty or old list, renamed or deleted files, and after a rebase.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fork seams: shorten comments for the lint rule

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
scuttletoby added a commit that referenced this pull request Nov 18, 2026
* fork seams: lower counts after a rebase, shrink lowers its ceiling, tip copies spent

- fork-follow commits the list with each "pending" ceiling and "lost" count
  that fell after a rebase lowered (#325 item 1), checks it again, and names
  each change in the commit body and the run summary. It never raises one.
- fork-seam-check --lowered-list (only with --from-upstream --after-rebase)
  writes that list; lowerList keeps comments and order and only lowers.
- A pull request that removes lines from a "pending" file lowers its ceiling
  in its own list (#311 comment); its own list is read only for that.
- A listed seam copy the base branch's tip has since the merge-base is spent
  (#311 item 3, as far as code goes).
- A file named like an Object property is held to its "lost" count (#416
  item 3); the announcement lookup reads own keys too.
- Docs: the new rules, #416 item 2, and how the check behaves with a
  missing, empty or old list, renamed or deleted files, and after a rebase.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fork seams: shorten comments for the lint rule

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant