Skip to content

fix(pr-queue): lens rules follow-ups from #283's lenses (#303) - #591

Merged
scuttletoby merged 3 commits into
mainfrom
fix/303-lens-followups
Nov 20, 2026
Merged

scuttletoby merged 3 commits into
mainfrom
fix/303-lens-followups

Conversation

@scuttletoby

Copy link
Copy Markdown
Owner

What and why

Follow-ups from the lenses on #283 (all minor; none was a regression). Each point of #303:

1. Nested markup cleaned twice. no_lens_text cleans the text once, and the reader cleans the queue's comment again. With nested tags or links (lens<<a>b>= …, lens[[](z)](w)= …, also read<<a>b> at, safe<<a>b> to merge), the second pass made a lens field, so the queue's own turn-off comment was flagged. Now the <, >, [ and ] that one pass leaves become blanks before the rewrite, so another pass (either way of reading tags and links) changes nothing. A 20,000-case random test checks exactly that, and that the queue's comment stays clear.
Also: keep_markdown (the queue's own advice) now keeps the text as written only when both readings (narrow and not) hold no lens form. Before, only the wide reading was checked, so <lens= x 0123abcd99 <q></b> was kept although the narrow reading shows lens= with a sha. The advice comes from the config, so this was not reachable from outside.

2. A lens field and a sha on one line block for good. Doc fix, as the issue asks. The README now says that a > quote does not help a lens field (only a read at line), and that in a fix-round comment each sha goes on a line of its own. It has a short fix-round template. The queue's advice (NO_HEADER_WAY_OUT) now also says "no sha on a line with a lens field". A test pins the README rule.

3. Carried from round 1.

  • Fences follow GFM (plain_lines and unquoted_lines now share one fence_step). Only a line of the same character, at least as long, with nothing but blanks after it, closes a fence. So ```js inside an open fence is code. A backtick line with another backtick after it (```x```) is inline code, not a fence. Before, a fence line with an info string closed the fence, so a > line that GitHub shows as code was skipped as a quote. In plain_lines it went both ways, for example a plain safe to merge line after a ```py line inside a fence was hidden as code.
  • A second header in another form counts. Below a valid header, a line that starts with lens —, lens – or lens= (as well as lens:) and names a sha is a second header. = counts only when written right after lens: the replay found a real skyrpg lens report whose test code says LENS = 'lens: … read at: 1234567', and counting LENS = would have blocked it.
  • The two mutation gaps from the round-1 bypass lens. M9 (a quote at any indent): the 4-space case got a test in fix(pr-queue): lens opt-out blocks a read written another way; one-pass norm_line; null-cursor test #283; this adds the tab case. M10 (any fence line closes): new tests where a ~~~ fence is not closed by a backtick line, a fence is not closed by ``` ````, and the other way round.

Mutations (each run alone on lens.test.py; all killed)

mutation red tests
M9 quote at any indent (\s*>) 2
M10 any fence line closes / no length check / no kind check 4 / 2 / 2
closing fence may carry an info string (match for fullmatch) 4
backtick info string may hold a backtick 1
no flattening / flatten only <> / only [] 28 / 8 / 20
keep_markdown checks the wide reading only 1
second header lens: only / no — / no = / no – 4 / 1 / 2 / 1
= after blanks counts too 1
plain_lines with the old fence rule 3 (incl. the corpus)

Tests

New tests run against main's lens.py: 29 failures and 7 errors (nested markup, the random clean-twice test, info-string fences, plain_lines fences, second headers, narrow keep_markdown). Not red on main, by design: the tab and fence-kind tests (today's code was right; they hold it against the M9/M10 mutations) and the README-rule test.
At this head: lens.test.py 78 OK, pr_queue.test.py 389 OK, pr_queue_laws.test.py 3 OK, review_config/review_glob/review_judge OK, head_check.test.py 199 OK (1 skipped), head_gate.test.py 106 OK.
Time stays linear: 64 KB backtick or blank runs, 16K fence lines and 64 KB of nested < each take under 0.2 s for lens_problem plus no_lens_text.

State question

The rules are pure functions of the comments: the same comments give the same answer.

  • Runs again: same answer. Stops half-way: nothing new is stored; recover reads the comments again with these rules.
  • Recorded corpus: replayed, no recorded answer changes (2021 cases; still 54 + 2 deliberate differences). So the README's "Where the rules come from" needs no new entry.
  • Real comments, main against this head: 917 PR comments by people on this repository, plus 2,055 PR comments and 163 inline comments by people on the SkyRPG project since 2026-10-15. 0 answers change (74, 77 and 1 flagged before and after). The fence rule reads no real comment differently. For every comment, the queue's quote of it (as leave_off writes it) stays clear before and after.
  • Old state: old queue comments were written by the old no_lens_text. Only one with nested markup around a lens field could read differently, and it was already flagged before (the bug); it stays flagged until deleted. Empty or missing: an empty body or no comments behave as before.

Not changed

  • Not in pr-queue lens rules: follow-ups from #283's lenses #303: the round-1 bypass lens's note that the field spelling is a list of 4 characters (‒, ―, −, :… stay open). Not a regression; it can be its own issue.
  • Headings: a multi-line quote holding # … lens can still trip the heading rule in the queue's comment. The random test found it. It needs a gh error with a Markdown heading naming a lens on a later line, so I left it as is.
  • The SkyRPG project's own lens-format doc and fix-round brief template are project-side, outside this repository. They should get the same sentence: a sha goes on its own line, and a > quote does not help a lens field.

General vs project

All of this is general pr-queue code (fork-ci/actions/pr-queue): lens.py, its tests and README. It applies to every project that uses the queue or the head gate (head-check imports lens.py). The replay used this repository's and one project's PR comments as sample data only. Nothing project-specific was added.

Fixes #303

🤖 Generated with Claude Code

- no_lens_text: the <, >, [ and ] one clean pass leaves become blanks, so
  nested markup in a quoted comment cannot turn into a lens field when the
  reader cleans it again; keep_markdown checks both readings.
- Fences follow GFM: a line with an info string no longer closes a fence,
  and a backtick line with another backtick after it opens none
  (plain_lines and unquoted_lines share fence_step).
- A second header below a valid one counts in the forms `lens —`,
  `lens –` and `lens=` too (not `LENS = '...'`, test code in reports).
- README: in a fix-round comment, a sha goes on its own line; a quote does
  not help a lens field. The queue's advice says so.
- Tests for the two round-1 mutation gaps (tab before `>`, fence kind and
  length).

Fixes #303

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

not a lens: review of #591, lens B (merge-path attack, state question), read-only

The example lines below use <N> and <40-hex> in place of the real values, so this comment does not trip the rule it describes.

SAFE (the code). No new way to fake a lens read or to hide a blocking lens: lens_header is unchanged, the second-header rule only adds blocks, and the fence change moves both readers closer to GitHub's fences (every difference I found blocks more, or matches GitHub). One thing must be fixed before merge: CI is red.

Blocking (merge only, not the code)

  • CI Lint fails (so fork-ci-result fails): vp check reports a formatting problem in fork-ci/actions/pr-queue/README.md (the new section, lines 271-292). Fix: vp check --fix on that file and push.

Major: none.

Minor

  1. lens.py:31 and :386, the second-header rule blocks some real reports. In a report with a valid first-line header, any later line that, once cleaned, starts with lens=, lens — or lens – and holds any 7+ hex token with a digit now counts as a second header. The whole comment then reads nothing, and it keeps blocking until it is deleted. Old code accepted all four of these, new code blocks them (checked):

    • Python keyword arguments in a fenced code block (PEP 8 writes them with no blanks): lens='security', sha='<40-hex>',. The LENS = '...' exception does not cover this, the more common form.
    • Plain prose: Lens — security: ran <N> random cases, where <N> is any number with 7 or more digits (it counts as a sha).
    • A table row: | lens=security | <N> |, with <N> as above.

    It fails closed and your replay found no such comment, so it is not blocking. Cheapest fix: count the new =/—/– forms only on lines outside code blocks (plain_lines), as you already do for LENS = '...'. Or state in README:284 that any 7+ digit number counts as a sha.

  2. pr_queue.py:1484, multi-line why. Turning < > [ ] into blanks also removes > quote markers. So in a multi-line gh error, a quoted line > covered: … / > checks: … or > # … lens becomes a plain line. Two field lines or a heading then flag the queue's own comment. Fuzzing 200k random multi-line texts: 6,573 queue comments flagged at this head against 6,442 on main. 165 are flagged only here, and old rules flag the new text too, so the difference comes from the cleaned text. Single-line text stays clear, which matches your 20k test. This is the same kind of problem as the heading case you left open, and it fails closed (the write token is a person's, so the queue's comment counts). One fix closes the whole class: join why onto one line (' '.join(why.split())) before no_lens_text. A one-line text cannot hold a quote, a heading, a fence or two field lines.

  3. Mutants that still pass lens.test.py (each run alone):

    • a fence closes only on a line of the same length (>= changed to == in fence_step, lens.py:227)
    • fence opener at any indent (' {0,3}' changed to ' *' in FENCE_OPEN_RE, :219)
    • closer at any indent, or at none (FENCE_CLOSE_RE, :220)

    The PR says "fences follow GFM", but no test pins "at least as long" with a longer closer (e.g. closing a ``` ```` fence) or the 0-3 space indent. Two short plain_lines cases in `test_plain_lines_follow_githubs_fences` (`lens.test.py:114`) would cover them. The mutants I tried that are killed: closer with an info string, backtick info string holding a backtick, no narrow check in `keep_markdown`, flattening only `<>`, `=` after blanks, no `–`, no fence check in `unquoted_lines`, any fence kind closes.

  4. A gap that was already there, for the record: splitlines() also breaks on \x85, 
, 
, \x0b, \x0c and \x1c-\x1e, which GitHub does not treat as line ends. text
```` opens a fence for the reader, so plain_lineshides lines that GitHub shows as plain text. Only a counted author can write this, andread at ` is still checked on every line. This PR does not make it worse.

Checked and fine

  • Nested markup, quotes, tabs, odd dashes, look-alikes: the queue's quote of one-line text never reads as a field, either way of reading tags and links. I fuzzed 200k cases with lenſ (re.I matches it in both the writer and the reader), NBSP, tabs, —/–, nested <<a>b> and [[](z)](w), and code fences. The first line of the queue's comment is fixed text, so its comment can never become a valid read; at worst it blocks (minor 2).
  • keep_markdown: now checks both readings, and NO_HEADER_WAY_OUT stays as written.
  • Fences against GFM: opener indent 0-3, a backtick info string holds no backtick, a tilde info string may hold anything, the closer is the same character, at least as long, followed only by spaces or tabs, and an unclosed fence runs to the end. All match the spec. Fences in list items and HTML blocks are still not modelled, as before.
  • New tests fail on main's lens.py: 30 failures and 7 errors (the PR says 29/7; the subtest count differs a little). At this head: lens.test.py 78 OK, pr_queue.test.py OK, pr_queue_laws.test.py 3 OK.
  • State question:
    • The rules are pure functions of the comments, and the random test is seeded, so the same comments give the same answer.
    • A half-written comment with an unclosed fence reads to the end, as GitHub shows it. An edited comment hits the unchanged EDITED rule.
    • Old queue comments have no header, and the second-header rule only changes which message a headerless comment gets, not whether it is flagged.
    • Empty or missing body: unchanged.

General vs project: all general pr-queue code, nothing project-specific.

@scuttletoby

Copy link
Copy Markdown
Owner Author

lens: merge-path A · reviewer: opus · read at: 0b6ee5b

Verdict: SAFE for the lens rules. One item blocks the merge, and it is CI only: Lint is red (a one-line README format fix, details below). No finding fails open. Every new rule is stricter than main, or exactly as strict.

Blocking

  1. CI: Lint fails, so fork-ci-result fails. vp check reports a formatting issue in fork-ci/actions/pr-queue/README.md. Running vp fmt on that file changes one line (README.md:291): the four-backtick code span around ```js becomes ` ```js `. With that change, lens.test.py is still 78 OK. After the fix the head moves, so the queue needs a lens read of the new head or an override, as usual.

Major

None.

Minor

  1. The second-header rule can now block a real lens report that quotes code or prose (lens.py:31, lens.py:386). It fails closed. Below a valid header, FIELD_START_RE also reads lines inside fences, because every line is normalised and leading blanks are dropped. So a Python keyword-argument line inside a ```py block, such as lens='security', head='abc1234f',, counts as a second header. So does a prose line that starts with Lens — summary: fixed in abc1234f. Each case makes the report read nothing until it is deleted. Main read both cases as one valid read (I checked). The replay over real comments found 0 such cases, so I would not change the rule now. But the README sentence at README.md:284-287 should say that code blocks count too: "a line in a code block counts as well; only LENS = ... with a blank before the = is exempt".
  2. The queue's own comment can still be blocked by a multi-line why (pr_queue.py:1484). This is not new: main has the same rate. It fails closed and needs a counted write token. I fuzzed 100,000 leave_off bodies at this head (why + advice + the "Needs a lens read" line, with newlines, quotes, list markers, fences, tags, links, NBSP and odd dashes). Every flag came from one of two rules. One is the heading rule the PR names. The other is not named: two lines of a gh error that start checks: or covered: (FIELD_LINE_RE). Main gives the same counts (6534/6425 against 6520/6412). Nested markup no longer flags anything. One fix closes both: in leave_off, put why on one line (for example ' '.join(why.split())). Then nothing quoted can start a line. This can be a follow-up.
  3. "Read as GitHub renders them" holds only at the top level (README.md:289, lens.py:232-251). This is not new and is on main too. Two cases. (a) A fence inside a list item: in - ```` ``` ```` then > read at , GitHub shows code, but unquoted_linesskips the line as a quote. (b)str.splitlines()also splits on U+2028, U+0085 and similar characters, while GitHub splits only on\nand\r. So note<U+2028>> read at is one plain line on GitHub but a quote to the reader. In both cases anot a lens:comment with a hex read-at outside a real quote is not flagged. Only counted authors can do this, and the effect is "not a lens", never a counted read. A short follow-up: split on\r\n|\r|\n` only, and soften the README wording.
  4. Test gaps (mutants that survive): FENCE_CLOSE_RE with [ \t]* changed to \s* (NBSP after a closing fence would then close it, which GitHub does not do), and FENCE_OPEN_RE with {0,4} (4-space fence openers). Both are lens.py:219-220. Both survive the whole lens.test.py. Nit.

What I checked

  • Faking a field or verdict from outside: a stranger's comment is ignored (counts). Strangers' text reaches a counted comment only through the queue's quote (no_lens_text). After one clean, < > [ ] become blanks, so a second pass (narrow or wide) changes nothing. Then every field form is rewritten. Across 100,000 random texts (alphabet above plus &lt;, <!--, _/__, é, U+2012, U+2015, U+2212, fullwidth colon) nothing was read as a lens field, read at or a header. Look-alike letters and dashes are not read by the queue at all, so they can only mislead a human reader, never the gate. The quote is never on the first line, so it can never be a header. The verdict text itself is never read by the queue; only the shas are.
  • A real report missed: lens_header is unchanged. FIELD_START_RE is a superset of the old LENS_RE second-header rule. For a comment without a header, every FIELD_START_RE line is also a loose_header line, so flag/clear answers do not change there. The fence change in plain_lines only hides lines that GitHub also shows as code. ```x``` no longer opens a fence, which is stricter than before.
  • New tests on main's lens.py: 29 failures and 7 errors, as the PR says. Mutants I ran myself (each alone): rewrite-then-flatten 31 red; no = in the field forms 24; plain_lines ignoring fences 3; unquoted_lines ignoring fences 1+6; FIELD_START_RE .search 3, no re.I 2; closing fence with no length check 1+1, match instead of fullmatch 1+3, no trailing blanks 1; tilde info string with no tilde 1; keep_markdown wide-only 1; = after blanks 1; second header lens: only 4. Equivalent ones that survive: flatten to '' instead of ' ' (still safe), and counting the first line too.
  • Tests at this head: lens 78 OK, pr_queue 389 OK, pr_queue_laws, review_config, review_glob, review_judge OK, head_check OK (1 skipped), head_gate OK.
  • State: the rules are pure functions of the comments, and the random test is seeded. The same comments give the same answer on every run. Half-written or edited comments: the edit/hidden handling is unchanged (EDITED, HIDDEN). Old state: old queue comments have no header, and for them the new second-header rule changes only the message, never flagged/clear. Old nested-markup queue comments were already flagged and stay so. Empty body: fence_step and the readers handle '' and None, as before.

…-block note (#591 lenses)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

Copy link
Copy Markdown
Owner Author

Fix for the two lens reviews (#591 (comment), #591 (comment)), commit 6d0e969:

  • Lint: the README section is formatted (vp fmt --check clean).
  • Finding 1 (second-header rule can block a real report): the README now says lines in code blocks count too, so a report quotes sample headers with placeholders. The rule itself is unchanged; it fails safe.
  • Finding 2 (multi-line error text): leave_off joins the quoted error onto one line before cleaning it, so no line of it can start with covered:, checks: or a heading. New test in pr_queue.test.py; putting the old line back fails it.
  • Findings 3/4 (fence mutants): new tests for a longer closing fence, a no-break space after a closing fence, and the at-most-3-spaces indent for opening and closing. The >=→==, any-indent (open and close) and [ \t]*→\s* mutants now fail lens.test.py.
  • Not changed: Python's own line split on a few Unicode characters (an older gap, as both lenses said).

lens.test.py, pr_queue.test.py and pr_queue_laws.test.py pass.

@scuttletoby
scuttletoby merged commit 2b9ce33 into main Nov 20, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pr-queue lens rules: follow-ups from #283's lenses

1 participant