Skip to content

[Feat] Post surviving mutants as a PR comment - #2293

Merged
edelauna merged 3 commits into
mainfrom
feat/mutation-pr-comment-1xut0ew4vmm7i
Nov 4, 2026
Merged

[Feat] Post surviving mutants as a PR comment#2293
edelauna merged 3 commits into
mainfrom
feat/mutation-pr-comment-1xut0ew4vmm7i

Conversation

@zoomote2

@zoomote2 zoomote2 Bot commented Nov 1, 2026 •
edited by edelauna
Loading

Copy link
Copy Markdown
Contributor

​Created by Roomote. Follow up by mentioning @roomote, in the web UI, or in Discord.

Related GitHub Issue

Closes: #2287

Description

Changed-code mutation testing reports surviving mutants through annotations and a job summary. This change publishes one advisory PR comment and updates it after later valid runs.

  • Analysis and reproduction use the merge SHA. The comment marker uses the PR head SHA and is checked against the completed run and the live PR before writing.
  • The trusted consumer never checks out or executes PR code. The resolve job uses actions: read and pull-requests: read; the comment job uses actions: read and pull-requests: write. Cross-run artifact downloads pass an explicit token.
  • Fork targets are resolved by exact head owner, branch, SHA, and base repository. The pinned GitHub client lacks listArtifactsForWorkflowRun, so artifact lookup uses the documented REST route.
  • Counts remain visible. Full tables, focused tests, guidance, and reproduction steps appear inside a section collapsed by default.
  • The restricted renderer builds trusted HTML and escapes all artifact text. It supports headings, paragraphs, lists, emphasis, inline code, fenced code blocks, and tables. URLs, mentions, issue references, commit SHAs, and emoji shortcodes remain literal code. Reference matching can conservatively format harmless text as code.
  • Per-PR concurrency serializes writes. Old-head reports and closed PRs are ignored. Valid clean reports clear existing findings without creating a new clean comment.
  • Missing or invalid reports retain the last valid comment and its visible head SHA. Missing test-file names are reported separately from incomplete runs and zero-mutant results.

Validation

  • All 106 tests in scripts/stryker-diff.test.mjs pass on d29a17f5c4.
  • The repository pre-push type check passes. Prettier and git diff --check pass. The full test suite and Knip were not rerun locally for the final changes.
  • Private integration fixtures exercised real workflow_run triggers, read-only fork producer tokens, authenticated artifact retrieval, comment creation and updates, clean results, invalid/missing report retention, and delayed old-run rejection.
  • Same-repo preview and genuine private fork preview verify readable tables and prose inside collapsed details. The fork event supplied an empty pull_requests array and exercised the fallback.
  • Latest cross-reference protections were verified through GitHub's Markdown API using the actual workflow output: all six tested reference forms stayed literal in headings, paragraphs, and lists, with zero generated links.
  • Audited five real production workflow artifacts. Completed mutation reports included test-file metadata; this PR's original run was not applicable. Preview data is synthetic and does not represent Stryker execution.
  • The fixtures verify GitHub mechanics and rendering. Zoo-Code's own Actions policy and comment posting still need validation after deployment.

Pre-Submission Checklist

  • Issue Linked: This PR is linked to an approved GitHub Issue (see "Related GitHub Issue" above).
  • Scope: My changes are focused on the linked issue (one major feature/fix per PR).
  • Self-Review: I reviewed my own code with care.
  • Writing Style: The PR title and description use Simplified Technical English (ASD-STE100): short sentences, plain words, active voice, one idea per sentence, no contractions.
  • Testing: New and/or updated tests have been added to cover my changes (if applicable).
  • Test Helpers (test changes only): Shared test helpers (src/test-utils/, @/utils/test-utils) are used only for mechanical duplication; test intent and scenario-specific mocks remain visible.
  • Visual Snapshot (UI changes only): If a user would notice this change at a glance (layout, theme tokens, brand elements, empty/error states), I added or updated a *.visual.tsx snapshot in webview-ui/. See webview-ui/AGENTS.md → "When a UI change needs a snapshot".
  • Documentation Impact: I checked if my changes need documentation updates (see "Documentation Updates" section below).
  • Contribution Guidelines: I read and agree to the Contributor Guidelines.

Visual Snapshots

No UI change. This PR changes CI automation and a CI gate script.

Videos (interaction / animation only)

Not applicable. There is no interaction to record.

Documentation Updates

Added docs/mutation-testing.md to describe provenance, rendering, retained comments, focused-test metadata, and pre-merge integration validation.

Get in Touch

0x7777777_

@coderabbitai

coderabbitai Bot commented Nov 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one): 1
  • coderabbit-review-active

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@zoomote2

zoomote2 Bot commented Nov 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

No new code issues found. See session

  • .github/workflows/mutation-pr-comment.yml:30-31 — resolve job grants only actions: read, but its fork backfill calls github.rest.pulls.list, which needs pull-requests: read; on fork PRs the call returns 403, the job fails, and the mutation comment is never posted. — fixed: the resolve job now grants pull-requests: read and the static permissions test was updated to match.

Reviewed d29a17f

@github-actions

github-actions Bot commented Nov 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Review status

This PR was opened by an automated account. A human maintainer must verify the change intent, provenance, and validation before merging.

Current step: The required review sequence passed. Remaining merge requirements apply.

Review-state labels are managed by this workflow; do not edit them manually. community-approved is managed the same way — do not add or remove it manually. It signals a fresh community code approval for the current head as an advisory priority only; maintainer review is still required.

@codecov

codecov Bot commented Nov 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

zoomote2[bot]
zoomote2 Bot commented Nov 1, 2026
View reviewed changes
Comment thread .github/workflows/mutation-pr-comment.yml Resolved
@edelauna
edelauna marked this pull request as ready for review November 3, 2026 03:45
@github-actions github-actions Bot added the awaiting-maintainer CodeRabbit approved; waiting for a human maintainer label Nov 3, 2026
@github-actions github-actions Bot added awaiting-maintainer CodeRabbit approved; waiting for a human maintainer and removed awaiting-maintainer CodeRabbit approved; waiting for a human maintainer labels Nov 4, 2026
@edelauna
edelauna added this pull request to the merge queue Nov 4, 2026
@github-actions github-actions Bot removed the awaiting-maintainer CodeRabbit approved; waiting for a human maintainer label Nov 4, 2026
Merged via the queue into main with commit a15ba8c Nov 4, 2026
24 checks passed
@edelauna
edelauna deleted the feat/mutation-pr-comment-1xut0ew4vmm7i branch November 4, 2026 04:40
easonLiangWorldedtech pushed a commit to easonLiangWorldedtech/Zoo-Code that referenced this pull request Nov 4, 2026
@easonliang28
Merge upstream/main into fix/mutation-gate-stacked-base
f133c58 
The only conflict was the gate step env block in .github/workflows/mutation-testing.yml.
Both sides were kept: PR_BASE_REF (this branch, it tells the gate which branch the unit targets)
and PR_HEAD_SHA (upstream Zoo-Code-Org#2293, the pull request head used as provenance for the mutation PR
comment). The run block now takes the event value when it is present and derives the head from
the merge commit only as a fallback for a run with no pull_request payload, so one variable no
longer silently overrides the other. scripts/stryker-diff.mjs and scripts/stryker-diff.test.mjs
merged on their own; the pin for that derivation line was updated to the merged spelling, and the
continue-on-error count pin (3) already matches the merged workflow.
@easonLiangWorldedtech easonLiangWorldedtech mentioned this pull request Nov 4, 2026
Open
5 tasks done
easonLiangWorldedtech added a commit to easonLiangWorldedtech/Zoo-Code that referenced this pull request Nov 5, 2026
@easonLiangWorldedtech
Merge upstream/main (e943530) into vps2/f1c-webview-identity
3f99274 
Live merge base 3ee042d; main brought 6 commits (Zoo-Code-Org#2223 display settings, Zoo-Code-Org#2313
Keenable, Zoo-Code-Org#1309 abort-signal round 2, Zoo-Code-Org#2293 mutant comments, Zoo-Code-Org#2294 CI hygiene,
Zoo-Code-Org#1207 BM25 ranker) across 113 paths. Conflict prediction by blob identity against
the live merge base flagged 10 shared paths; 8 auto-merged and 2 needed a hand
resolution. Both hand resolutions keep the settled side and our side:

- src/core/webview/ClineProvider.ts: kept main's displaySettings import and our
  extra ProviderSettingsNotFoundError binding; took main's three new display keys
  (timeFormat/dateFormat/showRelativeTimes) into our getState() return, re-pointed
  at mergedStateValues because that is where our per-view merge feeds this block;
  and wrapped our setValues() sanitization with main's withoutInvalidDisplaySettings
  so an invalid display value is dropped before the shared write and before the
  view-local write, instead of only before the shared write.
- src/core/webview/webviewMessageHandler.ts: kept main's isInvalidDisplayValue
  guard and our viewStates guard in the updateSettings loop, both with their own
  continue, and kept main's whole failed-save/settingsSaveResult restructuring.

Fidelity: every line either side added since the merge base is present in the
result except 6, all of them intentional rewrites named above (1 ours, 5 main).
No file present on either side is absent from the result: 0 of main's 39 new files
missing, 0 of ours' 1 new file missing, 0 files main deleted since the base.
No conflict markers remain, and the recursive stage-1 blobs matched the real
merge-base blobs byte for byte with no committed markers in them.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@edelauna edelauna edelauna approved these changes
@taltas taltas Awaiting requested review from taltas taltas is a code owner
@JamesRobert20 JamesRobert20 Awaiting requested review from JamesRobert20 JamesRobert20 is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[ENHANCEMENT] Post surviving mutants as a PR comment

2 participants

@edelauna @roomote