[zoomote2]

Related GitHub Issue

Closes: #2291

Description

Documentation-only pull requests currently run expensive validation that does not depend on those files. This PR adds a conservative classifier and an always-running audit gate to code-qa, codeql, e2e, mutation-testing, and visual-regression.

Only a successful docs-only classification on a pull_request event can reduce validation. The classifier uses the complete merge-base range, NUL-delimited paths, and rename detection. It checks both sides of a rename. Unknown paths or statuses, empty ranges, policy changes, and missing history select full validation or trigger the full-validation fallback.

The allowlist contains four root guides and policies, Markdown under docs/, and Markdown under .github/. Fixture, test, example, snapshot, and generated directories stay excluded. Root README.md stays outside the allowlist because the extension build packages it. Formatting, Knip, and invisible-character checks always run. Dependency review retains its PR-only condition.

The required compile and both platform-unit-test jobs start for documentation-only changes. Their expensive steps read RUN_VALIDATION. The required compile job also fails unless the always-running formatting and classifier/gate tests passed. This preserves the existing required check names without a ruleset update.

Each audit gate replays classification and checks the outcomes of its watched jobs. The gate reports mismatches and records job outcomes and best-effort runner-minute metrics. It cannot inspect the step-level conditions inside compile or unit-test; workflow tests pin those conditions. Replay detects accidental drift, not hostile changes to PR-controlled validation code.

The ci:full-validation label forces full validation. Adding or removing labels starts new runs. The repository variable ZOO_CODE_REQUIRE_FULL_VALIDATION=true forces full validation on subsequent runs. Existing event and draft conditions still apply. Non-PR events never receive classification-based skips. CodeQL does not currently run for merge_group events.

Test Procedure

Run these commands from the repository root:

pnpm test:change-gate
pnpm test:mutation-ci
pnpm test:code-qa-ci
pnpm exec prettier --check docs/ci-change-based-validation.md

The four script suites passed 215 tests at 687f8489d. Coverage includes classifier results, independent replay-field mismatches, CLI override precedence, required check names, exact step-gating expressions, unusual real git paths, shallow history, renames, summary escaping, and gate CLI inputs. Two clean baselines passed and all 12 targeted mutation probes were caught. These probes are not broader Stryker coverage.

The five validation workflows passed for 687f8489d. The full-validation run confirms existing code validation, but it does not prove docs-only matrix check reporting. A separate docs-only PR must target this PR branch to test the new workflows before merge. Verify compile and both platform check contexts, skipped expensive steps, and all audit gates. Then add ci:full-validation and check the full-validation override.

Pre-Submission Checklist

• Issue Linked: This PR links to CI: add conservative change-based validation with a reliable required gate #2291.
• Scope: The change implements conservative change-based CI validation.
• Self-Review: The code, tests, workflows, and documentation received review.
• Writing Style: This description uses short, direct technical English.
• Testing: Focused script tests and workflow checks pass.
• Test Helpers: Not applicable. These root Node test suites use temporary repositories and local helpers.
• Visual Snapshot: Not applicable. No product UI changes.
• Documentation Impact: docs/ci-change-based-validation.md explains behavior and limits.
• Contribution Guidelines: The change follows the repository contribution rules.

Visual Snapshots

Not applicable. This change affects CI tooling.

Videos

Not applicable.

Documentation Updates

The CI document covers the allowlist, dependency audit, failure behavior, overrides, rollback, and open verification. CONTRIBUTING.md links to it. No product documentation changes are needed.

Additional Notes

• Live docs-only check reporting remains unverified until the separate test PR completes.
• format-check and validation-gate are not required checks in the current ruleset. Required compile carries formatting and policy-test failures. Audit-gate failures only block directly after a maintainer updates the ruleset.
• Keep the existing required checks. A maintainer can consider requiring validation-gate (code-qa) after live verification. Do not require the CodeQL gate for merge-queue entries until it handles merge_group.
• All label changes can replace in-flight PR runs. The runner-time cost is accepted for now.
• Shared gate extraction and broader Stryker integration remain follow-ups.
• The earlier historical sample reported one docs-only change among 500 first-parent commits. It is a historical snapshot, not a current measurement.

[coderabbitai]

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one): 1

• coderabbit-review-active

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

• Source mode: Auto
• Repository: Zoo-Code-Org/Zoo-Code
• Branch: main
• File: .coderabbit.yaml
• Commit: 5319fe9
• Layers: Repository YAML
• Review profile: ASSERTIVE
• Plan: Advanced
• Run ID: a13cd533-826e-4f1c-9f2e-13999171efcd

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

[zoomote2]

No code issues found in the latest update. See session

• Override ci:full-validation label / repo variable never triggers a run (no labeled trigger; docs call the variable override "immediate") — docs/ci-change-based-validation.md:96
• Gate replay trusts the head commit's own allowlist; allowlist-fingerprint never compared to base — scripts/change-gate.mjs:95
• history --output <path>.txt JSON sidecar write clobbers the markdown report — scripts/change-classification.mjs:394
• Docs claim dependency-review always runs; it is pull-request-only — docs/ci-change-based-validation.md:71
• Identical classify/validation-gate job names across five workflows are ambiguous as future required checks — .github/workflows/code-qa.yml:382
• Mutation workflow test still asserts the old pull_request trigger list, so pnpm test:mutation-ci fails — scripts/stryker-diff.test.mjs:84

_{Reviewed df28bfb}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[github-actions]

Review status

This PR was opened by an automated account. A human maintainer must verify the change intent, provenance, and validation before merging.

Current step: Mark the PR ready. Required CI must pass before CodeRabbit starts.

Review-state labels are managed by this workflow; do not edit them manually. community-approved is managed the same way — do not add or remove it manually. It signals a fresh community code approval for the current head as an advisory priority only; maintainer review is still required.

[edelauna]

Closing. A historical sample found one docs-only change in 500 first-parent commits. The runner time saved is small. The classifier, audit gates, and overrides add more maintenance cost than that saving. See #2291.