Skip to content

Unblock 1.7.10: opt-in signed release qualification, atomic installer write, faster CI - #283

Merged
Coding-Dev-Tools merged 5 commits into
codex/release-1-7-10-reconciledfrom
ci/release-gate-opt-in
Oct 25, 2026
Merged

Coding-Dev-Tools merged 5 commits into
codex/release-1-7-10-reconciledfrom
ci/release-gate-opt-in

Conversation

@Coding-Dev-Tools

Copy link
Copy Markdown
Owner

Why

Nothing has reached PyPI since 1.7.9 on 2026-10-01, and #270 (1.7.10) has been green but blocked since 2026-10-18.

  • Since Require durable writes and verified release qualification #217 (2026-09-13), every release.yml write to PyPI or GitHub has had to pass scripts.verify_release_qualification. That step checks an Ed25519-signed envelope covering all 11 RELEASE_GATES, including a multi-week human pilot (five developers for a clean week, then twenty repositories for two more weeks) and a usability gate (at least 11 of 12 first-time developers reach useful recall unaided). The gate definitions are in docs/RELEASE_READINESS.md.
  • The four secrets the verifier reads (ENGRAPHIS_RELEASE_QUALIFICATION, ENGRAPHIS_RELEASE_VERIFY_KEY, ENGRAPHIS_RELEASE_CANDIDATE_ID, ENGRAPHIS_RELEASE_LEDGER_SHA256) have never been configured. A tag push therefore always fails at the publisher.
  • 1.7.6, 1.7.8 and 1.7.9 shipped only through the hard-coded waive_v176/v178/v179_qualification repair inputs. Every patch release needed a new workflow edit just to waive the gate again.

For a patch release from a solo maintainer, the existing build gates plus the protected-environment approval are the right bar. The signed 11-gate qualification is still worth having for a major release or a public claim, so this PR keeps it as an opt-in switch rather than deleting it.

This is a stacked PR on codex/release-1-7-10-reconciled (#270's head, 79b0502). That branch rewrites release.yml, install_cc_hook.py and pyproject.toml heavily, so basing this PR on main would conflict.

What changed (one commit per theme)

1. ci(release): make signed full-product qualification opt-in

  • All four Require signed full-product qualification … steps (publish, github-release, and both writes in github-release-repair) now run only when vars.REQUIRE_SIGNED_QUALIFICATION == 'true'. The two repair steps' old !inputs.waive_* conditions are replaced by the same guard.
  • Removed the three waive_v17x_qualification dispatch inputs, the waiver-guard step, the pre-PyPI waiver-disclosure step, and the waiver branches, env and notes text in Repair GitHub Release. workflow_dispatch now takes only release_tag.
  • Unchanged: every build gate (tag/version match, tag-on-main, full pytest on 3.9–3.14, evals, pyright, CodeQL, pip/npm audits, browser/Pi/container gates, installed-artifact smokes), the two-builder reproducibility check, both SBOMs and the grype gate, trusted publishing, the post-publish PyPI file-set check, the retained-run hash verification in repair, and environment: release-qualification on all three writer jobs.
  • One judgement call to review: when repair has to create a release that doesn't exist yet, it still uses the numeric "never displace a newer Latest" comparison, which used to run only on the waiver path. Now that repair needs no qualification, that path is the one that will actually run, and a repair of an old tag would otherwise mark it Latest unconditionally. Repairing an existing release still only re-uploads assets. The post-upload "promote to Latest" branch was waiver-only and is gone.
  • Docs: docs/RELEASE_QUALIFICATION.md now opens with "What a release requires": green release gates plus release-qualification approval for a patch release. It explains setting REQUIRE_SIGNED_QUALIFICATION=true (repo or environment variable) to bring back the signed 11-gate qualification for major releases or public claims. It also records that the 1.7.6–1.7.9 waivers were removed and that skipping qualification is never recorded as a pass. The signed-envelope contract is kept as-is. docs/RELEASE_READINESS.md and docs/REWORK_EXECUTION.md are updated so they no longer say every release needs the signed approval.
  • Tests: the contract tests now assert the new contract instead of being deleted:
    • tests/test_release_qualification.py checks that each writer job is still in release-qualification, and that each qualification step exists, is guarded by exactly vars.REQUIRE_SIGNED_QUALIFICATION == 'true', reads only secrets.* (never vars.*) and precedes every PyPI or GitHub write. It also checks that dispatch inputs are exactly {release_tag}, that no waive text remains, and that repair writes only verified assets and generated notes.
    • The bash-executed Latest-guard tests are kept and adapted: 12 execution cases plus 8 numeric-comparison cases.
    • tests/test_release_infrastructure.py asserts the guard text inside the publish and GitHub-release qualification steps.
    • I mutation-checked this: removing one if: makes the contract test fail.
    • Tests for steps that no longer exist (waiver disclosure, waiver input guard, retained-candidate SHA binding) were removed together with those steps.

2. fix(install_cc_hook): keep the operator's key order and sync the rename

  • On this branch, _write_settings() was already atomic: temp file in the same dir, fsync, os.replace, mode preserved. The zero-filled-file crash risk in the audit applies to main's version, which Prepare 1.7.10: preserve historical memory and repair hosted client flows #270 already replaces.
  • Remaining fixes:
    • Dropped sort_keys=True, so the user's key order is preserved and an install→uninstall round trip is byte-identical.
    • Added a best-effort fsync of the parent directory after the replace (POSIX), so a power loss can't roll back the rename.
  • The script deliberately stays stdlib-only, so it mirrors engraphis.private_state.atomic_private_text rather than importing it. That helper also defaults to 0o600 and raises on concurrent edits, which isn't the semantics this installer needs.
  • New tests: key order preserved through install and uninstall; an interrupted replace leaves the previous file intact with no temp leftovers; fsync order is staged file, then directory; a refused directory fsync doesn't fail a completed install.

3. ci: run the suite with pytest-xdist and raise the coverage floor

  • pytest-xdist>=3.6 is added to the test extra.
  • ci.yml's full-stack test matrix and the coverage job now run -n auto. Each worker gets its own --basetemp subdir, which stays inside ENGRAPHIS_INDEX_ROOTS.
  • --cov-fail-under goes from 60 to 80.
  • Left serial on purpose:
    • core-py39: numpy+pytest-only floor job; I'm not adding deps there.
    • encryption: one file, and kept short-lived because of SQLCipher.
    • release.yml: scripts/release_evidence.py records the exact pytest command in the public evidence, and the release path should favour fidelity over speed.
  • Measured coverage: 86.74% (41232 statements, 5467 missed; 86.7% on the base branch too). 80 is about 7 points under that, so the floor catches a real regression without failing on normal churn.
  • xdist safety: one test was not xdist-safe. tests/test_external_checkpoints.py::test_unrecognized_marker_fails_closed_without_changing_checkpoint_files put os.getpid() in its parametrize id, so the workers collected different test ids and xdist aborted ("Different tests were collected between gw0 and gw1"). The fix gives that case a fixed id="pid"; the payload is unchanged. No other test needed serialising.

Test results (local, Python 3.13, 2 cores, venv with pip install -e ".[test]")

Run Result Wall time
Base codex/release-1-7-10-reconciled @ 79b0502, serial + coverage (what CI does today) 10744 passed, 0 failed, 40 skipped, coverage 86.7% 1266 s
This branch, -n auto (2 workers) + --cov-fail-under=80 (the new CI coverage job) 10695 passed, 0 failed, 40 skipped, coverage 86.74%, floor met 657 s
ruff check . on this branch All checks passed
  • No failures existed on the base branch, so there was nothing pre-existing to separate out.
  • The 49 fewer tests are all accounted for. In the four changed test files, collection goes from 209 to 160 tests: −54 waiver-only cases whose steps were removed, +1 new repair-contract test, and +4 new installer tests. Every other file collects the same number of tests.
  • The 40 skips are the same on both runs and expected: 14 need the encryption extra (it has its own CI job), plus Windows/macOS-only tests, a release-only CycloneDX test, and a Docker-image test.
  • Hosted runners have more cores than this 2-core box, so CI should speed up more than the ~1.9× seen here.

How to ship 1.7.10 after this

  1. Review and merge this PR into codex/release-1-7-10-reconciled.
  2. Wait for Prepare 1.7.10: preserve historical memory and repair hosted client flows #270's CI to go green on the new head, then merge Prepare 1.7.10: preserve historical memory and repair hosted client flows #270 into main.
  3. Tag the merge commit on main: git tag v1.7.10 <merge-sha> && git push origin v1.7.10. The tag must be on main and match pyproject.toml (1.7.10).
  4. When Publish to PyPI and then Publish GitHub Release pause, approve the release-qualification environment for each.
    • Leave REQUIRE_SIGNED_QUALIFICATION unset for this patch release.
    • Please confirm in Settings → Environments that release-qualification has you as a required reviewer. This PR assumes that protection exists; I couldn't read environment settings from this session.

Notes / follow-ups (not done here)

  • AGENTS.md §1 still says --cov-fail-under=60. I was asked not to touch AGENTS.md, so it needs a one-word follow-up edit.
  • CHANGELOG.md is unchanged. Add a line under 1.7.10 if you want the release notes to mention that the qualification is now opt-in.
  • Nothing was merged, tagged, released or dispatched.

Generated by Claude Code

Since #217 every PyPI and GitHub release write required an Ed25519-signed
envelope covering all 11 RELEASE_GATES, including a multi-week human pilot
and a first-time-developer usability study. The four secrets it reads have
never been configured, so v1.7.6, v1.7.8 and v1.7.9 could only ship through
hard-coded per-release waiver inputs, and v1.7.10 has been blocked since
2026-10-18.

The four "Require signed full-product qualification" steps now run only when
the repository or environment variable REQUIRE_SIGNED_QUALIFICATION is
'true'. Unset, a release still needs every build gate (tests on 3.9-3.14,
evals, pyright, CodeQL, audits, browser/Pi/container gates, installed
artifact smokes, reproducibility) and approval of the protected
release-qualification environment.

The waive_v176/v178/v179_qualification inputs are removed, together with the
waiver guard, the pre-PyPI waiver disclosure step and the waiver branches
of the GitHub repair step. Repair keeps its retained-run artifact and evidence
hash checks. When it creates a release that does not exist yet, it still
refuses to displace a newer Latest.

The release-contract tests now assert the new contract: each qualification
step is present, reads only protected secrets, precedes every publication
write and is guarded by the variable, and no waiver inputs remain.
docs/RELEASE_QUALIFICATION.md documents the variable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0111byzLrKKTCJ23uTT6PyZq
_write_settings() already staged settings.json in a temp file, fsynced it and
moved it into place with os.replace(), so a crash can no longer leave a
truncated file. Two gaps remained:

- json.dumps(..., sort_keys=True) re-sorted every key in the user's
  hand-maintained settings file on each install and uninstall. Keys now keep
  their original order, so an install-then-uninstall round trip is
  byte-identical.
- The directory entry was never synced after the replace, so a power loss
  could still roll the rename back. The parent directory is now fsynced on
  POSIX. This is best effort: a filesystem that refuses a directory fsync
  does not turn a completed install into a reported failure.

The script stays stdlib-only, so it mirrors
engraphis.private_state.atomic_private_text instead of importing it. That
helper also defaults to 0o600 and treats a concurrent edit as a conflict.
Everything else behaves as before.

New tests cover key-order preservation through install and uninstall, an
interrupted replace leaving the previous file intact with no temp files,
fsync ordering (file, then directory), and a refused directory fsync.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0111byzLrKKTCJ23uTT6PyZq
CI ran the whole ~10.7k-test suite in one process, and the coverage gate
only failed below 60% while measured line coverage is 86.7%. That floor
would let roughly a quarter of the tested code lose its tests before CI
noticed.

- Add pytest-xdist>=3.6 to the test extra. ci.yml now runs the full-stack
  test matrix and the coverage job with `-n auto`. Each worker gets its own
  subdirectory under --basetemp, which stays inside ENGRAPHIS_INDEX_ROOTS.
  Locally on two cores the suite with coverage went from 1266s serial to
  657s, with identical results.
- Raise --cov-fail-under from 60 to 80, a few points below the measured
  86.74% (41232 statements, 5467 missed).
- Fix the one test that was not xdist-safe:
  test_unrecognized_marker_fails_closed_without_changing_checkpoint_files
  put os.getpid() in its parametrize id, so xdist workers collected
  different test ids and aborted. The pid case now has a fixed id, and the
  payload is unchanged.

These stay serial on purpose: core-py39 (a numpy+pytest-only floor job),
the single-file encryption job, and release.yml, whose public release
evidence records the exact pytest command.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0111byzLrKKTCJ23uTT6PyZq
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-25T07:10:55.088015Z
We couldn’t complete this review. Comment @codex review to try again.
aade8c6 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@Coding-Dev-Tools
Coding-Dev-Tools merged commit aade8c6 into codex/release-1-7-10-reconciled Oct 25, 2026
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants