[MaxGhenis]

What this fixes

In .github/workflows/push.yaml, the Publish job (PyPI upload and git tag) had no needs:, so on each release commit it ran beside Test. A release could reach PyPI while its tests were still running, or after they failed. In the 2026-10-06 13:11Z release run (37468917800), Publish finished at 13:40:07 and Test at 13:44:04.

Publish now has needs: Test. Both jobs have the same if: (the canonical repository and the "Update PolicyEngine Core" commit), so Publish still runs on every release commit; it now waits for Test to pass.

We merged master into the branch. Since the branch point (757147c7), master had added Test and Lint timeouts, a Docs job with needs: [Test, Publish], a parsed check of the Publish steps, and PyPI trusted publishing (#594, #597, #604, #610). The job graph is now Test → Publish → Docs, with no cycle; all three share the same if:, so Docs still runs on release commits. The Test comment that said Publish does not wait on it now says Publish waits on Test and Docs on both.

Gating won't hold up current releases: all 69 of the last 69 "Update PolicyEngine Core" push runs passed as of 2026-10-06, and the five most recent (2026-10-30 to 10-31) all passed too.

Tests

test_release_tagging.py::test_publish_workflow_publishes_only_after_tests_pass checks four things:

• every job that uses pypa/gh-action-pypi-publish needs Test;
• the only such job is Publish;
• Publish's if: is Test's;
• Publish's if: contains no status function (always(), failure(), cancelled(), success()), which could let it run after a failed Test.

Master's parsed check of the Publish steps (test_publish_workflow_tags_only_after_pypi_succeeds and its mutation tests, from #597) covers permissions, fetch depth and step order, not needs, so the PR keeps both. test_docs_deployment.py::test_documentation_deploy_cannot_block_pypi_publishing checks that Publish doesn't wait on Docs.

After the merge with master:

before (master workflow):  1 failed, 15 passed   (AssertionError: assert 'Test' in [])
after:                     16 passed
tests/core/test_docs_deployment.py: 5 passed
actionlint 1.7.12 (with shellcheck and pyflakes): 0 errors

Impact

The PR changes CI only, not package code. Each release now publishes once Test finishes: in the last five "Update PolicyEngine Core" push runs, Test took between 5 min 31 s and 7 min 40 s, while Publish finished 36 to 47 s after it started. Docs already waited on Test and Publish, so documentation now deploys one Publish run (36 to 47 s in those runs) later than before.

The Test job's "Deploy documentation" step had if: matrix.os == 'ubuntu-latest', but the job had no matrix, so GitHub skipped the step on every run. #594 moved the deploy into its own Docs job.

axiom: n/a: CI workflow change, no policy change

Other audit findings, handled elsewhere

The 2026-10-06 invariants audit of master 757147c7 also confirmed defects that this PR leaves alone, because other work owns them:

• Stock ADD changing later plain reads: fixed by Cache ADD and DIVIDE results only where a plain read returns them #571.
• Year deletion leaving monthly fast-cache entries: draft Drop every contained period from the fast cache when deleting or setting a period #566.
• Disk deletion of contained periods, and disk branch-name parsing and prefix collisions: draft Delete the periods within a deleted period from disk storage #565 (it inherits Keep branch period reads, dumps, and disk deletion consistent #552).
• Clone aliases bound to the parent: fixed by Bind a clone's calc and df to the clone, and give it its own system's entities #586.
• Serialization and subsample input provenance: Restore inputs as inputs, evict the fast cache on holder writes, reject non-numeric uprating and defined_for #576 and Rebuild a subsampled simulation from its inputs, not its calculated values #573.
• Restored entity counts: Fix restore_simulation entity counts #548.
• A monthly input leaving a cached annual sum stale (set_input for one period leaves the variable's cached sum or twelfth over an overlapping period in place #579): fixed by Drop sums and twelfths an own-period input makes stale #593.

The audit counted branch staleness after set_input and raw-file restore as documented behaviour.

🤖 Generated with Claude Code

[MaxGhenis]

Picking this up for the US hub. I'll post a hand-off comment here when it's ready.

---

Generated by Claude Code

[MaxGhenis]

Ready for hub review

Head: 2424738af6b1582da08bf5116680870e0730aae0. This is a merge commit of origin/master (3c89556f2d) into fef0d21f73, pushed as a fast-forward. There was no rebase or force-push. The branch head matched fef0d21f73 at pick-up, and its last push was 2026-10-06.

RESULT: FIXED

Conflicts and how I resolved them

• .github/workflows/push.yaml. Git's 3-way merge put this branch's needs: Test line into master's new Docs job, and left Publish with no needs. I resolved it this way:
  • Docs keeps master's needs: [Test, Publish] and its comment.
  • Publish gets needs: Test, with a one-line comment, at .github/workflows/push.yaml:115-116.
  • Everything else is master's, unchanged: Lint, versioning, Test with timeout-minutes: 45, Docs with its permissions and concurrency, and Publish with its environment, permissions, steps and comments.
• Stale comment. The Test comment said "Docs waits on this job (Publish does not)". It now reads "Publish waits on this job, and Docs waits on both" (push.yaml:60-62). I checked every other comment in the file. The Docs comment ("Deploy only a release that reached PyPI…") and the Publish trusted-publishing comment are still accurate.
• tests/core/test_release_tagging.py.
  • I took master's side in full: the parsed assert_publish_tags_only_after_pypi_succeeds and its mutation tests from Lint GitHub Actions workflows on pull requests, and test the parsed Publish job #597.
  • I dropped this branch's copy of the old text-slice test. Master's parsed version replaced it and covers the same checks (permissions, fetch depth, step order, no || true).
  • I kept test_publish_workflow_publishes_only_after_tests_pass (:315) and the import re it needs.
  • Master's Lint GitHub Actions workflows on pull requests, and test the parsed Publish job #597 test does not check needs, if: equality or status functions, so the two tests don't overlap and both stay.
• Checks on master's structure:
  • Publish's if: still equals Test's. Docs' if: equals both.
  • The status-function check is kept.
  • The job graph is Test → Publish → Docs. graphlib gives the topological order Lint, versioning, Test, Publish, Docs, so there is no cycle.
  • Docs shares the release-commit if:, so it still runs on release commits.
  • test_docs_deployment.py::test_documentation_deploy_cannot_block_pypi_publishing already asserts Docs is not upstream of Publish.

Tests (one file per process)

• tests/core/test_release_tagging.py: 16 passed.
• tests/core/test_docs_deployment.py: 5 passed. This is the only other test file that reads .github/workflows/.
• Mutant: I restored master's push.yaml, without needs: Test, and ran test_release_tagging.py. Result: 1 failed (AssertionError: assert 'Test' in []), 15 passed. The code was restored afterwards and the file gives 16 passed again.
• actionlint: pr.yaml runs the rhysd/actionlint 1.7.12 Docker image with -color. Docker has no daemon in this VM, so I ran the official 1.7.12 release binary with shellcheck and pyflakes on PATH (the image bundles both). Result: 0 errors in 2 files.
• ruff format --check and ruff check on the test file: clean.

CI at head

Pull request run 41548423680: completed, success. All 19 jobs passed, including Lint workflows, the 8 Test jobs and the 8 country-package jobs.

PR body

• I updated the body to master's structure.
• The "Noticed in passing" note now says Deploy the documentation site from its own release job #594 resolved the skipped deploy step.
• The test counts are updated.
• "Impact: CI only; no package code changes" and the axiom: line are kept.

Judgment calls

• Timing claim. I did not keep "about four minutes". In the last five "Update PolicyEngine Core" push runs (41183437332, 41165687527, 41093672475, 41091798445, 41060086107), Test took 5 min 31 s to 7 min 40 s, and Publish took 36–47 s. The body now quotes those figures.
• Test-job comment. I left master's "Slowest observed 2026-10-06..08: 4 minutes" in the Test comment. It is dated and still well under the 45-minute timeout, and this PR shouldn't rewrite Cancel superseded PR CI runs and add job timeouts #604's data.

Proposed follow-up

Remove actionlint ignores that no longer match anything. .github/actionlint.yaml ignores two push.yaml findings from the old Test-job deploy step, and its own comment says to delete them once #594 lands. actionlint 1.7.12 reports 0 errors with an empty config, so both entries are now unused.

---

Generated by Claude Code

[MaxGhenis]

Hub review in progress

Second pass, rechecking the first pass's findings at head 2424738af6b1582da08bf5116680870e0730aae0, which is still the PR head. Master's head is still 3c89556f2d. Still nothing at P1 or P2.

Rechecked at the head this pass

• Merge (section 2): git diff 3c89556f2d 2424738af6 --stat lists only the PR's three files. Against master (^2), only the Test comment changes and needs: Test plus a comment is added. The new test is identical to fef0d21f73's.
• Job graph (section 3): parsed with PyYAML and graphlib, the order is Lint, versioning, Test, Publish, Docs. Lint, Test, Publish and Docs share one if:, and versioning's is that string with the message clause negated.
• Other publishers (section 4): push.yaml:142 is the only PyPI upload under .github/.
• Tests and mutants (section 5): 16 passed and 5 passed. M1, M2, M4 and M5 are killed, and M3, M6 and M7 survive, as in the first pass. I validated a four-line guard that kills M6 and M7.
• Body numbers (section 7): every figure reproduces from the runs and jobs APIs.
• actionlint (section 8): 0 errors in 2 files with the shipped config and with an empty one. On the pre-Deploy the documentation site from its own release job #594 push.yaml, the two ignores hide exactly two errors.
• CI (section 9): run 41548423680, 19 of 19 jobs success.

Not rechecked

• docs.github.com is blocked from this session, so I couldn't re-fetch the documentation the first pass quoted for section 1.

The final review follows as a new comment.

---

Generated by Claude Code

[MaxGhenis]

Ready for hub review
VERDICT: APPROVE
2424738
METHODOLOGY_CHOICE: no

I reviewed head 2424738af6b1582da08bf5116680870e0730aae0. It is still the PR head, so nothing changed after it. Master's head is still 3c89556f2d.

The change does what the PR says. On a release commit, Publish now waits for Test, and nothing else under .github/ can upload to PyPI. The merge keeps everything master added, and CI is green. I found nothing at P1 or P2. All six findings are P3. None blocks the merge.

One limit: docs.github.com is blocked from this session, so I couldn't re-read the documentation for section 1. The semantics below rest on the passages an earlier pass quoted in the interim comment. I've labelled them so.

P3

1. No test stops Test from passing after its tests fail (tests/core/test_release_tagging.py:315).

• Failure scenario: someone adds continue-on-error: true to the Test job (M6) or to its "Run tests" step (M7). In both cases the existing tests pass.
• With the step-level form, the earlier pass quotes the docs as saying it lets "a job to pass when this step fails". Test would then succeed, and Publish would upload a release whose tests failed.
• The job-level form is unclear. The quoted passage says only that it "Prevents a workflow run from failing when a job fails", not what jobs that need it see.
• Neither form exists today.
• Command: each mutant on a scratch copy of push.yaml, then python -m pytest -q -p no:cacheprovider tests/core/test_release_tagging.py. Output: M6 16 passed, M7 16 passed. test_docs_deployment.py gives 5 passed for both.
• A cheap guard, which I validated: no mutant gives 17 passed, M6 gives 1 failed, 16 passed, and M7 gives 1 failed, 16 passed. Append it to tests/core/test_release_tagging.py:

  def test_test_job_cannot_pass_with_failing_tests():
      # continue-on-error on Test or one of its steps lets the job succeed
      # after a step fails, and Publish would then run.
      test_job = yaml.safe_load(PUSH_WORKFLOW.read_text())["jobs"]["Test"]
      assert not any(
          "continue-on-error" in item for item in [test_job, *test_job["steps"]]
      )

• The guard doesn't catch make test || true or a step-level if:. It's optional here or in a follow-up.

2. The PR body's claim that "documentation deploys no later than before" is wrong (PR body, "Impact").

• Before this PR, Docs needed Test and Publish, and Publish ran beside Test. Docs therefore started about 3 s after Test finished.
• Now Publish starts only after Test, and Docs waits for Publish too. So Docs starts one full Publish run later than before.
• Jobs API for the five runs: Test ends at 08:17:15, 03:50:59, 16:58:57, 16:44:34 and 13:10:58, and Docs starts 3–4 s later each time. Publish took 36, 41, 40, 43 and 47 s in those runs.
• Replacement: "Docs already waited on Test and Publish, so documentation now deploys one Publish run (36 to 47 s in those runs) later than before."

3. The body's merge summary is inaccurate (PR body, third paragraph).

• It says master "since added a Lint job". The merge-base 757147c7 already has Lint: at push.yaml:6.
• It also leaves out Publish to PyPI with trusted publishing #610 (trusted publishing). git log 757147c7..3c89556f2d -- .github/workflows/push.yaml .github/actionlint.yaml tests/core/test_release_tagging.py lists Lint GitHub Actions workflows on pull requests, and test the parsed Publish job #597, Deploy the documentation site from its own release job #594, Cancel superseded PR CI runs and add job timeouts #604 and Publish to PyPI with trusted publishing #610.
• Replacement: "Merged with master, which since added Test and Lint timeouts, a Docs job with needs: [Test, Publish], a parsed check of the Publish steps, and PyPI trusted publishing (Deploy the documentation site from its own release job #594, Lint GitHub Actions workflows on pull requests, and test the parsed Publish job #597, Cancel superseded PR CI runs and add job timeouts #604, Publish to PyPI with trusted publishing #610)."

4. Two statuses under "Other audit findings" are stale (PR body).

• Cache ADD and DIVIDE results only where a plain read returns them #571 is merged, not a draft (gh api repos/PolicyEngine/policyengine-core/pulls/571 gives closed merged=true).
• Issue set_input for one period leaves the variable's cached sum or twelfth over an overlapping period in place #579 is closed as completed (state_reason: completed). Master has Drop sums and twelfths an own-period input makes stale #593, "Drop sums and twelfths an own-period input makes stale".
• Replacement lines:
  • "Stock ADD changing later plain reads: fixed by Cache ADD and DIVIDE results only where a plain read returns them #571."
  • "A monthly input leaving a cached annual sum stale (set_input for one period leaves the variable's cached sum or twelfth over an overlapping period in place #579): fixed by Drop sums and twelfths an own-period input makes stale #593."

5. The changelog fragment type differs from the repository's convention (changelog.d/fix-publish-needs-test.fixed.md).

• docs/engineering/skills/github-prs.md:50-52 says to use changed for "tooling". The four earlier workflow PRs all used .changed.md: fix-docs-deploy, ci-actionlint, ci-capacity-ci-hygiene and trusted-publishing.
• It's arguable either way, since this fixes a release-pipeline defect. Both types give a patch bump (.github/bump_version.py:34-38), and fixed is a type configured in pyproject.toml, so the release is unaffected.
• Optional: rename the file to fix-publish-needs-test.changed.md.

6. The new test reads only push.yaml (tests/core/test_release_tagging.py:328).

• publishing == ["Publish"] lists only the jobs in PUSH_WORKFLOW. A PyPI upload added to a new workflow file would pass it.
• Today the grep below finds no such upload, so this is a note on scope, not a defect.

Checked and sound

• The rule (section 1). I couldn't re-fetch these docs. The earlier pass quoted:

  • jobs.<job_id>.needs: needed jobs "must complete successfully before this job will run", and "If a job fails or is skipped, all jobs that need it are skipped unless the jobs use a conditional expression that causes the job to continue".
  • Status check functions: "A default status check of success() is applied" when an if: has no status function.
  • timeout-minutes: GitHub will "automatically cancel" a job that runs too long.

  Publish's if: (push.yaml:117-119) has no status function, and the test at :337-341 enforces that. On those passages, Publish runs on a release commit exactly when Test succeeds.

• The merge (section 2).

  • git diff 3c89556f2d 2424738af6 --stat lists only push.yaml, the fragment and test_release_tagging.py.
  • Against ^2, the only changes to master's lines are the Test comment (push.yaml:60-62), plus needs: Test and its one-line comment (:115-116). So every master hunk survives, and Docs keeps needs: [Test, Publish] and its comment.
  • The new test is byte-identical to fef0d21f73's.
  • The comments match the merged jobs. The Test comment's "Publish waits on this job, and Docs waits on both" is true. The Docs comment ("Deploy only a release that reached PyPI") holds through needs: Publish. The trusted-publishing comment is unchanged.

• The job graph (section 3).

  • PyYAML and graphlib give: Lint [], versioning [], Test [], Docs [Test, Publish], Publish [Test]. The topological order is Lint, versioning, Test, Publish, Docs, so there is no cycle.
  • Lint, Test, Publish and Docs have identical if: strings. versioning's is the same string with !( on the message clause.
  • Docs is downstream of Publish only, so it can't block Publish.
  • On Lint: I'd leave Publish without needs: Lint. Lint only runs ruff format --check, and the PR workflow already gates formatting before merge. A formatting slip on a release commit doesn't make the package wrong, and gating on it would hold a tested release back.

• Other publishers (section 4). grep -rniE 'gh-action-pypi-publish|twine upload|uv publish|pypi' .github/ finds the upload only at push.yaml:142. The other hits are comments, environment: pypi, and a comment in .github/requirements/legacy-country-smoke.txt. pr.yaml doesn't publish, and make build is just python -m build.

• Tests and mutants (section 5).

  • Unmodified: 16 passed and 5 passed.
  • M1 is killed: AssertionError: assert 'Test' in [].
  • M2 is killed: assert 'Test' in ['Lint'].
  • M3 passes: 16 passed.
  • M4 and M5 are killed by the if: equality assertion at :333, not by the regex. When I put Always() && in both Test's and Publish's if:, the regex fires (match='Always('), so the case-insensitive match works.
  • M6 and M7 survive (P3 1).
  • test_docs_deployment.py gives 5 passed for every mutant.
  • I restored the file each time, and git status was clean afterwards.

• The changelog fragment (section 6). The name follows <branch>.<type>.md, fixed is a configured towncrier type, and the text describes the change. CI's "Check changelog fragment" job passed.

• Body numbers (section 7).

  • Run 37468917800: Publish 13:39:32–13:40:07 and Test 13:40:52–13:44:04.
  • Paging the push runs list by page= gives 222 runs. 97 are "Update PolicyEngine Core", and the 69 up to 2026-10-06 (from 2025-10-01, the oldest the API lists) are all success. The 28 after that are all success too.
  • In runs 41183437332, 41165687527, 41093672475, 41091798445 and 41060086107, Test took 460, 361, 439, 331 and 434 s (5 min 31 s to 7 min 40 s), and Publish 36 to 47 s. Every figure reproduces.

• actionlint (section 8).

  • Setup: the 1.7.12 release binary, with ShellCheck 0.11.0 (shellcheck-py) and pyflakes 4.1.0 on PATH. I confirmed both integrations run with a probe workflow that triggers [shellcheck] and [pyflakes].
  • Shipped config: "Found 0 errors in 2 files". Empty config: the same.
  • On push.yaml from before Deploy the documentation site from its own release job #594 with an empty config, it reports exactly the two ignored messages, at lines 79 and 82. The ignores are dead now, but they would hide those errors if they came back.
  • Removing them belongs in a follow-up, because this PR doesn't touch that step or that file.

• CI (section 9). Pull request run 41548423680 at the head is completed and success, with all 19 jobs success: Check changelog fragment, Lint, Lint workflows, 8 Test and 8 country-package jobs. It ran on merge commit e1b49a5, whose tree has no diff from the head's.

What I couldn't check

• docs.github.com: the proxy refuses the connection (403), so I couldn't confirm the section 1 passages or the job-level continue-on-error behaviour myself.
• The push workflow itself: it runs only on master in the canonical repository, so I couldn't run it to watch Publish wait on Test.
• Runs before 2025-10-01: the API lists none, so "69 of the last 69" means every release run the API still lists.

---

Generated by Claude Code

[MaxGhenis]

US + core hub merge audit, core#591 at 2424738af6b1582da08bf5116680870e0730aae0 (squash): Publish a release only after its tests pass.

• Review: an independent Opus 5.5 review approved this head with six P3 findings and no P1 or P2: Publish a release only after its tests pass #591 (comment)
  • Its mutants M1, M2, M4 and M5 failed the new test, and M3, the list form of needs, passed as intended.
  • It reproduced every timing in the description from the Actions API.
• P3s:
  • P3 2, 3 and 4 corrected the description, and the hub checked each correction against the GitHub API before applying it.
  • P3 1 (no test stops continue-on-error on Test) and P3 6 (the publisher check reads only push.yaml) go to a follow-up PR, with the two actionlint ignores that no longer match. No workflow sets continue-on-error today.
  • P3 5 stays: the fragment's fixed type and the convention's changed both give a patch bump.
• Classification: CORE_SEMANTICS no. The PR changes the release workflow and its tests, not package code.
• PE-US core gate: not run, since no package code changes.
• Gates checked live at merge:
  • every check passes (19 of 19 at the head);
  • MERGEABLE;
  • not a draft;
  • no CHANGES_REQUESTED review;
  • head pinned.
• Axiom: n/a. This is a CI workflow change, with no policy change.