Skip to content

Require the simulation API's latest PolicyEngine.py version before deploying - #4158

Merged
anth-volk merged 1 commit into
masterfrom
ci/check-latest-simulation-bundle
Nov 3, 2026
Merged

anth-volk merged 1 commit into
masterfrom
ci/check-latest-simulation-bundle

Conversation

@anth-volk

@anth-volk anth-volk commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #4157

Summary

The bundle check now fails unless the Public API's installed PolicyEngine.py version is the latest one the simulation API has published (policyengine.latest in the selected environment's /versions). It no longer compares the simulation API's US and UK routes, and it runs on every pull request.

  • .github/request-simulation-model-versions.sh: after confirming that /versions routes the installed PolicyEngine.py version, it reads .policyengine.latest. It fails when that's missing, or when it differs from the installed version, for example "The Public API is on PolicyEngine .py 6.11.1, but the simulation API's latest is 6.12.0". The unused -us and -uk options and their route comparison are removed.
  • policyengine_api/worker_spm_release.py: validate_installed_worker no longer requires the installed policyengine-us and policyengine-uk versions' routes to name the bundle's worker app. It still checks that the installed packages match the bundle manifest, and that the registry's SPM capability for the PolicyEngine.py version matches.
  • .github/check-policyengine-bundle-supported.sh: --if-changed-from-base, which skipped the check unless a pull request changed the bundle pin or one of seven listed files, is removed. The script now refuses any argument, so a stale caller fails instead of being ignored.
  • pr.yml: ensure-policyengine-bundle-supported-by-simulation-api runs the check with no condition and no longer fetches the full history. Its name and the step names in pr.yml and push.yml describe the new check.
  • Rules and docs: api.migration.bundle-supported-by-simulation-api states the new requirement, and its rationale now points to docs/canonical-spm.md, which describes the check. docs/generated/rules.md is regenerated.

Why the US and UK routes can be dropped: the Public API sends policyengine_version with every simulation request, so the gateway picks the worker by PolicyEngine.py version (_resolve_pinned_policyengine_route) and doesn't use the us or uk routes for those requests.

Expected effect

  • While the Public API is behind the simulation API, every pull request and every release fails this check until the bundle update merges, whatever US and UK versions the newer bundle contains. Before, a behind Public API was blocked only when the newer bundle kept the same US and UK versions, as 6.12.0 did on 2 November.
  • A simulation API rollback moves latest back to an older version, so releases also fail until the Public API's PolicyEngine.py version matches it.
  • Staging and production each compare with their own simulation API's latest. Pull requests use staging's.

Testing

Each test was run on its own, and the new ones failed before the change:

  • tests/unit/test_worker_spm_release.py::test_release_accepts_country_routes_to_a_newer_worker (replaces the test that required the country routes to match). Before: "The us route resolves to a different worker application".
  • tests/unit/test_cloud_run_deploy_scripts.py::test_simulation_version_guard_rejects_bundle_older_than_latest and ::test_simulation_version_guard_rejects_registry_without_latest. Before: both exited 0.
  • tests/unit/test_cloud_run_deploy_scripts.py::test_simulation_version_guard_uses_selected_endpoint (its registry now has latest).
  • tests/unit/test_cloud_run_workflow_configuration.py::test_every_pull_request_checks_the_bundle_against_the_simulation_api. Before: the step had --if-changed-from-base.

python scripts/run_quality_guards.py passes, and ruff passes on the changed Python files. This pull request's own run of ensure-policyengine-bundle-supported-by-simulation-api checked staging's /versions and passed: "PolicyEngine .py bundle 6.12.0 is the simulation API's latest". The simulation entrypoint relays the gateway's /versions body unchanged, and the gateway adds latest from its routing state.

Environment variables

No new secrets or variables. The jobs that run the check read the same ones as before:

  • SIM_ENTRYPOINT: GitHub Actions environment variable in staging and production (both cloud_run_simulation_entrypoint). It reaches the scripts through the job env: of pr.yml's ensure-policyengine-bundle-supported-by-simulation-api (environment staging) and push.yml's ensure-staging-model-version-aligns-with-sim-api and ensure-production-model-version-aligns-with-sim-api.
  • SIMULATION_ENTRYPOINT_URL: GitHub Actions environment secret in staging and production, passed through the same three job env: blocks.
  • OLD_SIMULATION_GATEWAY_URL: read only when SIM_ENTRYPOINT is old_gateway_direct, which neither environment selects. Unchanged.
  • github.base_ref is no longer read.

🤖 Generated with Claude Code

…ploying

The bundle check compared the simulation API's US and UK routes with the
installed bundle's worker, so a Public API behind the simulation API was
blocked only when the newer bundle kept the same US and UK versions. It now
fails unless the installed PolicyEngine.py version is the policyengine.latest
that the selected environment's /versions reports, and it runs on every pull
request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.22%. Comparing base (b27b89c) to head (eb96f3a).
⚠️ Report is 43 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff             @@
##           master    #4158       +/-   ##
===========================================
+ Coverage   51.68%   88.22%   +36.53%     
===========================================
  Files         267      285       +18     
  Lines       15193    16847     +1654     
  Branches     2435     2741      +306     
===========================================
+ Hits         7853    14863     +7010     
+ Misses       6685     1302     -5383     
- Partials      655      682       +27     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@anth-volk
anth-volk marked this pull request as ready for review November 3, 2026 20:17
@anth-volk
anth-volk requested a review from a team as a code owner November 3, 2026 20:17
@anth-volk
anth-volk merged commit a08df0f into master Nov 3, 2026
22 checks passed

This branch was previously deployed

1 inactive deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Require the simulation API's latest PolicyEngine.py version before the Public API deploys

1 participant