Skip to content

chore(deps): update policyengine to 6.20.0 - #1076

Merged
DTrim99 merged 6 commits into
mainfrom
auto/update-policyengine-6.20.0
Nov 16, 2026
Merged

DTrim99 merged 6 commits into
mainfrom
auto/update-policyengine-6.20.0

Conversation

@policyengine

@policyengine policyengine Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #805
Fixes #1075

Same-Repository Draft PR Check

  • This PR is a draft.
  • This PR is from a branch in PolicyEngine/policyengine-sim-api, not a personal fork.

Changelog

  • This PR adds a changelog fragment, changelog.d/805-spm-forecast-2036.changed.md, whose type is breaking, added, changed, fixed or removed. The update workflow's own fragment, changelog.d/update-policyengine-6.20.0.changed.md, is kept.

Summary

The update workflow opened this PR with the bundle update (first commit). The later commits add the changes #805 asks for and the fixes the update needs.

The bundle selects policyengine-core 3.33.3, policyengine-us 2.2.2, policyengine-uk 2.174.1 and spm-calculator 1.1.0, with the US data release populace-us-2024-spm-20260915 and the UK data release microcosm-uk-2024-25-national-20261107T020434Z-eeb6093e (dataset SHA-256 0a9b77db1c0dee2693cb2d47bad68da34cb662916eee4829942bcd45de581f25, unchanged). The workflow verified the release dispatch against the installed wheel.

  • Bundle update. Moves the simulation worker from policyengine.py 6.17.1 to 6.20.0, the first release with Certify the 2036 US bundle on policyengine-us 2.2.2, with the ACS-local release on an audited mismatch policyengine.py#750: policyengine-us 2.2.2 and spm-calculator 1.1.0, whose SPM forecast (content digest 335e6755dc8e0bd883e09b76c2bd53a5315683773e44d50085c7d6b8832b6291) covers 2022–2036. US runs for 2036 pass the worker's SPM year check, and 2037 is the first year it refuses with SPM_YEAR_UNAVAILABLE. pyproject.toml and uv.lock are the update workflow's; a local run of the same update produced identical files.
  • No forecast pin in source. The worker reads the forecast digest and its last year from the installed bundle. Two tests copied the old digest (3d86d5c4…) and now name 335e6755….
  • Move the worker to the bundle with the 2022–2036 SPM forecast #805's test changes in test_spm_runtime_native.py: the unavailable-year cases move from 2036 to 2037, and the valid-prevalidation test also covers 2036 alone and a 2027–2036 window. The year- and scenario-boundary tests now pass the observability runtime the worker entrypoints require; without it every case failed with a TypeError before checking a year. These tests run only with SPM_NATIVE_SMOKE_SOURCE set, so CI skips them.
  • Stage 12 data_version for default-dataset runs. Publish to PyPI with trusted publishing #750 addresses the US national dataset by a Hugging Face commit (0ff35303…) while the release keeps its version name (populace-us-2024-spm-20260915); before 6.20.0 they were the same string. The main path reports the release's data version for a run on the default dataset, while Stage 12 reported the dataset's revision, so every US national production/Stage 12 comparison would record a data_version difference and parity qualification would refuse every US national input with dataset_provenance_mismatch. Stage 12's output and the parity check now take the release's data version for the default dataset and keep the dataset's revision for any other; a new test checks both paths against the installed bundle with nothing stubbed.
  • Dataset addresses. Publish to PyPI with trusted publishing #750 moved the US datasets into build folders at commits (national 0ff35303…, ACS-local 28f355b8…; same bytes). The sim API parses these as ordinary hf://org/repo/path@revision URIs, and both commits list the paths the bundle names. One test hard-coded the old root-level path for a logical populace_us_2024@revision reference; it now takes the path from the bundle.

Deploy order

As #805 says: deploy this worker before or together with the Public API's bundle update (PolicyEngine/policyengine-api#3928), because the Public API checks that the worker's /versions match. After deploying, the staging checks from #805 apply: a national 2036 economy run completes with "2036" in its SPM receipt, budget_window:2027:10 completes all ten years, and 2037 returns SPM_YEAR_UNAVAILABLE.

Environment variables

No new ones. policyengine.py 6.17.1 → 6.20.0 adds no environment-variable reads, and spm-calculator 1.1.0 reads the same three as 1.0.0 (AXIOM_CORE_BIN, BLS_API_KEY, SPM_CALCULATOR_CE_CACHE). The UK data release is unchanged, and the US datasets are in the public policyengine/populace-us repository.

Known differences left as they were

  • A UK constituency or local-authority report on the local-area dataset: Stage 12 reports that dataset's revision, the main path the UK release's data version. This predates 6.20.0.

Testing

Run individually by node ID (no full suite), in the executor's locked environment:

  • On 6.17.1, before the bundle update, the changed tests failed for the reasons they target: 2036 refused ("Forecast has no entry for 2036"), the loader rejecting 335e6755…, the SPM stand-in's defaults differing from the installed ones. On 6.20.0 they pass: 75 cases across test_spm_runtime_native.py (with SPM_NATIVE_SMOKE_SOURCE set), test_household_spm.py and test_stage12_report_run_reuse.py.
  • Stage 12 data_version: before the fix, 9 cases in test_stage12_aggregate_reports.py and test_stage12_qualification.py failed (the builder got the dataset's revision; qualification refused the release's data version); after it, all 18 pass. The installed-bundle comparison test fails for the national run with the earlier rule and passes with the new one.
  • test_release_bundle.py::test_resolve_bundle_dataset_uri_maps_explicit_logical_revision_to_hf_uri failed on 6.20.0 and passes after the change.
  • Ruff (0.15.22) on every changed file; Pyright for projects/policyengine-simulation-executor passes.

🤖 Generated with Claude Code

github-actions Bot and others added 6 commits November 16, 2026 16:22
The forecast in spm-calculator 1.1.0 runs to 2036 and has content digest
335e6755…, so 2037 becomes the first year the worker refuses (#805).
The native year-boundary tests also pass the observability runtime the
worker entrypoints now require; without it every case failed with a
TypeError before checking any year.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…log fragment

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
policyengine.py 6.20.0 addresses the US national dataset by a Hugging
Face commit (0ff35303…) while the release keeps its version name
(populace-us-2024-spm-20260915). The main path reports the release's
data version for a run on the default dataset; Stage 12 reported the
dataset's revision. Until 6.20.0 the two were the same string, so every
US national Stage 12 comparison would now record a data_version
difference, and parity qualification would refuse every US national
input with dataset_provenance_mismatch.

Stage 12 output and the parity check now take the value the main path
reports: the release's data version for the default dataset, and the
dataset's revision for any other, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…led bundle

The new test reads the installed bundle with nothing stubbed and checks
that Stage 12 reports the data version the main path reports for US
national and state runs; with the earlier rule it fails for the national
run. The docstring now names the one kind of run where the two still
differ, a UK constituency or local-authority report, which predates
policyengine.py 6.20.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sion test

policyengine.py 6.20.0 moved the US national dataset into a build folder
(releases/microcosm-us-2024-national/populace-us-2024-spm-20260915/), so
the root-level path this test expected no longer matches the address the
resolver builds. The test now derives the repository and path from the
installed bundle and checks only that the requested revision replaces
the bundle's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@DTrim99 DTrim99 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at a0902df. The bump selects what policyengine.py 6.20.0 (#750) certifies: policyengine-us 2.2.2 and spm-calculator 1.1.0, forecast 335e6755…, dataset populace-us-2024-spm-20260915.

Thanks Anthony for the follow-ups:

  • reported_data_version keeps Stage 12 reporting the release's data version for default-dataset runs, now that the US default's revision is a Hugging Face commit. The unstubbed national/state test against the installed bundle pins it to the main path. UK and non-default datasets are unchanged.
  • The tests now follow 6.20.0: forecast pins, the 2037 refusal year, and the dataset address taken from the bundle.

All 28 checks pass. Approving and merging.

@DTrim99
DTrim99 marked this pull request as ready for review November 16, 2026 20:05
@DTrim99
DTrim99 merged commit ee856b6 into main Nov 16, 2026
30 checks passed
anth-volk added a commit that referenced this pull request Nov 16, 2026
`main` updated .py to 6.20.0 (#1076), which changes the executor's
`pyproject.toml` and `uv.lock` beside this branch's changes; they merged
without a conflict and the lock matches the project.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk added a commit that referenced this pull request Nov 17, 2026
Brings in policyengine.py 6.20.0 (#1076) and #1088, which removes the
staging household and economy check jobs from the deploy workflow,
because a new .py version has no Public API catalog rows until the
Public API deploys it.

Two conflicts, resolved to main's side:
- `simulation-deploy.reusable.yml`: both check jobs are removed, which
  also drops this branch's renamed economy check step and its comment.
  This branch's removal of `V2_SIMULATION_OUTPUT_BUCKET_VALUE` from the
  entrypoint's deploy step stays.
- `tests/test_staging_economy_check.py`: deleted with its job, as main
  deletes it. The script `.github/scripts/simulation-staging-economy-check.sh`
  keeps this branch's forced re-run; main keeps the script too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update policyengine to 6.20.0 Move the worker to the bundle with the 2022–2036 SPM forecast

2 participants