Skip to content

Certify the Microcosm UK pair built on policyengine-uk 2.177.7 - #759

Merged
juaristi22 merged 4 commits into
mainfrom
uk-certify-microcosm-pair-3a706
Nov 16, 2026
Merged

juaristi22 merged 4 commits into
mainfrom
uk-certify-microcosm-pair-3a706

Conversation

@juaristi22

@juaristi22 juaristi22 commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

This certifies the Microcosm UK pair built from PolicyEngine/microcosm main on 15–16 November, on policyengine-uk 2.177.7 and policyengine-core 3.34.10, so each build certifies on its own build. Both builds are published on policyengine/populace-uk-private and current in its releases.json.

This is the first UK certificate read from Microcosm's build folders. Each file is pinned at releases/<release>/<build>/ and at the hub commit that published it, not at a tag.

The national

uk-frs-calibration-attempt-20261115T104657Z-fcfac709 (microcosm_uk_2024_25, release microcosm-uk-2024-25-national):

  • Build: Microcosm main at 3a7067fa2, on Chronicle feed 4ce631f5, with policyengine-uk 2.177.7 and policyengine-core 3.34.10.
    • It certifies on its own build (built_with_model_package).
    • The core is the bundle's pin (core_binding: bundle_pin), as on main.
  • Certification in Microcosm: 58 spine gates, 7 terminal gates and 21 release-cut gates passed, and the release certification is shippable.
  • Publication: published 16 Nov at hub commit ad84c0602ed28f694ef52662eb9b063779e4e923 and made current at 12:48 UTC. Its build staged locally only, so it was published with --allow-missing-staging.
  • H5 sha256: 301d2fe5e7805c5a6242b37843cb118445aefd59a0593833362ab5a6e2f9ba90.

The local area

uk-local-candidate-f100-s42-20261116T102715Z-e2745e59 (microcosm_uk_2024_25_dense, release microcosm-uk-2024-25-dense). Constituency and local-authority regions route to it (regional_release_kind: local_area).

  • Build: the national's spine, on policyengine-uk 2.177.7. It also certifies on its own build, with no accepted model mismatch and no input-surface audit.
  • Gates:
    • Build 2 reused build 1's solve exactly: 0 rows differ.
    • It passed all 7 local gates and is shippable.
    • The diagnostic high-value-homes support check fails, as on earlier local cuts.
  • Publication: published 16 Nov at hub commit 4c594d7243fdaec8913ef2543207f1d0934174ab and made current at 12:49 UTC.
  • H5 sha256: 436a680b336106b5bf68f766daa8798c903dee701441b2acc67cb57a6da308b8.

Impact

  • The UK model pin moves from 2.174.1 to 2.177.7. UK baselines on this bundle carry policyengine-uk 2.174.2 to 2.177.7. Among those changes:
    • allowances and trade loss relief set against income in the order that gives the least tax (2.177.7);
    • the legacy enhanced disability premium for enhanced PIP daily living (2.177.6);
    • no income-related ESA at pensionable age (2.177.5);
    • real income variables priced to 2026 (2.177.4);
    • the carer qualifying condition, read from cares_for_severely_disabled_person (2.177.3);
    • non-dependant deductions under each scheme's rules (2.176.2).
  • The shared core pin moves from 3.33.3 to 3.34.10. policyengine-uk 2.177.4 and later require core 3.34.9 or later, and both builds were made on 3.34.10.
    • The US model, policyengine-us 2.2.2, accepts core 3.30.1 or later, and spm-calculator 1.1.0 doesn't depend on core.
    • The strict TRO check's warning, that the US core is outside the US release's ==3.32.5 claim, was already there at 3.33.3.
  • The data moves from eeb6093e and 38605444 to the builds above. UK results change with both.
  • Snapshots: only the UK model surface moves (its parameter count bucket goes from 48 to 50 hundreds). Every UK and US household value is unchanged.
  • One dataset name stops resolving: score_vs_enhanced_frs. It was an evidence file of the old national cut. The national build now ships score_vs_reference, since Microcosm scores a build against the current national build. Nothing loads it as a dataset, so no overlay keeps it.

Commits

The branch is rebased on main at 6c2ef70c (6.20.0), after #750 moved the US to policyengine-us 2.2.2. The lock, the bundle manifest and both TROs were regenerated on main by the commands below, not merged by hand. The manifest's UK release is the one reviewed at e3173d4, apart from main's 6.20.0 bundle version, and its US release is main's.

  • 10125b77: the pair, certified by one certify-data run after the core pin move. It also carries:
    • the lock, in which only policyengine-core and policyengine-uk move;
    • the release lock and both TROs;
    • the certified test constants and the changelog fragment.
  • 2c410ae0: the household calculator snapshots, regenerated on 2.177.7.
  • 536e1630: three bundle tests, moved from the tagged layout to the build-folder layout.
  • f90d3696: docs/households.md names policyengine-core 3.34.10 as the production pin.

The Test jobs already have 150 minutes (#744), so the workflows don't change.

How it was made

uv sync --python 3.13 --extra dev
python scripts/bundle.py update-packages --core 3.34.10
python scripts/bundle.py certify-data --country uk --data-producer populace \
  --manifest-uri "hf://dataset/policyengine/populace-uk-private@ad84c0602ed28f694ef52662eb9b063779e4e923/releases/microcosm-uk-2024-25-national/uk-frs-calibration-attempt-20261115T104657Z-fcfac709/release_manifest.json" \
  --regional-manifest-uri "hf://dataset/policyengine/populace-uk-private@4c594d7243fdaec8913ef2543207f1d0934174ab/releases/microcosm-uk-2024-25-dense/uk-local-candidate-f100-s42-20261116T102715Z-e2745e59/release_manifest.json" \
  --model-version 2.177.7
uvx --from uv==0.13.4 uv lock --no-config --no-sources --default-index https://pypi.org/simple --python 3.13
uv sync --python 3.13 --extra dev
python scripts/release_lock.py
python scripts/bundle.py generate --include-tros
PE_UPDATE_SNAPSHOTS=1 python -m pytest tests/test_household_calculator_snapshot.py -q
python scripts/bundle.py check
python scripts/bundle.py check --include-tros --strict-tros
python scripts/bundle.py check --published-spm --include-tros --strict-tros
python scripts/check_uk_uc_floor.py --dataset <national H5> --year 2025 --output uc_floor.json
  • update-packages' own changelog fragment was dropped. The certification's fragment covers the core move.
  • The certified constants were rewritten with rewrite_certified_constants. The URIs then needed setting by hand, to the build-folder form (releases/<release>/<build>/<file>@<hub commit>), with the local-area build's id and commit beside them.

Evidence

  • Bundle checks: bundle.py check, check --include-tros --strict-tros and check --published-spm --include-tros --strict-tros pass. Only the pre-existing US core warning above is raised.
  • Lock:
    • release_lock.py and uv lock --check pass.
    • In uv.lock, only policyengine-core (3.33.3 → 3.34.10) and policyengine-uk (2.174.1 → 2.177.7) move.
  • Lint: ruff 0.17.4 format and lint pass over the repository.
  • Tests, after the rebase: 3,060 tests over the 61 bundle, manifest, certification, receipt, SPM, UK and US region test files. 3,052 passed, 8 were skipped and none failed.
    • They ran in two processes. The four receipt test files ran on their own (985 passed, 3 skipped), since in one run with the rest they fail with a pytest assertion-rewriting error.
    • This includes the certified-constant tests and the 11 household calculator snapshots, which pass on policyengine-us 2.2.2 with core 3.34.10.
  • UC coverage floor (Bump to policyengine-uk 2.90.2 and revert the UK certified default to enhanced FRS 2024-25 #491): passes on the national build (Great Britain, 2025, policyengine-uk 2.177.7).
    • Benefit units with Universal Credit: 0.912 of DWP's 6.7m.
    • Universal Credit spend: 0.942 of the OBR's £79.3bn.
    • On eeb6093e they were 0.913 and 0.939.

🤖 Generated with Claude Code

@vahid-ahmadi vahid-ahmadi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code side looks right to me. The nested build-folder paths resolve correctly:

  • region_dataset_name and resolve_region_dataset_path match the new path_templates exactly, so constituency and local-authority regions still route to microcosm_uk_2024_25_dense at 4c594d72;
  • dataset_logical_name still gives microcosm_uk_2024_25;
  • _bundle_destination accepts the nested paths.

The pins agree across pyproject.toml, manifest.json, uv.lock and both TROs. Two points:

  1. The local-area build isn't from Microcosm main.
    • The body opens with "built from PolicyEngine/microcosm main". The dense build, though, is from 3dbf54e19, which GitHub compare puts 3 ahead of and 58 behind main.
    • That commit exists only on PolicyEngine/microcosm#1640, which is still open. Thanks for keeping it reachable with the merge in 65606b552.
    • Microcosm allows squash merges, though. If #1640 is squashed, the certified build's commit never reaches main, and the certificate points at a branch.
    • Could #1640 land with a merge commit before this PR, as microcosm#1535 did before the last certification? Otherwise, could the body record the exception? Either way, the opening line should say the dense build comes from #1640.
  2. docs/households.md:159-162 still says the production manifest pins policyengine-core 3.33.3. It now pins 3.34.10.

Review drafted with AI assistance and checked against the code at e3173d4.

@juaristi22

Copy link
Copy Markdown
Contributor Author

Thanks, Vahid.

Point 2: fixed in d88bd536. docs/households.md now names policyengine-core 3.34.10 as the production manifest's pin. No other doc names the old core or UK pins; the 3.33.3 in data-certification.md is the measurement in #699, which stays as it was.

I also rebased onto main after #750, which moved the US to policyengine-us 2.2.2. The lock, the bundle manifest and both TROs were regenerated on main rather than merged by hand. The manifest's UK release is identical to the one you reviewed, and its US release is main's. policyengine-us 2.2.2 accepts core 3.30.1 or later, so the core move still holds. The body's commit list and checks are updated.

juaristi22 and others added 4 commits November 16, 2026 16:31
scripts/bundle.py certify-data on the national build
uk-frs-calibration-attempt-20261115T104657Z-fcfac709 (hub commit
ad84c060), with the local-area build
uk-local-candidate-f100-s42-20261116T102715Z-e2745e59 (hub commit
4c594d72) as the regional release, at policyengine-uk 2.177.7, the
version both were built with. It is the first UK certificate read from
Microcosm's build folders: each file is pinned at
releases/<release>/<build>/ and the commit that published it.

The shared core pin moves from 3.33.3 to 3.34.10 (bundle.py
update-packages), the core both builds were made on: policyengine-uk
2.177.4 and later need 3.34.9 or later, and policyengine-us 2.2.1
accepts it. uv.lock was re-resolved with uv 0.13.4; only
policyengine-core and policyengine-uk move. Both TRO sidecars were
regenerated. The certified test constants move to the pair in the
build-folder form, with the local-area build's id and commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only the UK model surface moves: its parameter count bucket goes from
48 to 50 hundreds. Every household value is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A build-folder certificate names the build (data_build_id), not a cut
tag that starts with the release name, and its files keep their
releases/<release>/<build>/ folder, in the default dataset URI and under
the data directory a bundle install writes to.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/households.md listed the packages the production manifest pins for the
US household SPM contract. The core pin is now 3.34.10, not 3.33.3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22
juaristi22 force-pushed the uk-certify-microcosm-pair-3a706 branch from d88bd53 to f90d369 Compare November 16, 2026 16:32
@juaristi22
juaristi22 marked this pull request as ready for review November 16, 2026 17:12
@juaristi22
juaristi22 merged commit b6c2ed3 into main Nov 16, 2026
15 checks passed
MaxGhenis added a commit that referenced this pull request Nov 16, 2026
Main released 6.20.1 from #759's fragment before this PR merged, so the
PR's patch fragment now cuts 6.20.2, the first release that ships the
aliases. The 22 markers move from 6.20.1 to 6.20.2; nothing else changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants