Skip to content

Rename Populace to Microcosm in prose and code - #743

Merged
MaxGhenis merged 15 commits into
mainfrom
microcosm-name-sweep-20261106
Nov 16, 2026
Merged

MaxGhenis merged 15 commits into
mainfrom
microcosm-name-sweep-20261106

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

Finish the Microcosm rename while preserving certified release identifiers and existing public imports.

Counts

Counts compare main 7dc8b27 (the 6.20.1 release) with head 0d4ed6d. They count case-insensitive occurrences from git grep -I -i -o populace, and a filename alone doesn't count. Of the 914 occurrences at 57b5870, where the sweep began, main still holds 819 on unchanged lines; it rewrote or deleted the lines of the other 95 (93 R and 2 P, in its bundle manifest, TRACE files, tests and certification guide) and added 120.

Files Mentions
Before 64 939
After 59 860

The classification TSV (repos/policyengine-py/policyengine.py.classification.tsv in the sweep record) covers each of main's 939 occurrences at main's line numbers: the 819 originals (53 P, 70 C, 1 D, 617 R, 78 H) and the 120 that main added after 810a59e (111 R, 5 H, and the 2 P and 2 C that the merge renames: the certification guide's "US Populace certification" sentence and main's new import of a test helper this PR renamed). policyengine.py.classification.rewritten-by-main.tsv and policyengine.py.classification.rewritten-by-main-7dc8b27e.tsv keep the 95 original rows whose lines main rewrote or deleted (81 before 6.20.0, 14 in #759). Another 48 rows cover the occurrences the head adds, at the head's line numbers: 44 compatibility names and markers, 3 assertions that the serialized producer label stays populace, and the repository id policyengine/populace-us that the runbook's line 28 names in place of the prose phrase "the US populace repo". The 812 kept occurrences (696 original, 116 main-added) and the 48 new rows make the head's 860. local/reclassify743_r3.py in the sweep record regenerates the TSV and checks both totals; its --carry pass moved the table from 6c2ef70 to 7dc8b27.

What changed

  • Updated live documentation, docstrings, comments, example output and the current package-name example to Microcosm. Renamed test helpers, test descriptions, test constants and synthetic temporary filenames.
  • Introduced MicrocosmBelgiumDataset, MicrocosmDataProducerCertificationStrategy and MICROCOSM_* source constants. Updated internal references and preserved every prior public import with a marked alias.
  • Made the Belgium example prefer MICROCOSM_BE_DATASET, with a lazy fallback to the prior environment variable. Added regression tests for public imports, producer labels and environment precedence.
  • The branch holds 7734c12 (initial prose rename), 2cfd72a (changelog fragment), ca1b1bf (restore the required hosting link), 7b1f29f (finish code/prose renames and add compatibility coverage), 06d7897 (fragment text), 0f23ab9, 8baca66, e23c1a2, 39ccb9b and eebed8d (merges of main), 6a1d89f, 6f624ab, 3547c9f and 0d4ed6d (release markers), and 1c5c61a (review fixes, below).

Left as is

Category Remaining mentions Reason
R 732 Published HF repositories, release IDs, dataset keys, artifact filenames, package metadata, serialized producer labels and matching code/tests. Includes all 85 mentions in the bundle manifest and all 18 in the bundle TRACE files. The PR adds no dataset-key alias.
H 83 75 released changelog occurrences, 6 occurrences in the dated TRACE case study, and 2 historical public runbook-link paths that keep existing links working.
D 1 README calibration URL remains on its existing Vercel hosting address per the host's explicit instruction until the policyengine.org route is live.
C compatibility 44 Marked source aliases and tests that exercise them, including their required removal comments.

No actionable P or unaliased C occurrences remain, and the PR adds no mapping line. The PR's diff against main touches none of the 37 files whose every occurrence it keeps (R, H or D), so each is byte-identical to main; they include the certified manifest, the TRACE sidecars, the released changelog and the JSON/JCS evidence fixtures.

Compat aliases

Every alias carries compat: populace alias, remove after 6.20.2. Main released 6.17.0 through 6.20.1 without this PR. Main's changelog.d/ at 6.20.1 holds no fragment, and the head adds only this PR's .changed.md fragment, so .github/bump_version.py (infer_bump, bump_version) cuts patch release 6.20.2 from it; a read-only run of those functions at the head gives 6.20.1, patch, 6.20.2. 6.20.2 is therefore the first release that ships the aliases. Main releases most days, so I recompute that version just before merge. If another fragment lands first, one commit changes only the version in the 22 markers, and I update this paragraph, this table and the classification TSV's rows that name it.

Prior name Current name Location Remove after
PopulaceBelgiumDataset MicrocosmBelgiumDataset policyengine.tax_benefit_models.be, .be.datasets, .be.model; package __all__ 6.20.2
PopulaceDataProducerCertificationStrategy MicrocosmDataProducerCertificationStrategy policyengine.provenance.certification 6.20.2
POPULACE_US_SOURCE_COVERAGE_FILE MICROCOSM_US_SOURCE_COVERAGE_FILE policyengine.provenance.certification 6.20.2
POPULACE_DATA_PACKAGES MICROCOSM_DATA_PACKAGES policyengine.provenance.certification 6.20.2
POPULACE_RELEASE_REPOS MICROCOSM_RELEASE_REPOS policyengine.provenance.certification 6.20.2
POPULACE_US_2024_ACS_LOCAL_BUILD_O MICROCOSM_US_2024_ACS_LOCAL_BUILD_O policyengine.tax_benefit_models.us.legacy_spm.registry 6.20.2
POPULACE_US_2024_ACS_LOCAL_MICROCOSM_1263 MICROCOSM_US_2024_ACS_LOCAL_MICROCOSM_1263 policyengine.tax_benefit_models.us.legacy_spm.registry 6.20.2
POPULACE_BE_DATASET MICROCOSM_BE_DATASET examples/belgium_axiom_pilot.py; new environment variable wins 6.20.2

The CLI value --data-producer populace counts as R, because certified manifests record it. It stays as it is rather than becoming a new dataset or producer alias.

Tests

At head 0d4ed6d, after every merge of main (logs in reviews/policyengine-py/r4/ in the sweep record; r3/ holds the same runs at 3547c9f):

  • POLICYENGINE_SKIP_COUNTRY_IMPORTS=1 .venv/bin/python -m pytest --noconftest tests/test_microcosm_compatibility.py tests/test_be_axiom_pilot.py -q: 28 passed, 4 optional source-stack tests skipped. This selection includes the two registry cases and the new test for an unset dataset variable.
  • The same flags on the files main's merges touched or that cover them, tests/test_us_regional_input_surface.py, tests/test_us_regions.py, tests/test_uk_regions.py, tests/test_release_manifests.py, tests/test_certify_data_release.py, tests/test_bundle.py and tests/test_bundle_metadata.py, less the three slow or private-HF cases named below: 236 passed, 2 failed (at 3547c9f, without tests/test_uk_regions.py: 194 passed, 2 failed). Both failures are UK tests in tests/test_release_manifests.py, and this host's environment causes them: its policyengine-uk is 2.151.0, while the bundle pins 2.174.1. A clean checkout of main d3678b7f with the same environment fails the same two (94 passed, 2 failed, on the first three files).
  • ruff check and ruff format --check on the 20 Python files the branch changes: passed. git diff --check: passed.
  • CI: the checks on this PR, which install the pinned versions.

The runs below used 06d7897. All pytest runs were serial; at most two pytest runners ran concurrently. Completed focused selections recorded 487 passing tests and 4 optional skips. The access and timeout failures below also reproduce on the unchanged baseline. The normal 18-file regression command below stopped twice before collection finished, because pinned country-model initialization spent several minutes recursively importing US variable modules and loading parameter YAML. These are incomplete checks, not test passes. A retry using only POLICYENGINE_SKIP_COUNTRY_IMPORTS=1 also stopped during country fixture imports; focused runs use --noconftest as well and tests with self-contained fixtures.

.venv/bin/python -m pytest tests/test_certify_data_release.py tests/test_us_legacy_spm.py tests/test_us_legacy_spm_integration.py tests/test_us_legacy_inputs_integration.py tests/test_release_manifests.py tests/test_dataset_materialization.py tests/test_dataset_source.py tests/test_dataset_runtime.py tests/test_trace_tro.py tests/test_release_tro_generation.py tests/test_release_build_folders.py tests/test_uk_qualification_pipeline.py tests/test_execution_receipt.py tests/test_bundle.py tests/test_bundle_metadata.py tests/test_entity_utils.py tests/test_us_regions.py tests/test_uk_regions.py -q
  • Installed all extras using UV_CACHE_DIR=/private/tmp/policyengine-rename-uv-cache UV_PYTHON_INSTALL_DIR=/private/tmp/policyengine-rename-uv-python uv sync --all-extras: passed in about 4 minutes. The workspace sandbox denied the first attempt, which used the default cache; a retry with a writable cache succeeded. The PR leaves the dependency pins and the lockfile alone.
  • POLICYENGINE_SKIP_COUNTRY_IMPORTS=1 .venv/bin/python -m pytest --noconftest tests/test_microcosm_compatibility.py tests/test_be_axiom_pilot.py -k 'not registry' -q: 25 passed, 4 optional source-stack tests skipped, 2 registry cases deselected; a separate check below validates the registry module itself.
  • POLICYENGINE_SKIP_COUNTRY_IMPORTS=1 .venv/bin/python -m pytest --noconftest tests/test_dataset_materialization.py tests/test_dataset_source.py tests/test_bundle.py -q: 41 passed.
  • POLICYENGINE_SKIP_COUNTRY_IMPORTS=1 .venv/bin/python -m pytest --noconftest tests/test_certify_data_release.py tests/test_release_build_folders.py tests/test_uk_qualification_pipeline.py tests/test_trace_tro.py tests/test_release_tro_generation.py tests/test_bundle_metadata.py -k 'not test_preparation_generates_new_sidecars_and_verifies_the_exact_pair and not test_the_real_cli_generates_replay_requests_despite_country_import_suppression' -q --tb=short: 421 passed, 1 failed, 5 deselected. The remaining failure, test_the_certification_environment_falls_back_to_the_packaged_certificate, exceeded its unchanged 60-second subprocess timeout. The same test reproduced the identical timeout on the isolated origin/main archive (1 failed, 225.68 seconds), confirming the issue predates this branch. All 70 certification, 80 release-folder, 46 TRACE, 40 TRO-generation and 7 bundle-metadata tests passed, together with 178 UK qualification cases. The five excluded cases are the four private-HF variants and slow real CLI case described below.
  • The earlier lane's python -m pytest tests/test_entity_utils.py tests/test_us_regions.py -q failed because the environment lacked the package; the completion run installed the full development environment before retrying checks.
  • Lightweight registry check with .venv/bin/python and runpy.run_path: both aliases retain object identity, full serialized source records match origin/main, and registry/hash lookups return the same objects. This avoids initializing the entire country model.
  • Ruff lint and format checks on all changed Python files: passed.
  • AST/syntax checks on all changed Python files: passed.
  • git diff --check: passed.
  • Classification multiset comparison against origin/main: exactly 914 baseline occurrences, no missing/extra rows; all 45 added HEAD occurrences match their current file/line/token.
  • An initial certification/qualification/TRACE selection passed 168 tests and failed 4 parameterized preparation cases before the lane interrupted its real CLI case. All four failures hit HTTP 401 fetching policyengine/policyengine-uk-data-private release metadata. One representative failure reproduced identically on an isolated origin/main archive (57b5870, 1 failed), confirming pre-existing private-repository access requirements.
  • Byte comparisons against origin/main: all 39 frozen artifact/changelog files passed.

Host follow-ups (ca1b1bf, 06d7897, 0f23ab9, 6a1d89f, 1c5c61a, 8baca66, 6f624ab, e23c1a2, 39ccb9b, 3547c9f, eebed8d, 0d4ed6d)

  • README.md:10: the calibration dashboard link stays on its Vercel address until its policyengine.org route is
    live, under Max's link rule (ca1b1bf).

  • changelog.d/microcosm-name-sweep-20261106.changed.md: the fragment now names what this PR renames and the
    aliases' last release (06d7897). Its generic text had said "throughout docs, comments and code".

  • Merge with main (0f23ab9, 2026-11-09): main had moved 7 commits to f41c63a, including the 6.16.1 release.
    One file overlapped, tests/test_release_manifests.py, and git merged it without a conflict. Main's new code
    uses none of the names this PR renames, apart from the marked aliases.

  • Release markers (6a1d89f): every remove after marker and the fragment's last release read 6.17.0 instead of
    6.16.1.

  • Review fixes (1c5c61a), from the 2026-11-09 review below:

    • examples/belgium_axiom_pilot.py now reads MICROCOSM_BE_DATASET, then the former name, and exits with a
      message naming MICROCOSM_BE_DATASET when the environment sets neither. A new test,
      test_belgium_example_names_the_new_variable_when_neither_is_set, covers it.
    • scripts/certify_data_release.py:51 help text names the populace producer value again, since manifests
      record that value. The runbook names the published repo policyengine/populace-us again, in place of the prose phrase.
    • tests/test_release_manifests.py calls its 2023 UK constants UK_MICROCOSM_2023_RELEASE_ID and
      UK_MICROCOSM_2023_DATASET_URI. docs/regions.md drops a passive.
    • The changelog fragment no longer names a release, so it can't go stale.
  • Merge with main (8baca66, 2026-11-16): main had moved 18 commits to 810a59e, through the 6.19.0 release. No file
    changed on both sides, main added no occurrence of the old name, and main's new code uses none of the renamed
    names.

  • Release markers (6f624ab): the markers read 6.19.1.

  • Merge with main (e23c1a2, 2026-11-16): main took d3678b7f (Certify the 2036 US bundle on policyengine-us 2.2.2, with the ACS-local release on an audited mismatch #750), which certifies the 2036 US bundle. It
    conflicted in docs/engineering/skills/data-certification.md; the merge keeps main's build-folder manifest URIs
    and this PR's "Microcosm certification" wording. Main's new tests/test_us_regional_input_surface.py imported
    _populace_manifest_payload_without_regions from tests/test_certify_data_release.py, which this PR renames,
    so git merged it cleanly but the import would have failed; the merge renames it to
    _microcosm_manifest_payload_without_regions. No other name this PR renames reappears in main's code. Main's
    other 102 new occurrences are published ids, serialized values and its released changelog, which the PR keeps as
    R and H.

  • Merge with main (39ccb9b): main's 6.20.0 version commit, 6c2ef70, which released d3678b7f and two other
    fragments. No conflict.

  • Release markers (3547c9f): the 22 markers read 6.20.1.

  • Merge with main (eebed8d): main took Certify the Microcosm UK pair built on policyengine-uk 2.177.7 #759 (b6c2ed3, a UK certification) and released it as 6.20.1 (7dc8b27).
    No conflict, and git show --remerge-diff eebed8d is empty. Certify the Microcosm UK pair built on policyengine-uk 2.177.7 #759 changes only published Hugging Face ids: it
    rewrote 14 of the original R lines and added 14 R occurrences, so the totals stay 939 and 860. Its new code uses
    none of the names this PR renames.

  • Release markers (0d4ed6d): the 22 markers now read 6.20.2, for the reason under "Compat aliases".

Unsure

  • examples/belgium_axiom_pilot.py:22: retained populace_be_pilot_2026.h5 as R conservatively. The example refers to an existing pilot input artifact; this sweep has no evidence that its on-disk basename changed. The PR renames the environment variable and the surrounding prose.

Review

  • An Opus 5.5 in-session review of 6a1d89f (2026-11-09, while an operator hold kept every Subfleet lane closed)
    asked for one change, the release number, plus nits. 1c5c61a and 6f624ab answer them. It is supporting
    evidence, not an independent review round.
  • Round 1 of the independent review (GPT-6.1 Sol on a Subfleet lane, at 6f624ab): REQUEST_CHANGES, with two P1s, one P2 and two P3s. This body and the regenerated TSV fix four of them. The counts now treat the runbook's line 28 as P (renamed prose) plus one new R occurrence. "CLI help" leaves the list of changes, since scripts/certify_data_release.py:51 keeps its text. The TSV's 15 rows that said "through 6.16.1" now say 6.19.1, and the version note above lists everything a version change touches. Two passives go. The fifth finding, a P3, cites docs/engineering/skills/github-prs.md, which asks for an issue, Fixes #ISSUE as the body's first line and a fragment named for the issue. No issue tracks this sweep, which follows one instruction across the organization, and I didn't open one just for the link; the fragment keeps the name the sweep's other PRs use.
  • Round 2 (GPT-6.1 Sol, a full review, at 6f624ab): APPROVE, with two P3s. It found round 1's fixes sound, every frozen file byte-identical to main and no actionable P or unaliased C left. This body answers the first P3: the version paragraph now says that the head, not main, adds the fragment. The second P3 repeats the issue-link exception above, which stands: CONTRIBUTING.md's own example names a fragment after its branch (changelog.d/my-branch.fixed.md).
  • Round 3 (GPT-6.1 Sol, at 3547c9f) reviewed only the changes since round 2: the two merges of main, the conflict resolution, the renamed import and the 6.20.1 markers. APPROVE, with one P3 (two passives in this body), which this body fixes. It found the merges kept every frozen value, the 22 markers and 6.20.1 correct, the counts and both TSVs reproducible byte for byte, and all 37 frozen files byte-identical to main. The review has finished.
  • After round 3, main released 6.20.1. eebed8d merges it with no conflict and no manual change, and 0d4ed6d changes only the 22 markers' version, as the version paragraph provides. Nothing else changed since round 3.

Sweep record and rulebook: ~/reviews/populace-rename-2026-11-06/ (RULEBOOK.md).

🤖 Generated with Claude Code

MaxGhenis and others added 15 commits November 6, 2026 23:59
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rg route is live

Public links go to policyengine.org, never a hosting address, and when the
policyengine.org route isn't live the link is held rather than swapped for
another hosting address. policyengine.org/us/calibration-diagnostics returns
404 today, so README.md keeps calibration-diagnostics.vercel.app/populace.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The generic fragment said the stack was renamed "throughout docs, comments
and code", which claims more than this diff does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main released 6.16.1 without this change, and its pending added fragment makes
the next release 6.17.0, so the aliases first ship there. Move every
"remove after" marker and the changelog fragment to 6.17.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…taset

- examples/belgium_axiom_pilot.py exits with a message naming
  MICROCOSM_BE_DATASET when neither dataset variable is set, instead of a
  KeyError naming the former one; a new test covers it.
- The certification CLI help names the producer value it accepts.
- The runbook names the US release repository by its id.
- The 2023 UK test constants say so in their names.
- The changelog fragment no longer names a release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ps them

Main released 6.17.0 through 6.19.0 since the markers named 6.17.0, and
its changelog.d now holds only this PR's patch fragment, so
.github/bump_version.py cuts 6.19.1 from it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main's data-certification guide gained build-folder manifest URIs; the
merge keeps them and the sweep's "Microcosm certification" wording.
Main's new tests/test_us_regional_input_surface.py imported the
manifest-payload test helper under its former name, which this branch
renamed in tests/test_certify_data_release.py, so it now imports
_microcosm_manifest_payload_without_regions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main released 6.20.0 before this PR merged, so the PR's patch fragment
cuts 6.20.1, the first release that ships the aliases. The 22 markers
move from 6.19.1 to 6.20.1; nothing else changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main released 6.20.1 from #759's fragment before this PR merged, so the
PR's patch fragment now cuts 6.20.2, the first release that ships the
aliases. The 22 markers move from 6.20.1 to 6.20.2; nothing else changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis marked this pull request as ready for review November 16, 2026 22:04
@MaxGhenis
MaxGhenis merged commit 23c2223 into main Nov 16, 2026
16 checks passed
@MaxGhenis
MaxGhenis deleted the microcosm-name-sweep-20261106 branch November 16, 2026 22:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant