[anth-volk]

Fixes #4149

Summary

• New job full-catalog-integration ("Cross-database integration tests / Full installed PolicyEngine.py catalog") in .github/workflows/v2-integration-check.yml. It takes the two steps that load the complete installed catalog out of catalog-runtime-integration and runs them in one pytest process with --durations=10, so the catalog is extracted once (tests/fixtures/installed_catalog.py, a functools.cache around extract_installed_catalog()). The qualification test still starts tracemalloc after extraction and gc.collect(), so the 256 MB limit still measures only publish_catalog's additional memory. The job reports nproc, free -m and lscpu, prepares its own disposable v2 schema (PostgreSQL only; neither test needs MySQL, Redis or the managed roles) and uploads its own coverage file (coverage-v2-full-catalog.xml).

• Runs only when needed. The job's first step, .github/scripts/decide_full_catalog_tests.sh, lists what changed since the merge base of the pull request's base commit (or the commit before the push) and sets run_full_catalog_tests. Every later step has if: steps.full_catalog.outputs.run_full_catalog_tests == 'true'. The tests run when the change touches:

  • uv.lock, pyproject.toml (the .py and country model package versions)
  • policyengine_api/data/v2/catalog/*, and policyengine_api/data/v2/models/metadata.py (the models of the tables publication writes through)
  • migrations/v2/*, alembic-v2.ini
  • the two tests and tests/fixtures/installed_catalog.py
  • .github/workflows/v2-integration-check.yml and the script itself

  A moved file counts under its old path too (--no-renames). An empty or all-zero base, or one missing from the clone (a force push), runs the tests.

• Nightly and manual runs. A schedule trigger (04:23 UTC) and the existing workflow_dispatch always run them.

• The job always runs. It has no job-level if, so it succeeds whether or not the tests are skipped. check_ci_complete.sh fails on any job that didn't succeed, and .github/release_commit.py lets a release run skip the check only when every Cross-database integration tests / ... job of the merge run completed and succeeded. Both still hold.

• Pushes to master use the same rule against github.event.before. A merge that doesn't touch those paths skips the full-catalog tests, which also shortens the time from merge to the release commit, because versioning needs this check. When the release run can skip its checks (the usual case), it skips this whole check as before. When it has to run them itself, its diff includes the version lines in pyproject.toml and uv.lock, so it runs the full-catalog tests too, as it does today.

• Edge-case catalog on every pull request. tests/fixtures/v2_catalog_value_shapes.py adds these to the deterministic fixture catalog, as a separate catalog so that normalized_catalog()'s counts stay as test_catalog_extraction.py and test_v2_catalog_publication.py assert them:

  • one parameter value of each shape extraction accepts: True/False, 0 and negative integers, 0.30000000000000004, 12000.0 (which must stay a float), ±inf (stored as strings), text with tabs, newlines, backslashes, quotes and non-ASCII characters, an enum, a date and a datetime, lists and tuples (including empty), nested dicts (including empty), None, and NumPy float64/int64/True_
  • bracket parameters gov.value_shapes.scale[0].rate and similar, with their bracket nodes
  • date-only and naive start dates, a year-1 start (.py reads Core's 0000-01-01 that way), a one-day range and open-ended ranges
  • region filters: an integer, a scalar string and a JSON membership list, plus an unfiltered place

  test_publication_stores_every_value_shape_unchanged publishes it twice. The second publication compares every stored row with the catalog in both directions. The test then reads every parameter value, variable, parameter, node and region back from PostgreSQL and compares them as strict JSON, which tells 1, 1.0 and true apart. tests/unit/v2/test_catalog_value_shapes.py fails if a type in JSON_VALUE_NORMALIZERS has no example in the catalog.

• docs/engineering/skills/testing.md says where the path list and the edge-case catalog live.

Time saved on a pull request that doesn't touch the catalog

Before this change, over the last 13 successful pull request runs (2026-10-29 and 2026-10-30), "Cross-database integration tests / MySQL, PostgreSQL, and Redis integration" took 7.2 to 37.6 minutes (median 11.1). Its two full-catalog steps took 5.0 to 32.8 minutes of that; every other step together took 2.1 to 4.8 minutes (median 3.1).

This PR's first run (41119889262):

Job	Time
MySQL, PostgreSQL, and Redis integration (no full-catalog steps)	4.1 min (246 s)
Full installed PolicyEngine.py catalog (runs its tests, because this PR changes the workflow)	still running when this was written
the same job's steps before the tests: set up the job, start PostgreSQL, check out full history, decide	about 20 s

On a pull request that doesn't touch the listed paths, the full-catalog job stops after those first steps (about 20 s), so the check finishes when the runtime job does, in about 2 to 5 minutes. That saves about 5 to 33 minutes per run, 8 at the median.

Environment variables and secrets

Variable	Read by	Provider	Wiring
CODECOV_TOKEN	both jobs' Codecov upload	GitHub Actions secret, both a repository secret and an organization secret (visibility: all repositories); the repository secret takes precedence	pr.yml and push.yml pass it explicitly (secrets: CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}) to the reusable workflow, which declares it as an optional workflow_call secret. Scheduled and manual runs start the workflow directly, so secrets.CODECOV_TOKEN reads the secret itself. Unchanged for catalog-runtime-integration.
V2_MIGRATION_DATABASE_URL, V2_ALEMBIC_DISPOSABLE_TEST, UV_PYTHON	full-catalog-integration	literals in the job's env:; the URL points at the job's own postgres:17 service	job scope
RUN_V2_CATALOG_COMPATIBILITY, RUN_V2_CATALOG_PUBLICATION_QUALIFICATION	the test step	literals in the step's env:	step scope
EVENT_NAME, BASE_SHA	the decision step	the github context (github.event_name, github.event.pull_request.base.sha || github.event.before), which in a called workflow is the caller's event	step scope; empty on scheduled and manual runs, which always run the tests
GITHUB_OUTPUT	the decision step	the runner	set by the runner for every step

No other variable is read. Neither job uses pr.yml's or push.yml's workflow-level env: (OLD_SIMULATION_GATEWAY_URL), which GitHub doesn't pass to called workflows. That was already true before this change.

Testing

Run individually, by file or node ID (no full suite):

• tests/unit/test_decide_full_catalog_tests_script.py (35 tests: each listed path, similar paths that don't match, a move out of the catalog directory, a deletion, a test merge commit whose base changed uv.lock, a push, scheduled and manual runs, a missing or all-zero base, a base missing from the clone, required variables)
• tests/unit/test_full_catalog_integration_workflow.py, tests/unit/v2/test_catalog_value_shapes.py
• tests/unit/test_alembic_workflows.py::test_reusable_v2_integration_check_uses_disposable_databases_and_coverage (coverage counts updated: two jobs, one coverage file each)
• tests/unit/test_locked_test_environments.py::test_detects_every_known_test_job (EXPECTED_TEST_JOBS now lists full-catalog-integration)
• tests/integration/test_v2_catalog_publication.py::test_publication_stores_every_value_shape_unchanged against a local postgres:17; it also fails when one expected value is changed from 12000.0 to 12000
• both full-catalog tests in one process, as the new job runs them, against a local postgres:17 on macOS. The qualification test passed in 29.6 s with the shared catalog, staying under the 256 MB limit. The installed-interface test counted 29,361 parameter nodes on macOS instead of the 29,367 it pins. This change only caches the return value of extract_installed_catalog(), so master counts the same on that machine; the job on Linux in this PR's run is what counts
• python scripts/run_quality_guards.py (all guards pass, including ci-scripts with shellcheck) and python -m scripts.rules render (no changes; the [[rule.ci]] entries for pr.yml:v2-integration-check and v2-integration-check.yml:catalog-runtime-integration still match)

publish-git-tag's require-checks, pr.yml and push.yml are unchanged: both callers still call one reusable workflow under the job ID v2-integration-check.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.15%. Comparing base (c4e3a92) to head (7ee2746).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #4150       +/-   ##
===========================================
+ Coverage   50.84%   88.15%   +37.30%     
===========================================
  Files         270      286       +16     
  Lines       15221    16659     +1438     
  Branches     2443     2733      +290     
===========================================
+ Hits         7739    14685     +6946     
+ Misses       6841     1297     -5544     
- Partials      641      677       +36     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.