Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
eb8e42f
Test household calculation inputs, axis limits and pre-submission che…
anth-volk Nov 10, 2026
9218248
Add household calculation inputs, axis limits and pre-submission checks
anth-volk Nov 10, 2026
0fe120c
Test reading household calculations from household_jobs (failing)
anth-volk Nov 10, 2026
dba90da
Read household calculations from household_jobs through the v2 layers
anth-volk Nov 10, 2026
2599d22
Test submitting a household calculation after the Public API's checks…
anth-volk Nov 10, 2026
b19c779
Submit a household calculation only after the Public API's checks
anth-volk Nov 10, 2026
bb5f574
Test the simulation service submission of household calculations (fai…
anth-volk Nov 10, 2026
a905ae3
Submit household calculations to the simulation service behind a prot…
anth-volk Nov 10, 2026
6d3db8a
Test the /v2/household-calculations routes and their error answers (f…
anth-volk Nov 10, 2026
4b24d1b
Serve POST and GET /v2/household-calculations
anth-volk Nov 10, 2026
b13d762
Test household calculations' route group, request log and contracts (…
anth-volk Nov 10, 2026
291e230
Log household calculations' sources and add them to the migration con…
anth-volk Nov 10, 2026
7af81ff
Test reading a succeeded job's single-policy result document (failing)
anth-volk Nov 10, 2026
365fd86
Read a succeeded job's single-policy result document
anth-volk Nov 10, 2026
161cf4b
Test recording SPM availability at publication and serving it (failing)
anth-volk Nov 10, 2026
1f7b701
Record each catalog version's SPM availability and serve it as spm
anth-volk Nov 10, 2026
d35e697
Test household calculations live against the staging candidate (failing)
anth-volk Nov 10, 2026
188832b
Run the household calculation checks after the other live staging tests
anth-volk Nov 10, 2026
f687a6f
Document household calculations and add a changelog fragment
anth-volk Nov 10, 2026
ed03c25
Name the household functions' unexpected-error messages in the error …
anth-volk Nov 10, 2026
1fe8057
Merge master into v2plan/c1-3-household-calculations after #4259 and …
anth-volk Nov 10, 2026
813ca7c
Test submitting through the database-backed services (failing)
anth-volk Nov 10, 2026
e000885
Answer 503 when the policy check's catalog query fails
anth-volk Nov 10, 2026
e559ef7
Test the simulation submission's IDs, sim's codes and an unreadable 2…
anth-volk Nov 10, 2026
cfc7460
Send the observability ID to sim, name sim's codes, and answer 502 fo…
anth-volk Nov 10, 2026
9ca15ae
Test which answers the live household checks send again (failing)
anth-volk Nov 10, 2026
c6b7cc0
Send a live household submission again only after a 503 or a failed c…
anth-volk Nov 10, 2026
76b5e07
Test reporting SPM availability publication left unknown (failing)
anth-volk Nov 10, 2026
3ecd5f8
Report SPM availability publication left unknown, and serve it as unk…
anth-volk Nov 10, 2026
37f0c91
Test that only version 1 result documents are read (failing)
anth-volk Nov 10, 2026
cfd8be9
Read household result documents only at contract version 1
anth-volk Nov 10, 2026
c74667c
Name the generated client's household modules and document the new an…
anth-volk Nov 10, 2026
75afad4
Test that a 500 from the simulation service is answered 502 (failing)
anth-volk Nov 10, 2026
2be97a7
Answer the simulation service's 500 with 502
anth-volk Nov 10, 2026
ad1e18d
Test that the 202 returns the observability ID sent to sim (failing)
anth-volk Nov 10, 2026
f497ec5
Return the observability ID sent to sim on the 202
anth-volk Nov 10, 2026
031b249
Test listing only versions still unrecorded after a publication (fail…
anth-volk Nov 10, 2026
86d3b1a
List as not recorded only versions still unrecorded after a publication
anth-volk Nov 10, 2026
77d13fa
Test that a live submission isn't sent again once it may have arrived…
anth-volk Nov 10, 2026
51daa7d
Send a live submission again only after a 503 or a connection never made
anth-volk Nov 10, 2026
1582923
Test lost answers, observability IDs on errors and quiet SPM output (…
anth-volk Nov 10, 2026
fe0d0e3
Answer a lost answer 502, name the observability ID on every answer a…
anth-volk Nov 10, 2026
7e74679
Test the SPM warning, proxy answers and transport-error headers (fail…
anth-volk Nov 10, 2026
f4bd858
Warn only when SPM availability stays unrecorded, and answer proxy 50…
anth-volk Nov 10, 2026
b2f587e
Merge remote-tracking branch 'origin/master' into v2plan/c1-3-househo…
anth-volk Nov 10, 2026
f51e658
Test that every 502 asks the caller to submit the household calculati…
anth-volk Nov 11, 2026
200b527
Ask the caller to submit the household calculation again after a 502
anth-volk Nov 11, 2026
ad91764
Name every 502 case in the submit docstrings, and say why they aren't…
anth-volk Nov 11, 2026
5e8ebbf
Merge remote-tracking branch 'origin/master' into v2plan/c1-3-househo…
anth-volk Nov 11, 2026
a10a733
Test that the parallel live check's own clients send the CI server cr…
anth-volk Nov 11, 2026
40e55db
Send the CI server credential from the parallel live check's own clients
anth-volk Nov 11, 2026
f995a64
Test that the credentialed client is built where the api_client fixtu…
anth-volk Nov 11, 2026
cf05901
Build the api_client fixture's client and the parallel check's client…
anth-volk Nov 11, 2026
1bccf66
Say the parallel check opens one client per thread, and state the red…
anth-volk Nov 11, 2026
e324a2b
Test that unexpected statuses answer 502 and that sim's 500 and 503 t…
anth-volk Nov 11, 2026
756dddd
Answer unexpected statuses 502, and keep sim's 500 and 503 text from …
anth-volk Nov 11, 2026
e04a04e
Test that a 503 without sim's code is a proxy's, and that codes are r…
anth-volk Nov 11, 2026
a9e3175
Answer a 503 without sim's code as a proxy's, and read codes apart fr…
anth-volk Nov 11, 2026
9f5291c
Test that a deeply nested or non-object body reads as unreadable, not…
anth-volk Nov 11, 2026
3dc1f8b
Parse each answer once with pydantic, so no body can turn a 502 or 50…
anth-volk Nov 11, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .github/scripts/run_live_staging_tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,19 @@ uv run --frozen --extra dev python -m pytest \
tests/integration/test_live_calculate.py \
tests/integration/test_live_economy.py \
-v

# The household calculation checks run afterwards, on 4 workers, so they never
# share the candidate with the tests above. They call the simulation service's
# household calculation route, so they pass only once that route is deployed
# to staging; staging promotion waits on them, so the change that adds them
# merges only after that deployment. Their requests are short (a
# submission, then a poll every few seconds) apart from the parity tests'
# /calculate requests, but the parallel check submits 6 calculations at once:
# with the 3 other workers' requests, at most 3 wait for a free slot, as above.
uv run --frozen --extra dev python -m pytest \
-n 4 \
--dist load \
--maxschedchunk=1 \
tests/integration/test_live_v2_household_calculations.py \
tests/integration/test_live_v2_household_parity.py \
-v
4 changes: 3 additions & 1 deletion .github/workflows/push.yml
Original file line number Diff line number Diff line change
Expand Up @@ -643,7 +643,9 @@ jobs:
parallel-live-simulation-tests-staging-cloud-run:
name: Run live simulations against the staging candidate
runs-on: ubuntu-latest
timeout-minutes: 20
# The household calculation checks run after the other live tests and
# wait on Modal.
timeout-minutes: 30
needs: deploy-cloud-run-staging
if: |
!cancelled()
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/seed-v2-database.yml
Original file line number Diff line number Diff line change
Expand Up @@ -73,3 +73,6 @@ jobs:
# Reads the certified UK data release, whose repository is private,
# to check each new catalog version's region filters.
HUGGING_FACE_TOKEN: ${{ secrets.PE_UK_PRIVATE_HF_READ_TOKEN }}
# Each version's SPM availability comes from this environment's
# simulation service's public versions answer.
SIMULATION_ENTRYPOINT_URL: ${{ secrets.SIMULATION_ENTRYPOINT_URL }}
6 changes: 3 additions & 3 deletions .github/workflows/v2-integration-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,15 +74,15 @@ jobs:
- name: Verify the managed v2 roles hold their declared privileges
run: uv run --frozen --extra dev python scripts/reconcile_v2_privileges.py verify
- name: Test v2 metadata publication, resource routes and the countries list
run: uv run --frozen --extra dev coverage run --branch -m pytest -q tests/integration/test_v2_catalog_publication.py tests/integration/test_v2_catalog_publication_time_periods.py tests/integration/test_v2_catalog_publication_pruning.py tests/integration/test_v2_metadata_routes.py tests/integration/test_v2_country_routes.py
run: uv run --frozen --extra dev coverage run --branch -m pytest -q tests/integration/test_v2_catalog_publication.py tests/integration/test_v2_catalog_publication_time_periods.py tests/integration/test_v2_catalog_publication_pruning.py tests/integration/test_v2_catalog_spm_availability.py tests/integration/test_v2_metadata_routes.py tests/integration/test_v2_country_routes.py
- name: Test v2 policy persistence and immediate v1 mirroring
run: uv run --frozen --extra dev coverage run -a --branch -m pytest -q tests/integration/test_v2_policy_persistence.py tests/integration/test_v1_policy_dual_write.py tests/integration/test_mysql_policy_dual_write.py tests/integration/test_v2_user_policy_mirroring.py tests/integration/test_v1_user_policy_dual_write.py
- name: Test signed-in identity provisioning, user associations and user types
run: uv run --frozen --extra dev coverage run -a --branch -m pytest -q tests/integration/test_v2_user_identity_persistence.py tests/integration/test_v2_user_type_persistence.py
- name: Test comparison-run persistence, the simulation API's roles and privilege reconciliation as the managed roles
run: uv run --frozen --extra dev coverage run -a --branch -m pytest -q tests/integration/test_v2_stage12_comparison_persistence.py tests/integration/test_v2_simulation_role_privileges.py tests/integration/test_v2_privilege_reconciliation.py
- name: Test v2 household persistence and immediate v1 copying
run: uv run --frozen --extra dev coverage run -a --branch -m pytest -q tests/integration/test_v2_household_persistence.py tests/integration/test_v1_household_dual_write.py tests/integration/test_mysql_household_spm_dual_write.py
- name: Test v2 household persistence, immediate v1 copying and household calculation reads
run: uv run --frozen --extra dev coverage run -a --branch -m pytest -q tests/integration/test_v2_household_persistence.py tests/integration/test_v1_household_dual_write.py tests/integration/test_mysql_household_spm_dual_write.py tests/integration/test_v2_household_calculation_reads.py
- name: Test the row counts v1 ORM updates report on MySQL
run: uv run --frozen --extra dev coverage run -a --branch -m pytest -q tests/integration/test_mysql_update_row_counts.py
# The real US and UK models, reduced to a few parameter folders: the
Expand Down
1 change: 1 addition & 0 deletions changelog.d/v2-household-calculations.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add `POST /v2/household-calculations`, which calculates a stored household through the simulation service after checking the country, the household and policy, the household's default year and the policy's parameters, and `GET /v2/household-calculations/{calculation_id}`, which reads the calculation's status, result and error. `/v2/model-catalog` now says whether each catalog version offers SPM settings, as `spm`.
18 changes: 14 additions & 4 deletions docs/engineering/migration-contracts.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ Generated from `policyengine_api/migration_registry.py` and `tests/contract/regi
| Metric | Count |
| --- | ---: |
| route group count | 9 |
| workflow count | 16 |
| request count | 90 |
| workflow count | 17 |
| request count | 92 |
| db entity count | 6 |
| sim flow count | 3 |

Expand All @@ -20,7 +20,7 @@ Generated from `policyengine_api/migration_registry.py` and `tests/contract/regi
| `specification` | `specification` | `none` | `none` |
| `metadata` | `metadata`, `countries`, `datasets`, `economy-options`, `model-catalog`, `parameter-values`, `parameters`, `regions`, `tax-benefit-model-versions`, `tax-benefit-models`, `variables` | `metadata` | `none` |
| `policy` | `policy`, `policies`, `user-policy`, `user-policies` | `policy` | `none` |
| `household` | `household`, `households`, `user-household`, `user-households`, `calculate`, `calculate-full` | `household` | `household` |
| `household` | `household`, `households`, `household-calculations`, `user-household`, `user-households`, `calculate`, `calculate-full` | `household` | `household` |
| `economy` | `economy` | `simulation` | `economy` |
| `simulation` | `simulation`, `simulations`, `user-simulations` | `simulation` | `economy` |
| `report` | `report`, `user-reports`, `shared-reports` | `report` | `report` |
Expand Down Expand Up @@ -88,6 +88,16 @@ Generated from `policyengine_api/migration_registry.py` and `tests/contract/regi
| `GET` | `/v2/households/{household_id}?country_id=us` | 200 | `household` | none | `status`, `message`, `result.item.id`, `result.item.country_id`, `result.item.default_year`, `result.item.household_data`, `result.item.created_at`, `result.item.updated_at` |
| `GET` | `/v2/households?country_id=us` | 200 | `household` | none | `status`, `message`, `result.items`, `result.offset`, `result.limit`, `result.has_more` |

### `household_calculations_v2`

- Current contract: `typed_v2_resources`
- Future owner: Stored-household calculations through the simulation service

| Method | Path | Status | Route group | Required headers | Stable response fields |
| --- | --- | ---: | --- | --- | --- |
| `POST` | `/v2/household-calculations` | 202 | `household` | none | `status`, `message`, `result.item.calculation_id`, `result.item.status`, `result.item.country_id`, `result.item.household_id`, `result.item.policy_id`, `result.item.year` |
| `GET` | `/v2/household-calculations/{calculation_id}` | 200 | `household` | none | `status`, `message`, `result.item.calculation_id`, `result.item.country_id`, `result.item.household_id`, `result.item.policy_id`, `result.item.year`, `result.item.catalog_version_id`, `result.item.country_package_version`, `result.item.policyengine_version`, `result.item.status`, `result.item.created_at`, `result.item.started_at`, `result.item.completed_at`, `result.item.result`, `result.item.error` |

### `user_household_associations_v2`

- Current contract: `typed_v2_resources`
Expand Down Expand Up @@ -213,7 +223,7 @@ Generated from `policyengine_api/migration_registry.py` and `tests/contract/regi
| `GET` | `/v2/parameters/search-file?country_id=us` | 200 | `metadata` | none | `format`, `country_id`, `policyengine_version`, `parameters`, `other_parameters`, `nodes` |
| `GET` | `/v2/tax-benefit-models/by-country/us` | 200 | `metadata` | none | `status`, `message`, `result.policyengine_version`, `result.model`, `result.model_version` |
| `GET` | `/v2/economy-options?country_id=us` | 200 | `metadata` | none | `status`, `message`, `result.policyengine_version`, `result.current_law_id`, `result.region`, `result.time_period`, `result.datasets` |
| `GET` | `/v2/model-catalog?country_id=us` | 200 | `metadata` | none | `status`, `message`, `result.policyengine_version`, `result.catalog_format`, `result.entities`, `result.basic_inputs`, `result.regions`, `result.datasets`, `result.economy_options` |
| `GET` | `/v2/model-catalog?country_id=us` | 200 | `metadata` | none | `status`, `message`, `result.policyengine_version`, `result.catalog_format`, `result.entities`, `result.basic_inputs`, `result.regions`, `result.datasets`, `result.economy_options`, `result.spm` |
| `GET` | `/v2/model-catalog/variables?country_id=us` | 200 | `metadata` | none | `status`, `message`, `result.policyengine_version`, `result.catalog_format`, `result.variables`, `result.part_parameter_values` |
| `GET` | `/v2/model-catalog/parameters?country_id=us&name=gov.irs.income.bracket.rates.1` | 200 | `metadata` | none | `status`, `message`, `result.policyengine_version`, `result.catalog_format`, `result.parameters`, `result.node_labels`, `result.missing_names` |

Expand Down
30 changes: 29 additions & 1 deletion docs/engineering/skills/alembic-migrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,9 @@ check the declarations:
`policyengine_stage12_test`, holding only their declared privileges.
`tests/integration/test_v2_simulation_role_privileges.py` runs the
simulation API's writes as its two roles; their code lives in the simulation
API's repository.
API's repository. `tests/integration/test_v2_household_calculation_reads.py`
reads the household jobs they write through the household calculation
service as `policyengine_v2_runtime`.
- `api.v2.deployed-privileges-match`: every deploy runs
`scripts/reconcile_v2_privileges.py apply` right after the schema upgrade,
then `verify`, before the catalog is seeded.
Expand Down Expand Up @@ -321,6 +323,7 @@ When a pull request adds or changes a v2 table, or a connector's use of one:
tests/integration/test_v2_catalog_publication.py \
tests/integration/test_v2_catalog_publication_pruning.py \
tests/integration/test_v2_simulation_role_privileges.py \
tests/integration/test_v2_household_calculation_reads.py \
tests/integration/test_v2_privilege_reconciliation.py
```

Expand Down Expand Up @@ -797,3 +800,28 @@ The previous Public API revision never uses the table. The downgrade drops it
with every recorded change. `test_user_type_changes_revision_adds_only_its_table`
in `tests/integration/test_alembic_v2_user_types.py` checks that nothing else
changes.

### API v2 catalog SPM availability

Revision `0c136a278f29` (parent `970be5d29283`) was autogenerated, with no
post-generation corrections, after `tax_benefit_model_versions` gained the
nullable JSON column `spm_availability`. It was generated on the saved-item
keys revision `81b728a088e7`; when `master` gained the user types revisions
on the same parent, its `down_revision` moved to `970be5d29283`, and
generating it again from that revision gives the same operations. Catalog publication records it for
each catalog version of the installed PolicyEngine.py version that has none
yet, after publishing the catalog
(`policyengine_api/data/v2/catalog/spm_availability.py`): whether the
version offers SPM settings, from the simulation service's public
`GET /versions` (at `SIMULATION_ENTRYPOINT_URL`), with the settings' JSON
schema and the certified defaults. A US version the simulation service does
not serve yet, and every version when its answer cannot be read, stays
unrecorded for the next publication run. `/v2/model-catalog` serves it as
`spm`, and a version without it reports `available` null, unknown, which a
recorded absence (`available` false) is not. The catalog writer role already
holds `UPDATE` on the table, so no privilege changes.

The revision writes no data, and the previous Public API revision, which
doesn't map the column, keeps working at it. The downgrade drops the column
with its values, which the next publication run of each PolicyEngine.py
version records again.
8 changes: 5 additions & 3 deletions docs/engineering/skills/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -443,9 +443,11 @@ avoid depending on specific production data fixtures. The deployed-API tests
send the CI server credential from `CI_SERVER_ACCESS_TOKEN` through the
`api_client` fixture, and fail without it: a client-credentials token of a
client on that environment's `PUBLIC_API_SERVER_CLIENT_IDS`
(`tests/integration/deployed_api_credentials.py`). Only refusal checks use
`uncredentialed_api_client`, which sends no credential. In push.yml each job that
sets `API_BASE_URL` requests one with
(`tests/integration/deployed_api_credentials.py`). A check that needs clients
of its own, such as one per thread of a parallel check, builds them with that
module's `deployed_api_client`, which also builds the fixture's client. Only
refusal checks use `uncredentialed_api_client`, which sends no credential. In
push.yml each job that sets `API_BASE_URL` requests one with
`.github/scripts/request_ci_server_token.sh`, and
`tests/unit/test_ci_server_credential_workflow.py` fails for a job that
doesn't:
Expand Down
5 changes: 3 additions & 2 deletions docs/engineering/skills/v2-code-organization.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,8 +128,9 @@ A resource may also add a `process_state.py` module for process-wide state its
service reuses across requests, such as the metadata package's (each catalog
version a request has selected; after publication its row changes only in
its time-period options, its `catalog_last_published_at`, which no read uses,
and its `catalog_values_pruned_at`, which the service checks again before or
after the reads of the rows pruning deletes). Such a module holds plain data, constructs no
its `catalog_values_pruned_at`, which the service checks again before or
after the reads of the rows pruning deletes, and its `spm_availability`,
which publication records once after publishing it). Such a module holds plain data, constructs no
SQL and owns no session; the composition code creates one instance per process
and injects it into the service.

Expand Down
57 changes: 54 additions & 3 deletions docs/generated/migration_contracts.json
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
{
"metadata": {
"db_entity_count": 6,
"request_count": 90,
"request_count": 92,
"route_group_count": 9,
"sim_flow_count": 3,
"workflow_count": 16
"workflow_count": 17
},
"route_groups": [
{
Expand Down Expand Up @@ -62,6 +62,7 @@
"path_segments": [
"household",
"households",
"household-calculations",
"user-household",
"user-households",
"calculate",
Expand Down Expand Up @@ -485,6 +486,55 @@
}
]
},
{
"current_contract": "typed_v2_resources",
"future_owner_pr": "Stored-household calculations through the simulation service",
"name": "household_calculations_v2",
"requests": [
{
"expected_status": 202,
"headers": [],
"method": "POST",
"path": "/v2/household-calculations",
"route_group": "household",
"stable_response_fields": [
"status",
"message",
"result.item.calculation_id",
"result.item.status",
"result.item.country_id",
"result.item.household_id",
"result.item.policy_id",
"result.item.year"
]
},
{
"expected_status": 200,
"headers": [],
"method": "GET",
"path": "/v2/household-calculations/{calculation_id}",
"route_group": "household",
"stable_response_fields": [
"status",
"message",
"result.item.calculation_id",
"result.item.country_id",
"result.item.household_id",
"result.item.policy_id",
"result.item.year",
"result.item.catalog_version_id",
"result.item.country_package_version",
"result.item.policyengine_version",
"result.item.status",
"result.item.created_at",
"result.item.started_at",
"result.item.completed_at",
"result.item.result",
"result.item.error"
]
}
]
},
{
"current_contract": "typed_v2_resources",
"future_owner_pr": "PR 12A: User Login and v1-Anchored User Associations",
Expand Down Expand Up @@ -1613,7 +1663,8 @@
"result.basic_inputs",
"result.regions",
"result.datasets",
"result.economy_options"
"result.economy_options",
"result.spm"
]
},
{
Expand Down
Loading
Loading