Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
068572b
Test read-only models of the Public API's input tables (failing)
anth-volk Nov 10, 2026
03822b7
Model the Public API's input tables read-only, and compare their columns
anth-volk Nov 10, 2026
6be02ef
Show how the household functions apply each reform period form in 2200
anth-volk Nov 10, 2026
b22c25a
Test reading a stored policy as the reform values a calculation appli…
anth-volk Nov 10, 2026
50bcd88
Read a stored policy as the reform values a calculation applies
anth-volk Nov 10, 2026
ac61072
Test the household job writes and reads the simulation service makes …
anth-volk Nov 10, 2026
b0d142a
Create, record and fail household jobs as the simulation service's role
anth-volk Nov 10, 2026
3dfbee9
Test the household job store and a transaction that limits its statem…
anth-volk Nov 10, 2026
d675b87
Run household job reads and writes in bounded transactions of their own
anth-volk Nov 10, 2026
f2a2778
Test the household path's settings, worker lookup and household conve…
anth-volk Nov 10, 2026
c4fca54
Give the household path its own v2 manifest settings, worker lookup a…
anth-volk Nov 10, 2026
65a3078
Test submitting and reading household calculations (failing)
anth-volk Nov 10, 2026
7964b03
Submit a household calculation as a household job, and read the job back
anth-volk Nov 10, 2026
3f56df4
Test the household calculation routes and their OpenAPI answers (fail…
anth-volk Nov 10, 2026
537ac7e
Serve POST /household-calculations and GET /household-calculations/{id}
anth-volk Nov 10, 2026
9833b50
Test that every deploy gives the entrypoint its Modal token and v2 ma…
anth-volk Nov 10, 2026
85b7b1f
Test a household calculation on every staging deploy (failing)
anth-volk Nov 10, 2026
6a57f58
Mount the Modal token and v2 manifest settings on every deploy, and c…
anth-volk Nov 10, 2026
471c5c7
Merge c1-1/household-functions (#903's one-policy household contract)…
anth-volk Nov 10, 2026
03e996e
Send the household function one policy, as the household contract now…
anth-volk Nov 10, 2026
24a5b00
Check the household routes' generated client, and describe the change
anth-volk Nov 10, 2026
4d81b0f
Test the household request and answer shape the Public API sends and …
anth-volk Nov 10, 2026
84dea74
Accept axes, refuse a null spm, and name the household job job_id
anth-volk Nov 10, 2026
3df1678
Merge c1-1/household-functions at 58cf2344 (#903's newest head) into …
anth-volk Nov 10, 2026
e89dd79
Test that open-ended policy values follow .py's own rule (failing)
anth-volk Nov 10, 2026
77602a8
Stop an open-ended policy value where the parameter's next value star…
anth-volk Nov 10, 2026
b3af4cf
Test the late Modal answer, the failed failure write and internal ref…
anth-volk Nov 10, 2026
c96c236
Answer a late-confirmed household call as it stands, and name interna…
anth-volk Nov 10, 2026
d569b45
Merge origin/main (#903 merged as 560b104e) into v2plan/c1-2-househol…
anth-volk Nov 10, 2026
855b579
Test JSON 500s, a pending answer for an unknown job state, and same-d…
anth-volk Nov 10, 2026
f0e3062
Answer every household route 500 as JSON, a lost job state as pending…
anth-volk Nov 10, 2026
9f844ac
Test that the policy reader gives .py's periods for same-day values, …
anth-volk Nov 10, 2026
163bc18
Give .py's periods for same-day policy values, keeping bounded ones i…
anth-volk Nov 10, 2026
fcea9fd
Log household route failures as a fixed event with their frames, and …
anth-volk Nov 10, 2026
696ee3e
Test that the household 500 log's emitted record keeps its stack fram…
anth-volk Nov 11, 2026
a6f261a
Log a household route failure's stack frames as one string within the…
anth-volk Nov 11, 2026
73dcafc
Test each policy period through the reader, and a bounded value takin…
anth-volk Nov 11, 2026
f2a8073
Build each policy period in the reader alone, and say policyengine-uk…
anth-volk Nov 11, 2026
60bb070
Describe how the observability runtime shortens a long string attribute
anth-volk Nov 11, 2026
d53d74c
Test that the household 500 log keeps the service's frames before a l…
anth-volk Nov 11, 2026
114de18
Keep the service's frames in the household 500 log, with shorter paths
anth-volk Nov 11, 2026
5fe2a58
Say which open-ended values the policy reader leaves out after 2200
anth-volk Nov 11, 2026
a9c8dea
Count the staging household check among the jobs that declare the rel…
anth-volk Nov 11, 2026
f6e5dc1
Say which jobs read the release environment without ordering them by …
anth-volk Nov 11, 2026
5fc12f3
Start the staging household scripts with #!/bin/bash, like every othe…
anth-volk Nov 11, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions .github/scripts/deploy-cloud-run-simulation-entry.sh
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,8 @@ required_variables=(
OLD_GATEWAY_AUTH_CLIENT_ID_VALUE
OLD_GATEWAY_AUTH_CLIENT_SECRET_SECRET_NAME
SIMULATION_SERVICE_DATABASE_URL_SECRET_NAME
MODAL_TOKEN_ID_SECRET_NAME
MODAL_TOKEN_SECRET_SECRET_NAME
OBSERVABILITY_SERVICE_NAMESPACE
OBSERVABILITY_TRACE_PROJECT_ID
OTEL_EXPORTER_OTLP_ENDPOINT
Expand Down Expand Up @@ -144,13 +146,17 @@ add_runtime_secret OLD_GATEWAY_AUTH_CLIENT_SECRET \
# API's tables, on every deploy whatever the Stage 12 setting.
add_runtime_secret SIMULATION_SERVICE_DATABASE_URL \
"${SIMULATION_SERVICE_DATABASE_URL_SECRET_NAME}"
# The Modal token the household routes call the v2 workers' household
# functions with, on every deploy whatever the Stage 12 setting. The Modal
# client reads it from MODAL_TOKEN_ID and MODAL_TOKEN_SECRET, so the Stage 12
# comparison path, in the same process, uses this mount too.
add_runtime_secret MODAL_TOKEN_ID "${MODAL_TOKEN_ID_SECRET_NAME}"
add_runtime_secret MODAL_TOKEN_SECRET "${MODAL_TOKEN_SECRET_SECRET_NAME}"

if [[ "${DEPLOY_STAGE12_V2}" == "true" ]]; then
stage12_variables=(
STAGE12_ENABLED_VALUE
STAGE12_ARTIFACT_BUCKET_VALUE
MODAL_TOKEN_ID_SECRET_NAME
MODAL_TOKEN_SECRET_SECRET_NAME
STAGE12_DATABASE_URL_SECRET_NAME
)
for variable_name in "${stage12_variables[@]}"; do
Expand All @@ -162,8 +168,6 @@ if [[ "${DEPLOY_STAGE12_V2}" == "true" ]]; then
|| fail "STAGE12_ARTIFACT_BUCKET_VALUE is invalid"
validate_secret_reference "${STAGE12_DATABASE_URL_SECRET_NAME}"

add_stage12_runtime_secret MODAL_TOKEN_ID "${MODAL_TOKEN_ID_SECRET_NAME}"
add_stage12_runtime_secret MODAL_TOKEN_SECRET "${MODAL_TOKEN_SECRET_SECRET_NAME}"
add_stage12_runtime_secret STAGE12_DATABASE_URL "${STAGE12_DATABASE_URL_SECRET_NAME}"
fi

Expand All @@ -189,6 +193,8 @@ jq -n '
OTEL_METRICS_EXPORTER: "otlp",
OTEL_TRACES_SAMPLER_ARG: "1.0",
POLICYENGINE_OTEL_GOOGLE_AUDIENCE: env.POLICYENGINE_OTEL_GOOGLE_AUDIENCE,
SIMULATION_V2_MODAL_ENVIRONMENT: env.MODAL_ENVIRONMENT,
SIMULATION_V2_MANIFEST_NAME: "simulation-api-v2-version-manifest",
STAGE12_ENABLED: (
if env.DEPLOY_STAGE12_V2 == "true" then env.STAGE12_ENABLED_VALUE else "0" end
)
Expand Down
112 changes: 112 additions & 0 deletions .github/scripts/simulation-staging-household-check.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
#!/bin/bash
# Calculate the fixed staging household through a deployed Simulation
# Entrypoint, and require its household job to succeed.
# Usage: simulation-staging-household-check.sh <base-url>
# Required env vars: STAGING_HOUSEHOLD_ID (the ID of a US household stored in
# the staging Public API, from the beta variable
# SIMULATION_STAGING_HOUSEHOLD_ID), and the test client's
# SIMULATION_TEST_AUTH_ISSUER, SIMULATION_TEST_AUTH_AUDIENCE,
# SIMULATION_TEST_AUTH_CLIENT_ID and SIMULATION_TEST_AUTH_CLIENT_SECRET.
# Optional: HOUSEHOLD_CHECK_DEADLINE_SECONDS (600) and
# HOUSEHOLD_CHECK_POLL_SECONDS (5).
# The access token is sent from a file and the client secret on standard
# input, so neither appears on a command line.

set -euo pipefail

fail() {
printf 'Staging household check failed: %s\n' "$1" >&2
exit 1
}

base_url="${1:?Simulation Entrypoint base URL is required}"
base_url="${base_url%/}"
household_id="${STAGING_HOUSEHOLD_ID:-}"
deadline_seconds="${HOUSEHOLD_CHECK_DEADLINE_SECONDS:-600}"
poll_seconds="${HOUSEHOLD_CHECK_POLL_SECONDS:-5}"
uuid_pattern='^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'

[[ -n "${household_id}" ]] \
|| fail "STAGING_HOUSEHOLD_ID is empty: set the beta environment variable SIMULATION_STAGING_HOUSEHOLD_ID to the ID of a US household stored in the staging Public API"
[[ "${household_id}" =~ ${uuid_pattern} ]] \
|| fail "STAGING_HOUSEHOLD_ID is not a UUID in lowercase hexadecimal"
for variable_name in SIMULATION_TEST_AUTH_ISSUER SIMULATION_TEST_AUTH_AUDIENCE \
SIMULATION_TEST_AUTH_CLIENT_ID SIMULATION_TEST_AUTH_CLIENT_SECRET; do
[[ -n "${!variable_name:-}" ]] || fail "${variable_name} must be set"
done
[[ "${deadline_seconds}" =~ ^[0-9]+$ ]] \
|| fail "HOUSEHOLD_CHECK_DEADLINE_SECONDS must be a whole number of seconds"
[[ "${poll_seconds}" =~ ^[0-9]+$ ]] \
|| fail "HOUSEHOLD_CHECK_POLL_SECONDS must be a whole number of seconds"

work_directory="$(mktemp -d)"
trap 'rm -rf "${work_directory}"' EXIT
chmod 700 "${work_directory}"

# Send one request; write its body to <name>.json and print its HTTP status.
send() {
local name="$1"
shift
curl --silent --show-error --output "${work_directory}/${name}.json" \
--write-out '%{http_code}' "$@"
}

token_status="$(
jq -n \
--arg client_id "${SIMULATION_TEST_AUTH_CLIENT_ID}" \
--arg client_secret "${SIMULATION_TEST_AUTH_CLIENT_SECRET}" \
--arg audience "${SIMULATION_TEST_AUTH_AUDIENCE}" \
'{client_id: $client_id, client_secret: $client_secret, audience: $audience, grant_type: "client_credentials"}' \
| send token --request POST --header 'Content-Type: application/json' \
--data @- "${SIMULATION_TEST_AUTH_ISSUER%/}/oauth/token"
)"
[[ "${token_status}" == "200" ]] || fail "the token request answered ${token_status}"
access_token="$(jq -r '.access_token // empty' "${work_directory}/token.json")"
[[ -n "${access_token}" ]] || fail "the token response has no access_token"
authorization_header="${work_directory}/authorization-header"
printf 'Authorization: Bearer %s\n' "${access_token}" > "${authorization_header}"

submission_status="$(
jq -n --arg household_id "${household_id}" \
'{country: "us", household_id: $household_id}' \
| send submission --request POST --header "@${authorization_header}" \
--header 'Content-Type: application/json' --data @- \
"${base_url}/household-calculations"
)"
[[ "${submission_status}" == "202" ]] \
|| fail "POST /household-calculations answered ${submission_status}: $(cat "${work_directory}/submission.json")"
job_id="$(jq -r '.job_id // empty' "${work_directory}/submission.json")"
[[ "${job_id}" =~ ${uuid_pattern} ]] || fail "the submission named no household job"
printf 'Submitted household job %s for staging household %s\n' "${job_id}" "${household_id}"

started_at="${SECONDS}"
while true; do
job_status_code="$(
send job --header "@${authorization_header}" \
"${base_url}/household-calculations/${job_id}"
)"
job_body="${work_directory}/job.json"
if [[ "${job_status_code}" == "200" ]]; then
job_status="$(jq -r '.status' "${job_body}")"
case "${job_status}" in
succeeded)
[[ "$(jq -r '.result.status // empty' "${job_body}")" == "succeeded" ]] \
|| fail "household job ${job_id} succeeded without a succeeded result document"
printf 'Household job %s succeeded on .py %s with %s\n' "${job_id}" \
"$(jq -r '.policyengine_version' "${job_body}")" \
"$(jq -r '.country_package_version' "${job_body}")"
exit 0
;;
failed)
fail "household job ${job_id} failed: $(jq -r '.error_message // "no error message"' "${job_body}")"
;;
pending | running) ;;
*) fail "household job ${job_id} has an unknown status: ${job_status}" ;;
esac
elif [[ "${job_status_code}" != "503" ]]; then
fail "GET /household-calculations/${job_id} answered ${job_status_code}: $(cat "${job_body}")"
fi
((SECONDS - started_at < deadline_seconds)) \
|| fail "household job ${job_id} did not finish within ${deadline_seconds} seconds"
sleep "${poll_seconds}"
done
13 changes: 13 additions & 0 deletions .github/scripts/simulation-staging-household-ready.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#!/bin/bash
# Require the staging household check to have succeeded in a staging
# deployment, and to have been skipped in any other.
# Required env vars: REQUESTED ("true" in a staging deployment) and
# CHECK_RESULT (the staging_household_check job result).

set -euo pipefail

if [[ "${REQUESTED}" == "true" ]]; then
test "${CHECK_RESULT}" = "success"
else
test "${CHECK_RESULT}" = "skipped"
fi
43 changes: 42 additions & 1 deletion .github/workflows/simulation-deploy.reusable.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1215,6 +1215,41 @@ jobs:
MODAL_ENVIRONMENT: ${{ inputs.modal_environment }}
run: ../../.github/scripts/stage12-v2-default-restore.sh

staging_household_check:
name: Calculate the staging household through the candidate
# Every staging deployment calculates the stored staging household that
# the beta variable SIMULATION_STAGING_HOUSEHOLD_ID names, through the
# candidate's household routes, and requires its household job to
# succeed. The routes use only the v2 manifest's default worker, so the
# check runs after the default has moved to the release's worker (or, in
# a deployment without Stage 12, with the default as it is). It calls the
# tagged candidate URL, which serves this release's revision whether or
# not it was promoted. The job's call writes a household_jobs row in the
# staging database, which the household function completes.
if: >-
${{
!cancelled() &&
inputs.deployment_environment == 'staging' &&
needs.deploy_entrypoint.result == 'success' &&
needs.authenticated_test.result == 'success' &&
((inputs.deploy_stage12_v2 && needs.publish_stage12_v2_default.result == 'success') ||
(!inputs.deploy_stage12_v2 && needs.publish_stage12_v2_default.result == 'skipped'))
}}
needs: [deploy_entrypoint, authenticated_test, publish_stage12_v2_default]
runs-on: ubuntu-latest
environment: ${{ inputs.release_environment }}
steps:
- uses: actions/checkout@v6

- name: Calculate the staging household and require it to succeed
env:
STAGING_HOUSEHOLD_ID: ${{ vars.SIMULATION_STAGING_HOUSEHOLD_ID }}
SIMULATION_TEST_AUTH_ISSUER: ${{ secrets.SIMULATION_ENTRYPOINT_AUTH_ISSUER }}
SIMULATION_TEST_AUTH_AUDIENCE: ${{ secrets.SIMULATION_ENTRYPOINT_AUTH_AUDIENCE }}
SIMULATION_TEST_AUTH_CLIENT_ID: ${{ secrets.SIMULATION_ENTRYPOINT_TEST_AUTH_CLIENT_ID }}
SIMULATION_TEST_AUTH_CLIENT_SECRET: ${{ secrets.SIMULATION_ENTRYPOINT_TEST_AUTH_CLIENT_SECRET }}
run: .github/scripts/simulation-staging-household-check.sh "${{ needs.deploy_entrypoint.outputs.candidate_url }}"

finalize_routing:
name: Finalize the Modal routing publication
# Finish the durable journal after every gate, including failed uploads
Expand Down Expand Up @@ -1273,7 +1308,7 @@ jobs:
# The reporter only reads job results. Journal completion happens before
# it, so a failure or cancellation here cannot strand a publication.
if: ${{ !cancelled() }}
needs: [deploy_entrypoint, update_routing, integration, authenticated_test, promote_entrypoint, publish_latest, finalize_routing, publish_stage12_v2_default]
needs: [deploy_entrypoint, update_routing, integration, authenticated_test, promote_entrypoint, publish_latest, finalize_routing, publish_stage12_v2_default, staging_household_check]
runs-on: ubuntu-latest
defaults:
run:
Expand Down Expand Up @@ -1302,6 +1337,12 @@ jobs:
DEFAULT_RESULT: ${{ needs.publish_stage12_v2_default.result }}
run: .github/scripts/stage12-v2-default-ready.sh

- name: Check the staging household check result
env:
REQUESTED: ${{ inputs.deployment_environment == 'staging' }}
CHECK_RESULT: ${{ needs.staging_household_check.result }}
run: .github/scripts/simulation-staging-household-ready.sh

cleanup_entrypoint_revisions:
name: Remove superseded Cloud Run revisions
if: ${{ !cancelled() && inputs.promote_entrypoint && needs.deployment_ready.result == 'success' }}
Expand Down
1 change: 1 addition & 0 deletions changelog.d/c1-2-household-calculations.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Added `POST /household-calculations` and `GET /household-calculations/{household_job_id}` to the Simulation Entrypoint. A submission names a household, and optionally a policy, stored in the Public API, with optional axes and SPM settings; the service reads them, creates a `household_jobs` row for the household's default year on the v2 version manifest's default worker, and starts its household function's call, answering 202 with the job's `job_id` and status, or 400, 422, 500 or 503 with the reason. A 202 `pending` may also be a job whose call Modal didn't confirm and whose state couldn't then be read: poll it rather than submitting again. A read returns the job's status, result document and error message. A stored policy value with no end date applies to the end of 2200. Every deploy now gives the entrypoint the Modal token and the v2 manifest's Modal environment, and every staging deploy calculates the household that the `beta` variable `SIMULATION_STAGING_HOUSEHOLD_ID` names.
5 changes: 5 additions & 0 deletions docs/engineering/skills/database-access.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,11 @@ and this repository has no migration chain of its own
`household_jobs`), modelled in `libs/policyengine-simulation-persistence`.
The service connects as its own role, which policyengine-api's privilege
manifest grants.
- the policyengine-api tables a calculation is read from (`households`,
`policies`, `parameters`, `parameter_values`, `tax_benefit_models` and
`tax_benefit_model_versions`), which the simulation service only reads.
Their read-only models in that library's `input_tables.py` name only the
columns the service reads, apart from the models of the tables it writes.
- the run tables the v2 Modal coordinator, workers, household functions and
stuck-run sweep write as the Modal worker role: `report_runs`, `simulation_runs`, the
result tables and `household_jobs`. Their mappings in
Expand Down
2 changes: 1 addition & 1 deletion docs/generated/rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ flowchart LR
| Rule | Statement | Tier | Status | Blocking | Enforced by | CI | Proof |
| --- | --- | --- | --- | --- | --- | --- | --- |
| `sim.persistence.missing-privileges-reported` | The simulation service's privilege check, which its readiness check uses, names every table privilege and every column UPDATE privilege its role needs on the Public API's tables and lacks, as PostgreSQL reports it, counts a table missing from the public schema as lacking all of them and a missing column as lacking its own, and names a missing USAGE privilege on that schema. | mechanical | enforced | yes | pytest `libs/policyengine-simulation-persistence/tests/test_access.py` | `pr.yml:simulation-database` | yes |
| `sim.persistence.models-match-public-api` | Sim's models of the Public API's tables match the tables the Public API's v2 migration chain builds at the pinned Public API commit, which is on the Public API's default branch: every column's type, nullability and default, every constraint and index by name and definition, and every enum's values; and the simulation service's role, with the privileges the Public API's manifest declares, writes every modelled table. | mechanical | enforced | yes | pytest `libs/policyengine-simulation-persistence/tests/test_public_api_schema.py`, `libs/policyengine-simulation-persistence/tests/test_service_writes.py` | `pr.yml:public-api-schema`, `public-api-schema-drift.yml:public-api-schema-drift` | yes |
| `sim.persistence.models-match-public-api` | Sim's models of the Public API's tables match the tables the Public API's v2 migration chain builds at the pinned Public API commit, which is on the Public API's default branch: every column's type, nullability and default, every constraint and index by name and definition, and every enum's values; every column its read-only models of the Public API's input tables name exists there with the same type and nullability; and the simulation service's role, with the privileges the Public API's manifest declares, writes every modelled table. | mechanical | enforced | yes | pytest `libs/policyengine-simulation-persistence/tests/test_public_api_schema.py`, `libs/policyengine-simulation-persistence/tests/test_service_writes.py` | `pr.yml:public-api-schema`, `public-api-schema-drift.yml:public-api-schema-drift` | yes |

```mermaid
flowchart LR
Expand Down
Loading
Loading