Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 0 additions & 8 deletions .github/scripts/deploy-cloud-run-simulation-entry.sh
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,6 @@ required_variables=(
SIMULATION_SERVICE_DATABASE_URL_SECRET_NAME
MODAL_TOKEN_ID_SECRET_NAME
MODAL_TOKEN_SECRET_SECRET_NAME
V2_SIMULATION_OUTPUT_BUCKET_VALUE
OBSERVABILITY_SERVICE_NAMESPACE
OBSERVABILITY_TRACE_PROJECT_ID
OTEL_EXPORTER_OTLP_ENDPOINT
Expand All @@ -129,12 +128,6 @@ esac
|| fail "TAG is invalid"
[[ "${ENTRYPOINT_RUNTIME_SERVICE_ACCOUNT}" =~ ^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.gserviceaccount\.com$ ]] \
|| fail "ENTRYPOINT_RUNTIME_SERVICE_ACCOUNT is invalid"
# The bucket that keeps v2 simulation output files for good. The service
# reads it into its settings and uses it for nothing (a report joins its
# sides' simulation runs by version set); it never reads the bucket. The
# setting and this check go with the run-reuse search.
[[ "${V2_SIMULATION_OUTPUT_BUCKET_VALUE}" =~ ^[a-z0-9][a-z0-9._-]{1,61}[a-z0-9]$ ]] \
|| fail "V2_SIMULATION_OUTPUT_BUCKET_VALUE is invalid"
[[ "${ENTRYPOINT_MIN_INSTANCES}" =~ ^[0-9]+$ ]] \
|| fail "ENTRYPOINT_MIN_INSTANCES must be a non-negative integer"
[[ "${ENTRYPOINT_MAX_INSTANCES}" =~ ^[0-9]+$ ]] \
Expand Down Expand Up @@ -202,7 +195,6 @@ jq -n '
POLICYENGINE_OTEL_GOOGLE_AUDIENCE: env.POLICYENGINE_OTEL_GOOGLE_AUDIENCE,
SIMULATION_V2_MODAL_ENVIRONMENT: env.MODAL_ENVIRONMENT,
SIMULATION_V2_MANIFEST_NAME: "simulation-api-v2-version-manifest",
V2_SIMULATION_OUTPUT_BUCKET: env.V2_SIMULATION_OUTPUT_BUCKET_VALUE,
STAGE12_ENABLED: (
if env.DEPLOY_STAGE12_V2 == "true" then env.STAGE12_ENABLED_VALUE else "0" end
)
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/simulation-deploy.reusable.yml
Original file line number Diff line number Diff line change
Expand Up @@ -247,7 +247,6 @@ jobs:
STAGE12_DATABASE_URL_SECRET_NAME: ${{ vars.STAGE12_DATABASE_URL_SECRET_NAME }}
MODAL_TOKEN_ID_SECRET_NAME: ${{ vars.MODAL_TOKEN_ID_SECRET_NAME }}
MODAL_TOKEN_SECRET_SECRET_NAME: ${{ vars.MODAL_TOKEN_SECRET_SECRET_NAME }}
V2_SIMULATION_OUTPUT_BUCKET_VALUE: ${{ vars.V2_SIMULATION_OUTPUT_BUCKET }}
run: .github/scripts/deploy-cloud-run-simulation-entry.sh

- name: Resolve exact candidate metadata
Expand Down
1 change: 1 addition & 0 deletions changelog.d/1081.removed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The Simulation Entrypoint's deployment no longer sets `V2_SIMULATION_OUTPUT_BUCKET` on its Cloud Run service or requires the `beta` and `prod` GitHub variable of that name for the entrypoint's deploy step; the service never used it. The Stage 12 deployment steps still pass it to the Modal coordinator and workers, which write simulation output files to that bucket. The simulation persistence library drops the create functions that use no definition keys, the search for an earlier succeeded simulation run to reuse, and the `reuse` parameters of the economy store's input reads, none of which the service has called since reports, simulations and runs are found or created by key. Requests and answers are unchanged.
31 changes: 19 additions & 12 deletions docs/migration/stage-12-modal-worker-foundation.md
Original file line number Diff line number Diff line change
Expand Up @@ -288,12 +288,15 @@ creates or claims a `stage12_evaluation_*` record. Only comparison-path calls,
which pass an evaluation record, write those, until the comparison path is
removed. The household functions work the same way on `household_jobs`.

**A report run.** The simulation service creates, in one transaction, the
`simulations` and `reports` rows, a `simulation_runs` row for the baseline and
one for the reform (pending, with no `modal_call_id`; or a run that already
succeeded, to reuse it), and the `report_runs` row naming both (pending, with no
`modal_call_id`). It then starts the manifest's `report_coordinator_callable`
(`coordinate_report`) with two arguments and no evaluation record:
**A report run.** The simulation service finds or creates by definition key,
in one transaction, the `simulations` and `reports` rows, a `simulation_runs`
row for the baseline and one for the reform (pending, with no `modal_call_id`;
or a run of the same simulation on the same versions that it joins, which may
have succeeded already), and the `report_runs` row naming both (pending, with
no `modal_call_id`, or a run on the same versions that it joins). For a report
run it created, it then starts the manifest's `report_coordinator_callable`
(`coordinate_report`) with two arguments and no evaluation record; a joined
report run gets no call:

- the report input, a `ReportExecutionInput` whose `evaluation_id` is the report
run's ID and whose baseline and reform `simulation_execution_id`s are their
Expand All @@ -313,7 +316,7 @@ run row naming other simulation runs, fails the report run with
`report_run_input_mismatch`. If a worker it starts finds its run held by a
worker an earlier attempt started, which Modal never cancelled, the coordinator
waits for that worker instead. It uses the stored output of each simulation run
that already succeeded (see the reuse of succeeded runs above), fails the
that already succeeded (see the joined runs above), fails the
report run with the run's own recorded failure if a run it must calculate
already failed (`simulation_run_failed` if it recorded none), and starts a
worker for each other one with a context naming only that
Expand All @@ -322,8 +325,10 @@ the report run with its result rows. It returns
`{"report_run_id", "status"}`, or the same marked `deduplicated` without running
when another call holds the report run or it has finished.

**A simulation run on its own.** The service creates the `simulations` row and
its `simulation_runs` row (pending, no call), starts the country's
**A simulation run on its own.** The service finds or creates by definition
key the `simulations` row and its `simulation_runs` row (pending, no call, or
a run on the same versions that it joins, which gets no call). For a run it
created, it starts the country's
`single_simulation_callable` with a `SimulationExecutionInput` (role
`standalone`, no output plan, `evaluation_id` and `simulation_execution_id`
both the run's ID) and a context naming the run in `simulation_run_id`, with
Expand Down Expand Up @@ -394,9 +399,11 @@ window of 1 to 75 consecutive years. It is an economy report whose
the year, with its own simulation runs of that year, so a year can reuse a
single-year report's runs, and the reverse.

- **Creating and starting.** The simulation service creates every year's runs
in one transaction, all pending with no call, then starts the earliest
years within one ten-second wait for Modal, counted from the first start;
- **Creating and starting.** The simulation service finds or creates the
window and its window run by key, in one transaction; a window run it
creates names every year's report run, each on its year's simulation runs,
found or created, and every run it creates is pending with no call. For a
window run it created, it then starts the earliest years within one ten-second wait for Modal, counted from the first start;
a later year not confirmed within it is deferred (`YearStartDeferred`)
and waits for a started year's coordinator. The contract's
`year_calculation` derives each year's coordinator
Expand Down
81 changes: 29 additions & 52 deletions libs/policyengine-simulation-persistence/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,32 +83,16 @@ runs too long and the limit never stays on a pooled connection.

## Economy simulations and reports

`economy_run_records.py` holds the simulation service's reads and writes for
an economy report, or a standalone economy simulation. It reads the
country's model (`read_model_id`) and a catalog version's region by code
(`read_region`) and dataset by name or ID (`read_dataset`,
`read_dataset_by_id`). `insert_economy_report` creates, in the caller's
transaction, each side's new simulation and first run, the report naming
both simulations and the reform's policy with its options in `inputs`, and
its first report run, trigger `initial`; every run is `pending` with no
Modal call ID. A side may instead name a run that already succeeded
(`ReportSimulation.reused`), for which nothing is created.
`insert_economy_simulation` creates a standalone simulation and its first
run. Both create unkeyed rows, as `insert_budget_window_report` and
`insert_household_report` do; the Simulation Entrypoint calls none of them
now, creating every record by key ("Shared simulations, reports and runs"
below), and tests use them to store records created before keys. A row
naming one that does not exist is refused by its foreign key with
SQLSTATE `23503`; `EconomyRecordReferenceError` names the table and column.

`find_reusable_run` returns the run a new report's side may reuse: the
newest, by completion, of the same economy simulation (policy, dynamic,
catalog version, dataset, region and year) that succeeded on the same
country package and .py versions, whose output file is under the output
bucket's `gs://<bucket>/`, that a succeeded report run with the same options
used, and that no report run refused to reuse (an error message starting
`reused_simulation_`). So a submission after a refused reuse calculates the
simulation again.
`economy_run_records.py` holds the simulation service's catalog reads and
run writes for an economy report, or a standalone economy simulation. It
reads the country's model (`read_model_id`) and a catalog version's region
by code (`read_region`) and dataset by name or ID (`read_dataset`,
`read_dataset_by_id`), and builds a simulation run `pending` with no Modal
call ID (`simulation_run_row`, `insert_simulation_run`). The Simulation
Entrypoint creates every simulation, report and run by key ("Shared
simulations, reports and runs" below). A row naming one that does not exist
is refused by its foreign key with SQLSTATE `23503`;
`EconomyRecordReferenceError` names the table and column.

`record_report_run_call` and `record_simulation_run_call` record a Modal
call's ID, and `fail_unstarted_report_run` and
Expand All @@ -119,9 +103,9 @@ with its newest run's state.

`EconomyRunStore` runs them as the Simulation Entrypoint does, each step in
a transaction of its own: `read_report_inputs` reads the catalog version,
model, region and dataset, each side's policy and dynamic values, and each
side's reusable run together, never giving both sides the same run;
`read_simulation_inputs` does the same for one simulation; the writes and
model, region and dataset, and each side's policy and dynamic values
together; `read_simulation_inputs` does the same for one simulation, and
`read_budget_window_inputs` for a budget window's two sides; the writes and
reads each take their own. A missing region or dataset raises
`UnknownRegionError` or `UnknownDatasetError`, and a report side's refused
policy or dynamic raises `ReportSideError`, naming the side. A create whose
Expand Down Expand Up @@ -157,7 +141,7 @@ report and run rows (C1 · 24). These modules do it:
weren't forced, outside budget windows), then a read of the row that
won, retried when that run fails in between (`SharedRowNotSettledError`
after three tries). A forced run is inserted directly and never looked
up, joined or offered for reuse.
up or joined.
- `keyed_economy_records.py` runs a report's find-or-create in one
transaction: each side's simulation by key, in key order; the report by
key; the report's joinable run on the current versions, joined with the
Expand Down Expand Up @@ -190,23 +174,23 @@ report and run rows (C1 · 24). These modules do it:

## Household reports

`household_report_records.py` holds the simulation service's writes for a
household report: one stored household, for one year, under a baseline and,
optionally, a reform. `insert_household_report` creates, in the caller's
transaction, a household simulation (no dataset or region) and its first
run for each side, never reusing one; the report, of type `household`,
naming the household, both simulations and the reform's policy, with its
options in `inputs`; and its first report run, trigger `initial`, naming no
reform run for a report of the baseline alone. Every run is `pending` with
no Modal call ID, and a row naming one that does not exist is refused as
`EconomyRecordReferenceError`.
A household report calculates one stored household, for one year, under a
baseline and, optionally, a reform. Its report has type `household`
(`HOUSEHOLD_REPORT_TYPE`) and names the household, and each side is a
household simulation (no dataset or region) naming the side's policy.
`keyed_household_records.py` finds or creates the report, its simulations
and its run by key, as an economy report's are (C1 · 24's answer 8);
`household_report_records.py` builds each side's run, `pending` with no
Modal call ID (`household_simulation_run_row`,
`insert_household_simulation_run`). A row naming one that does not exist
is refused as `EconomyRecordReferenceError`.

`HouseholdReportStore` runs the steps in transactions of their own:
`read_inputs` reads the catalog version for the .py version, the country's
model, the stored household and each side's policy values together,
raising `UnknownCatalogVersionError`, `UnknownHouseholdError` or
`ReportSideError` for the side whose policy is missing or sets parameters
the catalog version lacks; `create_report` raises
the catalog version lacks; `find_or_create_report` raises
`EconomyRecordOutcomeUnknownError` when the commit fails after the rows were
inserted; and `record_report_call` and `fail_unstarted_report` use the
economy report run writes.
Expand All @@ -223,11 +207,6 @@ version, dataset and region it calculates on.
record calculates: its kind (`economy`, `budget_window`, `household` or
`unsupported`), country, year, its region's code and dataset's name, and
each simulation's policy and dynamic.
- `find_reusable_run_of` searches one simulation's runs, by its ID, under
`find_reusable_run`'s conditions on a run (versions, the run's own catalog
version, dataset, region and year, the output file, a succeeded report run
with the same options, no refused reuse, not forced); it doesn't compare the
simulation row, which keeps the catalog rows it was created on.
- `insert_report_rerun` joins the report's run on the re-run's versions
that hasn't failed and wasn't forced (pending, running or succeeded),
creating nothing, or else creates the report's new run, naming no
Expand All @@ -239,8 +218,8 @@ version, dataset and region it calculates on.
a transaction-level advisory lock on the report's ID, so concurrent
re-runs of one report wait for each other and end on one run. No re-run
is refused while a run is under way. A forced one (`NewReportRerun.forced`) takes no lock, always
creates a new run of each side, marked `forced`, and joins none. No side
of a report's re-run names a reused run. `insert_simulation_rerun` joins or creates the
creates a new run of each side, marked `forced`, and joins none.
`insert_simulation_rerun` joins or creates the
simulation's run on the re-run's versions, catalog version and year
through `shared_rows`, with no lock. Each returns the run, its status and
whether it created it. When the database refuses the new run's key to the
Expand Down Expand Up @@ -274,10 +253,8 @@ in a transaction of its own and returns the run; the store's `read_inputs`
reads the re-run's household and policies as it does a new report's.

`EconomyRunStore.read_report_rerun_inputs` reads a re-run's catalog rows by
the stored region code and dataset name, and each side's values, with, given
`reuse`, its reusable run of its own simulation, never giving both sides the
same run; the service passes none, as its re-runs join or force their runs.
`read_window_rerun_inputs` does the same for each year of a window.
the stored region code and dataset name, and each side's values.
`read_window_rerun_inputs` does the same for a window's two sides.
`create_report_rerun`, `create_window_rerun` and `create_simulation_rerun`
run the inserts in a transaction of their own.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,8 @@
read_missing_table_privileges,
)
from policyengine_simulation_persistence.budget_window_records import (
BudgetWindowYear,
NewBudgetWindowReport,
WindowSide,
WindowYearRuns,
insert_budget_window_report,
)
from policyengine_simulation_persistence.definition_keys import (
KEY_VERSION,
Expand Down Expand Up @@ -57,18 +54,10 @@
CatalogRegion,
EconomyPopulation,
EconomyRecordReferenceError,
NewEconomyReport,
NewEconomySimulation,
ReportSimulation,
ReusableRun,
RunVersions,
SimulationChoice,
fail_unstarted_report_run,
fail_unstarted_simulation_run,
find_reusable_run,
find_reusable_run_of,
insert_economy_report,
insert_economy_simulation,
read_dataset,
read_dataset_by_id,
read_model_id,
Expand All @@ -86,7 +75,6 @@
EconomySimulationInputs,
ReportSide,
ReportSideError,
RunReuse,
SideRefusal,
SimulationInputs,
UnknownDatasetError,
Expand Down Expand Up @@ -117,9 +105,6 @@
)
from policyengine_simulation_persistence.household_report_records import (
HOUSEHOLD_REPORT_TYPE,
HouseholdReportSimulation,
NewHouseholdReport,
insert_household_report,
)
from policyengine_simulation_persistence.household_report_store import (
HouseholdReportInputs,
Expand Down Expand Up @@ -197,7 +182,6 @@
"SERVICE_TABLE_PRIVILEGES",
"SERVICE_UPDATABLE_COLUMNS",
"BudgetWindowInputs",
"BudgetWindowYear",
"CatalogDataset",
"CatalogRegion",
"EconomyCatalogRows",
Expand Down Expand Up @@ -229,7 +213,6 @@
"HouseholdJobStore",
"HouseholdReportDefinition",
"HouseholdReportInputs",
"HouseholdReportSimulation",
"HouseholdReportStore",
"HouseholdRerunSide",
"HouseholdSimulationDefinition",
Expand All @@ -239,11 +222,7 @@
"KeyedHouseholdReport",
"KeyedWindowYear",
"MissingPolicyParametersError",
"NewBudgetWindowReport",
"NewEconomyReport",
"NewEconomySimulation",
"NewHouseholdJob",
"NewHouseholdReport",
"NewHouseholdReportRerun",
"NewReportRerun",
"NewSimulationRerun",
Expand All @@ -258,11 +237,8 @@
"ReportRunTrigger",
"ReportSide",
"ReportSideError",
"ReportSimulation",
"RerunRecordDeletedError",
"RerunSide",
"ReusableRun",
"RunReuse",
"RunVersions",
"SharedRowNotSettledError",
"SideDefinition",
Expand Down Expand Up @@ -303,15 +279,9 @@
"find_or_insert_simulation",
"find_or_insert_simulation_run",
"find_or_insert_simulation_runs",
"find_reusable_run",
"find_reusable_run_of",
"household_jobs",
"input_metadata",
"insert_budget_window_report",
"insert_economy_report",
"insert_economy_simulation",
"insert_household_job",
"insert_household_report",
"insert_household_report_rerun",
"insert_report_rerun",
"insert_simulation_rerun",
Expand Down
Loading
Loading