Repository navigation
Re-run a stored economy, budget-window or household report, or a simulation, on the current .py version - #1059
Merged
Merged
Conversation
…log version (failing) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ent catalog version A re-run keeps the record and every earlier run. It reads the record's region code, dataset name and each simulation's policy and dynamic, so the current catalog version's rows can be found; a report side may reuse a run of its own simulation that succeeded on the current versions, compared on the run's own catalog version, dataset and region. Nothing is created while the record has a run pending or running, checked under a transaction-level advisory lock on the record's ID. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…/{id}/runs and POST /simulations/{id}/runs (failing)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s and POST /simulations/{id}/runs
A re-run creates the stored record's new run on the v2 version manifest's
default worker and starts it by run ID, as a new record's first run is
started; every earlier run is kept. A report's re-run reuses a run of each
side's own simulation that succeeded on the current versions unless force
is set, and its run's trigger is manual. A simulation's re-run is always
calculated. A record with a run pending or running, a budget-window or
household record, or one missing a row it calculates on is answered 409;
an unknown record 404.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…log fragment Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… definition reads (failing) The entry tests fail: a record deleted after a re-run's reads should be answered 404, and a row deleted meanwhile should be refused naming the report. The persistence tests cover existing behaviour and pass at once: the reuse search's catalog-version condition on the run row, and the household, filtered, other-model and other-type definition reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ored record's A record deleted after a re-run read it is answered 404, and any other row the database refuses meanwhile is named as the report's or simulation's. The economy submission service is generic over its store, so the re-run service needs no constructor of its own; the re-run records share the economy report type; the docs say how the caller-row codes apply to a re-run, that the report body is required, and how the in-progress check differs between a report and a simulation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s own error (failing) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d re-run record apart insert_report_rerun and insert_simulation_rerun raise RerunRecordDeletedError when the database refuses the new run's key to the record itself, so the re-run service answers 404 without naming tables or columns. The docs say a report deleted with its simulations answers 400 missing_reference, and the re-run routes' 400 names every code that refers to the stored record's rows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, and when a deleted report answers 404 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in the budget windows (#1054). Both sides' imports are kept; the call-refused answer is public under main's name with the re-run keyword; the report route dispatches budget windows as main does, with the public answer helpers. The budget-window service is generic over its store, and the re-run service extends it, so the app builds one service that creates, re-runs and starts budget windows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ion (failing) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s (failing) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A budget window's re-run creates a budget-window run (trigger manual, recalculate set by force) with a report run per year, each year on the simulations that year's report run of the report's newest window run names, reusing a run of them that succeeded on the current versions unless forced. Its first years start as a new window's do: the budget-window service's start of the first years now takes the window's IDs. A report re-run, of either kind, is refused while a budget-window run or a report run of the report is pending or running, and a window whose newest run lacks a year answers 409 rerun_not_supported. The re-run answers with the budget-window create's shape. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fail: a window re-run checks its years are its window's, and a year naming no reform is left out of a window's definition. Pass at once, covering existing behaviour: a window re-run of a deleted report or year simulation, one refused while a year still runs after its window ended, and the budget-window store starting only a report's later window run's years. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s years A window re-run's dataclass checks its years as a new window's does, with the same check; a window's definition leaves out a year naming no reform, so the service needs no copy of each year. A new window and a re-run share the per-year reuse search, the year's report run row and the year's answer; the budget-window service reads its store by one name. The docs say which run a 409 run_in_progress names, that up to five years start at once, and that a deleted budget-window report always answers 404. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ad, and test the insert order The shared year row and year answer take a protocol of the fields they read, so the new-window modules no longer name the re-run's year type. A new test, which passes at once, shows a window re-run of a report deleted with a year's simulation is refused as a deleted report, since its window run is inserted first. The years check, the window years read and the README say what they do in full. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts, each kept from both sides: - POST /reports keeps main's household-report dispatch and the branch's public answer helpers, which the re-run routes use. - The app builds the re-run service and the household report service and adds the re-run routes after the economy routes. - The 422 codes gain household_not_calculable beside the 409 re-run codes. - The persistence package exports both sides' records, and each README keeps both sections. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The re-run's report-kind check kept its own copy of the type that #1057's household report records now export as HOUSEHOLD_REPORT_TYPE. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…runs (failing) - Persistence: a household report's definition names its household; its re-run is a manual report run with a new run of each side's own household simulation, on the current catalog version and the report's year; it is refused while a run is under way, or by its key for a deleted report or simulation. - Service: the re-run reads the stored household and policies, creates the runs and starts one household function call; force changes nothing; the refusals are located at the stored report. - Route, OpenAPI and generated client: the re-run's 201 may be HouseholdReportCreated. - The tests that replace economy_submissions_for accept the household service the app now passes it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lation
POST /reports/{report_id}/runs now re-runs a household report instead of
answering 409 rerun_not_supported:
- Persistence: a report's definition names its household.
insert_household_report_rerun creates a manual report run and a new run of
each side's own household simulation, on the current catalog version and
the report's year, under the re-run lock and the run-under-way check the
other re-runs use. It shares the household simulation run insert with
#1057's create, and HouseholdReportStore.create_report_rerun runs it.
- Service: the re-run service hands a household report to
HouseholdReportRerunService, which extends #1057's household report
service. It reads the stored household and policies, builds the payload
with household_report_calculation in the report's year, creates the runs
and starts one household function call through the household service's
own read, create and start, now shared rather than copied. Refusals are
located at the stored report; force changes nothing, since household runs
are never reused.
- Route: the 201 may be HouseholdReportCreated, and the answers declare the
household codes. The app builds one household report service, for
POST /reports and the re-run.
- A household simulation still answers 409; it is re-run through its report.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A household report whose side names a dynamic is refused (409), since a household calculation applies none (fails until the fix). - The app passes its household report service to the re-run service, and the configured builder makes a service that re-runs. - A persistence test comment no longer suggests the restart check covers household rows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…re the answer - A household report's re-run refuses a side naming a dynamic (409 rerun_not_supported) rather than calculating it without one. - household_report_created builds the 201 answer for a household report's create and re-run; HouseholdReportCreated's description covers a re-run's answer (the report's own simulations, in the report's year). - call_refused moves to the submission errors module, beside the other refusal helpers, so the household report service no longer imports the economy submission service. - The re-run's declared 400 names a household that no longer exists once, without repeating that other codes are the service's state. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fore 503 (failing) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hold 409s - The re-run service refuses a household side naming a dynamic with its other household 409s, before the 503 a service without household reports answers, so a report that can never be re-run is not answered as one to retry. - The household re-run protocol lives beside its service, keeping the re-run service module under the 800-line limit. - The re-run's declared 400, the errors module's summary and the entry README's 409 list read plainly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The entry README's refusal paragraph is wrapped like the rest of the file, and it, the re-run service's and the app's docstrings say a household report answers 503 without household reports only when it could be re-run. StoredHouseholdReport says neither side names a dynamic. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk
added a commit
to PolicyEngine/policyengine-api
that referenced
this pull request
Nov 13, 2026
The simulation service's re-run of a household report (PolicyEngine/policyengine-sim-api#1059) gives each side a new run of its own household simulation, so force changes nothing for it. The route, request model, service and re-run input texts say so, and the changelog no longer says the simulation service refuses household re-runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk
marked this pull request as ready for review
November 13, 2026 18:51
anth-volk
added a commit
that referenced
this pull request
Nov 13, 2026
…runs Conflicts kept both sides: the persistence library's README holds the re-run section and the restart section, its exports list both sides' names, and `_start_first_years`'s docstring names both the restart check and `rerun`. The merged `economy_run_store.py` was 801 lines, over the 800-line limit, so `read_simulation_reform`, which this branch added, moves unchanged to `policy_reform.py`, beside the policy and dynamic readers it combines. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1056
Summary
The simulation service re-runs a stored economy, budget-window or household report, or an economy simulation, on the current .py version (the v2 version manifest's default worker), as a new run started by run ID, and keeps every earlier run. A report's re-run can force a recalculation of both simulations.
POST /reports/{report_id}/runs(simulation:submit), body{"force": false}. The service reads what the report calculates (its country, year, region code, dataset name, options and each simulation's policy and dynamic), finds the rows with that region code and dataset name in the catalog version of the default worker's .py version, and reads each stored policy and dynamic on it. It then creates, in one transaction, the report's new run (triggermanual) and a new run of each side's own simulation, and starts the coordinator with the run IDs, asPOST /reportsdoes (Coordinate economy runs through the run tables instead of the evaluation records #1051's coordination). Withoutforce, a side reuses its own simulation's newest run that succeeded on the current versions, under the same conditions as a new report's reuse;forcecalculates both simulations again. It answers 201 with the create's fields, naming the new run.budget_window_runsrow (triggermanual,recalculateset byforce) with a report run per year. Each year's sides are the simulations that year's report run of the report's newest window run names (a year that reused another report's run names that report's simulation), each reusing a run of them unless forced. The first years start through the budget-window service's own start (_start_first_years,_YearStarts): up to five at once under one shared wait for Modal. It answers 201 withBudgetWindowReportCreated.mainmerged in at20b48ddc): the same route re-runs one in the report's own year (not the household's currentdefault_year), under each side's stored policy read on the current catalog version. Each side gets a new run of its own household simulation, recording that catalog version; household runs are never reused, soforcechanges nothing. The new report run (triggermanual) is started by one household function call with the payload Create household reports through POST /reports and calculate them in one household function call #1057'shousehold_report_calculationbuilds, and the re-run answers 201 withHouseholdReportCreated. The route's 201 is the union of the three report answers. The household function claims and records a re-run's runs as it does a new report's.POST /simulations/{simulation_id}/runs(simulation:submit), no body: a stored economy simulation's new run, always calculated, started on the country's simulation function.reportsorsimulations, so a report keeps its simulations; each new run records the catalog version, dataset and region it calculates on.unknown_report/unknown_simulation, also for a record deleted between the re-run's read and its insert (the persistence library raisesRerunRecordDeletedErrorwhen the database refuses the new run's key to the record); 409run_in_progress, naming the run inrun_id(a report's budget-window run, or else its report run; a simulation's run), while the record has a run pending or running (checked under a transaction-level advisory lock on the record's ID, so two simultaneous re-runs of one record create one run); 409rerun_not_supportedfor a household simulation (re-run only through its report), a budget window whose newest window run lacks a year, or one naming no simulation, region, dataset, household, year or country the service calculates; and the create's 400 and 422 codes, worded for the stored record (a 422 reason'slocstarts withreportorsimulation), with a household report's own: 400unknown_householdandno_household_function, and 422household_not_calculableat the report'shousehold_idor a side'spolicy_id. Without household reports configured, a household report's re-run answers 503not_configured. A report's re-run checks only the report's own runs, so a side whose simulation has a run under way for another report still gets a run of its own; a simulation's re-run checks all of its runs. When Modal refuses the new run's call, the run is failed and the 503modal_call_refusednames the record and the run.find_reusable_runshares its run conditions withfind_reusable_run_of, which searches one simulation's runs and compares the catalog version on the run row. The household re-run service extends Create household reports through POST /reports and calculate them in one household function call #1057's household report service (now generic over its store), whose input read, row creation and call start take what names the rows and the run IDs; a household report's re-run and create share the household simulation run insert; the app builds one household report service forPOST /reportsand the re-run.POST /reportsandPOST /simulationsanswer as before.docs/migration/stage-12-modal-worker-foundation.mddescribe re-runs. The changelog fragments arechangelog.d/c1-17-reruns.added.md,changelog.d/c1-17-budget-window-reruns.added.mdandchangelog.d/c1-17-household-reruns.added.md.Environment variables and secrets
None added or changed. The re-run routes use the economy routes' existing settings, which #1052 wired and this change doesn't touch:
SIMULATION_SERVICE_DATABASE_URL: a Secret Manager secret named by the GitHub variableSIMULATION_SERVICE_DATABASE_URL_SECRET_NAME, attached as a Cloud Run runtime secret bydeploy-cloud-run-simulation-entry.sh.SIMULATION_V2_MODAL_ENVIRONMENTandSIMULATION_V2_MANIFEST_NAME: set as Cloud Run environment variables by the same script, from the job's Modal environment and a fixed manifest name.V2_SIMULATION_OUTPUT_BUCKET: the GitHub environment variable of the same name, passed to the script asV2_SIMULATION_OUTPUT_BUCKET_VALUEbysimulation-deploy.reusable.yml.A household report's re-run uses the household report service #1057 configured, from the same database URL, Modal environment and manifest name; this change adds no setting to it.
Database grants
None needed. A re-run inserts into
simulation_runs,report_runsandbudget_window_runs(the service role holdsSELECT, INSERTon each), readsreports,simulations,regions,datasets,tax_benefit_modelsandtax_benefit_model_versions(SELECT), and records or fails the new run's call through the column-levelUPDATEs the creates already use; a window re-run's first years start through the budget-window store's advance, which writes only the window columns #1054's Public API grant gives the service. A household report's re-run reads the household and policies as #1057's create does, and inserts householdsimulation_runsand areport_runsrow with the same columns. The advisory lock needs no privilege.Testing
Each test this PR adds or changes, by node ID:
masterwith its privileges reconciled: the 23test_economy_reruns.pytests (reading a stored report's and simulation's definition, reuse of a simulation's own run on a later catalog version, the re-run inserts and their refusal while a run is under way, the advisory lock held until the transaction ends, the re-run reads on the current catalog version) and the 11test_economy_window_reruns.pytests (a window's newest window run's years, the window re-run's rows, years check and refusals, each year's reusable runs). The existing reuse, records, store and budget-window record tests that use the reworked reuse search, the shared window code and the test row helpers were run by node ID too.test_economy_reruns.py(service, store and Modal faked, with the budget-window service's fake advancer for windows),test_economy_rerun_routes.py, and the budget-window submission tests whose start path changed.mainwith Create household reports through POST /reports and calculate them in one household function call #1057 was merged in):test_household_report_reruns.pytests, against the same disposable PostgreSQL 17 database (a household report's definition and its household; the re-run's report run and new household simulation runs on a later catalog version, in the report's year; a report of the baseline alone; refusal while a run is under way; a deleted report or simulation; a lost commit).test_household_report_reruns.pytests (22 cases: the re-run's reads, rows, payload, call and answer, with and withoutforce; a report of the baseline alone; each refusal, including a side naming a dynamic; a run under way; a deleted report or simulation; an unconfirmed commit; Modal's refusal; the app's one household report service), the household re-run route test, the changed OpenAPI and re-run refusal tests, and one test of each route test file whose fake service builder now takeshousehold_reports../scripts/generate-clients.sh; the four re-run tests intest_economy_generated_client.pyand the household re-run test intest_household_report_generated_client.pyran against it.mainwith Create household reports through POST /reports and calculate them in one household function call #1057 and the household re-runs. None found a bug; every quality finding was fixed, test-first where it changed code. Round 11's findings were wording only and were fixed without a further round.🤖 Generated with Claude Code