Skip to content

Remove the Simulation Entrypoint's unused create functions without keys, run-reuse search and output-bucket setting - #1082

Merged
anth-volk merged 2 commits into
v2plan/c1-27-find-or-create-by-keyfrom
claude/c1-27-reuse-cleanup-f9622a
Nov 17, 2026
Merged

anth-volk merged 2 commits into
v2plan/c1-27-find-or-create-by-keyfrom
claude/c1-27-reuse-cleanup-f9622a

Conversation

@anth-volk

Copy link
Copy Markdown
Contributor

Fixes #1081

Stacked on #1074 (base v2plan/c1-27-find-or-create-by-key), and merges after it. #1074 makes the Simulation Entrypoint find or create every simulation, report and run by its definition key. After that, nothing the service runs calls the code that created those rows without keys, or the search for an earlier succeeded simulation run to reuse. This PR removes that code. Requests, answers and the rows the service writes don't change.

What is removed

Persistence library (libs/policyengine-simulation-persistence)

  • The create functions that use no keys: insert_economy_report, insert_economy_simulation, insert_household_report and insert_budget_window_report, with insert_simulation_with_run, which only they called.
  • Their input types: NewEconomyReport, NewEconomySimulation, ReportSimulation, NewHouseholdReport, HouseholdReportSimulation, NewBudgetWindowReport and BudgetWindowYear. Nothing else used them.
  • The store methods that wrapped them: EconomyRunStore.create_report, create_simulation and create_budget_window_report, and HouseholdReportStore.create_report.
  • The run-reuse search: find_reusable_run, find_reusable_run_of, ReusableRun, RunReuse, search_by_definition, search_own_runs and REFUSED_REUSE_PREFIX.
  • The reuse parameter of EconomyRunStore.read_report_inputs, read_report_rerun_inputs, read_window_rerun_inputs and read_budget_window_inputs. The service always passed None. The window reads' years parameter goes too, because it only fed the search.
  • The SimulationInputs.reusable_run and BudgetWindowInputs.reusable_runs fields.
  • RerunSide.reused, and the __post_init__ checks in NewReportRerun and NewWindowRerun that refused a side with it set. The reused member of the WindowSide protocol goes too: year_created now takes a FoundWindowYear, the only type it is called with, whose sides still say whether this request inserted the run.

Simulation Entrypoint (projects/policyengine-simulation-entry)

  • The reuse and years arguments in the store protocols and their call sites.
  • ServiceSettings.output_bucket and Settings.v2_simulation_output_bucket, which read V2_SIMULATION_OUTPUT_BUCKET. Nothing used them.

Deployment

  • .github/scripts/deploy-cloud-run-simulation-entry.sh no longer requires V2_SIMULATION_OUTPUT_BUCKET_VALUE, no longer checks its format, and no longer writes V2_SIMULATION_OUTPUT_BUCKET into the Cloud Run runtime variables.
  • The "Deploy Cloud Run entrypoint candidate" step of .github/workflows/simulation-deploy.reusable.yml no longer passes it.

What stays

The Modal coordinator and workers still write simulation output files to the bucket, so these keep it:

  • the Stage 12 steps of the same workflow, "Validate pre-provisioned Stage 12 deployment resources" and "Synchronize the separate Stage 12 runtime secret";
  • .github/scripts/stage12-validate-infrastructure.sh and .github/scripts/stage12-sync-modal-secrets.sh;
  • the stage12-evaluation-runtime Modal secret and the executor's stage12_runtime/dependencies.py.

Environment variables

Variable Provider Before After
V2_SIMULATION_OUTPUT_BUCKET GitHub Actions environment variable, vars.V2_SIMULATION_OUTPUT_BUCKET, set in beta (policyengine-v2-staging-simulation-outputs) and prod (policyengine-v2-production-simulation-outputs). No repository-level variable of that name exists. Job deploy_entrypoint runs in environment: ${{ inputs.release_environment }} (beta, then prod, from simulation-deploy.yml). Its step "Deploy Cloud Run entrypoint candidate" passed the variable as V2_SIMULATION_OUTPUT_BUCKET_VALUE to deploy-cloud-run-simulation-entry.sh. The script required it, checked its format, and wrote V2_SIMULATION_OUTPUT_BUCKET into the --env-vars-file of gcloud run deploy. Settings.from_env read it, and nothing used the value. The entrypoint deploy step and script don't read it. --env-vars-file replaces all of a new revision's variables, so revisions deployed from this commit on have no V2_SIMULATION_OUTPUT_BUCKET. Revisions deployed earlier keep theirs. The service no longer reads it.
V2_SIMULATION_OUTPUT_BUCKET (Stage 12) The same GitHub Actions environment variable. Job deploy_stage12_v2, in the same environment, passes it to stage12-validate-infrastructure.sh and to stage12-sync-modal-secrets.sh. The sync script writes it into the stage12-evaluation-runtime Modal secret. src/modal/v2_app.py attaches that secret, and stage12_runtime/dependencies.py reads V2_SIMULATION_OUTPUT_BUCKET from it. Unchanged.

The change adds no variable and removes no provider. Not verified: the runtime variables of the Cloud Run revisions now serving. I didn't read them; they don't affect this change, because --env-vars-file sets each new revision's variables in full.

Tests

Tests that stored a record created before keys now insert its rows directly, with the same values the removed functions inserted:

  • tests/economy_rows.py gains insert_pending_report.
  • The new tests/household_report_rows.py inserts household reports.
  • test_economy_window_reruns.py has its own window helper.

Deleted tests:

  • tests/test_economy_run_reuse.py and the reuse cases in test_economy_reruns.py, test_economy_run_store.py and test_budget_window_records.py.
  • The tests of only the removed creators, in test_economy_run_records.py, test_household_report_store.py and test_budget_window_records.py. check_window_years remains covered by the window re-run tests.
  • The bucket tests: test_economy_deployment.py and the setting's case in test_config.py.

Some of the deleted tests checked behaviour that remains, so the keyed creators now test it:

  • A refused region, dataset, dynamic, policy or household reference names its column. The Simulation Entrypoint turns that column into its unknown_* refusals. test_keyed_economy_records.py, test_keyed_household_records.py and test_keyed_window_records.py now check this against find_or_create_economy_report, find_or_create_household_report and find_or_create_budget_window.
  • A refused window writes nothing.

These tests pass, because the behaviour hasn't changed. The economy store's commit-failure tests now use a forced report re-run, whose create only inserts rows.

The Stage 12 migration doc's report-run, standalone-simulation and budget-window passages now say that the service finds or creates these rows by key, and starts a call only for a run it created.

Checks run locally:

  • Ruff format and lint, the rules registry and quality guards, uv lock --check, and Pyright for both packages all pass.
  • The changelog fragment check against this PR's base passes.
  • The changed and added tests, selected by node ID, pass: 68 persistence tests against a PostgreSQL 17 database built from the Public API's v2 schema at e96e52ffb (Share detailed-save reports and runs by key, and record each user's report runs policyengine-api#4303's merge), and 56 entry tests.
  • One review round found no bugs. Its two quality findings, the coverage and the migration doc above, are fixed.

The "Public API schema" check's last step requires the pinned commit on the Public API's master. It fails on #1074's branch until #1074 moves the pin, and so it fails here too. #1074 fixes it.

🤖 Generated with Claude Code

anth-volk and others added 2 commits November 17, 2026 02:07
…e entrypoint's output-bucket setting

After #1074 the Simulation Entrypoint finds or creates every simulation,
report and run by its definition key, so nothing it runs calls the code
that created those rows without keys or searched for an earlier succeeded
simulation run to reuse. Remove that code, with no change in behaviour:

- the persistence library's insert_economy_report,
  insert_economy_simulation, insert_household_report and
  insert_budget_window_report, their input types, and the store methods
  that wrapped them (EconomyRunStore.create_report, create_simulation and
  create_budget_window_report, HouseholdReportStore.create_report);
- find_reusable_run, find_reusable_run_of, ReusableRun, RunReuse and the
  searches built on them, the store's `reuse` parameters (and the window
  reads' `years`, which only fed the searches), and the reusable_run and
  reusable_runs fields;
- RerunSide.reused and the checks that refused it;
- ServiceSettings.output_bucket, Settings.v2_simulation_output_bucket, and
  the entrypoint deploy's V2_SIMULATION_OUTPUT_BUCKET format check, runtime
  variable and workflow input. The Stage 12 steps, scripts and Modal
  workers keep the bucket.

Tests that store a record created before keys now insert its rows
directly (economy_rows.insert_pending_report, household_report_rows and a
window helper in the window re-run tests). The economy store's
commit-failure tests use a forced report re-run, whose create only
inserts rows.

Fixes #1081

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…orrect the Stage 12 creation passages

Review round 1: the tests deleted with the create functions without keys
were the only database tests that a refused region, dataset, dynamic,
policy or household reference names its column, which the Simulation
Entrypoint turns into its unknown_* refusals. The same checks now call
find_or_create_economy_report, find_or_create_household_report and
find_or_create_budget_window, and pass, as the behaviour is unchanged.
The Stage 12 migration doc's standalone-simulation and budget-window
passages now say the service finds or creates the rows by key and starts
a call only for a run it created.

Refs #1081

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk
anth-volk merged commit 81ee633 into v2plan/c1-27-find-or-create-by-key Nov 17, 2026
2 checks passed
@anth-volk

Copy link
Copy Markdown
Contributor Author

Folded into #1074 at the user's request: this PR's two commits (9c5ab3ac, 81ee633e) were fast-forwarded onto #1074's branch v2plan/c1-27-find-or-create-by-key, which is why GitHub shows this PR as merged. #1074's description now covers the removal, and #1081 closes when #1074 merges.

@anth-volk
anth-volk deleted the claude/c1-27-reuse-cleanup-f9622a branch November 17, 2026 07:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant