Skip to content

Start each side's cliff calculation when a report asks for cliffs - #1099

Draft
anth-volk wants to merge 11 commits into
v2plan/c1-38-run-cliff-resultsfrom
v2plan/c1-39-include-cliffs
Draft

anth-volk wants to merge 11 commits into
v2plan/c1-38-run-cliff-resultsfrom
v2plan/c1-39-include-cliffs

Conversation

@anth-volk

@anth-volk anth-volk commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #1098

POST /reports (economy) and POST /reports/{report_id}/runs accept include_cliffs: true again. A request with it starts a cliff calculation for each side's simulation run that has none, and include_cliffs is never stored or keyed.

Stacked on #1096 (C1 · 38: the v2 worker calculates a simulation run's cliff result), and based on its branch. Retarget to main once #1096 merges.

Deploy order:

  1. API v1's revision bc334f9877b7 (cliff_impacts, Store one cliff result per simulation run in the v2 database, with grants policyengine-api#4327, C1 · 37) must be applied in each environment first.
  2. Then a sim release whose v2 app has Calculate each simulation run's cliff gap and share as its own cliff result #1096 must be the default worker.
  3. Then this.

What a request asking for cliffs does

A simulation run has one cliff result, its cliff gap and cliff share, in a cliff_impacts row that every report naming the run reads (C1 · 24's answer 4b).

  1. Rows. The create or re-run ends on its report run as before, created or joined. In the same transaction, it finds or inserts the cliff calculation of each simulation run that report run names (find_or_insert_report_run_cliffs):

    • INSERT … ON CONFLICT (simulation_run_id) WHERE status <> 'failed' DO NOTHING against uq_cliff_impacts_unfailed, in simulation run ID order;
    • a calculation that's pending, running or succeeded is joined;
    • a failed one never counts, so a new row is inserted;
    • one run on both sides gets one row.

    The rows go through the CliffImpact mapped class on the request's SQLAlchemy Session, as the rest of the service's database code does since the ORM conversion (Answer 500 when a household job's commit is unconfirmed, and move sim's database code to the SQLAlchemy ORM #1090). A re-run asking for cliffs goes through insert_report_rerun_with_cliffs, which returns FoundReportRerun: its report run and its calculations.

    Two identical requests at once end on one calculation per run. A request the database refuses wrote neither the report run nor its calculations, so a 503 database_unavailable still means nothing was written.

  2. Calls. Only for a calculation it inserted, and before the report run's coordinator, the service spawns the country's simulation function in its cliff mode (Calculate each simulation run's cliff gap and share as its own cliff result #1096). The payload is the run's own SimulationExecutionInput, built as a standalone run's; the context names the calculation (cliff_result_id) under the run's artifact prefix. The service then records the call's ID while the row is pending with none. A calculation runs beside its simulation run, whatever that run's state, and recalculates the simulation from its dataset and policy.

  3. Refused calls. When Modal refuses a cliff call, or doesn't confirm it in 10 s, the service fails that row while it's still pending with no call. The answer is still 201, with that side failed, and the next request asking for cliffs starts another. Because cliff calls start first, a coordinator that Modal refuses (503 modal_call_refused) still leaves them started. The three log records for these problems (cliff_calculation_call_unconfirmed, cliff_calculation_not_failed, cliff_calculation_call_not_recorded) pass their fields through log_extra and name the cliff_impacts row with database_row_attributes, as Name the database row and Modal call in v2 telemetry, and emit the Simulation Entrypoint's log fields #1092's logging rule requires.

  4. Answer. EconomyReportCreated gains cliffs: each side's cliff_result_id and status. The field is left out when the request didn't ask.

include_cliffs is never stored or keyed

  • EconomyReportRequest.options(), BudgetWindowReportRequest.options() and simulation_options no longer carry it. The coordinator's payload options hold only the SPM selection, and the coordinator never plans cliff columns.
  • The keys and reports.inputs are unchanged by it, which the tests check.
  • A report stored before keys whose inputs hold include_cliffs: true used to answer 422 on re-run. Its re-run now asks for its runs' cliffs. Its stored option is only read, and rerun_scope no longer passes it to the coordinator.

Refusals

  • The interim 422 cliffs_not_available on POST /reports, and on a re-run of such a stored report, is gone (C1 · 24's interim answers A and B).
  • POST /reports/{report_id}/runs takes include_cliffs. A budget window's or household report's re-run asking for it is refused with 422 cliffs_not_available at ["body", "include_cliffs"], since neither calculates cliffs. A budget window's create already refuses the field.

Tests

Red-green throughout: each "(failing)" commit precedes its change. Only the tests this PR adds or changes were run, each by node ID, not whole files or suites. The Postgres tests ran as the simulation service's role, against a database built from #4327's branch (a4be63110). In CI they need public-api-commit to name #4327's merge, which #1096 moves to.

  • Persistence (test_cliff_calculations.py, 17 tests):
    • find or insert per run, in simulation run ID order; a failed row never joined; a succeeded row is the result;
    • a row naming a missing simulation run names the cliff_impacts.simulation_run_id column in its error;
    • two transactions at once insert one row, and two concurrent creates asking for cliffs insert one per run;
    • a request that loses the report run to an identical one still writes the winner's runs' calculations;
    • creates and re-runs (joined, created, forced, and over two different simulation runs) write their runs' calculations with the report run;
    • a refused calculation writes no report;
    • a call's ID recorded, or the row failed, only while it's pending with none.
  • Entry (53 test cases):
    • creates and re-runs, created or joined, start a call only for an inserted calculation, and none for one being calculated or calculated;
    • one run on both sides gets one calculation, and two identical requests start one per run;
    • keys and stored inputs don't change with the flag, and no coordinator payload carries it;
    • a refused cliff call, an unrecordable call ID, and a failure write that fails, each logged once per row naming that row;
    • every log call passes its fields through log_extra (test_log_fields.py);
    • the old 422s are gone, and window and household re-runs are refused;
    • the route and OpenAPI shapes.

Sim's checks (pinned Ruff, rules, quality guards, lock checks, Pyright for the entry and persistence packages) pass, and so does the changelog fragment check against #1096's head.

Commits

The change was first written on SQLAlchemy Core and has been rebuilt on mapped classes and Sessions on top of #1096's ORM version. Each "(failing)" commit adds its tests before the change that makes them pass.

  1. 1232cb68 (failing), ad1442b9: persistence, each run's cliff calculation written with its report run, on ORM Sessions.
  2. 4c727839 (failing), 1d0b7825: the Simulation Entrypoint starts each side's cliff calculation.
  3. 7c591174: READMEs, the migration document and the changelog fragment.
  4. 721f775e (failing), 02e9f483: a re-run's cliffs returned beside its run, insert order, and the refused column named.
  5. f2382482 (failing), 0a8115fc: the cliff-call problem logs go through log_extra.
  6. 0c5206d7: this PR's text calls PolicyEngine/policyengine-api "API v1".
  7. 49858c36: the log test states each case's error and fails on a row logged twice.

Review

Four read-only review rounds:

  1. Round 1 found that the cliff rows were written in a transaction of their own, after the report's commit and the coordinator's start. A failure there answered 503 "nothing written" although the report run existed, and forced retries added extra runs. Fixed test-first by writing them in the report run's transaction and starting them first.
  2. Round 2 found no bugs. Its quality findings are fixed: calculations name their run, there is one cliff step for creates and re-runs, and there are tests of concurrent creates and of a refused coordinator on re-run.
  3. Round 3, of the rebuild on mapped classes and Sessions, found no bugs and no change in behaviour. Its quality findings are fixed (commit group 4): re-runs return their cliffs in their own result type, FoundReportRun is as on main, and there are new tests of the insert order, a request losing the report run, and a re-run over two different simulation runs.
  4. Round 4, of the logging change, the "API v1" wording and the rebase, found no bugs. Its quality finding on the log test is fixed (commit 7).

Deployment and environment variables

No workflow, Dockerfile, Terraform, v2_app.py or staging check script changes, and the code reads no new environment variable or secret. The service's privileges on cliff_impacts, which /ready checks, come from #1096's access.py.

🤖 Generated with Claude Code

anth-volk and others added 11 commits November 19, 2026 20:49
A request asking for cliffs finds or inserts one CliffImpact per
simulation run its report run names, in the report run's own
transaction: pending once, then joined; a failed row never joined; a
succeeded row is the run's result; two transactions at once insert one.
A calculation the database refuses leaves no report. The store records
a calculation's Modal call, or fails it, while it is pending with none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A create or re-run asking for cliffs proposes its sides' calculation
IDs (ProposedCliffs). In the transaction that settled its report run,
find_or_insert_report_run_cliffs finds or inserts the CliffImpact of
each simulation run that report run names, against
uq_cliff_impacts_unfailed, in simulation run ID order, so a calculation
the database refuses leaves nothing written. The economy store records a
calculation's Modal call, or fails it, while it is pending with none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A report created or re-run with include_cliffs writes its runs' cliff
calculations with its report run and starts, before the coordinator,
only those it created, through the country's simulation function with a
context naming the calculation. Two identical requests start one per
run; include_cliffs is in no key, stored input or calculation option; a
refused cliff call fails that calculation and the report is answered
201. The 422s of C1 · 24's interim answers A and B go, a stored report
asking for cliffs asks for its new runs' cliffs, and a budget window's
or household report's re-run asking for them is refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
POST /reports with include_cliffs, and a re-run with the re-run body's
new include_cliffs, propose their sides' calculation IDs with their
report run. Before the report run's coordinator, the service starts only
the calculations its request inserted: the country's simulation
function, given the run's own input and a context naming the cliff
calculation. A refused call fails that calculation and the report is
still answered 201, naming each side's calculation and status in
cliffs, a field left out when the request didn't ask.

include_cliffs is never stored, keyed or in any calculation's options.
The 422s of C1 · 24's interim answers A and B go: a report stored before
keys asking for cliffs is re-run with its runs' cliffs, and a budget
window's or household report's re-run asking for them is refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Simulation Entrypoint's README describes what a create or re-run
asking for cliffs does and answers, and drops the interim 422s; the
persistence README names the cliff calculation lookup and its writes;
the Stage 12 foundation document says the service now starts each
run's cliff calculation. Adds the changelog fragment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…iling)

From the ORM redo's review: a re-run asking for cliffs returns its
report run and their calculations together, leaving FoundReportRun as
every other lookup returns it. Also covers the simulation run ID order
of the inserts, a request losing the report run taking the winner's
runs' calculations, a re-run over two distinct simulation runs, and a
refused calculation naming the column that refused it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the ORM redo's review: insert_report_rerun and FoundReportRun stay
as every other caller uses them, and a re-run asking for cliffs goes
through insert_report_rerun_with_cliffs, which returns its report run
and their calculations together (FoundReportRerun). A cliff calculation
naming a missing run now names cliff_impacts.simulation_run_id in
EconomyRecordReferenceError, and the run classes' comments include it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each log of a cliff calculation's unconfirmed call, unrecorded call ID
or unwritten failure carries the error type and the cliff_impacts row
through log_extra, the only fields that reach Cloud Logging.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The three logs of a cliff calculation's call now reach Cloud Logging,
with the error's type and the cliff_impacts row, as #1092's rule
requires of every Simulation Entrypoint log.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant