Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/scripts/test_alembic_v2_lifecycle.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,6 @@ uv run --frozen --extra dev pytest -q \
tests/integration/test_alembic_v2_saved_item_keys.py \
tests/integration/test_alembic_v2_user_types.py \
tests/integration/test_alembic_v2_withheld_area_codes.py \
tests/integration/test_alembic_v2_shared_report_keys.py
tests/integration/test_alembic_v2_shared_report_keys.py \
tests/integration/test_alembic_v2_cliff_impacts.py \
tests/integration/test_alembic_v2_dynamics.py
4 changes: 2 additions & 2 deletions .github/workflows/v2-integration-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -75,8 +75,8 @@ jobs:
run: uv run --frozen --extra dev python scripts/reconcile_v2_privileges.py verify
- name: Test v2 metadata publication, resource routes, the countries list, reports, simulations and users' report runs
run: uv run --frozen --extra dev coverage run --branch -m pytest -q tests/integration/test_v2_catalog_publication.py tests/integration/test_v2_catalog_publication_time_periods.py tests/integration/test_v2_catalog_publication_pruning.py tests/integration/test_v2_catalog_spm_availability.py tests/integration/test_v2_catalog_statistics.py tests/integration/test_v2_metadata_routes.py tests/integration/test_v2_country_routes.py tests/integration/test_v2_report_reads.py tests/integration/test_v2_report_creation.py tests/integration/test_v2_budget_window_report_reads.py tests/integration/test_v2_budget_window_report_creation.py tests/integration/test_v2_report_runs.py tests/integration/test_v2_report_reads_by_run.py tests/integration/test_v2_user_report_runs.py tests/integration/test_v2_user_report_run_requesters.py tests/integration/test_v2_user_budget_window_runs.py tests/integration/test_v2_forced_report_reruns.py tests/integration/test_v2_user_report_views.py
- name: Test v2 policy persistence and immediate v1 mirroring
run: uv run --frozen --extra dev coverage run -a --branch -m pytest -q tests/integration/test_v2_policy_persistence.py tests/integration/test_v1_policy_dual_write.py tests/integration/test_mysql_policy_dual_write.py tests/integration/test_v2_user_policy_mirroring.py tests/integration/test_v1_user_policy_dual_write.py
- name: Test v2 policy and dynamic persistence and immediate v1 mirroring
run: uv run --frozen --extra dev coverage run -a --branch -m pytest -q tests/integration/test_v2_policy_persistence.py tests/integration/test_v2_dynamic_persistence.py tests/integration/test_v1_policy_dual_write.py tests/integration/test_mysql_policy_dual_write.py tests/integration/test_v2_user_policy_mirroring.py tests/integration/test_v1_user_policy_dual_write.py
- name: Test signed-in identity provisioning, user associations, saved detailed-save records and user types
run: uv run --frozen --extra dev coverage run -a --branch -m pytest -q tests/integration/test_v2_user_identity_persistence.py tests/integration/test_v2_user_type_persistence.py tests/integration/test_v2_saved_detailed_save_records.py tests/integration/test_v2_saved_budget_window_reports.py
- name: Test comparison-run persistence, the simulation API's roles, shared reports and runs, and privilege reconciliation as the managed roles
Expand Down
1 change: 1 addition & 0 deletions changelog.d/simulation-run-cliff-impacts.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The v2 database can store one cliff result per simulation run, its cliff gap and cliff share, with the state of its calculation. The simulation service may start each calculation, its Modal workers may write the result, and API v1 may read it.
80 changes: 78 additions & 2 deletions docs/engineering/skills/alembic-migrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -255,8 +255,8 @@ and a revision runs once per database, so it can't repair drift.
| `policyengine_v2_runtime` | The Public API on Cloud Run |
| `policyengine_v2_catalog_writer` | `scripts/publish_v2_metadata_catalog.py`, which also runs `ANALYZE` on seven catalog tables after every publication, with `MAINTAIN` (PostgreSQL 17) |
| `policyengine_stage12_<environment>` | The simulation API and its Modal worker, on the temporary comparison-run tables |
| `policyengine_v2_simulation_service` | The simulation API on Cloud Run: creates simulations, reports, their runs, budget-window runs and household jobs, and reads the policy, dynamic, household and catalog rows a run needs. It updates only the status, Modal call ID, error and completion columns of the runs and household jobs whose Modal calls it starts, and the status, error and completion columns of a budget-window run, which it fails at once when Modal refuses the calls of the window's first years |
| `policyengine_v2_modal_worker` | The simulation API's Modal workers: write their runs' and household jobs' status and results, and budget-window runs' status, and read the rows a run needs. It updates only a run's status, Modal call ID, error, time and result columns, and a budget-window run's status, error and time columns: each year's coordinator starts the window's next unstarted year when its own year finishes, the last year to finish marks the window succeeded, and a failed year fails it. The workers' code, not the database, keeps a worker to its own run |
| `policyengine_v2_simulation_service` | The simulation API on Cloud Run: creates simulations, reports, their runs, budget-window runs, household jobs and simulation runs' cliff calculations, and reads the policy, dynamic, household and catalog rows a run needs. It updates only the status, Modal call ID, error and completion columns of the runs, household jobs and cliff calculations whose Modal calls it starts, and the status, error and completion columns of a budget-window run, which it fails at once when Modal refuses the calls of the window's first years |
| `policyengine_v2_modal_worker` | The simulation API's Modal workers: write their runs', household jobs' and cliff calculations' status and results, and budget-window runs' status, and read the rows a run needs. It updates only a run's or cliff calculation's status, Modal call ID, error, time and result columns, and a budget-window run's status, error and time columns: each year's coordinator starts the window's next unstarted year when its own year finishes, the last year to finish marks the window succeeded, and a failed year fails it. The workers' code, not the database, keeps a worker to its own run |

Every v2 table has an entry for every role, so "no access" is a decision, not
an omission. A privilege is declared on the whole table, or with
Expand Down Expand Up @@ -457,6 +457,8 @@ it publishes that version, with nothing to restore it. Downgrading past the
revision is therefore safe only while no version is pruned (no
`tax_benefit_model_versions` row has `catalog_values_pruned_at` set).
From the head, such a downgrade first runs the later revisions' downgrades:
`bc334f9877b7`'s drops `cliff_impacts` with every simulation run's cliff
results (see "API v2 simulation-run cliff impacts"),
`9d1d12a7ac58`'s needs `dynamics` empty (see "API v2 dynamics stored like
policies"),
`f519a77ba122`'s drops `user_invitations` and needs no recorded change with
Expand Down Expand Up @@ -1073,3 +1075,77 @@ row. `tests/integration/test_alembic_v2_dynamics.py` checks that nothing else
changes, the downgrade and upgrade, both refusals, and the keys and unique
rules.

### API v2 simulation-run cliff impacts

Revision `bc334f9877b7` (parent `9d1d12a7ac58`) was autogenerated, with its
ID given by `--rev-id`, after the `CliffImpact` model was added. It was
generated on `f519a77ba122`; when `master` gained the dynamics revision
`9d1d12a7ac58` on the same parent, its `down_revision` moved to
`9d1d12a7ac58`, and generating it again from there gives the same
operations. Each
simulation run can have one cliff result, linked to the run and not to any
report, which every report naming the run shares: cliffs need marginal tax
rates, which recalculate each household with extra earnings, so a shared run
never stores the cliff columns (US national runs keep their region-group
split), and a request that asks for cliffs starts a separate calculation of
the run on its dataset, policy and dynamic.

It creates `cliff_impacts`, one row per cliff calculation of a simulation run:

- `simulation_run_id`, a foreign key to `simulation_runs` with `CASCADE`, as
`poverty` and `inequality` have: the result is part of its run, and nothing
references it. A run can be deleted only once no report run names it.
- The calculation's state: `status` (`pending`, `running`, `succeeded` or
`failed`, in the `v2_simulation_status` type simulation runs use),
`modal_call_id`, `error_message` (`code: message`, as runs record theirs),
`started_at` and `completed_at`, with the run tables' check that a
succeeded or failed row has `completed_at`
(`ck_cliff_impacts_finished_completion`).
- The result: `cliff_gap`, the weighted total of the net income people would
lose from a small rise in their earnings, and `cliff_share`, the weighted
number of people on a cliff divided by the weighted number of adults, both
double precision. A succeeded row has both, and no other row has either
(`ck_cliff_impacts_result_when_succeeded`). Neither is range-checked: .py
doesn't keep the share to at most 1, and a filtered simulation with no
adults gives NaN.
- `uq_cliff_impacts_unfailed`, a partial unique index on `simulation_run_id`
where `status <> 'failed'` (also declared for SQLite): a run has at most one
calculation that hasn't failed, and so at most one result. The simulation
service is to start one with `INSERT ... ON CONFLICT (simulation_run_id)
WHERE status <> 'failed' DO NOTHING`, so that two requests at once start
one calculation and the second joins it. A failed calculation never counts:
it stays as a record, and the next request inserts another.
- An index on `(status, started_at)`, as `simulation_runs` has, for finding
unfinished calculations by age, as the simulation API's stuck-run sweep
does for runs: a calculation whose worker ended without recording its
outcome stays unfinished, and keeps later requests from starting another,
until something fails it.

A result has no versions of its own: it inherits its run's catalog and
package versions, so it is calculated only for a run on the versions the
simulation API serves. The simulation service may read and insert rows and update their
status, Modal call ID, error and completion columns; the Modal worker may read
them and update their state and result columns, never insert them; the
runtime role may read them; neither simulation role may change the run a row
names or delete a row. The reconcile step after each schema upgrade grants
them. Until the simulation API writes the table, no run has a cliff
calculation, and its cliffs stay refused. `report_runs`' four cliff columns
stay until a later revision drops them, once API v1 reads cliffs from
here.

It has one post-generation correction, commented in the revision: `status`
reuses the baseline's `v2_simulation_status` type
(`postgresql.ENUM(..., create_type=False)`), as `1356f658347d` does for
`simulation_runs.status`; the generated `create_table` would create the type
again and fail. So the downgrade, which drops only the table, leaves the type
to `simulation_runs`.

The revision writes no data and changes no other table, and API v1's
previous release, which doesn't map the table, keeps working at it. The
downgrade drops the table with every calculation and result; a later
calculation of the same runs on the same versions gives the same numbers.
`tests/integration/test_alembic_v2_cliff_impacts.py` checks that nothing
else changes, that the downgrade leaves the type and an upgrade works again,
and the index, checks and delete rule;
`tests/integration/test_v2_simulation_role_privileges.py` runs the writes
and reads as the roles that do them.
5 changes: 4 additions & 1 deletion docs/migration/stage-12a-user-login.md
Original file line number Diff line number Diff line change
Expand Up @@ -770,7 +770,10 @@ admin; `scripts/set_v2_user_type.py` makes another admin afterwards.
`users.primary_country`, so it fails while legacy-only rows, any simulation
or report associations, or any unauthenticated user exist. Removing them
first deletes users' saved work. From the head, the later revisions'
downgrades run first: the user invitations revision (`f519a77ba122`) drops
downgrades run first: the cliff impacts revision (`bc334f9877b7`) drops
every simulation run's cliff results, the dynamics revision
(`9d1d12a7ac58`) needs `dynamics` empty, the user invitations revision
(`f519a77ba122`) drops
every invitation and dismissal and needs no change with source
`invitation`, the shared reports revision (`94dc43079186`) drops
every user's report-run record, every simulation and report key, the runs'
Expand Down
7 changes: 5 additions & 2 deletions docs/migration/stage-9-v2-metadata.md
Original file line number Diff line number Diff line change
Expand Up @@ -835,8 +835,11 @@ pruning").

A schema downgrade past revision `334859ef4fec` drops
`catalog_last_published_at` and `catalog_values_pruned_at`. From the head,
it first runs the later revisions' downgrades: `f519a77ba122`'s drops
`user_invitations` and needs no recorded change with source `invitation`,
it first runs the later revisions' downgrades: `bc334f9877b7`'s drops
`cliff_impacts` with every simulation run's cliff results, `9d1d12a7ac58`'s
needs `dynamics` empty, `f519a77ba122`'s
drops `user_invitations` and needs no recorded change with source
`invitation`,
`94dc43079186`'s drops
`user_report_run_associations` with every user's report-run record, the
simulation and report keys, the runs' `forced` markers, the window runs' .py
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
"""record each simulation run's cliff calculations

Revision ID: bc334f9877b7
Revises: 9d1d12a7ac58
Create Date: 2026-11-17 17:38:09.806260
Generation: uv run alembic -c alembic-v2.ini revision --autogenerate --rev-id bc334f9877b7

Generated on f519a77ba122; its down_revision moved to 9d1d12a7ac58 when the
dynamics revision reached master first. Generating it again from
9d1d12a7ac58 gives the same operations.

One post-generation correction, commented where it is made:
cliff_impacts.status reuses the existing v2_simulation_status type instead of
creating it again.

Each simulation run can have one cliff result: its cliff gap and cliff share,
linked to the run and not to any report. It adds cliff_impacts, one row per
cliff calculation of a simulation run, holding the calculation's state
(status, Modal call, error and times) and, once it succeeds, the two numbers.
A partial unique index allows one calculation per run that hasn't failed, so
identical requests join it; a failed calculation never counts. Deleting a run
deletes its calculations. A result has no versions of its own: it inherits its
run's.

It writes no data and changes no other table, and API v1's previous
release, which doesn't map the table, keeps working at it. The downgrade
drops the table with every calculation and result, and leaves the type,
which simulation_runs still uses.
"""

from typing import Sequence, Union

from alembic import op
import sqlalchemy as sa
import sqlmodel
from sqlalchemy.dialects import postgresql


revision: str = 'bc334f9877b7'
down_revision: Union[str, None] = '9d1d12a7ac58'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None


def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
op.create_table('cliff_impacts',
sa.Column('id', sa.Uuid(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('simulation_run_id', sa.Uuid(), nullable=False),
sa.Column(
'status',
# Dialect correction: a cliff calculation has a simulation run's
# states, so the column reuses the baseline's v2_simulation_status
# type instead of creating it again, which would fail.
postgresql.ENUM(
'pending',
'running',
'succeeded',
'failed',
name='v2_simulation_status',
create_type=False,
),
nullable=False,
),
sa.Column('modal_call_id', sqlmodel.sql.sqltypes.AutoString(length=255), nullable=True),
sa.Column('error_message', sqlmodel.sql.sqltypes.AutoString(), nullable=True),
sa.Column('started_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('completed_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('cliff_gap', sa.Float(), nullable=True),
sa.Column('cliff_share', sa.Float(), nullable=True),
sa.CheckConstraint("(cliff_gap IS NULL) = (cliff_share IS NULL) AND (status = 'succeeded') = (cliff_gap IS NOT NULL)", name=op.f('ck_cliff_impacts_result_when_succeeded')),
sa.CheckConstraint("status NOT IN ('succeeded', 'failed') OR completed_at IS NOT NULL", name=op.f('ck_cliff_impacts_finished_completion')),
sa.ForeignKeyConstraint(['simulation_run_id'], ['simulation_runs.id'], name=op.f('fk_cliff_impacts_simulation_run_id_simulation_runs'), ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_cliff_impacts'))
)
op.create_index(op.f('ix_cliff_impacts_simulation_run_id'), 'cliff_impacts', ['simulation_run_id'], unique=False)
op.create_index('ix_cliff_impacts_status_started_at', 'cliff_impacts', ['status', 'started_at'], unique=False)
op.create_index('uq_cliff_impacts_unfailed', 'cliff_impacts', ['simulation_run_id'], unique=True, postgresql_where=sa.text("status <> 'failed'"), sqlite_where=sa.text("status <> 'failed'"))
# ### end Alembic commands ###


def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
op.drop_index('uq_cliff_impacts_unfailed', table_name='cliff_impacts', postgresql_where=sa.text("status <> 'failed'"), sqlite_where=sa.text("status <> 'failed'"))
op.drop_index('ix_cliff_impacts_status_started_at', table_name='cliff_impacts')
op.drop_index(op.f('ix_cliff_impacts_simulation_run_id'), table_name='cliff_impacts')
op.drop_table('cliff_impacts')
# ### end Alembic commands ###
2 changes: 1 addition & 1 deletion policyengine_api/data/v2/migration_target.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
from collections.abc import Mapping

V2_ALEMBIC_DISPOSABLE_TEST = "V2_ALEMBIC_DISPOSABLE_TEST"
V2_ALEMBIC_HEAD_REVISION = "9d1d12a7ac58"
V2_ALEMBIC_HEAD_REVISION = "bc334f9877b7"
DISPOSABLE_DATABASE_NAME = "policyengine_v2_alembic_test"
DISPOSABLE_HOSTS = frozenset({"localhost", "127.0.0.1", "::1", "postgres"})
MIGRATION_ROLE = "policyengine_v2_migrator"
Expand Down
2 changes: 2 additions & 0 deletions policyengine_api/data/v2/models/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@
AggregateType,
BudgetSummary,
ChangeAggregate,
CliffImpact,
CongressionalDistrictImpact,
ConstituencyImpact,
DecileImpact,
Expand Down Expand Up @@ -124,6 +125,7 @@
"CatalogEntity",
"CatalogEnumValue",
"ChangeAggregate",
"CliffImpact",
"CongressionalDistrictImpact",
"ConstituencyImpact",
"Country",
Expand Down
Loading
Loading