Skip to content

Invite users by email, email every type change, and report the latest change on /v2/me - #4297

Merged
anth-volk merged 12 commits into
masterfrom
feat/v2-user-invitations
Nov 16, 2026
Merged

anth-volk merged 12 commits into
masterfrom
feat/v2-user-invitations

Conversation

@anth-volk

@anth-volk anth-volk commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #4292

Summary

Admins invite an email address to become an author or admin, and the signed-in invitee accepts once Auth0 reports their verified email matching the address. Every committed type change emails the user through Postmark, and GET /v2/me reports the latest change until the user dismisses it. The Public API stores no user's email and adds none to access tokens: the invitee's and each user's email are read from Auth0's Management API through the existing read-only profile reader (build_auth0_user_profile_reader). Acceptance reads the invitee with a new read_user_profile, GET /api/v2/users/{id}, which Auth0 answers at once, rather than the user search, which may not yet see someone who signed up or verified their email moments ago. The settings code of the Auth0 management module moves unchanged to auth0_management_settings, which it re-exports, to keep it under 800 lines.

Schema

  • Autogenerated v2 revision f519a77ba122 on 1e1e3f079f2c, no post-generation changes. It was generated on 0c136a278f29; when master gained the withheld area codes revision (Record the UK area codes disclosure control withholds from a report run #4305) on the same parent, its down_revision moved to 1e1e3f079f2c, and generating it again from that revision gives identical operations. It adds user_invitations and user_type_changes.dismissed_at, and renames the source check to ck_user_type_changes_source_allowed, which also allows invitation (Alembic compares checks by name only). Deleting the inviting admin keeps an invitation (SET NULL); deleting the user who accepted it deletes it (CASCADE), since its address was theirs.
  • TABLE_ACCESS: the runtime role reads, inserts and updates user_invitations (never deletes), and updates only user_type_changes.dismissed_at.
  • Rollback: the previous revision reads every change's source as admin_route or script, so there an admin's read of the type history of a user who accepted an invitation answers 500. The downgrade drops user_invitations and dismissed_at and fails while any change's source is invitation.

Routes

  • POST /v2/admin/invitations {email, user_type} (201), GET /v2/admin/invitations?offset=&limit=, POST /v2/admin/invitations/{invitation_id}/resend, DELETE /v2/admin/invitations/{invitation_id} (204): admins only, every write under the user-type change lock. Inviting an address again revokes its open invitation; resending replaces the token and the 7-day expiry. A failed send leaves the invitation open with last_send_error.
  • POST /v2/me/invitations/accept?country_id= {token}: reads the caller's Auth0 profile first (503 auth0_unavailable), then in one transaction under the lock checks the token (410 invitation_not_found, invitation_revoked, invitation_accepted, invitation_expired), the inviting admin (409 inviter_not_admin), verification (409 email_not_verified) and the address, whole and ignoring case (403 email_mismatch). It never lowers a type, and records a change with source invitation and the inviting admin. Refusals carry a code next to status and message.
  • GET /v2/me: result.latest_type_change (id, previous_user_type, user_type, changed_at) until dismissed; POST /v2/me/type-changes/{type_change_id}/dismiss?country_id= (204).
  • The app's half (C3 · 16) builds against these routes.

Emails

  • After every committed change (admin route, invitation, scripts/set_v2_user_type.py) the user is emailed at their first verified Auth0 email; an acceptance reuses the address it read. The script says whether it emailed the user, and why not. Postmark's POST /email/withTemplate, template aliases user-invitation and user-type-changed, outbound stream; a timeout, connection error, 429 or 5xx is retried, three attempts in all, after 0.5 s and 1 s.
  • An unsent email never fails or undoes the change: the WARNING user_type_change_email_not_sent (v2 user ID and cause only) or user_invitation_email_not_sent (invitation ID and cause). Without Postmark settings nothing is sent.
  • The template data each email receives is pinned in the tests and in the Stage 12A doc.

Docs and contracts

Stage 12A doc (routes, emails, settings, Postmark operator steps, rollback), the Alembic and testing skills, OpenAPI descriptions, the regenerated migration contracts (GET /v2/me gains the stable field result.latest_type_change; the accept and dismiss routes are added), and a changelog fragment.

Settings and their wiring

The Postmark setup (C3 · 14) created the values in both GitHub environments: production on 2026-11-13, staging on 2026-11-16. Each setting is optional and all-or-none; with none set the Public API sends nothing and records a WARNING, and partial or invalid values fail readiness and the deploy validation.

Variable Provider How it reaches the code State
POSTMARK_SERVER_TOKEN_SECRET_RESOURCE GitHub Actions secret in the staging and production environments, naming a Secret Manager version: projects/policyengine-api/secrets/policyengine-api-staging-postmark-server-token/versions/latest and …-prod-postmark-server-token/versions/latest. The token itself is in Secret Manager (project policyengine-api, automatic replication). push.yml jobs deploy-cloud-run-staging (environment: staging) and deploy-cloud-run-candidate (environment: production), job-level env: → the "Deploy tagged Cloud Run … candidate" step inherits it → deploy_cloud_run_candidate.sh runs validate_cloud_run_deploy_env.sh and passes it in --set-env-vars → the revision's environment → load_postmark_settings → read from Secret Manager when an email is sent, as the revision's runtime service account Set in both environments. Each secret has one enabled version, and its only binding is roles/secretmanager.secretAccessor for its environment's runtime account: policyengine-api-cr-staging@ for staging, policyengine-api-cr-runtime@ for production, the accounts the current Cloud Run services run as. Unverified: Cloud Run's own read of the secret, until this PR deploys.
POSTMARK_SENDER_ADDRESS GitHub Actions variable in staging and production Same path, without Secret Manager Set in both: PolicyEngine <hello@policyengine.org>
POLICYENGINE_APP_URL GitHub Actions variable in staging and production Same path, without Secret Manager Set. Production: https://app.policyengine.org. Staging: the Vercel preview of PolicyEngine/policyengine-app-v2#1562, the app's half of this work, since there is no staging app address; it needs a new value once #1562 merges.
POSTMARK_SERVER_TOKEN Local development only Read by load_postmark_settings; no workflow or deploy script passes it Not deployed

Acceptance and type-change emails also use the existing AUTH0_USER_PROFILES_* settings and PUBLIC_API_AUTH_ISSUER; this PR changes nothing in how they reach the revision. release-tests receives none of the new settings.

Postmark, as set up in C3 · 14: each server has the templates user-invitation and user-type-changed on the outbound stream, and one test email per template per server, sent with the token read from Secret Manager, was delivered. Two limits remain:

  • Postmark hasn't approved the account (approval isn't requested yet), so it sends only to @policyengine.org addresses and refuses others with error code 412. The Public API records such a refusal as postmark_412 and doesn't retry it.
  • policyengine.org has no Postmark DKIM or Return-Path DNS records yet.

Inline suppressions added

  • # noqa: S105 on the two constants that name the Postmark token's environment variables: ruff reads a constant named …TOKEN… as a hard-coded secret; it is a variable name, as for the Auth0 constants. The two Auth0 constants' existing # noqa: S105 moved, unchanged, with the settings code to auth0_management_settings.
  • # noqa: BLE001 four times in the email service module: the sender factory, the send, the profile source factory and the Auth0 profile read. A committed type change or invitation must not fail because its email can't be sent, whatever the error; each failure is recorded by cause instead.
  • # noqa: BLE001 in one integration test, which keeps the waiting acceptance's error for the assertion, as the neighbouring lock tests do.
  • # type: ignore[assignment] in one unit test, which sets the stand-in sender to None to play "no Postmark settings".

Testing

  • master is merged in, most recently at d7042b226 (Serve household reports through /v2/reports #4293, Answer 400 for a household calculation whose policy can't be read #4279, Log refused economy requests at WARNING instead of ERROR #4220, Run reports again and list their runs in /v2/reports #4298 and Record the UK area codes disclosure control withholds from a report run #4305). Record the UK area codes disclosure control withholds from a report run #4305's revision shares this PR's parent, so this PR's revision moved onto it (see Schema). Two counts that both sides had raised by the same amount, and Git merged silently, now add up: 49 v2 tables in the import side-effects test, and 101 requests in the migration contracts, regenerated from the merged registry.
  • Existing tests updated for the new routes: the v2 OpenAPI path list, the migration contract request count (101 with master's), and the oversized-body test, which now fills in {type_change_id} too; the dismiss and resend routes apply the 256 KiB request limit like every other write.
  • Added or changed tests only, by node ID: the Postmark client, the email service, the invitation service, the admin invitation and /v2/me route contracts, the account service's latest change and dismissal, the user-type service's and the script's emails, the models, privileges, revision chain, readiness, the deploy validation and deploy script, push.yml, the app workflow contracts; and on disposable PostgreSQL 17 the revision's upgrade from 1e1e3f079f2c and refused downgrade, and as the runtime role the whole invitation flow, dismissal, the deletion of an accepted invitation with its account, and an acceptance that waits on the user-type change lock while its admin is demoted.
  • ruff check, ruff format --check, mypy (configured scope), lint-imports, python -m scripts.rules validate, python scripts/run_quality_guards.py, python scripts/export_migration_contracts.py (committed output), the changelog fragment check, and alembic check against the upgraded database (no differences).
  • mypy now also checks policyengine_api/libs/auth0_management_settings.py and policyengine_api/libs/postmark_email.py, so the api.quality.mypy-v2 rule in rules/registry.toml names them (the rendered rules docs are regenerated).
  • mypy on the changed policyengine_api/data/v2 model module reports only call-overload for sa_type=sa.DateTime(timezone=True), the convention every v2 timestamp column uses (49 such errors in data/v2/models on master, 54 with the five new columns); the typecheck job skips that package.

🤖 Generated with Claude Code

anth-volk and others added 6 commits November 13, 2026 00:08
Admins invite an email address to become an author or admin; a signed-in
user whose verified Auth0 email matches the address accepts the invitation
with the token from its link. Every committed type change, through an
invitation, the admin routes or the type script, emails the user through
Postmark's template API, and GET /v2/me reports the latest change until the
user dismisses it.

- user_invitations, user_type_changes.dismissed_at and the invitation
  source, in autogenerated v2 revision bb09a374231f, with the runtime
  role's privileges.
- Admin routes to create, list, resend and revoke invitations, and
  POST /v2/me/invitations/accept, whose refusals name a code.
- POST /v2/me/type-changes/{type_change_id}/dismiss.
- POSTMARK_SERVER_TOKEN_SECRET_RESOURCE, POSTMARK_SENDER_ADDRESS and
  POLICYENGINE_APP_URL in readiness, the deploy validation, the deploy
  script and both deploy jobs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Acceptance reads the invitee's Auth0 profile with GET /api/v2/users/{id},
  which Auth0 answers at once, instead of the eventually consistent user
  search, so a user who signed up or verified their email moments ago is
  not refused as unverified.
- The Auth0 settings code moves unchanged to auth0_management_settings,
  which auth0_management re-exports, keeping that module under 800 lines;
  mypy and its rule now also cover that module and the Postmark library.
- Deleting an account deletes the invitations it accepted (CASCADE), whose
  address was the user's; the revision is regenerated as f519a77ba122.
- The acceptance route documents both error shapes for 409 and 503.
- The type script says whether it emailed the user, and why not.
- Tests for a send whose invitation a resend replaced or that is gone, and
  docs and docstrings brought in line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…me role

- The acceptance route's 403 also documents a machine principal's refusal,
  which has no code.
- A PostgreSQL test deletes, as the runtime role, an account that accepted
  an invitation, and the foreign key deletes that invitation.
- Auth0 docstrings cover the read by ID, and the admin profile tests'
  stand-in reader has the protocol's new method.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keeps both sides' route dependency imports, and regenerates the migration
contracts from the merged registry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The dismiss and resend routes refuse bodies over 256 KiB with 413, like
  every other user write, before the service runs.
- Existing tests that list routes and counts include the new routes: the v2
  OpenAPI paths, the migration contract request counts, the v2 table count,
  and the oversized-body test, which now fills in {type_change_id}.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.11155% with 29 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.60%. Comparing base (e96e52f) to head (a481977).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
policyengine_api/libs/postmark_email.py 91.09% 9 Missing and 4 partials ⚠️
...ngine_api/services/v2/user_invitations/services.py 96.00% 4 Missing and 1 partial ⚠️
policyengine_api/fastapi_routes/dependencies.py 60.00% 4 Missing ⚠️
...ine_api/services/v2/user_invitations/validators.py 95.23% 2 Missing and 1 partial ⚠️
...licyengine_api/services/v2/user_emails/services.py 97.53% 2 Missing ⚠️
...api_routes/v2/admin_invitations/response_models.py 97.29% 0 Missing and 1 partial ⚠️
...s/v2/user_invitations/database_connectors/reads.py 94.44% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4297      +/-   ##
==========================================
+ Coverage   93.48%   93.60%   +0.11%     
==========================================
  Files         369      387      +18     
  Lines       21478    22380     +902     
  Branches     3401     3543     +142     
==========================================
+ Hits        20079    20948     +869     
- Misses        816      843      +27     
- Partials      583      589       +6     
Flag Coverage Δ
integration 56.96% <62.94%> (+0.29%) ⬆️
unit 91.25% <97.01%> (+0.20%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Keeps both sides in the user-type service, the service protocols, the
testing skill's module list, the route list test and the type script's
tests, where master's new import check sits next to the branch's email
tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in admins' account deletion (#4272) and budget-window reports
(#4290). Keeps both sides in the user-type service's and validators'
docstrings, the account service protocol (dismissal and admin deletion),
the Stage 12A doc's account of the user-type change lock, and the runtime
role's persistence tests, whose module now holds both sides' tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Master gained the withheld area codes revision 1e1e3f079f2c (#4305) on
0c136a278f29, this branch's revision's parent, so f519a77ba122's
down_revision moves to 1e1e3f079f2c, as 0c136a278f29's did; generating it
again from 1e1e3f079f2c gives the same operations. The head constant, the
revision-chain and revision tests, the invitations revision's integration
tests and the Alembic skill doc follow it, keeping both sides' sections and
tests.

Two counts that both sides had raised by the same amount, and Git merged
silently, now combine: 49 v2 tables, and 101 migration contract requests
(the contracts are regenerated).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…test

The required "Test environment variables" check failed on every pull
request: the test asserted that the main branch of
policyengine/populace-uk-private lists populace_uk_2023.h5, which it no
longer does. The test now finds the installed bundle's default UK dataset,
with the policyengine package's fallbacks and without importing that
package, and lists the files at the revision the bundle pins.

Applied unchanged from #4303, so both pull requests carry identical files.

(cherry picked from commit 6191dde)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk and others added 2 commits November 16, 2026 21:29
Master gained the shared reports revision 94dc43079186 (#4303) on
1e1e3f079f2c, this branch's revision's parent, so f519a77ba122's
down_revision moves to 94dc43079186; generating it again from 94dc43079186
gives the same operations. The head constant, the revision-chain and
revision tests, the invitations revision's integration tests, and the
rollback order in the Alembic skill doc and the Stage 9 and Stage 12A
runbooks follow it, keeping both sides' sections and tests.

Both sides had raised the v2 table count from 48 to 49, which Git merged
silently; with both tables it is 50.

With both sides' revision tests, tests/unit/v2/test_alembic_v2.py reached
1,504 lines, over the 1,500-line limit for test code. The revision order
test and each revision's contents test, with the constants and helpers they
use, move unchanged to tests/unit/v2/test_alembic_v2_revisions.py; the
Alembic environment's tests stay. The v2 Alembic lifecycle script, its
workflow test and the testing guide's commands name the new module.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tion

The invitations revision's contents test found its file by the head
revision's ID, so it would read the wrong file once another revision lands
on it, as the shared reports test did in this merge. It now uses its own
USER_INVITATIONS_REVISION constant. The revision's docstring also says that
generating it again from 94dc43079186 gives the same operations, as the
Alembic skill doc and the shared reports revision say of theirs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk
anth-volk marked this pull request as ready for review November 16, 2026 17:58
@anth-volk
anth-volk requested a review from a team as a code owner November 16, 2026 17:58
@anth-volk
anth-volk merged commit a507e21 into master Nov 16, 2026
23 checks passed
@anth-volk
anth-volk deleted the feat/v2-user-invitations branch November 16, 2026 17:58
anth-volk added a commit that referenced this pull request Nov 16, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk added a commit that referenced this pull request Nov 16, 2026
…branch

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was previously deployed

1 inactive deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Email invitations to become an author or admin, type-change emails, and the latest type change on /v2/me

1 participant