[anth-volk]

Fixes #4197

Summary

DELETE /<country_id>/economy/<policy_id>/over/<baseline_policy_id>

• Request (Flask, economy_routes.py). The query is parsed by AnnualEconomyQuery through parse_multidict_query, the GET's model, and both methods build the request with one helper, _annual_impact_request.
• Selection. EconomyService.delete_failed_economic_impact validates the request as the GET does and selects the stored record with the GET's _get_reusable_impact, including its worker-app check, on a fresh copy of the options for each attempt. It submits and polls nothing.
• Answers:
  • 204, no body: the record was a failure recorded at least 10 seconds earlier (FAILED_IMPACT_DELETE_LOCKOUT_SECONDS, from its end_time). It is removed with its scope and recent index entries, so the next GET submits a new job.
  • 409 {"status": "error", "message": …, "result": null, "delete_after_seconds": N}: a more recent failure.
  • 409 with delete_after_seconds: null: a computing or completed record.
  • 404 {"status": "error", "message": …, "result": null}: nothing is stored that the GET would answer from. That includes a record whose value cannot be decoded, and a failure stored under a worker app the request no longer resolves to; the GET would submit a new job for both.
  • 400: invalid query parameters, with the GET's own body.
  • 502 with only status, message and result: a request to the simulation service got no response.
  • 500, through the error handlers: the cache cannot be reached, a record cannot be read from it, or the record changed three times in a row (CacheCoordinationError). The selection runs inside ReformImpactCache.reads_fail_closed(), a ContextVar mode in which the GET's own lookups raise instead of reading an error as nothing stored. Outside that block the lookups behave as before.
• Scope. Every failed annual economy job: US national, state, congressional district and place, UK national, and UK local-area breakdowns, plus typed SPM failures. A typed SPM failure's GET answers 400 rather than 502, so its body has no delete_after_seconds.
• No authentication. Like the GET, the DELETE needs none, so any client can remove a stored failure 10 seconds after it was recorded, including a 30-day one, and the next GET reruns the job. This is what tracker task 4A · 1 decided; the 10-second wait is the only limit.

Removing only the failure that was checked

• The script. ReformImpactCache.delete_failure finds the failure's record key through its scope index, which also finds typed failures whose worker handle was cleared. One Lua script, DELETE_UNCHANGED_RECORD, deletes the record and both index entries only if the record still holds exactly the value read.
• What it never touches. A job a concurrent GET submits gets its own record key, so it is never touched. A record rewritten after it was read is left alone, and the service selects and checks again, up to FAILED_IMPACT_DELETE_ATTEMPTS (3) times.
• Writes after a DELETE. ReformImpactCache.update, used for every completion and failure write, now writes the record with Redis SET … XX. A late status poll that read the failed job before the DELETE can therefore no longer write it back after the 204, and update returns None, as it does for a missing record. The index entries such a refused write adds point at no record and are skipped on read, as an expired record's are. First writes (set_reform_impact) are unchanged.

delete_after_seconds on the failed annual GET's 502

• The value. It is the number of whole seconds, rounded up, until a DELETE is accepted, and 0 once it is. It is null whenever retry_after_seconds is null, because it is computed only when that field is.
• The clock. ReformImpactsService takes an injectable now clock, which stamps end_time and counts the lockout. Each instance counts on its own clock, and the value is capped at 10. The docs say what clock skew between instances does to the wait.
• Unreachable simulation service. The annual 502 carries delete_after_seconds: null; the budget-window 502 is unchanged.
• Retry note. The state and local-area failure message now ends "… 2 minutes after this failure, or 10 seconds after it by first sending DELETE to this URL."

Other

• CORS. The ASGI CORSMiddleware already allows DELETE and answers the preflight before Flask, and Flask-CORS's default methods include DELETE. A test covers both preflights and the CORS headers on a 409 and a 204.
• Docs. docs/canonical-spm.md, where Say in a failed economy answer how long its failure is kept #4172 documented retry_after_seconds, now describes the DELETE and the new field. tests/contract/registry.py has no entry for the annual GET, and migration_registry.py assigns requests by path segment, so scripts/export_migration_contracts.py produces no change.
• Observability. The cache records failure-delete (deleted, delete-skipped, coordination-failed) and fail-closed index-read events. The service logs Deleted a stored economic impact failure with the removed run's deleted_observability_id. The DELETE adds no stage span, and no span names.
• In-memory cache stand-in. InMemoryCacheBackend now runs the new script, and set accepts xx. Any other script text is still taken as the claim store's compare-and-delete.
• Imports. The route imports the new result types from economy_impact_types, because the v1 route contract test loads the routes with a stand-in economy_service module.

No inline noqa suppressions are added.

Testing

Only the added and modified tests were run, each selected by node ID (103 cases):

• tests/unit/routes/test_economy_failure_deletion.py (new, 40 cases). It covers:

  • the countdown inside the lockout, and the last half second;
  • 204, then a GET that submits a new run;
  • each job kind: US national, state, district and place; UK national and local-area breakdown; and a typed SPM failure;
  • computing and completed records refused and kept;
  • an older failure behind a newer job;
  • 404, a second DELETE, and DELETEs for another region, year, policy, baseline or version;
  • query validation identical to the GET's, an omitted version, and a bare state code;
  • a failure under a worker app no longer in use, and an undecodable record;
  • concurrent changes: a GET submitting a job mid-delete, the failure expiring and being replaced, and a late poll recording it again or after a 204;
  • giving up after three changed attempts;
  • an unreachable cache and one unreadable record (500, including the body through error_bp);
  • the DELETE's 502;
  • CORS through ASGI and through Flask alone.

• tests/unit/runtime_cache/test_reform_impact_failure_deletion.py (new, 17 cases). It covers:

  • removing the record and both index entries, including a typed failure without a worker handle, and leaving the scope's other jobs;
  • a record rewritten before the scan, between the scan and the script, or after completing;
  • a record that is already gone, non-failures refused, and cache errors;
  • a late write after a DELETE;
  • fail-closed lookups only inside the block;
  • a NaN in a typed failure's result;
  • ordinary and disabled-backend writes keeping their result.

• Real Redis, in tests/integration/test_runtime_cache_redis.py (3 new tests, run against a local Redis with RUNTIME_CACHE_TEST_URL=redis://127.0.0.1:6391/0):

  • the script removes only the unchanged failure, and both its index entries;
  • it refuses a record rewritten just before it runs;
  • a late write cannot put a deleted failure back.

  Script mutations that delete unconditionally or skip the index removal fail these tests.

• New unit tests elsewhere: the stand-in's script and xx emulation, the service clock and countdown, the pass-through delete, and a stored failure whose lifetime cannot be read giving no waits.

• Existing tests updated for the new field and note text:

  • the failure replay tests, whose service clock now follows the in-memory cache's clock, so both waits count down together;
  • the request-plumbing and impact-execution failure tests, and the service mock's default;
  • the to_dict tests;
  • the unreachable-simulation tests (annual body with the field, budget-window body without);
  • the dataset-validation and refused-breakdown 502 bodies.

• ruff check, ruff format --check, mypy and scripts/run_quality_guards.py pass.

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 98.62069% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.63%. Comparing base (6d91ba0) to head (3c35f87).
⚠️ Report is 41 commits behind head on master.

Files with missing lines	Patch %	Lines
policyengine_api/runtime_cache/reform_impacts.py	95.45%	0 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4198      +/-   ##
==========================================
+ Coverage   88.52%   88.63%   +0.11%     
==========================================
  Files         304      304              
  Lines       18313    18442     +129     
  Branches     3057     3084      +27     
==========================================
+ Hits        16211    16346     +135     
+ Misses       1363     1357       -6     
  Partials      739      739              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.