Skip to content

Copy v1 households that leave people out of group collections to v2 - #4165

Closed
anth-volk wants to merge 3 commits into
masterfrom
fix/household-copy-translation-failures
Closed

anth-volk wants to merge 3 commits into
masterfrom
fix/household-copy-translation-failures

Conversation

@anth-volk

@anth-volk anth-volk commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #4164

Summary

  • The v1-to-v2 household translation now creates the groups that policyengine-core creates when it builds a simulation from a v1 household:
    • an omitted or null group collection gets one group of every person;
    • each person that a present collection's groups leave out gets a group of their own.
  • These groups get stable IDs after the collection's named groups (<collection>-<n>, numbered in person-name order), no source_name and no values. Named groups translate exactly as before.
  • Households that policyengine-core cannot parse, such as ones with a top-level key that is not an entity collection, are still rejected.

Two of the four production events in #4164 have this shape. With this change, their stored households translate, and each translated collection has the same number of groups as the policyengine-core simulation built from the v1 household. The other two events are households that policyengine-core rejects, and they still fail translation.

Commits

  1. Reproduce v1 household copies rejected …: the new tests alone. They fail on master with the error production logged, people record memberships must identify every country-supported entity collection (and legacy households must be an object for a null collection).
  2. Give people a v1 household leaves out …: the fix and the changelog fragment.
  3. Fail the integration test's household copy …: tests/integration/test_v1_household_dual_write.py::test_translation_failure_retains_one_pending_event_until_explicit_replay used a household of people alone as its translation failure. policyengine-core calculates that household, so it now copies. The test now uses a top-level key that is not an entity collection, which policyengine-core and the translation both reject.

Testing

Ran only the new tests, by node ID:

  • tests/unit/v2/test_household_translation.py::test_omitted_collection_becomes_one_unnamed_group_of_everyone (US with three collections omitted, US with a null households and every other collection omitted, UK with benunits omitted)
  • tests/unit/v2/test_household_translation.py::test_person_left_out_of_a_collection_gets_an_unnamed_group_of_their_own

All failed on the first commit and pass on the second. The modified integration test needs the disposable databases that CI provides, so it ran only in CI; locally, its new household fails translation with legacy household contains unsupported collections. ruff format, ruff check and mypy pass for the changed files. The rest of the suite runs in CI.

After merge

Once this is deployed to production, each of the two affected events is copied by dispatching process-v1-household-mirror-event.yml for its ID, one run at a time.

🤖 Generated with Claude Code

anth-volk and others added 2 commits November 4, 2026 01:05
…aves out

policyengine-core gives every person a group in each collection: one group of
everyone when a collection is omitted or null, and a group of their own when a
present collection's groups leave them out. The v1-to-v2 translation requires
an explicit membership instead and rejects these households, so their copies
to v2 fail with failure_category "translation".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… gives them

translate_legacy_household now creates the groups policyengine-core creates
when it builds a simulation from a v1 household: one group of everyone for an
omitted or null collection, and a group of their own for each person a present
collection's groups leave out. The groups have stable ids after the named
groups, no source_name and no values.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.90909% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 88.20%. Comparing base (f29abfa) to head (7f56b4c).
⚠️ Report is 37 commits behind head on master.

Files with missing lines Patch % Lines
...gine_api/services/v2/households/transformations.py 90.90% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4165      +/-   ##
==========================================
- Coverage   88.22%   88.20%   -0.02%     
==========================================
  Files         285      286       +1     
  Lines       16843    16886      +43     
  Branches     2741     2755      +14     
==========================================
+ Hits        14859    14895      +36     
- Misses       1302     1304       +2     
- Partials      682      687       +5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…ects

A household of people alone now translates, because policyengine-core gives
them one group of everyone in each omitted collection. The test that keeps a
failed copy pending now uses a top-level key that is not an entity collection,
which policyengine-core and the translation both reject.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was previously deployed

1 inactive deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

v1 household copies to v2 fail when the household leaves out group collections

1 participant