Skip to content

Move the simulation executor to policyengine.py 6.14.0 and keep accepting US 2022 and 2023 #1012

Open

Description

@anth-volk

Problem

Since policyengine.py 6.13.0 (PolicyEngine/policyengine.py#730), create_datasets and ensure_datasets raise UnsupportedDatasetYearError when a requested year is outside the years the installed country model supports the source dataset for. For the US, that refuses 2022 and 2023, which come before the 2024 data year of the certified dataset. Passing allow_unsupported_years=True skips the check. Year files that already exist are loaded without it, so the check applies whenever a year file has to be created.

The executor loads the dataset year for every simulation job through country_module.ensure_datasets(...) in _load_dataset (projects/policyengine-simulation-executor/src/policyengine_simulation_executor/simulation_runtime.py). That covers the main simulation runtime and the Stage 12 society-wide worker. The Public API and the app still offer US 2022 and 2023 for society-wide runs. If the executor moved to policyengine.py 6.13.0 or later without a change, those runs would fail partway through with an error the caller is not told the reason for.

The executor pins policyengine[models]==6.12.0, whose ensure_datasets has no allow_unsupported_years argument, so the opt-out and the version change have to ship in the same pull request. The automated update pull requests (#1011 for 6.14.0, #1002 for 6.13.0 and #990 for 6.12.1) change only the version.

Plan

  1. Move the executor to policyengine.py 6.14.0, and pass allow_unsupported_years=True from _load_dataset, so the simulation service keeps accepting every year the Public API offers. The precompute steps that build dataset year files keep the check; they build 2025, 2026 and 2027, which are supported.
  2. The Public API stops offering US 2022 and 2023 for society-wide runs and refuses them with a 400.
  3. This repository then removes the opt-out, so the executor refuses unsupported years as well.

This issue covers step 1. Until step 3, policyengine.py updates are not held back by the year check.

Acceptance criteria

  • projects/policyengine-simulation-executor/pyproject.toml requires policyengine[models]==6.14.0 in both of its dependency lists, and uv.lock is regenerated.
  • _load_dataset passes allow_unsupported_years=True to ensure_datasets, with a comment saying the opt-out is temporary and when it is removed.
  • The precompute calls to ensure_datasets are unchanged.
  • A unit test with a fake country module shows that a request for US 2022 or 2023 loads its dataset.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions