Skip to content

Codegen: anyOf branch conditions are discarded — only the branches' top-level required survives #1414

Closed

Description

@KonstantinMirin

Pin measured: adcp==9.0.0 (AdCP spec 3.2.1)
Blast radius measured in this pin: 42 models emit the flattened validator; 14 of those have
branches carrying nested conditions; 12 of those drop a real constraint.

On urgency — our own exposure is one model, so please don't read the 42 as 42 broken
adopters.
We walked every request DTO our seller validates and only CreateMediaBuyRequest
was actually exposed; CreativeAsset is the other reachable one and it turns out to be saved by
a different generated mechanism (see below). The rest of the 12 are response-side or belong to
operations we don't implement. We have a local workaround in place. Filing this because the
defect is in the generator rather than in any one schema, and the next adopter to hit it may be
exposed somewhere we are not.

What the generator emits

For a schema whose root carries an anyOf, the generator emits a single presence check.
media-buy/create-media-buy-request.json →
adcp/types/domains/media_buy/create_media_buy_request.py:331:

@model_validator(mode='after')
def _require_schema_required_group(self) -> CreateMediaBuyRequest:
    for group in (('packages',), ('packages', 'total_budget', 'budget_allocation'), ('proposal_id', 'total_budget')):
        if all(name in self.model_fields_set for name in group):
            return self
    raise ValueError(
        'CreateMediaBuyRequest requires at least one of these field groups: '
        'packages | packages+total_budget+budget_allocation | proposal_id+total_budget'
    )

Each tuple is one branch's top-level required array. Everything else in the branch is
discarded.

What that drops, for this one schema

The 3.2.1 anyOf has three branches. Alongside their required arrays they carry:

dropped condition branch
packages.items.allOf[{"required": ["budget"]}] 0 — explicit packages, fixed allocation
packages.items.allOf[{"not": {"required": ["min_spend_target"]}}] 0
properties.budget_allocation.properties.mode.const = "fixed" 0
properties.budget_allocation.properties.mode.const = "seller_optimized" 1 — seller-optimized
not: {"required": ["proposal_id"]} 0 and 1
not: {"anyOf": [{"required":["packages"]}, {"required":["budget_allocation"]}, {"required":["frequency_cap"]}]} 2 — committed proposal

Reproduction

from adcp.types import CreateMediaBuyRequest

doc = {
    "account": {"account_id": "acct_test"},
    "brand": {"domain": "example.com"},
    "idempotency_key": "0123456789abcdef0123",
    "start_time": "2026-10-07T00:00:00+00:00",
    "end_time": "2026-11-05T00:00:00+00:00",
    "packages": [{"product_id": "prod_1", "pricing_option_id": "cpm_usd_fixed"}],
}

CreateMediaBuyRequest.model_validate(doc)   # ACCEPTED

packages is present, so group ('packages',) matches. But the document satisfies no
branch of the schema:

  • branch 0 requires every package to carry budget — this one does not;
  • branch 1 requires total_budget + budget_allocation — neither is present;
  • branch 2 requires proposal_id + total_budget — neither is present.

jsonschema over the same pinned file agrees it is invalid:

import jsonschema, json
schema = json.load(open(".../adcp/_schemas/3.2/media-buy/create-media-buy-request.json"))
list(jsonschema.Draft7Validator(schema).iter_errors(doc))   # non-empty

So the generated validator and the schema it was generated from disagree about this document.

Why it matters downstream

In our seller the accepted document reached a business rule, which summed the absent package
budget to 0 and answered BUDGET_TOO_LOW — a budget-floor code for a document that supplied
no budget at all. Per enums/error-code.json, INVALID_REQUEST is the code whose condition is
"Request is malformed, missing required fields, or violates schema constraints", and
BUDGET_TOO_LOW is "Budget is below the seller's minimum". Both are recovery: correctable, so
recovery is unaffected, but the suggestion the buyer receives becomes "increase budget or
check capabilities.media_buy.limits" for a request that supplied none.

The business rule was correct. It was answering a question it should never have been asked,
because validation let the document through.

Blast radius in this pin

Measured over adcp/types/**:

  • 42 models emit _require_schema_required_group.
  • 14 of those have a root anyOf/oneOf whose branches carry anything beyond required.
  • 12 of those drop a condition that is a real constraint rather than an annotation
    (Product and Transformer carry only deprecated / description on their branches, so
    flattening them is harmless).

The 12: AudienceCharacteristic, Catchment, CreateMediaBuyRequest, CreativeAsset,
CreativeManifest, FrequencyCapImpressionConstraints, FrequencyCapIntervalConstraints,
GetReportingStatusResponse, ListAccountChangesResponse, RequestProposalsRequest,
SearchBrandsResponse, VerifyBrandClaimRequest.

The dropped conditions are mostly not clauses — i.e. the mutual exclusions between
branches. CreativeAsset and CreativeManifest are the clearest shape: format_id | format_kind
where each branch carries not: {required: [the other]}. Those two happen to be saved by a
separate generated mechanism (a oneOf validator that does emit "provide exactly one of
format_id or format_kind"), which is worth noting as the pattern that works — the two mechanisms
are not consistent with each other.

Suggested fix

Generate the branch's full subschema rather than only its required list. Where a condition is
not expressible as a pydantic validator, the honest options are to raise at import time or to
emit a documented gap, rather than to silently narrow the contract to field presence — a
validator that is more permissive than its schema is worse than no validator, because callers
reasonably treat model_validate as the contract.

If a full implementation is out of scope, the oneOf path already emits mutual exclusion
correctly; routing the anyOf not clauses through the same code would close the largest
share of the 12.

What we did locally

Re-stated the three branches by hand as a model_validator on our subclass, and fire-probed it
against the pinned schema across 18 documents covering every branch and every dropped
condition — the DTO and jsonschema now agree on all 18. We would rather delete that code than
maintain it, hence this report.

Activity

  1. added
    claude-triagingTriage routine is actively working on this issue (1-3 min)
    on Oct 6, 2026
  2. bokelley commented on Oct 6, 2026

    @bokelley
    Contributor

    Triage

    Classification: Bug
    Bucket(s): validation (no validation label in repo — gap noted in run summary)
    Status: ready-to-implement
    Milestone: omitted (no version target stated)

    What the experts said:

    • ad-tech-protocol-expert: High-severity correctness bug confirmed — the CreateMediaBuyRequest schema carries x-mutates-state: true + x-governed-commitment: {scope: "always"}, so the SDK validator is the client-side pre-screen that governance agents and sellers rely on; a silent pass on a mutually-exclusive field combination puts a structurally invalid request into the governance pipeline. Fix is a bug fix (patch-level), not a breaking change; callers following the spec are unaffected.
    • code-reviewer: (review in progress at post time — finding pending; implementation brief below is based on confirmed root cause)

    My take: The root cause is fully confirmed by code inspection: _root_required_groups in scripts/post_generate_fixes.py:7029 extracts only arm["required"] from each branch and passes the result to enforce_root_required_groups, which injects the validator — the not, const, and nested allOf conditions are never read. The right fix scope is targeted: extend _root_required_groups to return per-arm not.required exclusions (the dominant dropped condition across the 12 models) and have enforce_root_required_groups emit the exclusion checks alongside the presence check. Full-subschema reconstruction (nested allOf item validators, const property checks) requires traversing $ref chains across generated files and belongs in validators on the sub-models directly, not in this post-processor.

    Implementation scope:

    • scripts/post_generate_fixes.py — two changes:

      1. _root_required_groups (line ~7029): extend return type to also carry per-arm not.required exclusion lists (not: {required: [field]} and not: {anyOf: [{required: [field]}, ...]} shapes, both present in the 12 affected schemas).
      2. enforce_root_required_groups (line ~7217): (a) remove the already += 1; continue early return — the function must strip and re-inject validators to update already-emitted but incomplete ones; (b) extend the emitted validator body to check all(ex not in self.model_fields_set for ex in exclusions) conditioned on the arm's required group having matched, before the early return self.
    • Re-run post-generation step (python scripts/post_generate_fixes.py) to update the 12 files in src/adcp/types/domains/. Do not re-run datamodel-code-generator itself — that risks class-name renumber churn (see CLAUDE.md).

    • Tests (in tests/): (a) unit test for _root_required_groups covering not.anyOf compound shape; (b) integration test using the reporter's exact reproduction case (CreateMediaBuyRequest with packages: [{product_id, pricing_option_id}] — should raise ValidationError); (c) at least one positive-path test per branch.

    Required checks before PR: make lint && make typecheck-all && make test

    Non-breaking rationale: The changed validators reject documents that already violate the spec. Callers constructing valid requests are unaffected. The create_media_buy operation is itself deprecated since spec 3.2.0, which limits blast radius further. Release note should call out the tightened validation explicitly.


    Triaged by Claude Code. Session: https://claude.ai/code/cse_01JugFAj3xRfR8Jwf3opHSwr


    Generated by Claude Code

  3. added
    bugSomething isn't working
    and removed
    claude-triagingTriage routine is actively working on this issue (1-3 min)
    on Oct 6, 2026
  4. KonstantinMirin commented on Nov 11, 2026

    @KonstantinMirin
    CollaboratorAuthor

    This issue's own reproduction still holds on adcp==9.6.1, and on main at bef739e02

    Re-ran the payload from the report verbatim against the PyPI wheel adcp 9.6.1 (CPython 3.12.13), with jsonschema over the wheel's own bundle copy as ground truth:

    from adcp.types import CreateMediaBuyRequest
    doc = {
        "account": {"account_id": "acct_test"},
        "brand": {"domain": "example.com"},
        "idempotency_key": "0123456789abcdef0123",
        "start_time": "2026-10-07T00:00:00+00:00",
        "end_time": "2026-11-05T00:00:00+00:00",
        "packages": [{"product_id": "prod_1", "pricing_option_id": "cpm_usd_fixed"}],
    }
    CreateMediaBuyRequest.model_validate(doc)      # ACCEPTED
    import json, jsonschema, pathlib
    B = pathlib.Path(adcp.__file__).parent / "_schemas" / "3.2"
    s = json.loads((B / "bundled/media-buy/create-media-buy-request.json").read_text())
    sorted({e.validator for e in jsonschema.Draft7Validator(s).iter_errors(doc)})
    # ['anyOf']   -> the document refuses it

    What did change since the close: the refusal now names the keyword — PydanticCustomError('anyOf', ...) instead of ValueError — which is a real improvement and is the convention we ended up at independently. What did not change is the thing the title names: _require_schema_required_group in src/adcp/types/domains/media_buy/create_media_buy_request.py still carries only the three top-level required tuples, and every other condition in those branches is still discarded.

    Not sure whether this was closed as fixed or closed as "the model layer's contract is required-groups only". If the latter, that is a reasonable position and this comment is a request to say so in the docs — but then adcp.validation is the only arm that answers for these documents, and it is opt-in and keyed by tool name, so nothing on the model_validate path reaches it.

    The half that is cheapest to carry, and the measurement for it

    Of the dropped conditions in the table above, one is a single keyword in the same place the existing validator already walks: not over required, per branch.

    Measured on the wheel's own bundle copy (adcp/_schemas/3.2, manifest.json says adcp_version 3.2.7), excluding the bundled/ and mcp/ per-tool projections so nothing is counted twice:

    positions documents
    combinator-branch not over required 118 74
    root-level not over required 11 11
    total 129 85

    (of 1,057 documents; with the bundled/ and mcp/ projections included the same scan reads 1,473 positions in 315 documents, because those re-cut the same nodes per tool.)

    Scan used:

    for pointer, node in walk(doc):           # every mapping, with its JSON pointer
        for comb in ("anyOf", "oneOf", "allOf"):
            for i, branch in enumerate(node.get(comb) or []):
                if isinstance(branch.get("not"), dict) and "required" in branch["not"]:
                    hit(f"{pointer}/{comb}/{i}/not")

    A second exhibit, smaller than create_media_buy and entirely self-contained

    core/frequency-cap-interval-constraints.json — its own description says "The two forms are mutually exclusive":

    "oneOf": [
      {"required": ["allowed_intervals"],
       "not": {"anyOf": [{"required": ["minimum_interval"]}, {"required": ["maximum_interval"]}]}},
      {"required": ["minimum_interval", "maximum_interval"],
       "not": {"required": ["allowed_intervals"]}}
    ]
    payload the document FrequencyCapIntervalConstraints.model_validate
    {unit, allowed_intervals} ACCEPTED ACCEPTED
    {unit, minimum_interval, maximum_interval} ACCEPTED ACCEPTED
    {unit, allowed_intervals, minimum_interval, maximum_interval} REFUSED on oneOf ACCEPTED
    {unit, allowed_intervals, minimum_interval} REFUSED on oneOf ACCEPTED
    {unit} REFUSED on oneOf REFUSED on oneOf

    So the require half lands and the forbid half does not, in the same node. FrequencyCapIntervalConstraints.model_json_schema() publishes {"required": ["unit"]} and no oneOf at all, so a consumer generating a validator from the model's own published schema loses it too.

    Where it would go

    The generated validator already walks exactly the structure the nots sit on (src/adcp/types/domains/core/frequency_cap_interval_constraints.py, 9.6.1):

    @model_validator(mode='after')
    def _require_schema_required_group(self) -> FrequencyCapIntervalConstraints:
        for group in (('allowed_intervals',), ('minimum_interval', 'maximum_interval'),):
            if all(name in self.model_fields_set for name in group):
                return self
        raise PydanticCustomError('oneOf', '... requires at least one of these field groups: ...')

    42 generated modules carry this validator. A branch's forbid set is the same model_fields_set question with the sense reversed, so a branch would match when its require group is present and its forbid group is absent.

    One rider, and it is load-bearing: the two halves must stay paired per branch and must never be hoisted into one forbid set. core/catchment.json is the case that shows why — the union of its branches' nots forbids every one of its method keys, so a hoisted forbid set would refuse every payload the document admits.

    A question beside it, from the same code

    The message says "requires at least one of these field groups" and the keyword it raises is oneOf, which is exactly-one. Where branches are not mutually exclusive by their own nots, a payload satisfying two of them is one the document refuses and the validator admits — adagents.json's oneOf over required: [authoritative_location] / required: [authorized_agents] is that shape, with no not on either branch. 56 documents in this bundle have a oneOf whose branches carry distinct required groups and where not every branch carries a not. Is at-least-one deliberate here, or is the exactly-one check part of the same gap?

    Happy to send a PR for the per-branch forbid half if you want it in this shape.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions