Skip to content

fix(schema): keep structured output JSON serializable - #7591

Open
BlueRaddish wants to merge 2 commits into
google:mainfrom
BlueRaddish:fix/oct17-ao-adk
Open

BlueRaddish wants to merge 2 commits into
google:mainfrom
BlueRaddish:fix/oct17-ao-adk

Conversation

@BlueRaddish

@BlueRaddish BlueRaddish commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

Related to #7480. This covers date-valued structured output both through normal schema validation and the set_model_response fallback used with tools.

Problem: Pydantic validation can turn JSON strings into Python date objects. Dumping those models in Python mode puts non-JSON values into AgentTool responses or final structured output. The fallback used when a model cannot combine tools with an output schema has the same problem, as identified in this review.

Solution: Dump validated single models and list items in JSON mode in both validate_schema and SetModelResponseTool.run_async. Retain exclude_none=True, existing validation feedback, and primitive/non-model behavior. Schema-declaration metadata is unchanged. No API, dependency or provider change is introduced.

Testing Plan

The regression results use two distinct baselines:

  • Original three cases: against a6ed85905d4c1d1f285ba9d4f76a9d801b66d17f, model/list schema output retains Python dates and a real Runner/AgentTool response fails in the public Anthropic converter. All three fail there and pass after the original fix. Historical verification.
  • New fallback cases: against the previously published head a01591c78332256ca14579184f4e9091779d3851, both single/list Runner cases with tools raise TypeError: Object of type date is not JSON serializable; the matching no-tools controls pass. All four pass with this follow-up. They assert explicit output_schema_and_tools=False, the actual injected tool, final JSON and output_key session state.

The exact final normally-hooked native commit bb7472c3fa73aa305a662381b63bca965be56956 passes the unchanged full supported tox matrix in isolated public Ubuntu jobs:

Python Full-suite job Passed Skipped Xfailed Xpassed
3.11 job 20211 93 19 2
3.12 job 20202 94 19 2
3.13 job 20202 94 19 2
3.14 job 20203 93 19 2

Each job also repeats the two genuine fallback failures and two controls on the previous published head, passes all four fixed cases and the complete schema/tool/flow owning paths, and verifies actual import origins, source/configuration hashes and dependency-lock preservation. Normal scoped hooks and the actual Git pre-commit hook pass. Raw native commit bytes verify the author, committer, parent, tree and message; all four jobs produce the same commit. Test-defined skips, xfails, xpasses and warnings are retained in the raw logs.

Unit Tests:

  • Added regression coverage for single/list schema output, real AgentTool conversion and real Runner fallback paths.
  • All four required full unit-test tox environments pass with the final patch.

Manual End-to-End (E2E) Tests:

The deterministic consuming tests use the repository's scripted model with its real InMemoryRunner, tool dispatch and session state; the original consuming case also calls the real public Anthropic converter. No live model/provider, browser UI or manual cloud acceptance test was run.

Checklist

  • Read CONTRIBUTING.md and reviewed the source change.
  • Added tests that reproduce each defect against its stated unchanged baseline.
  • The full supported tox matrix passes with the final patch.
  • Manually tested against a live provider or UI.

Additional context

The initial follow-up pilot proved both fallback failures, both controls, all four fixed cases and 1,904 relevant owning tests, then failed only because Pyink wrapped three test regions. Its exact formatter output was applied with unchanged test AST and production bytes before the full matrix above.

OpenAI Codex assisted with diagnosis, implementation, deterministic verification and this description. A fresh complete open-PR path inventory and matching diffs were checked for supersession; no other current patch touches the fallback tool's production or owning-test file. The separate non-model schema proposal #7285 leaves these model dumps unchanged.

Serialize validated model output in JSON mode so date-valued AgentTool
results survive the real Runner and tool-result serialization. Keep the
existing exclusion of None fields for model and list output schemas.

Related to google#7480.

AI-assisted with OpenAI Codex.

@Somay-kousis Somay-kousis left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked the three new tests against the base (a01591c78~1) and the head: test_structured_output_preserves_json_date[False], [True] and test_agent_tool_structured_date_output_survives_anthropic_conversion fail on the base and pass on the head.

One sibling path still has the bug. When an agent has both output_schema and tools and its model can't do both natively (capabilities.output_schema_and_tools is False, which is the case for claude-* names and the LiteLlm providers listed in lite_llm.py), the final answer goes through SetModelResponseTool.run_async, not validate_schema. That tool still dumps in Python mode (set_model_response_tool.py:304 and :313), and get_structured_model_response then does json.dumps(response) (flows/llm_flows/prompt/_schema.py:120). So a date field makes the run raise, on any model, before AgentTool or the Anthropic converter come into it.

Repro: one LlmAgent with output_schema=DatedOutput (day: date), output_key="out", a fake model, InMemoryRunner. With tools=[lookup] the model answers by calling set_model_response with {"day": "2026-10-17"}.

On this branch:

--- output_schema=DatedOutput, tools=[]
capabilities.output_schema_and_tools = False
  final text: {"day": "2026-10-17"}
  state out: {'day': '2026-10-17'}
--- output_schema=DatedOutput, tools=[lookup]
capabilities.output_schema_and_tools = False
  run raised: TypeError: Object of type date is not JSON serializable

Adding mode="json" to the two model_dump calls in SetModelResponseTool.run_async fixes it (I tried it on your branch and reverted):

--- output_schema=DatedOutput, tools=[lookup]
capabilities.output_schema_and_tools = False
  final text: {"day": "2026-10-17"}
  state out: {'day': '2026-10-17'}

With that change tests/unittests/tools/test_set_model_response_tool.py, tests/unittests/utils, tests/unittests/tools/test_agent_tool.py and tests/unittests/flows give 1900 passed. Might be worth folding in here with a test in test_set_model_response_tool.py, since it's the same dump and the same #7480 case.

Probe script: probe_7591.py (outputs probe_7591_head.out, probe_7591_base.out, probe_7591_suggested.out).

Serialize validated BaseModel output and list items in JSON mode before
set_model_response enters event and final-response JSON serialization.
Preserve validation feedback and omission of None fields.

Refs google#7480

AI-assisted with OpenAI Codex.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants