Repository navigation
fix(schema): keep structured output JSON serializable - #7591
BlueRaddish wants to merge 2 commits into
Conversation
Serialize validated model output in JSON mode so date-valued AgentTool results survive the real Runner and tool-result serialization. Keep the existing exclusion of None fields for model and list output schemas. Related to google#7480. AI-assisted with OpenAI Codex.
Somay-kousis
left a comment
There was a problem hiding this comment.
Checked the three new tests against the base (a01591c78~1) and the head: test_structured_output_preserves_json_date[False], [True] and test_agent_tool_structured_date_output_survives_anthropic_conversion fail on the base and pass on the head.
One sibling path still has the bug. When an agent has both output_schema and tools and its model can't do both natively (capabilities.output_schema_and_tools is False, which is the case for claude-* names and the LiteLlm providers listed in lite_llm.py), the final answer goes through SetModelResponseTool.run_async, not validate_schema. That tool still dumps in Python mode (set_model_response_tool.py:304 and :313), and get_structured_model_response then does json.dumps(response) (flows/llm_flows/prompt/_schema.py:120). So a date field makes the run raise, on any model, before AgentTool or the Anthropic converter come into it.
Repro: one LlmAgent with output_schema=DatedOutput (day: date), output_key="out", a fake model, InMemoryRunner. With tools=[lookup] the model answers by calling set_model_response with {"day": "2026-10-17"}.
On this branch:
--- output_schema=DatedOutput, tools=[]
capabilities.output_schema_and_tools = False
final text: {"day": "2026-10-17"}
state out: {'day': '2026-10-17'}
--- output_schema=DatedOutput, tools=[lookup]
capabilities.output_schema_and_tools = False
run raised: TypeError: Object of type date is not JSON serializable
Adding mode="json" to the two model_dump calls in SetModelResponseTool.run_async fixes it (I tried it on your branch and reverted):
--- output_schema=DatedOutput, tools=[lookup]
capabilities.output_schema_and_tools = False
final text: {"day": "2026-10-17"}
state out: {'day': '2026-10-17'}
With that change tests/unittests/tools/test_set_model_response_tool.py, tests/unittests/utils, tests/unittests/tools/test_agent_tool.py and tests/unittests/flows give 1900 passed. Might be worth folding in here with a test in test_set_model_response_tool.py, since it's the same dump and the same #7480 case.
Probe script: probe_7591.py (outputs probe_7591_head.out, probe_7591_base.out, probe_7591_suggested.out).
Serialize validated BaseModel output and list items in JSON mode before set_model_response enters event and final-response JSON serialization. Preserve validation feedback and omission of None fields. Refs google#7480 AI-assisted with OpenAI Codex.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
Related to #7480. This covers date-valued structured output both through normal schema validation and the
set_model_responsefallback used with tools.Problem: Pydantic validation can turn JSON strings into Python
dateobjects. Dumping those models in Python mode puts non-JSON values into AgentTool responses or final structured output. The fallback used when a model cannot combine tools with an output schema has the same problem, as identified in this review.Solution: Dump validated single models and list items in JSON mode in both
validate_schemaandSetModelResponseTool.run_async. Retainexclude_none=True, existing validation feedback, and primitive/non-model behavior. Schema-declaration metadata is unchanged. No API, dependency or provider change is introduced.Testing Plan
The regression results use two distinct baselines:
a6ed85905d4c1d1f285ba9d4f76a9d801b66d17f, model/list schema output retains Python dates and a real Runner/AgentTool response fails in the public Anthropic converter. All three fail there and pass after the original fix. Historical verification.a01591c78332256ca14579184f4e9091779d3851, both single/list Runner cases with tools raiseTypeError: Object of type date is not JSON serializable; the matching no-tools controls pass. All four pass with this follow-up. They assert explicitoutput_schema_and_tools=False, the actual injected tool, final JSON andoutput_keysession state.The exact final normally-hooked native commit
bb7472c3fa73aa305a662381b63bca965be56956passes the unchanged full supported tox matrix in isolated public Ubuntu jobs:Each job also repeats the two genuine fallback failures and two controls on the previous published head, passes all four fixed cases and the complete schema/tool/flow owning paths, and verifies actual import origins, source/configuration hashes and dependency-lock preservation. Normal scoped hooks and the actual Git pre-commit hook pass. Raw native commit bytes verify the author, committer, parent, tree and message; all four jobs produce the same commit. Test-defined skips, xfails, xpasses and warnings are retained in the raw logs.
Unit Tests:
Manual End-to-End (E2E) Tests:
The deterministic consuming tests use the repository's scripted model with its real InMemoryRunner, tool dispatch and session state; the original consuming case also calls the real public Anthropic converter. No live model/provider, browser UI or manual cloud acceptance test was run.
Checklist
Additional context
The initial follow-up pilot proved both fallback failures, both controls, all four fixed cases and 1,904 relevant owning tests, then failed only because Pyink wrapped three test regions. Its exact formatter output was applied with unchanged test AST and production bytes before the full matrix above.
OpenAI Codex assisted with diagnosis, implementation, deterministic verification and this description. A fresh complete open-PR path inventory and matching diffs were checked for supersession; no other current patch touches the fallback tool's production or owning-test file. The separate non-model schema proposal #7285 leaves these model dumps unchanged.