Repository navigation
Send assistant messages in Responses requests - #401
Merged
Merged
Conversation
The OpenAI Responses and Open Responses builders dropped every assistant message, so a later turn showed the model only the user's side. Send each answer as an input item, and keep a tool round's text with its calls so that replay sends each piece of text once, in order.
Replay used a response's metadata even after a caller edited the response, so removed text could reach the server. Apply the metadata only to a response that is unchanged since it was written; otherwise send its segments, and the turn's tool rounds send only their calls. A handoff also lost text that no tool round carried. Keep that text on the response.
Gemini restored stored text parts and their signatures without checking the response's current segments, so an edited answer still sent the original text. Use stored parts only for text the segments still contain. A stopped structured stream's JSON was both the response and the tool round's text, so it could be sent twice or survive an edit. Treat it as the response's.
A stopped string stream marked its response with the OpenAI metadata keys, so Chat Completions, Responses, and Open Responses replayed an empty response and dropped the round's text. Gemini's replay doesn't need the marker for a string response, so leave it out. Add tests of several tool rounds in one turn.
A stop returns the round's JSON as the response, but the tool round kept the same text in its metadata. Rebuilding or removing the response discarded the marker that replay used to clean it up, so the original text was still sent. Don't keep the text on the round, and remove the marker and that cleanup.
mattt
added a commit
to gtokman/AnyLanguageModel
that referenced
this pull request
Nov 6, 2026
Replay treated each prompt as the start of a turn, so a message from pendingMessages(in:) ended the turn early. The rounds before the message lost their text on a later turn, which breaks "each piece of text is sent once and in order" from huggingface#401. Mark these prompts, and keep the mark in saved transcripts, because a prompt's position can't separate them from a later turn's prompt.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The OpenAI Responses and Open Responses request builders dropped every assistant message, so in a multi-turn session the model saw only the user's side of the conversation. Both builders now send each earlier answer as an input item, and a tool round's text stays with that round's calls, so a later turn sends each piece of text once, including after a model handoff or from a saved transcript. A transcript saved by 1.1.1 replays a tool round's text in the response rather than with the calls. Replay also now follows history edits: if a response has changed, or a finished turn has no response, only the current text and the calls are sent. This applies to Chat Completions and Gemini as well, where an edited answer could otherwise still send the original text. In Gemini, an edited answer is sent without its old text signature, and a tool round's unsigned text is no longer sent twice. Text that a model wrote after a function call replays before that call, and a cancelled turn replays its calls and outputs without the round's text.