[gtokman]

Resolves #359.

The calls a model makes in one round now run concurrently, in the shared resolver every provider's tool loop goes through (resolveProfileToolCalls in ProfileToolExecution.swift). MLX and llama.cpp now use it too, in place of their own copies.

As discussed in the issue:

• Always on. This matches Foundation Models, which requires tools to be Sendable and may call a tool concurrently with itself or with other tools. A tool that isn't safe to run alongside others can serialize its own work. There's no setting.
• parallel_tool_calls is unchanged. It governs what the model may ask for, not how the calls run.
• Decisions and reports in the model's order. The delegate and the profile decide each call in order before any runs, and a .stop still ends the round first. Once the calls are done, the profile's tool-output callbacks and didExecuteToolCall hear about them in the model's order, one at a time. Results keep that order.
• Errors. When a call throws, the round is cancelled first: the other calls are cancelled and waited for. Only then does the delegate hear about the error, and only the one the round throws.

Each call sees the history as the round began, since the calls before it in the round may still be running.

With MLX and llama.cpp on the shared resolver, a dynamic profile's tool callbacks run for them too.

Tests

ToolRoundTests uses gates rather than timings, so they don't depend on how fast the runner is:

• Calls run at once. Each call waits for the other to start.
• Model order. The first call finishes last, yet the results and the delegate keep the model's order, one callback at a time.
• Cancellation first. A call that throws cancels the others before the delegate hears about it.
• One error. Two calls fail while the delegate is slow; it hears about only the one the round throws.
• .stop. A stop decision ends the round before any call runs.
• Provided output and a missing tool. Both answer without running a tool.

The full suite has the same failures as main here: the 13 tests that need live services. It builds with --traits MLX,Llama.

The README's Tool Calling section notes that calls run at once and when the delegate hears about them.

🤖 Generated with Claude Code

[mattt]

Thanks, @gtokman. It's great to have seven copies of the tool loop down to one. Two things before we can merge this:

1. When a call throws, invokeToolCall awaits didFailToolCall before it throws, so the group can't cancel the other calls until the delegate returns. If another call fails during that wait, its error can reach the group first, and the delegate hears about both failures. Could the failure cancel the round first, and then report to the delegate only the error that the round throws? A test with a slow didFailToolCall and two failing tools would cover it.
2. didExecuteToolCall now runs as each call finishes, so the delegate hears about calls in the order they complete, and the callbacks can overlap. Before, it heard about them in the model's order, one at a time, and I'd like to keep that. Could the calls still run at once, but report to the delegate in the model's order, one at a time? A test where the calls finish in reverse order would cover it.

[gtokman]

Thanks, @mattt. I rebased on main, and since the dynamic profiles work had already given the tool loops one shared resolver, I moved the concurrency into resolveProfileToolCalls and pointed MLX and llama.cpp at it too.

1. Errors: a failing call now cancels the round first. The other calls are cancelled and waited for, and only then does didFailToolCall hear about the error the round throws, and only that one. onlyTheRoundsErrorReachesTheDelegate has two failing tools and a slow didFailToolCall.
2. Order: the calls still run at once, but didExecuteToolCall, and the profile's tool-output callbacks, hear about them in the model's order, one at a time, after the calls are done. theDelegateHearsInTheModelsOrderOneAtATime has the calls finish in reverse order.

The tests now coordinate with gates rather than timing the round, which is what failed on CI.

Two things worth a look:

• History: each call now sees the history as the round began, rather than with the outputs of the calls before it.
• Profiles on local models: with MLX and llama.cpp on the shared resolver, a dynamic profile's tool callbacks run for them too.

[mattt]

Thanks, @gtokman! I merged main and added two commits on top of yours.

The first fixes a bug with MLX and llama.cpp. Each resolver made a ToolCalls entry for callbacks, and its caller made another when it recorded the results, so a two-round response kept two copies of the first round, and a callback that wrote history could save the temporary one. Now each round's entry is created once and kept through to the transcript.

The second updates the docs. A Tool's history now reads as earlier rounds plus the current round's calls, without the outputs of sibling calls, which matches Foundation Models 27. MLX and llama.cpp are now listed with the providers that run onToolCall and onToolOutput.