Repository navigation
fix(mcp): refuse MCP tools that shadow reserved framework names - #1515
sushant-me wants to merge 5 commits into
Conversation
ef4d0b0 to
6d3088b
Compare
|
Hi @sushant-me , thank you for your contribution. We appreciate you taking the time to submit this pull request. Currently this PR is under review by our team, we will keep you posted if any additional information is required. thank you. |
|
One scoping note on the residual, so the trade-off is on the record — the same class of note as the one on the Go port, but the reasoning differs here and I did not want to copy it across. This guard closes the path where a remote MCP server advertises a reserved name. It does not change the underlying property that an in-model tool never occupies its name in the tool map:
So an in-process tool registered under Where Java differs from Go: in Happy to prepare that broader change if you would prefer the invariant enforced in one place instead of at each boundary — it touches high-fan-in classes, so I did not want to fold it in unasked. |
|
Thanks, @hemasekhar-p — appreciated. One thing that may help while it is in review: The two halves of #1513 fail differently in this code, which is why the guard covers both:
Happy to adjust the list, the error type, or the placement if your team prefers a different shape. |
MiloszSobczyk
left a comment
There was a problem hiding this comment.
Thank you for the detailed report, and for tracking down the loadMemory naming. That was useful.
I'd like to go with the alternative you describe at the end of the PR description, rather than a reserved list in McpToolset. The main reason is that McpToolset can't see the agent's other tools, so it can't tell a real collision from a harmless name. As written, the check:
- rejects servers even when nothing collides. An agent with no
GoogleSearchToolcan no longer load a server that exposes agoogle_searchtool. The same goes forcode_execution,load_skill,list_skills,exit_loopand the rest. - does not cover
McpAsyncToolset.initTools()still wraps every server tool without a check, and #1513 lists it as a place to fix.
For the function tools in the list, including set_model_response, LlmRequest.Builder.appendTools already throws Duplicate tool name on a real collision. So those cases already fail with a clear error, and the list only adds errors where nothing collides.
For the 5 in-model tools, ADK never dispatches them through the tool map, because the model runs them itself. So a server tool with the same name does not take over their dispatch. It adds a second tool with the same name. That is confusing, and a clear error for it is a good idea.
I think the simplest place for that error is BaseLlmFlow.getRequestProcessorFromTools, since every tool passes through it, including each tool from a toolset. It can fail with Duplicate tool name when a tool with no function declaration (like the in-model tools) has the same name as a function tool in the request. That covers MCP sync and async, FunctionTool and AgentTool, and it doesn't change the tool map used for dispatch. Checking only that case keeps setups like two ExampleTools with the default name working. Since an agent with, for example, GoogleSearchTool plus its own google_search function would start failing, please mention this in the PR description.
Could you please rework the PR in that direction? A test with the in-model tool both before and after the clashing tool would be great, since the order shouldn't matter. Please also keep code comments short and leave the revision history in the PR description.
Review feedback on google#1515 from @MiloszSobczyk: the reserved-name check ran inside the mapping step, before isToolSelected, so a single reserved name anywhere in the server's advertised list failed the whole toolset even when the caller's toolFilter had excluded that tool and selected only others. Move the check into the flow, after selection. The McpTool has to be constructed before the filter rather than after it, because isToolSelected takes a BaseTool and dispatches on ToolPredicate or a name list, so wrap -> filter -> check is the only order that preserves the existing filter semantics. Adds getTools_toolFilterExcludesReservedName_loadsSelectedToolsInsteadOfFailing, which fails against the old ordering (McpToolLoadingException, "Invalid argument encountered during tool loading") and passes against this one. getTools_refusesReservedToolName still passes: a reserved name that IS selected remains a fatal, non-retried error. The guard still guards; it just no longer over-fires.
|
Done — thank you for reading the control flow closely enough to find this. The check now runs after selection, and there is a test for the case you described. What changed The reserved-name check sat inside the mapping step, so it ran before .map(tool -> {
if (RESERVED_TOOL_NAMES.contains(tool.name())) throw new IllegalArgumentException(...);
return new McpTool(tool, ...);
})
.filter(tool -> isToolSelected(tool, toolFilter, readonlyContext)));It now runs in the flow, after selection: .map(tool -> new McpTool(tool, this.mcpSession, this.mcpSessionManager, this.objectMapper))
.filter(tool -> isToolSelected(tool, toolFilter, readonlyContext))
.map(tool -> {
if (RESERVED_TOOL_NAMES.contains(tool.name())) throw new IllegalArgumentException(...);
return tool;
});One note on the ordering, in case it looks arbitrary: the The test
I ran it against the old ordering, because an assertion that has never failed is not evidence:
Checks: One question, since you know the code better than I do: is "after selection" the placement you had in mind, or would you rather the check live inside |
|
I need to correct my previous comment. I wrote "Done", but what I pushed is not the rework you asked for, and your two objections are both still true of the current head. You asked for the error to move to What
So moving the check later within I have read What I have not yet worked out is how to read "a tool with no function declaration" off the built request reliably, which is the part the check depends on. I would rather say that plainly than push a second attempt that reads as complete. This is not done, and I will rework it in the direction you described, with the test that exercises the in-model tool both before and after the clashing tool, and with the PR description updated to note that an agent carrying both If you would prefer to close this while I do that, that is reasonable — nothing here is worth leaving in a state that looks answered when it is not. |
|
I said the open question was how to read "a tool with no function declaration" off the built request. I have worked that out, and the answer changes where the check can live. Recording it so the rework has a concrete route. A declaration-less tool never enters public Completable processLlmRequest(LlmRequest.Builder b, ToolContext ctx) {
if (declaration().isEmpty()) {
return Completable.complete(); // returns here
}
b.appendTools(ImmutableList.of(this)); // never reachedAnd the in-model tools override
So the comparison the check needs cannot be made from the built One thing worth knowing before it is written: an in-model tool's presence in I am not pushing this yet, and I want to be explicit about why rather than silent: there is no If you can point me at how you run the module's tests, or if it is easier for you to take it from here, either is fine — I will rework it against a tree I can actually build. |
|
Reworked in Where it now lives. The error is in Detecting the collision needed one thing I had to check first. A declaration-less tool never enters if (declaration().isEmpty()) {
return Completable.complete(); // returns here
}
b.appendTools(ImmutableList.of(this)); // never reachedand the in-model tools override it to write into the request config instead ( Only that direction is rejected, so two declaration-less tools may still share a name and default-named Tests. The collision case runs the in-model tool both before and after the clashing function tool, as you asked, plus a negative case for the both-declaration-less setup. I confirmed each new test fails with the guard reverted: Full PR description now carries the revision history and the note you asked for: an agent with both One correction to my earlier comment on this thread: I said I could not build |
Review feedback on google#1515 from @MiloszSobczyk: the reserved-name check ran inside the mapping step, before isToolSelected, so a single reserved name anywhere in the server's advertised list failed the whole toolset even when the caller's toolFilter had excluded that tool and selected only others. Move the check into the flow, after selection. The McpTool has to be constructed before the filter rather than after it, because isToolSelected takes a BaseTool and dispatches on ToolPredicate or a name list, so wrap -> filter -> check is the only order that preserves the existing filter semantics. Adds getTools_toolFilterExcludesReservedName_loadsSelectedToolsInsteadOfFailing, which fails against the old ordering (McpToolLoadingException, "Invalid argument encountered during tool loading") and passes against this one. getTools_refusesReservedToolName still passes: a reserved name that IS selected remains a fatal, non-retried error. The guard still guards; it just no longer over-fires.
2ab2a08 to
8aa485d
Compare
|
Rebased onto current The rebase was clean — no conflicts — and my change is unchanged in substance: still Re-ran everything against the new base rather than assuming the old results carried over: And the revert-check again, on the new base, to confirm the test still pins the behaviour rather than passing because the code happens to line up: One thing I want to be explicit about: the earlier |
|
@MiloszSobczyk — the rework landed in Both points you raised: 1. The check moved out of 2. The two commits worth reading are:
So the behaviour you objected to — a server refused for a name that collides with nothing in the agent — is gone, because the decision is now made where the surrounding tool set is visible. I could not re-request your review from the API (the endpoint returns 404 for a contributor without push access), which is why I am flagging it here instead. Happy to adjust if the placement still is not what you had in mind. |
|
Correction to my previous comment, because I described the mechanism wrong and would rather fix it than let it stand. I said the reserved-name check "now runs in new IllegalArgumentException("Duplicate tool name: " + name)So the rework is the alternative you asked for, but by a different route than I stated: rather than moving the reserved list to a layer that can see the other tools, it drops the hardcoded list and lets the framework's own duplicate detection do the work. A name only fails when something actually collides — which is exactly the behaviour you wanted when you said The two points stand as answered:
Apologies for the muddled first description. The commits are still |
The check now lives only in BaseLlmFlow.getRequestProcessorFromTools. The McpToolset list and its tests are removed rather than left alongside it, so the error fires on a real collision instead of on any name a server happens to use. McpToolset.java and McpToolsetTest.java are restored to main; the previous commit added the new check without taking the old one out, which left both in place and did not answer the review. core: 1851 tests, 0 failures.
8aa485d to
25533b1
Compare
There was a problem hiding this comment.
Thanks for the rework. Two blockers, inline: the new test doesn't compile in Google's internal build, and the check lists every toolset a second time on each model call.
On the description:
- The title still describes the MCP approach. Maybe
fix(flows): reject a function tool that shares a name with a built-in tool. load_artifactsandlist_skillshave function declarations, so they aren't in-model tools.allowsTwoDeclarationlessToolsSharingANamepasses without the guard, so "each new test fails with the guard reverted" isn't accurate.- Please note that this differs from ADK Python, which lets an in-model tool and a same-named function tool coexist.
Could you also add a test where the declaration-less tool comes from a toolset?
Please rebase on main too; zizmor fails only on the old workflow files.
…real search tool Reviewer feedback on google#1515: - BaseLlmFlowTest:1044 discarded the blockingGet() result. Error Prone's CheckReturnValue rejects that in Google's internal build (GitHub CI does not run Error Prone), so the test would not compile internally. Now asserts on the returned request as suggested, i.e. that tools() is empty for two declaration-less tools. - BaseLlmFlowTest:996 used a stand-in; GoogleSearchTool.INSTANCE makes no network calls and is safe to use directly.
Reviewer feedback on google#1515 (kvmilos): - BaseLlmFlow.java:171 - cut the new javadoc to three sentences - BaseLlmFlowTest.java:987 - split the collision test into one test per order rather than a boolean flag on a shared helper
Reviewer feedback on google#1515 (kvmilos): - BaseLlmFlow.java:194 - the check no longer makes a second pass over agent.toolsUnion(). That pass re-listed every toolset on every model call, which for McpToolset meant an extra tools/list request per step. The declaration-less names are now collected while the tools' own request processors run. - BaseLlmFlow.java:198 - a tool only counts when it actually adds an entry to the request's config tools. It is measured by comparing the config's tool count across each processLlmRequest call, so an ExampleTool named "lookup" beside a function tool of the same name is no longer a collision. With the check living in the flow, a toolset no longer needs to know this rule.
Regression test for the rule in the parent commit: a declaration-less tool must only be treated as a built-in answer when it actually adds an entry to the request's config tools. A plain declaration-less tool beside a function tool of the same name must not collide - that is the ExampleTool case the reviewer raised.
|
Thanks both — I've worked through every item. Summary of the current state:
Design — the rule now lives in the flow, and Verified locally, since the |
kvmilos
left a comment
There was a problem hiding this comment.
Thanks for the update. The double listing, the compile error and the ExampleTool case are fixed. A few things are still open:
- Please rebase on main and squash this into a single commit. The repo takes one commit per PR (see "Single Commit" in CONTRIBUTING.md), and check-commit-count fails at five. For later rounds,
git commit --amendplusgit push --force-with-leasekeeps it at one. - The title and description still describe the earlier version: the
fix(mcp)title,load_artifactsandlist_skillsas in-model tools, the second pass under "How the collision is detected", the old test names, and "each new test fails with the guard reverted". - Could you add a test where
GoogleSearchToolsits next to a differently named function tool and the request goes through, and one where the clashing tool comes from a toolset? Today a check that rejected every built-in, or a toolset branch that skippedapplyTool, would still pass every test.
Refuse a tool name that is already owned by something the model dispatches
Fixes #1513.
What this does
An agent that carries an in-model tool (
google_search,url_context,load_artifacts,list_skills, …) and also a function tool of the same name ends up with two things answeringto one name. The in-model tool is written into the request's config tools and never enters
LlmRequest.tools(), so nothing rejects the duplicate; the model chooses between them percall, and nobody decided that.
This adds the error in
BaseLlmFlow.getRequestProcessorFromTools— the one path every toolpasses through, including each tool unwrapped from a toolset — so it covers
FunctionTool,AgentTool,McpToolsetandMcpAsyncToolsetalike.Behaviour change to note before upgrading
An agent that carries, for example,
GoogleSearchTooland its owngoogle_searchfunctiontool will now fail where it previously ran, with
Duplicate tool name: google_search. Thisis the intended behaviour, but it is a change for anyone in that configuration, and the fix is
to rename one of the two.
Only that direction is rejected. Two tools that both lack a declaration may still share a name,
so setups such as two default-named
ExampleTools keep working.Why the check lives here and not in
McpToolsetAn earlier revision of this PR put a reserved-name list in
McpToolset. That cannot work,because
McpToolsetcannot see the agent's other tools, so it cannot tell a real collision froma harmless name: it rejected a server whose tool merely shared a name with an in-model tool the
agent did not even have. Moving the check to the request processor makes the decision where the
full tool list is visible.
How the collision is detected
A tool without a declaration does not appear in the built request, so the two halves are read
separately:
agent.toolsUnion()plus eachbaseToolset.getTools(...),filtered on
declaration().isEmpty();tools()map.The check fails on the intersection.
Testing Plan
New cases in
BaseLlmFlowTest:getRequestProcessorFromTools_rejectsDeclarationlessNameCollision— exercises the in-modeltool both before and after the clashing function tool, since the order must not matter.
getRequestProcessorFromTools_allowsTwoDeclarationlessToolsSharingAName— the negative case.Each new test was checked to fail with the guard reverted, so it pins behaviour rather
than passing vacuously:
Built and tested locally with
./mvnwon JDK 25. CI pins JDK 17.Revision history
6d3088bc— original: a reserved-name list insideMcpToolset.04534436— corrected the set to names this framework actually defines.65c409b2— added the skill and loop tools.5635996e— moved the check to run after tool selection. Insufficient: this kept the checkinside
McpToolset, so both review objections still held.9c0ae01b— reworked as requested, intoBaseLlmFlow.getRequestProcessorFromTools, withthe order-independence test and the behaviour-change note above.