fix: Fix usage of SearchableToolset with Agent when selecting a subset of tools to be active#11564
Open
sjrl wants to merge 8 commits into
Open
fix: Fix usage of SearchableToolset with Agent when selecting a subset of tools to be active#11564sjrl wants to merge 8 commits into
SearchableToolset with Agent when selecting a subset of tools to be active#11564sjrl wants to merge 8 commits into
Conversation
…en using SearchableToolset
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Contributor
Coverage reportClick to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||||||||||||||||||||
Code reviewNo issues found. Checked for bugs and CLAUDE.md compliance. |
…ols is being restricted
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
…. Remove .reset() since no longer needed.
Contributor
Author
|
@julian-risch based on our convo offline I've gone ahead and updated this PR to create a shallow copy of Toolset at Agent runtime using a new |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related Issues
Proposed Changes:
Before trying to restrict a
SearchableToolsetto a subset of its catalog by name at Agent runtime did not work:Before:
ValueError: The following tool names are not valid: {'get_weather', 'get_stock_price'}. Valid tool names are: {'search_tools'}SearchableToolsetonly exposed itssearch_toolsbootstrap tool (not its catalog) when flattened, so catalog tools couldn't be selected by name.tools=["search_tools"]instead would have collapsed the toolset into a single item list of justsearch_toolswhich disabled discovery.After: the
SearchableToolsetis kept intact, and when a user restricts the tools toget_weatherandget_stock_pricesearch and lazy-loading still work over the restricted subset.How this is fixed: instead of flattening the toolset into a static list of tools, the
Agentnow keeps theToolsetintact and applies the name selection to it:Toolset.get_selectable_tools().SearchableToolsetoverrides this to return its entire catalog, so catalog tools are now resolvable by name (previously only itssearch_toolstool was, since iterating the toolset only exposes that bootstrap tool plus already-discovered tools).list[Tool], theAgentregisters the requested names as a per-run selection on the liveToolset(viaToolset._selected_tool_names) and keeps that object as the run's tools.Agentre-reads its tools each step, theToolsetyields only the selected tools. ForSearchableToolsetthis means thesearch_toolstool is still exposed and search/discovery is restricted to the selected catalog tools — so search and lazy-loading keep working over the subset instead of being disabled.AgentcallsToolset.reset(), which clears the per-run selection.SearchableToolset.reset()additionally clears its discovered tools, so the selection does not leak into later runs.Changes Made:
Toolsetgained two methods:get_selectable_tools()returns every tool available for name-basedselection (ignoring any active selection restriction, e.g. the one used by
SearchableToolset), andreset()clears a Toolset's per-run state. Subclasses can overridereset()to reset additional state.SearchableToolsetnow exposes its full catalog of tools viaSearchableToolset.get_selectable_tools().Agent.run(tools=["tool_a", "tool_b"])now resolves correctly when aSearchableToolsetis configured. Previously theSearchableToolsetwas flattened into a single-itemlist (just its search tool), which broke its search and lazy-loading behavior.
SearchableToolsetnow clears its discovered tools at the end of eachAgentrun (viaSearchableToolset.reset()). Discovered tools no longer persist across runs, so each run starts fresh.How did you test it?
New tests
Notes for the reviewer
I did check locally that these changes work with our existing MCPToolset integration without any need for changes there.
Checklist
fix:,feat:,build:,chore:,ci:,docs:,style:,refactor:,perf:,test:and added!in case the PR includes breaking changes.