You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/features/routing-modes.md
+81-23Lines changed: 81 additions & 23 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -52,36 +52,51 @@ This means you can point different AI clients at different endpoints. For exampl
52
52
53
53
### retrieve_tools (Default)
54
54
55
-
The default mode uses BM25 full-text search to help AI agents discover relevant tools without exposing the entire tool catalog. This is the most token-efficient mode.
55
+
The default mode uses BM25 full-text search to help AI agents discover relevant tools without exposing the entire tool catalog. This approach — sometimes called "lazy tool loading" or "tool search" — is used by Anthropic's own MCP implementation and is the recommended pattern for large tool sets.
56
+
57
+
**Endpoint:**`/mcp/call`
56
58
57
59
**Tools exposed:**
58
-
-`retrieve_tools` -- Search for tools by natural language query
1. AI agent calls `retrieve_tools` with a natural language query
67
-
2. MCPProxy returns matching tools ranked by BM25 relevance
68
-
3. AI agent calls the appropriate `call_tool_*` variant with the tool name
68
+
2. MCPProxy returns matching tools ranked by BM25 relevance, with `call_with` recommendations
69
+
3. AI agent calls the appropriate `call_tool_*` variant with the exact tool name from results
70
+
71
+
**Pros:**
72
+
- Massive token savings: only matched tools are sent to the model, not the full catalog
73
+
- Scales to hundreds of tools across many servers without context window bloat
74
+
- Intent-based permission control (read/write/destructive variants) enables granular IDE approval flows
75
+
- Activity logging captures operation type and intent metadata for auditing
76
+
77
+
**Cons:**
78
+
- Two-step workflow (search then call) adds one round-trip compared to direct mode
79
+
- BM25 search quality depends on tool descriptions — poorly described tools may not surface
80
+
- The AI agent must learn the retrieve-then-call pattern (most modern models handle this well)
69
81
70
82
**When to use:**
71
83
- You have many upstream servers with dozens or hundreds of tools
72
-
- Token usage is a concern (only tool metadata for matched tools is sent)
73
-
- You want intent-based permission control (read/write/destructive variants)
84
+
- Token usage is a concern (common with paid API usage)
85
+
- You want intent-based permission control in IDE auto-approve settings
86
+
- Production deployments where audit trails matter
74
87
75
88
### direct
76
89
77
-
Direct mode exposes every upstream tool directly to the AI agent. Each tool is named `serverName__toolName` (double underscore separator).
90
+
Direct mode exposes every upstream tool directly to the AI agent. Each tool appears in the standard MCP `tools/list` with a `serverName__toolName` name. This is the simplest mode and the closest to how individual MCP servers work natively.
91
+
92
+
**Endpoint:**`/mcp/all`
78
93
79
94
**Tools exposed:**
80
95
- Every tool from every connected, enabled, non-quarantined upstream server
81
96
- Named as `serverName__toolName` (e.g., `github__create_issue`, `filesystem__read_file`)
82
97
83
98
**How it works:**
84
-
1. AI agent sees all available tools in the toolslist
99
+
1. AI agent sees all available tools in `tools/list`
85
100
2. AI agent calls tools directly by their `serverName__toolName` name
86
101
3. MCPProxy routes the call to the correct upstream server
87
102
@@ -94,31 +109,74 @@ Direct mode exposes every upstream tool directly to the AI agent. Each tool is n
94
109
- Agent tokens with server restrictions are enforced (access denied if token lacks server access)
95
110
- Permission levels are derived from tool annotations (read-only, destructive, etc.)
96
111
112
+
**Pros:**
113
+
- Zero learning curve: tools work exactly like native MCP tools
114
+
- Single round-trip: no search step needed, call any tool directly
115
+
- Maximum compatibility: works with any MCP client without special handling
116
+
- Tool annotations (readOnlyHint, destructiveHint) are preserved from upstream
117
+
118
+
**Cons:**
119
+
- High token cost: all tool definitions are sent in every request context
120
+
- Does not scale well beyond ~50 tools (context window fills up, model accuracy degrades)
121
+
- No intent-based permission tiers (the model just calls tools)
122
+
- All tools visible upfront increases attack surface for prompt injection
123
+
97
124
**When to use:**
98
-
- You have a small number of upstream servers (fewer than 50 total tools)
99
-
- You want maximum simplicity and compatibility
100
-
- AI clients that work better with a flat tool list
125
+
- Small setups with fewer than 50 total tools
126
+
- Quick prototyping and testing
127
+
- AI clients that don't support the retrieve-then-call pattern
128
+
- CI/CD agents that know exactly which tools they need
101
129
102
130
### code_execution
103
131
104
-
Code execution mode is designed for multi-step orchestration workflows. It exposes the `code_execution` tool with an enhanced description that includes a catalog of all available upstream tools.
132
+
Code execution mode is designed for multi-step orchestration workflows. Instead of making separate tool calls for each step, the AI agent writes JavaScript or TypeScript code that chains multiple tool calls together in a single request. This is inspired by patterns from OpenAI's code interpreter and similar "tool-as-code" approaches.
133
+
134
+
**Endpoint:**`/mcp/code`
105
135
106
136
**Tools exposed:**
107
-
-`code_execution`-- Execute JavaScript/TypeScript that orchestrates upstream tools
108
-
-`retrieve_tools`-- Search for tools (useful for discovery before writing code)
137
+
-`code_execution`— Execute JavaScript/TypeScript that orchestrates upstream tools (includes a catalog of all available tools in the description)
138
+
-`retrieve_tools`— Search for tools (instructs to use `code_execution`, not `call_tool_*`)
109
139
110
140
**How it works:**
111
141
1. AI agent sees the `code_execution` tool with a listing of all available upstream tools
112
142
2. AI agent writes JavaScript/TypeScript code that calls `call_tool(serverName, toolName, args)`
113
-
3. MCPProxy executes the code in a sandboxed VM, routing tool calls to upstream servers
143
+
3. MCPProxy executes the code in a sandboxed ES2020+ VM, routing tool calls to upstream servers
144
+
4. Results are returned as a single response
145
+
146
+
**Pros:**
147
+
- Minimal round-trips: complex multi-step workflows execute in one request
148
+
- Full programming power: conditionals, loops, error handling, data transformation
149
+
- TypeScript support with type safety (auto-transpiled via esbuild)
150
+
- Sandboxed execution: no filesystem or network access, timeout enforcement
151
+
- Tool catalog in description means no separate search step needed
152
+
153
+
**Cons:**
154
+
- Requires the AI model to write correct JavaScript/TypeScript code
155
+
- Debugging is harder: errors come from inside the sandbox, not from MCP tool calls
- Must be explicitly enabled (`"enable_code_execution": true`)
158
+
- Not all AI models are equally good at writing code for tool orchestration
114
159
115
160
**When to use:**
116
-
- Workflows that require chaining 2+ tool calls together
117
-
- You want to minimize model round-trips
118
-
- Complex conditional logic or data transformation between tool calls
161
+
- Workflows that chain 2+ tool calls with data dependencies between them
162
+
- Batch operations (e.g., "for each repo, check CI status and create issue if failing")
163
+
- Complex conditional logic that would require many round-trips in other modes
164
+
- Data transformation pipelines (fetch from one tool, transform, send to another)
119
165
120
166
**Note:** Code execution must be enabled in config (`"enable_code_execution": true`). If disabled, the `code_execution` tool appears but returns an error message directing the user to enable it.
121
167
168
+
## Choosing the Right Mode
169
+
170
+
| Factor | retrieve_tools | direct | code_execution |
0 commit comments