You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
- Update MCP server fields to match Create MCP Server dialog (Server ID,
Display Name, Server Address, Defer Loading Override, Forward OIDC
Token Override)
- Update orchestrator section to reflect system-managed config with
configurable blocked tool patterns
- Update deferred loading config to use per-server Defer Loading Override
dropdown instead of gateway-level toggle
- Update observability references to point to gateway Overview tab
- Comment out references to UI features not yet available
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: modules/ai-agents/pages/ai-gateway/admin/setup-guide.adoc
+82-42Lines changed: 82 additions & 42 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -26,7 +26,7 @@ After completing this guide, you will be able to:
26
26
27
27
Providers represent upstream services (Anthropic, OpenAI, Google AI) and associated credentials. Providers are disabled by default and must be enabled explicitly by an administrator.
28
28
29
-
. In the Redpanda Cloud Console, navigate to *AI Gateway* → *Providers*.
29
+
. In the Redpanda Cloud Console, navigate to *Agentic AI* → *Providers*.
30
30
. Select a provider (for example, Anthropic).
31
31
. On the Configuration tab for the provider, click *Add configuration*.
32
32
. Enter your API Key for the provider.
@@ -43,17 +43,15 @@ The model catalog is the set of models made available through the gateway. Model
43
43
44
44
The infrastructure that serves the model differs based on the provider you select. For example, OpenAI has different reliability and availability metrics than Anthropic. When you consider all metrics, you can design your gateway to use different providers for different use cases.
45
45
46
-
. Navigate to *AI Gateway* → *Models*.
46
+
. Navigate to *Agentic AI* → *Models*.
47
47
. Review the list of available models from enabled providers.
48
-
. For each model you want to expose through gateways, toggle it to *Enabled*.
49
-
+
50
-
Common models to enable:
48
+
. For each model you want to expose through gateways, toggle it to *Enabled*. For example:
51
49
+
52
50
--
53
-
* `openai/gpt-5.2` - OpenAI's most capable model
54
-
* `openai/gpt-5.2-mini` - Cost-effective OpenAI model
55
-
* `anthropic/claude-sonnet-4.5` - Balanced Anthropic model
56
-
* `anthropic/claude-opus-4.6` - Anthropic's most capable model
51
+
* `openai/gpt-5.2`
52
+
* `openai/gpt-5.2-mini`
53
+
* `anthropic/claude-sonnet-4.5`
54
+
* `anthropic/claude-opus-4.6`
57
55
--
58
56
59
57
. Click *Save changes*.
@@ -62,9 +60,7 @@ Only enabled models will be accessible through gateways. You can enable or disab
62
60
63
61
=== Model naming convention
64
62
65
-
Model requests must use the `vendor/model_id` format in the model property of the request body. This format allows AI Gateway to route requests to the appropriate provider.
66
-
67
-
Examples:
63
+
Model requests must use the `vendor/model_id` format in the model property of the request body. This format allows AI Gateway to route requests to the appropriate provider. For example:
68
64
69
65
* `openai/gpt-5.2`
70
66
* `anthropic/claude-sonnet-4.5`
@@ -109,7 +105,7 @@ endif::[]
109
105
110
106
A gateway is a logical configuration boundary (policies + routing + observability) on top of a single deployment. It's a "virtual gateway" that you can create per team, environment (staging/production), product, or customer.
111
107
112
-
. Navigate to *AI Gateway* → *Gateways*.
108
+
. Navigate to *Agentic AI* → *Gateways*.
113
109
. Click *Create Gateway*.
114
110
. Configure the gateway:
115
111
+
@@ -128,10 +124,12 @@ TIP: A workspace is conceptually similar to a resource group in Redpanda streami
128
124
. After creation, note the following information:
129
125
+
130
126
--
131
-
* *Gateway Endpoint*: URL for API requests (for example, `https://example/gateways/gw_abc123/v1`) - the gateway ID is embedded in the URL
127
+
* *Gateway endpoint*: URL for API requests (for example, `https://example/gateways/d633lffcc16s73ct95mg/v1`)
128
+
+
129
+
The gateway ID is embedded in the URL.
132
130
--
133
131
134
-
You'll share the Gateway Endpoint with users who need to access this gateway.
132
+
You'll share the gateway endpoint with users who need to access this gateway.
135
133
136
134
== Configure LLM routing
137
135
@@ -228,49 +226,78 @@ TIP: Provider pool (UI) = Backend pool (API)
228
226
229
227
If a provider pool contains multiple providers, you can distribute traffic to balance load or optimize for cost/performance:
230
228
231
-
* *Round-robin*: Distribute evenly across all providers
232
-
* *Weighted*: Assign weights (for example, 80% to Anthropic, 20% to OpenAI)
233
-
* *Least latency*: Route to fastest provider based on recent performance
234
-
* *Cost-optimized*: Route to cheapest provider for each model
229
+
* Round-robin: Distribute evenly across all providers
230
+
* Weighted: Assign weights (for example, 80% to Anthropic, 20% to OpenAI)
231
+
* Least latency: Route to fastest provider based on recent performance
232
+
* Cost-optimized: Route to cheapest provider for each model
235
233
236
234
== Configure MCP tools (optional)
237
235
238
236
If your users will build glossterm:AI agent[,AI agents] that need access to glossterm:MCP tool[,tools] via glossterm:MCP[,Model Context Protocol (MCP)], configure MCP tool aggregation.
239
237
240
238
On the gateway details page, select the *MCP* tab to configure tool discovery and execution. The MCP proxy aggregates multiple glossterm:MCP server[,MCP servers], allowing agents to find and call tools through a single endpoint.
241
239
240
+
=== Configure MCP rate limits
241
+
242
+
Rate limits for MCP work the same way as LLM rate limits.
243
+
244
+
. In the *MCP* tab, locate the *Rate Limit* section.
245
+
. Click *Add rate limit*.
246
+
. Configure the maximum requests per second and optional burst allowance.
247
+
. Click *Save*.
248
+
242
249
=== Add MCP servers
243
250
244
-
. In the *MCP* tab, click *Add MCP server*.
251
+
. In the *MCP* tab, click *Create MCP Server*.
245
252
. Configure the server:
246
253
+
247
254
--
248
-
* *Server name*: Human-readable identifier (for example, `database-server`, `slack-server`)
249
-
* *Server URL*: Endpoint for the MCP server (for example, `https://mcp-database.example.com`)
250
-
* *Authentication*: Configure authentication if required (bearer token, API key, mTLS)
251
-
* *Enabled tools*: Select which tools from this server to expose (or *All tools*)
255
+
* *Server ID*: Unique identifier for this server
256
+
* *Display Name*: Human-readable name (for example, `database-server`, `slack-server`)
257
+
* *Server Address*: Endpoint URL for the MCP server (for example, `https://mcp-database.example.com`)
252
258
--
253
259
254
-
. Click *Test connection* to verify connectivity.
255
-
. Click *Save* to add the server to this gateway.
260
+
. Configure server settings:
261
+
+
262
+
--
263
+
* *Timeout (seconds)*: Maximum time to wait for a response from this server
264
+
* *Enabled*: Whether this server is active and accepting requests
265
+
* *Defer Loading Override*: Controls whether tools from this server are loaded upfront or on demand
266
+
+
267
+
[cols="1,2"]
268
+
|===
269
+
|Option |Description
256
270
257
-
Repeat for each MCP server you want to aggregate.
271
+
|Inherit from gateway
272
+
|Use the gateway-level deferred loading setting (default)
258
273
259
-
=== Configure deferred tool loading
274
+
|Enabled
275
+
|Always defer loading from this server. Agents receive only a search tool initially and query for specific tools when needed. This can reduce token usage by 80-90%.
260
276
261
-
Deferred tool loading dramatically reduces token costs by initially exposing only a search tool and orchestrator, rather than listing all available tools.
277
+
|Disabled
278
+
|Always load all tools from this server upfront.
279
+
|===
262
280
263
-
. In the *MCP* tab, locate *Deferred Loading*.
264
-
. Toggle *Enable deferred tool loading* to *On*.
265
-
. Configure behavior:
281
+
* *Forward OIDC Token Override*: Controls whether the client's OIDC token is forwarded to this MCP server
266
282
+
267
-
--
268
-
* *Initially expose*: Search tool + orchestrator only
269
-
* *Load on demand*: Tools are retrieved when agents query for them
270
-
* *Token savings*: Expect 80-90% reduction in token usage for tool definitions
283
+
[cols="1,2"]
284
+
|===
285
+
|Option |Description
286
+
287
+
|Inherit from gateway
288
+
|Use the gateway-level OIDC forwarding setting (default)
289
+
290
+
|Enabled
291
+
|Always forward the OIDC token to this server
292
+
293
+
|Disabled
294
+
|Never forward the OIDC token to this server
295
+
|===
271
296
--
272
297
273
-
. Click *Save*.
298
+
. Click *Save* to add the server to this gateway.
299
+
300
+
Repeat for each MCP server you want to aggregate.
274
301
275
302
See xref:ai-gateway/mcp-aggregation-guide.adoc[] for detailed information about MCP aggregation.
276
303
@@ -280,11 +307,24 @@ The MCP orchestrator is a built-in MCP server that enables programmatic tool cal
280
307
281
308
Example: A workflow requiring 47 file reads can be reduced from 49 round trips to just 1 round trip using the orchestrator.
282
309
283
-
The orchestrator is enabled by default when you enable MCP tools. You can configure:
310
+
The orchestrator is pre-configured when you initialize the MCP gateway. Its server configuration (Server ID, Display Name, Transport, Command, and Timeout) is system-managed and cannot be modified.
284
311
285
-
* *Execution timeout*: Maximum time for orchestrator workflows (for example, 30 seconds)
286
-
* *Memory limit*: Maximum memory for JavaScript execution (for example, 128MB)
287
-
* *Allowed operations*: Restrict which MCP tools can be called from orchestrator workflows
312
+
You can configure blocked tool patterns to prevent specific tools from being called through the orchestrator:
313
+
314
+
. In the *MCP* tab, select the orchestrator server to edit it.
315
+
. Under *Blocked Tools*, click *Add Pattern* to add glob patterns for tools that should be blocked from execution.
316
+
+
317
+
Example patterns:
318
+
+
319
+
--
320
+
* `server_id:*` - Block all tools from a specific server
321
+
* `*:dangerous_tool` - Block a specific tool across all servers
322
+
* `specific:tool` - Block a single tool on a specific server
323
+
--
324
+
+
325
+
NOTE: The orchestrator's own tools are blocked by default to prevent recursive execution.
Copy file name to clipboardExpand all lines: modules/ai-agents/pages/ai-gateway/builders/discover-gateways.adoc
+3-12Lines changed: 3 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -27,19 +27,10 @@ After reading this page, you will be able to:
27
27
====
28
28
Using the Console::
29
29
+
30
-
. Navigate to *AI Gateway* in the Redpanda Cloud Console.
31
-
. View the *My Gateways* tab (or *Gateways* if you're an administrator).
32
-
. Review the list of gateways you can access:
30
+
. Navigate to *Gateways* in the Redpanda Cloud Console.
31
+
. Review the list of gateways you can access. For each gateway, you'll see the gateway name, ID, endpoint URL, status, available models, and provider performance.
33
32
+
34
-
For each gateway, you'll see:
35
-
+
36
-
--
37
-
* *Gateway Name*: Human-readable name (for example, `production-gateway`, `team-ml-gateway`)
38
-
* *Gateway Endpoint*: URL for API requests, with the gateway ID embedded in the path (for example, `https://example/gateways/gw_abc123/v1`)
39
-
* *Status*: Whether the gateway is active and accepting requests
40
-
* *Available Models*: Which LLM models you can access
41
-
* *MCP Tools*: Whether MCP tool aggregation is enabled
42
-
--
33
+
Click the Configuration, API, MCP Tools, and Changelog tabs for additional information.
Copy file name to clipboardExpand all lines: modules/ai-agents/pages/ai-gateway/gateway-quickstart.adoc
+4-11Lines changed: 4 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -78,10 +78,10 @@ ifdef::ai-hub-available[]
78
78
endif::[]
79
79
. Configure the gateway:
80
80
+
81
-
* Display name: Choose a descriptive name (for example, `my-first-gateway`)
82
-
* Workspace: Select a workspace (conceptually similar to a resource group)
83
-
* Description: Add context about this gateway's purpose
84
-
* Optional metadata for documentation
81
+
** Display name: Choose a descriptive name (for example, `my-first-gateway`)
82
+
** Workspace: Select a workspace (conceptually similar to a resource group)
83
+
** Description: Add context about this gateway's purpose
84
+
** Optional metadata for documentation
85
85
86
86
After creation, copy the gateway endpoint from the overview page. You'll need this for sending requests. The gateway ID is embedded in the endpoint URL. For example:
87
87
@@ -213,7 +213,6 @@ If your request fails, check these common issues:
213
213
214
214
Confirm your request was routed through AI Gateway.
215
215
216
-
. In the sidebar, navigate to *Agentic AI > Gateways*, then select your gateway.
217
216
. On the *Overview* tab, check the aggregate metrics:
218
217
+
219
218
* *Total Requests*: Should have incremented
@@ -225,12 +224,6 @@ Confirm your request was routed through AI Gateway.
225
224
+
226
225
The model you used in your request should appear with its request count, token usage (input/output), estimated cost, latency, and error rate.
227
226
228
-
If your request doesn't appear:
229
-
230
-
* Wait a few seconds for metrics to update
231
-
* Verify the gateway endpoint in your request matches the gateway you're viewing
232
-
* Check that your client received a successful response
233
-
234
227
== Configure LLM routing (optional)
235
228
236
229
Configure rate limits, spend limits, and provider pools with failover.
0 commit comments