@@ -309,9 +309,14 @@ so no coordination is needed.
309309Requires Envoy Gateway's global rate limit service (Redis-backed) for
310310cross-instance quota enforcement.
311311
312- Per-user or per-tenant rate limiting (multi-tenant cost allocation) is
313- a separate concern. If needed, a future policy CRD could attach to the
314- Gateway to layer per-client quotas on top of per-model limits.
312+ The initial design treats the namespace as a single tenant — rate
313+ limit counters are scoped per model, not per client. Per-user or
314+ per-tenant rate limiting is a separate concern. Multi-tenancy could
315+ be addressed by extracting rate limiting into a separate policy CRD
316+ with precedence rules akin to [BackendTrafficPolicy][] — where
317+ more specific policies override broader defaults.
318+
319+ [BackendTrafficPolicy] : https://gateway.envoyproxy.io/docs/concepts/gateway_api_extensions/backend-traffic-policy/
315320
316321# ## AIAccessPolicy
317322
@@ -505,6 +510,43 @@ proposal.
505510
506511Not in scope for the initial implementation.
507512
513+ # ## Future: protocol extensibility
514+
515+ The provider `schema` field declares the protocol a backend speaks.
516+ The initial implementation supports inference schemas (OpenAI,
517+ AWSBedrock, etc.), but this maps directly to the [Backend protocol
518+ model][Proposal 10] emerging in WG AI Gateway — where protocol is a
519+ property of the destination, not the route.
520+
521+ The `models` list is inference-specific : it defines virtual model
522+ names, maps them to provider backends, and configures per-model
523+ failover and rate limits. Non-inference protocols like MCP don't have
524+ a model selection concept. Extending AIRoutingPolicy to support MCP
525+ providers would add protocol-appropriate routing entries alongside
526+ `models`, driven by the provider's schema :
527+
528+ ` ` ` yaml
529+ providers:
530+ - name: ollama
531+ endpoint: http://ollama.svc:11434
532+ schema: OpenAI # inference — uses models[]
533+ - name: tool-server
534+ endpoint: http://tools.svc:8080
535+ schema: MCP # MCP — no model mapping needed
536+
537+ models: # inference-specific
538+ - name: qwen2.5:3b
539+ backends:
540+ - provider: ollama
541+ model: qwen2.5:3b
542+ ` ` `
543+
544+ The renderer would generate `AIGatewayRoute` for inference providers
545+ and `MCPRoute` for MCP providers. Credential injection, mTLS, and
546+ rate limiting apply uniformly regardless of protocol. The exact shape
547+ of non-inference routing entries is left to a future proposal once
548+ the WG AI Gateway Backend specification matures.
549+
508550# # Example: complete deployment
509551
510552A platform admin sets up a Gateway with mTLS. A team lead configures
0 commit comments