Skip to content

Commit a9a3263

Browse files
committed
docs: fix multiple endpoints route reference
Signed-off-by: Daneyon Hansen <daneyon.hansen@solo.io>
1 parent 7594b01 commit a9a3263

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

assets/agw-docs/pages/agentgateway/llm/providers/multiple-endpoints.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ Configure access to multiple OpenAI API endpoints such as for chat completions,
44

55
To set up multiple LLM endpoints, use the `ai.routes` field in the `policies` section of the {{< reuse "agw-docs/snippets/backend.md" >}} resource. This field maps the API paths to supported route types. The keys are URL suffix matches, like `/v1/models`. The values are the route types, like `Completions` or `Passthrough`.
66

7-
Alternatively, you can define the routes in a separate {{< reuse "agw-docs/snippets/policy.md" >}} resource attached to an {{< reuse "agw-docs/snippets/agentgateway.md" >}} {{< reuse "agw-docs/snippets/gateway-api.md" >}}. This approach decouples routing configuration from the backend definition, which can simplify management when you have multiple routes pointing to the same backend.
7+
Alternatively, you can define the routes in a separate {{< reuse "agw-docs/snippets/policy.md" >}} resource attached to an {{< reuse "agw-docs/snippets/agentgateway.md" >}} `HTTPRoute`. This approach decouples routing configuration from the backend definition, which can simplify management when you have multiple routes pointing to the same backend.
88

99
- `Completions`: Parses the request, translates it to the LLM provider format, and fully processes it as an LLM request. This route type unlocks the full set of {{< reuse "agw-docs/snippets/agentgateway.md" >}} LLM features, such as tokenization and token-based rate limiting, prompt guards, prompt enrichment, model aliasing, transformations, cost tracking, and detailed observability.
1010
- `Detect`: Forwards the request as-is, but makes a best effort to extract the model and token counts so that a subset of policies still applies, specifically token-based rate limiting and telemetry. Guardrails and other request-shaping policies do not apply. Use this route type for endpoints with a format that cannot be automatically parsed by {{< reuse "agw-docs/snippets/agentgateway.md" >}}, but where you still want metrics and rate limiting.

0 commit comments

Comments
 (0)