Skip to content

Commit 49ebb83

Browse files
committed
Lay the foundation for a future PayloadProcessingPipeline resource.
Signed-off-by: usize <mofoster@redhat.com>
1 parent 6e8e71c commit 49ebb83

1 file changed

Lines changed: 80 additions & 0 deletions

File tree

docs/plans/ai-gateway.md

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -547,6 +547,86 @@ rate limiting apply uniformly regardless of protocol. The exact shape
547547
of non-inference routing entries is left to a future proposal once
548548
the WG AI Gateway Backend specification matures.
549549

550+
### Future: PayloadProcessingPipeline
551+
552+
The WG AI Gateway's [Proposal 7: Payload Processing] defines
553+
[`PayloadProcessingPipeline`][PPP] — a resource for
554+
well-ordered payload processing policies such as semantic caching,
555+
model selection, guardrails, PII redaction, and prompt injection
556+
detection. These policies could apply at a per-provider or
557+
per-model level.
558+
559+
In the upstream proposal, processing pipelines attach to `Backend`
560+
resources and `Route` resources. In this proposal, we've abstracted
561+
networking-specific concepts behind terms of art associated with
562+
Generative AI. `Backend` maps onto "provider" — where a provider is
563+
a backend that communicates via an AI protocol (e.g. Responses API,
564+
MCP, A2A). `Route` is implicit in the model-to-provider mapping.
565+
The user never sees the generated HTTPRoute or Backend resources
566+
directly.
567+
568+
This presents a tension. Consider a user who wants PII redaction on
569+
requests to OpenAI but not to Ollama (different trust boundaries).
570+
They need a PayloadProcessingPipeline that targets only the OpenAI
571+
provider's traffic. Three approaches:
572+
573+
**Target by name string.** The user references a provider or model
574+
by name within the AIRoutingPolicy:
575+
576+
```yaml
577+
kind: PayloadProcessingPipeline
578+
spec:
579+
targetRef:
580+
kind: AIRoutingPolicy
581+
name: my-routing
582+
provider: openai # string reference into the spec
583+
```
584+
585+
The controller looks up the AIRoutingPolicy, finds the provider
586+
named `openai`, determines which HTTPRoute rules correspond to
587+
models using that provider, and attaches the processing pipeline to
588+
those rules. This works, but the controller does significant
589+
indirection, and the targeting vocabulary grows with every new
590+
concept (provider, model, etc.).
591+
592+
**Expose providers and models as resources.** Instead of one
593+
AIRoutingPolicy containing everything, the user creates individual
594+
resources that map to the networking primitives:
595+
596+
```yaml
597+
kind: AIProvider # generates Backend + AIServiceBackend
598+
spec:
599+
name: openai
600+
endpoint: https://api.openai.com/v1
601+
schema: OpenAI
602+
---
603+
kind: AIModel # generates a rule in AIGatewayRoute
604+
spec:
605+
name: gpt-4o
606+
providerRef: openai
607+
model: gpt-4o
608+
```
609+
610+
PayloadProcessingPipeline can now target `AIProvider` or `AIModel`
611+
directly using standard `targetRef` — no string indirection. This is
612+
more composable but the user manages many resources instead of one
613+
for a simple setup.
614+
615+
**Hybrid: compact CR with optional break-out.** AIRoutingPolicy
616+
stays as-is for simple deployments. When a user needs per-provider
617+
processing, they extract that provider into a standalone `AIProvider`
618+
CR and reference it by name from the AIRoutingPolicy. The standalone
619+
resource is targetable by PayloadProcessingPipeline. This is similar
620+
to how Kubernetes handles inline vs referenced specs — like a pod
621+
template inline in a Deployment vs a standalone Pod.
622+
623+
The right answer likely depends on real usage patterns. This
624+
proposal establishes the routing and access control foundation;
625+
payload processing attachment semantics will be addressed in a
626+
follow-up once the WG specification matures.
627+
628+
[PPP]: https://github.com/kubernetes-sigs/wg-ai-gateway/blob/main/proposals/7-payload-processing.md
629+
550630
## Example: complete deployment
551631

552632
A platform admin sets up a Gateway with mTLS. A team lead configures

0 commit comments

Comments
 (0)