Skip to content

Commit 371dd51

Browse files
committed
Explain the value of mTLS based inference access.
Signed-off-by: usize <mofoster@redhat.com>
1 parent 46f6905 commit 371dd51

1 file changed

Lines changed: 35 additions & 0 deletions

File tree

docs/plans/ai-gateway.md

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -327,6 +327,41 @@ API key authentication.
327327
This is a gateway-level concern — it applies uniformly to all traffic
328328
entering the gateway, independent of which provider handles the request.
329329

330+
#### Why: tokenless inference access
331+
332+
Today, giving an agent access to an LLM provider means provisioning
333+
an API key, storing it in a Secret, mounting it into the workload,
334+
and building rotation and revocation processes around it. API keys
335+
are bearer tokens — if leaked, anyone can use them. In multi-team
336+
environments, shared keys make attribution and cost accounting
337+
difficult.
338+
339+
With AIAccessPolicy, inference access is tied to workload identity.
340+
A platform admin deploys an AI Gateway with mTLS. Agent workloads
341+
in the namespace have SPIFFE identities provisioned by SPIRE. When
342+
an agent calls the gateway, its X.509 SVID is validated against the
343+
trust bundle — no API keys to provision, rotate, or leak. Workloads
344+
without a SPIFFE identity in the trust domain are rejected at the
345+
TLS handshake. The agent code is unchanged; it calls an
346+
OpenAI-compatible endpoint. The credential management is pushed from
347+
application teams to infrastructure.
348+
349+
Provider-level API keys (for external providers like OpenAI) still
350+
exist, but they live in one place — the AIRoutingPolicy's credential
351+
configuration — managed by the platform team. Individual agents
352+
never see them.
353+
354+
This mechanism also opens the door to per-workload policy. After
355+
mTLS validation, the proxy sets the `x-forwarded-client-cert`
356+
header with the client's SPIFFE ID (e.g.
357+
`spiffe://localtest.me/ns/team1/sa/weather-agent`). This header
358+
cannot be spoofed — the proxy strips any client-supplied XFCC and
359+
replaces it with data extracted from the validated certificate. Our
360+
initial data plane (Envoy) supports this natively; the mechanism
361+
may be replicated in future supported proxies. A future policy CRD
362+
could match on XFCC to apply per-workload rate limits, model access
363+
lists, or processing pipelines.
364+
330365
```yaml
331366
apiVersion: aigateway.kagenti.dev/v1alpha1
332367
kind: AIAccessPolicy

0 commit comments

Comments
 (0)