docs: surface PassthroughModel and make examples runnable and drift-proof

README.md

@@ -45,9 +45,9 @@ spec:
 
   sources:
     # Source 1: LLM serving pack Helm chart
-    - repoURL: https://github.com/nebari-dev/nebari-llm-serving-pack.git
-      targetRevision: v0.1.0-alpha.7
-      path: charts/nebari-llm-serving
+    - repoURL: quay.io/nebari/charts
+      chart: nebari-llm-serving
+      targetRevision: "0.1.2"
       helm:
         releaseName: nebari-llm-serving
         values: |

docs/astro.config.mjs

@@ -1,5 +1,6 @@
 // docs/astro.config.mjs
 import { defineConfig } from 'astro/config';
+import { fileURLToPath } from 'node:url';
 import starlight from '@astrojs/starlight';
 import { nebari } from '@nebari/starlight';
 import rehypeMermaid from 'rehype-mermaid';
@@ -47,4 +48,16 @@ export default defineConfig({
     remarkPlugins: [[remarkBaseLinks, { base: process.env.BASE || '/' }]],
     rehypePlugins: [[rehypeMermaid, { strategy: 'inline-svg' }]],
   },
+  vite: {
+    server: {
+      fs: {
+        // Defense-in-depth: allow the repo root (parent of docs/) so Vite's
+        // dev static-file middleware can serve examples/*.yaml if a future
+        // pattern ever needs it. The current embed resolves examples/*.yaml
+        // via the Astro SSR module graph (`?raw` import), which is NOT gated
+        // by fs.allow, so this is precautionary, not load-bearing today.
+        allow: [fileURLToPath(new URL('..', import.meta.url))],
+      },
+    },
+  },
 });

docs/src/content/docs/architecture.md → docs/src/content/docs/architecture.mdx

@@ -1,6 +1,9 @@
 ---
 title: Architecture
 ---
+import { Code } from '@astrojs/starlight/components';
+import passthroughExample from '../../../../examples/passthrough-openrouter.yaml?raw';
+
 This page explains how the Nebari LLM serving pack is designed: what components it deploys, how the operator reconciles `LLMModel` resources, how the two authentication paths work, how the key manager operates, and what security guarantees the design provides.
 
 For configuration knobs (Helm values, CRD fields), see [Configuration](/configuration/).
@@ -239,29 +242,9 @@ The operator combines these pack-level values with each `LLMModel`'s `access.gro
 
 Issue [#95](https://github.com/nebari-dev/llm-serving-pack/issues/95). A PassthroughModel routes the shared `llm.<baseDomain>` / `llm-internal.<baseDomain>` endpoints to an external OpenAI-compatible provider (OpenRouter, api.openai.com, a remote vLLM) instead of a locally served model. The operator provisions gateway plumbing and the same two auth layers served models get; the key-manager lists PassthroughModels next to LLMModels so users mint API keys for external providers through the same UI.
 
-```yaml
-apiVersion: llm.nebari.dev/v1alpha1
-kind: PassthroughModel
-metadata:
-  name: openrouter
-  namespace: nebari-llm-serving-system
-spec:
-  provider:
-    hostname: openrouter.ai          # bare hostname, TLS assumed
-    port: 443                        # default
-    schemaVersion: api/v1            # upstream path prefix (default v1)
-    credentialSecretName: openrouter-api-key   # same-namespace Secret, key "apiKey"
-  models:
-    catchAll: true                   # any model id not claimed elsewhere
-    declared:                        # explicit routes, advertised by /v1/models
-      - openai/gpt-5.2
-      - anthropic/claude-opus-4.6
-  access:
-    groups: [llm]                    # same semantics as LLMModel access
-  endpoints:
-    external: {enabled: true}
-    internal: {enabled: true}
-```
+<Code code={passthroughExample} lang="yaml" title="examples/passthrough-openrouter.yaml" />
+
+For a step-by-step walkthrough see [Route to an external provider](/quickstart/#route-to-an-external-provider); for the full field reference see the [PassthroughModel CRD reference](/configuration/#passthroughmodel-crd-reference).
 
 Generated resources (all in the CR's namespace, no serving stack):
 

docs/src/content/docs/configuration.md → docs/src/content/docs/configuration.mdx

@@ -1,6 +1,11 @@
 ---
 title: Configuration
 ---
+import { Code } from '@astrojs/starlight/components';
+import minimalModel from '../../../../examples/models/minimal.yaml?raw';
+import advancedModel from '../../../../examples/models/advanced-scheduling.yaml?raw';
+import passthroughExample from '../../../../examples/passthrough-openrouter.yaml?raw';
+
 This page is the authoritative reference for every knob the pack exposes.
 It covers the Helm chart values you set at install time, the `LLMModel` custom resource you create per model, and the `NebariApp` fields the chart uses to wire the key-manager UI into the Nebari platform.
 
@@ -50,7 +55,7 @@ Pass these with `--set key=value` or in a `values.yaml` override file.
 | Key | Default | Description |
 |-----|---------|-------------|
 | `keyManager.enabled` | `true` | Deploy the key-manager service. |
-| `keyManager.image.repository` | `ghcr.io/nebari-dev/nebari-llm-serving-pack/key-manager` | Container image repository. |
+| `keyManager.image.repository` | `ghcr.io/nebari-dev/llm-serving-pack/key-manager` | Container image repository. |
 | `keyManager.image.tag` | `""` | Image tag. Defaults to `.Chart.AppVersion` when empty, keeping chart and image versions in sync. Override only when testing a specific build. |
 | `keyManager.image.pullPolicy` | `Always` | Image pull policy. |
 | `keyManager.auditInterval` | `5m` | How often the scaffolded audit loop runs (only when `oidcUserinfoURL` is set). Group-change revocation is not yet implemented (see below), so this loop performs no revocation today. |
@@ -83,7 +88,7 @@ The React SPA served by nginx. It serves the SPA and a runtime `/config.json`, a
 | Key | Default | Description |
 |-----|---------|-------------|
 | `frontend.enabled` | `true` | Deploy the frontend (nginx) Deployment, Service, and config ConfigMap. |
-| `frontend.image.repository` | `ghcr.io/nebari-dev/nebari-llm-serving-pack/frontend` | Frontend container image repository. |
+| `frontend.image.repository` | `ghcr.io/nebari-dev/llm-serving-pack/frontend` | Frontend container image repository. |
 | `frontend.image.tag` | `""` | Image tag. Defaults to `.Chart.AppVersion` when empty. |
 | `frontend.image.pullPolicy` | `Always` | Image pull policy. |
 | `frontend.port` | `8080` | Container port nginx listens on. |
@@ -97,7 +102,7 @@ The React SPA served by nginx. It serves the SPA and a runtime `/config.json`, a
 
 | Key | Default | Description |
 |-----|---------|-------------|
-| `operator.image.repository` | `ghcr.io/nebari-dev/nebari-llm-serving-pack/operator` | Operator container image repository. |
+| `operator.image.repository` | `ghcr.io/nebari-dev/llm-serving-pack/operator` | Operator container image repository. |
 | `operator.image.tag` | `""` | Image tag. Defaults to `.Chart.AppVersion` when empty. Override only when testing a specific build. |
 | `operator.image.pullPolicy` | `Always` | Image pull policy. |
 
@@ -197,6 +202,10 @@ Escape hatches for power users. All fields optional.
 | `spec.advanced.vllm.extraEnv` | []EnvVar | Additional environment variables on the vLLM container (supports `value`, `valueFrom`, `secretKeyRef`, etc.). |
 | `spec.advanced.inferencePool.schedulerConfig` | object | EPP scheduler plugin configuration (free-form, `x-kubernetes-preserve-unknown-fields`). |
 
+A complete manifest exercising these fields:
+
+<Code code={advancedModel} lang="yaml" title="examples/models/advanced-scheduling.yaml" />
+
 ### Status fields
 
 The operator writes these fields to `status`; they are read-only.
@@ -213,6 +222,45 @@ The operator writes these fields to `status`; they are read-only.
 
 ---
 
+## PassthroughModel CRD reference
+
+`PassthroughModel` (`passthroughmodels.llm.nebari.dev`, group `llm.nebari.dev/v1alpha1`) routes the shared endpoints to an external OpenAI-compatible provider instead of a locally served model. It provisions gateway routing and the same two auth layers as `LLMModel`, but no storage, serving, or scheduling. Required fields: `spec.provider`, `spec.access`.
+
+### spec.provider
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `spec.provider.hostname` | string | Yes | Provider hostname, e.g. `openrouter.ai` (bare host, TLS assumed). |
+| `spec.provider.port` | integer | No (default `443`) | Provider port. |
+| `spec.provider.schemaVersion` | string | No (default `v1`) | Upstream API path prefix, e.g. `api/v1` for OpenRouter, `v1` for api.openai.com. |
+| `spec.provider.credentialSecretName` | string | Yes | Secret in the PassthroughModel's namespace whose `apiKey` key holds the provider API key. Injected upstream; end users never see it. |
+
+### spec.models
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `spec.models.catchAll` | boolean | No | Route any model id not claimed by a more specific route. Served LLMModels always win (they match Host and `x-ai-eg-model`). |
+| `spec.models.declared` | []string | No | Explicit model ids; also advertised by the gateway's `/v1/models` (external route only) so UIs can build a picker. |
+
+### spec.access
+
+Same semantics as `LLMModel`: set `public: true` or list `groups` (see [spec.access](#specaccess) above).
+
+### spec.endpoints
+
+| Field | Type | Default | Description |
+|-------|------|---------|-------------|
+| `spec.endpoints.external.enabled` | boolean | `true` | Create the external (API-key) endpoint. |
+| `spec.endpoints.internal.enabled` | boolean | `true` | Create the internal (JWT) endpoint. |
+
+> **Naming constraint:** an LLMModel and a PassthroughModel cannot share a name in the same namespace (both derive the `<name>-api-keys` Secret). Only one catch-all PassthroughModel per gateway is supported.
+
+### Minimal PassthroughModel example
+
+<Code code={passthroughExample} lang="yaml" title="examples/passthrough-openrouter.yaml" />
+
+---
+
 ## NebariApp fields (key-manager)
 
 The chart renders a `NebariApp` CR (API group `reconcilers.nebari.dev/v1`) when `keyManager.nebariApp.enabled=true`. The `nebari-operator` consumes this CR to provision an HTTPRoute, a cert-manager Certificate, a Keycloak OIDC client, and a landing page tile.
@@ -251,27 +299,4 @@ For the full NebariApp CRD schema, see the [nebari-operator](https://github.com/
 
 The example below is the smallest valid manifest. See [Quickstart](/quickstart/) for an annotated walkthrough and [Shared Storage](/shared-storage/) for PVC options.
 
-```yaml
-apiVersion: llm.nebari.dev/v1alpha1
-kind: LLMModel
-metadata:
-  name: my-model
-  namespace: llm-serving
-spec:
-  access:
-    public: true
-  model:
-    name: mistralai/Devstral-Small-2505
-    source: huggingface
-    authSecretName: hf-token
-    preload: true
-    storage:
-      type: pvc
-      size: 200Gi
-  resources:
-    gpu:
-      count: 1
-      type: nvidia
-  serving:
-    replicas: 1
-```
+<Code code={minimalModel} lang="yaml" title="examples/models/minimal.yaml" />

docs/src/content/docs/installation.md

@@ -837,7 +837,8 @@ non-empty `id`.
 
 ## 8. Install the nebari-llm-serving pack
 
-The pack itself ships as a single Helm chart that reconciles three things
+The pack itself ships as a single Helm chart, published to the
+`quay.io/nebari/charts` OCI registry, that reconciles three things
 into the cluster:
 
 - The **pack operator** (`nebari-llm-serving-operator`) which watches
@@ -875,9 +876,9 @@ metadata:
 spec:
   project: foundational
   source:
-    repoURL: https://github.com/nebari-dev/llm-serving-pack.git
-    targetRevision: v0.1.0-alpha.9
-    path: charts/nebari-llm-serving
+    repoURL: quay.io/nebari/charts
+    chart: nebari-llm-serving
+    targetRevision: "0.1.2"
     helm:
       releaseName: nebari-llm-serving
       values: |
@@ -925,6 +926,10 @@ spec:
       backoff: { duration: 5s, factor: 2, maxDuration: 3m }
 ```
 
+A generic multi-source template (chart + a cluster-config repo for
+LLMModel CRs) is at
+[`examples/argocd-application.yaml`](https://github.com/nebari-dev/llm-serving-pack/blob/main/examples/argocd-application.yaml).
+
 The `platform.gateway.external` and `platform.gateway.internal` blocks
 both point at `nebari-gateway` because this runbook uses a single
 shared Gateway. With `manageSharedListeners: true` the operator adds

docs/src/content/docs/quickstart.md → docs/src/content/docs/quickstart.mdx

@@ -1,6 +1,9 @@
 ---
 title: Quickstart
 ---
+import { Code } from '@astrojs/starlight/components';
+import passthroughExample from '../../../../examples/passthrough-openrouter.yaml?raw';
+
 Get a model serving in five minutes. This page covers the fast path: deploy the pack, apply one `LLMModel`, and call the API.
 
 For a full production-grade setup (air-gapped clusters, HuggingFace token secrets, monitoring, and all values), see [Installation](/installation/) and [Configuration](/configuration/).
@@ -144,7 +147,12 @@ spec:
       enabled: true
 ```
 
-For gated HuggingFace models, create a Secret with your HuggingFace token and reference it:
+For gated HuggingFace models, create a Secret holding your token, then reference it from the model spec:
+
+```bash
+kubectl create secret generic hf-token -n nebari-llm-serving-system \
+  --from-literal=HF_TOKEN=hf_your_token_here
+```
 
 ```yaml
 spec:
@@ -190,3 +198,29 @@ response = client.chat.completions.create(
     messages=[{"role": "user", "content": "Hello"}],
 )
 ```
+
+## Route to an external provider
+
+Instead of serving a model locally, a `PassthroughModel` points the shared endpoints at an external OpenAI-compatible provider (OpenRouter, api.openai.com, a remote vLLM). Users get the same two auth layers and mint keys in the same UI; the platform's provider key is injected upstream and never exposed.
+
+First create a Secret holding the platform's provider key (the `apiKey` field), in the operator namespace:
+
+```bash
+kubectl create secret generic openrouter-api-key -n nebari-llm-serving-system \
+  --from-literal=apiKey='sk-or-v1-...'
+```
+
+Then apply the PassthroughModel:
+
+<Code code={passthroughExample} lang="yaml" title="examples/passthrough-openrouter.yaml" />
+
+Once it reconciles, any provider model id works against the shared external endpoint with a minted key (or against `llm-internal.<baseDomain>` with a Keycloak JWT):
+
+```bash
+curl https://llm.your-cluster.example.com/v1/chat/completions \
+  -H "Authorization: Bearer sk-your-api-key" \
+  -H "Content-Type: application/json" \
+  -d '{"model": "openai/gpt-5.2", "messages": [{"role": "user", "content": "Hello"}]}'
+```
+
+See the [PassthroughModel CRD reference](/configuration/#passthroughmodel-crd-reference) for every field.

docs/src/content/docs/shared-storage.md → docs/src/content/docs/shared-storage.mdx

@@ -1,6 +1,10 @@
 ---
 title: Shared Storage
 ---
+import { Code } from '@astrojs/starlight/components';
+import ociModel from '../../../../examples/models/oci-model.yaml?raw';
+import devstralModel from '../../../../examples/models/devstral-small.yaml?raw';
+
 Model files are large. Downloading a 30-80 GB model on every pod start is slow, wastes bandwidth, and delays requests. The LLM Serving Pack solves this with persistent model storage managed by the operator: models are downloaded once, cached on a PVC, and shared across replicas.
 
 ## How model storage works
@@ -18,7 +22,7 @@ All model loading paths - HuggingFace and OCI - mount the model at `/model-cache
 When `model.source` is `huggingface` (or any value other than `oci`, including an empty string when the field is left unset - the operator treats any non-`oci` source as HuggingFace), the operator:
 
 1. Creates a PVC named `<llmmodel-name>-model-storage` in the same namespace
-2. Adds an init container running `ghcr.io/nebari-dev/nebari-llm-serving-pack/model-downloader:latest`
+2. Adds an init container running `ghcr.io/nebari-dev/llm-serving-pack/model-downloader:latest`
 3. The init container runs `hf download <model-name> --local-dir /model-cache`
 4. Mounts the PVC at `/model-cache` in both the init container and the vLLM container
 
@@ -57,6 +61,10 @@ kubectl create secret generic hf-token -n <operator-namespace> \
 
 The operator injects `HF_TOKEN` as an environment variable into the init container.
 
+A complete gated-model manifest (create the `hf-token` Secret first, as shown above):
+
+<Code code={devstralModel} lang="yaml" title="examples/models/devstral-small.yaml" />
+
 ### PVC access mode
 
 The PVC is created with `accessMode: ReadWriteOnce`. Most cloud block storage classes only support `ReadWriteOnce`, which means all replicas must be scheduled on the same node when sharing the PVC. If you need replicas on different nodes, use a storage class backed by a network filesystem (NFS, EFS, Filestore) that supports `ReadWriteMany`.
@@ -91,6 +99,10 @@ Tags are mutable; a digest pins the exact image layer set.
 
 Note: Kubernetes image volumes (alpha in 1.31, beta in 1.35) would eliminate the copy step, but many clusters do not have the `ImageVolume` feature gate enabled. The init-container copy path is the only supported OCI mechanism in v0.1.
 
+A complete OCI manifest:
+
+<Code code={ociModel} lang="yaml" title="examples/models/oci-model.yaml" />
+
 ## Storage type: PVC vs emptyDir for HuggingFace models
 
 For HuggingFace models you can choose between `pvc` (default) and `emptyDir`:

docs/src/content/docs/troubleshooting.md

@@ -239,7 +239,7 @@ Check the LLMModel status conditions for a machine-readable message:
 
 ```bash
 kubectl get llmmodel -n nebari-llm-serving-system <model-name> \
-  -o jsonpath='{.status.conditions}' | python3 -m json.tool
+  -o jsonpath='{range .status.conditions[*]}{.type}={.status} {.reason}: {.message}{"\n"}{end}'
 ```
 
 See the [configuration reference](/configuration/) for valid LLMModel field values.
@@ -274,3 +274,35 @@ If either flag is missing, update the ConfigMap (or the ArgoCD Application value
 ```bash
 kubectl rollout restart deploy -n envoy-gateway-system envoy-gateway
 ```
+
+---
+
+## External provider (PassthroughModel) not working
+
+**Upstream 401 / auth errors from the provider.** The gateway injects the key from `spec.provider.credentialSecretName`. Confirm the Secret exists in the operator namespace and has an `apiKey` key:
+
+```bash
+kubectl -n nebari-llm-serving-system get secret <credentialSecretName> \
+  -o jsonpath='{.data.apiKey}' | base64 -d | head -c 8; echo
+```
+
+If empty or the wrong key name, recreate it: the key MUST be named `apiKey`.
+
+**PassthroughModel stuck with `ApplyFailed`.** This condition usually means the Envoy AI Gateway CRDs are not installed; the operator requeues every minute rather than failing outright. Check the CR and the CRDs:
+
+```bash
+kubectl -n nebari-llm-serving-system get passthroughmodel <name> \
+  -o jsonpath='{range .status.conditions[*]}{.type}={.status} {.reason}: {.message}{"\n"}{end}'
+kubectl get crd | grep aigateway
+```
+
+Install the AI Gateway (see [Installation](/installation/)) and the next reconcile clears the condition.
+
+**A model id you expected to hit the provider is served locally instead.** Per-LLMModel routes match Host and `x-ai-eg-model`, so they always win over a catch-all PassthroughModel. If a declared provider model id collides with a served model name, the served model wins. Rename or remove the conflicting LLMModel, or declare the provider model under a distinct id.
+
+```bash
+# If the model id you expected to reach the provider appears here, that
+# LLMModel's route wins over the catch-all. Rename or remove the LLMModel,
+# or declare the provider model under a different id.
+kubectl get llmmodels -n nebari-llm-serving-system
+```