Skip to content

Commit 7293c88

Browse files
authored
docs: address #137 follow-ups (helm install, key-manager API, anchors, gpu-operator note) (#142)
* docs: address issue #137 follow-ups (helm install, key-manager API, revocation, index links, spec.access anchor, gpu-operator note) * docs: fix gpu-operator AMI framing, use version placeholder, clarify key revocation (review feedback)
1 parent f208392 commit 7293c88

6 files changed

Lines changed: 52 additions & 7 deletions

File tree

docs/src/content/docs/cicd-and-releasing.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -141,6 +141,25 @@ helm show values oci://quay.io/nebari/charts/nebari-llm-serving --version <versi
141141

142142
To install a released version, see the [Installation](/installation/) guide. To work with a local development build, which reads `values.yaml` as committed (so `latest`-tagged images), see the [Local Development](/local-development/) page.
143143

144+
## Installing a released chart directly
145+
146+
The [Installation](/installation/) guide's ArgoCD Application is the recommended path on Nebari - it keeps the release version under GitOps and gets you `selfHeal`/drift correction for free. For a local or manual install (testing a release, a non-ArgoCD cluster, or just poking around), install the published chart directly with `helm install`:
147+
148+
```bash
149+
# From the OCI registry (no repo add needed):
150+
helm install nebari-llm-serving \
151+
oci://quay.io/nebari/charts/nebari-llm-serving --version <version> \
152+
-n nebari-llm-serving-system --create-namespace -f my-values.yaml
153+
154+
# Or via the Helm repo index:
155+
helm repo add nebari https://raw.githubusercontent.com/nebari-dev/helm-repository/gh-pages/
156+
helm repo update
157+
helm install nebari-llm-serving nebari/nebari-llm-serving --version <version> \
158+
-n nebari-llm-serving-system --create-namespace -f my-values.yaml
159+
```
160+
161+
Replace `<version>` with a released version (the newest is on the repository's Releases page). Both forms install the same packaged chart with the same pinned image shas (see [Releasing](#releasing) above); `my-values.yaml` should set at least `platform.baseDomain` and the other required values covered in [Configuration](/configuration/).
162+
144163
## Known Gaps
145164

146165
- There is no chart-testing (`ct lint`) step that validates the chart against multiple Kubernetes versions. Contributions adding `helm/chart-testing-action` are welcome.

docs/src/content/docs/configuration.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -242,7 +242,7 @@ The operator writes these fields to `status`; they are read-only.
242242
| `spec.models.catchAll` | boolean | No | Route any model id not claimed by a more specific route. Served LLMModels always win (they match Host and `x-ai-eg-model`). |
243243
| `spec.models.declared` | []string | No | Explicit model ids; also advertised by the gateway's `/v1/models` (external route only) so UIs can build a picker. |
244244

245-
### spec.access
245+
### spec.access (external provider)
246246

247247
Same semantics as `LLMModel`: set `public: true` or list `groups` (see [spec.access](#specaccess) above).
248248

docs/src/content/docs/index.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,8 @@ Apply it, and the operator downloads the weights, starts vLLM, registers the mod
4040
inference scheduler, creates the internal and external routes, and gates them on the groups you
4141
listed. See the [Quickstart](/quickstart/) for the end-to-end flow.
4242
43+
Ready-to-apply manifests are in [`examples/models/`](https://github.com/nebari-dev/llm-serving-pack/tree/main/examples/models) (minimal, gated-HuggingFace, OCI, quantized, and advanced-scheduling).
44+
4345
## What you get
4446

4547
- **Declarative model serving** - one `LLMModel` CR per model; the operator reconciles the rest.

docs/src/content/docs/installation.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -350,6 +350,17 @@ spec:
350350
maxDuration: 3m
351351
```
352352
353+
> **Which values block to use.** The distinction is what your GPU node's AMI
354+
> already ships. The manifest above disables only the driver
355+
> (`driver.enabled: false`), so the operator still installs the container
356+
> toolkit and device plugin: use it when the AMI ships the NVIDIA driver but
357+
> not the toolkit, which is the case for NIC's AL2023 NVIDIA AMI. If your AMI
358+
> ships BOTH the driver and the toolkit, use the values in
359+
> [`examples/nvidia-gpu-operator.yaml`](https://github.com/nebari-dev/llm-serving-pack/blob/main/examples/nvidia-gpu-operator.yaml)
360+
> instead, which also sets `toolkit.enabled: false` so the operator adds only
361+
> the device plugin. If your nodes ship neither, set `driver.enabled: true` so
362+
> the operator installs the driver as well.
363+
353364
`git push` the file. ArgoCD's `nebari-root` app-of-apps picks it up on
354365
its next refresh (typically within a minute; you can force it with
355366
`kubectl annotate application -n argocd nebari-root argocd.argoproj.io/refresh=hard --overwrite`).

docs/src/content/docs/quickstart.mdx

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -183,6 +183,18 @@ curl https://llm.your-cluster.example.com/v1/chat/completions \
183183
-d '{"model": "Qwen/Qwen3.5-35B-A3B-GPTQ-Int4", "messages": [{"role": "user", "content": "Hello"}]}'
184184
```
185185

186+
You can also mint a key from the command line instead of the UI. The key-manager exposes a REST API under the same hostname (`keyManager.nebariApp.hostname`, e.g. `keys.llm.<baseDomain>`):
187+
188+
```bash
189+
# Mint a key via the API instead of the UI (bearer token = a Keycloak JWT):
190+
curl -X POST https://keys.llm.<baseDomain>/api/keys \
191+
-H "Authorization: Bearer <your-jwt>" \
192+
-H "Content-Type: application/json" \
193+
-d '{"modelName": "Qwen/Qwen3.5-35B-A3B-GPTQ-Int4"}'
194+
```
195+
196+
To revoke a key, delete it in the key manager UI, or call `DELETE /api/keys/{namespace}/{model}/{clientID}` with the same bearer token (the `namespace`, `model`, and `clientID` come from the mint response, or from `GET /api/keys`). A revoked key immediately returns 401 on the external route.
197+
186198
### Internal access (JWT from JupyterLab or in-cluster service)
187199

188200
```python

examples/nvidia-gpu-operator.yaml

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,11 @@
22
# Required for GPU nodes to advertise nvidia.com/gpu as an allocatable
33
# resource. Without this, GPU-requiring pods stay Pending.
44
#
5-
# If your nodes use a pre-installed NVIDIA driver AMI (like AWS
6-
# AL2023_x86_64_NVIDIA), disable driver and toolkit installation since
7-
# the AMI already includes both.
5+
# Use this values block when your GPU nodes boot from an AMI that ships BOTH
6+
# the NVIDIA driver and the container toolkit: it disables both so the
7+
# operator only adds the device plugin. If your AMI ships the driver but not
8+
# the toolkit (e.g. NIC's AL2023 NVIDIA AMI), keep toolkit enabled instead
9+
# (see the Installation guide's nvidia-gpu-operator step).
810
apiVersion: argoproj.io/v1alpha1
911
kind: Application
1012
metadata:
@@ -24,9 +26,8 @@ spec:
2426
helm:
2527
releaseName: nvidia-gpu-operator
2628
values: |
27-
# Disable driver and toolkit if using a pre-installed NVIDIA AMI.
28-
# Remove these overrides if your nodes need the operator to install
29-
# NVIDIA drivers.
29+
# Disable driver and toolkit when the AMI ships both. Remove these
30+
# overrides if your nodes need the operator to install the driver.
3031
driver:
3132
enabled: false
3233
toolkit:

0 commit comments

Comments
 (0)