Skip to content

Commit 59e76f4

Browse files
authored
docs(dgdr): clarify rapid fallback gates (#11040)
Signed-off-by: Dr. Stefan Schimanski <sschimanski@nvidia.com>
1 parent 7bb3a08 commit 59e76f4

2 files changed

Lines changed: 20 additions & 10 deletions

File tree

docs/components/profiler/profiler-guide.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -78,14 +78,18 @@ flowchart TD
7878

7979
### Rapid
8080

81-
Uses AIC's performance simulation to estimate optimal configurations without deploying real engines. Completes in ~30 seconds.
81+
Uses AIC's performance simulation to estimate optimal configurations without
82+
deploying real engines. Completes in ~30 seconds; see the
83+
[AIC support matrix](https://ai-dynamo.github.io/aiconfigurator/support-matrix/).
8284

8385
```yaml
8486
searchStrategy: rapid
8587
```
8688
8789
- Supports all backends: vLLM, SGLang, TensorRT-LLM
88-
- If the model/hardware/backend combination is not supported by AIC, falls back to a naive config (memory-fit TP calculation)
90+
- Falls back to a naive config (memory-fit TP calculation) only after DGDR
91+
accepts the GPU SKU. Fallback sizing depends on AIC system metadata and does
92+
not add support for additional GPU SKUs.
8993
- No GPU resources consumed during profiling
9094
9195
### Thorough

docs/kubernetes/model-deployment-guide.md

Lines changed: 14 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -159,9 +159,10 @@ GPU resources consumed during profiling.
159159
- Your GPU SKU is in the [AIC support matrix](#aic-support-matrix)
160160
161161
**Limitations:**
162-
- If AIC does not support your model/hardware/backend combination, the profiler
163-
falls back to a naive memory-fit config (basic TP calculation) which may not
164-
be optimal.
162+
- Fallback to a naive memory-fit config only applies after DGDR accepts
163+
`hardware.gpuSku`. Fallback sizing depends on AIC system metadata and does
164+
not add support for additional GPU SKUs; unsupported model/GPU/backend
165+
combinations can fall back but may be suboptimal.
165166
- Simulated results may differ from real-hardware performance for unusual
166167
configurations.
167168

@@ -177,23 +178,28 @@ benchmarks with AIPerf. Takes 2–4 hours.
177178

178179
**Use thorough when:**
179180
- Tuning for production and you need the most optimal configuration
180-
- Your hardware is not supported by AIC (e.g., PCIe GPUs)
181181
- You want measured rather than simulated performance data
182182

183183
**Constraints:**
184184
- **Disaggregated mode only** — thorough does not run aggregated configurations.
185185
- **`backend: auto` is not supported** — you must specify `vllm`, `sglang`, or
186186
`trtllm`. The DGDR will be rejected if you use `auto` with `thorough`.
187+
- **Still requires AIC generator support** — thorough measures candidates on
188+
real GPUs, but uses AIC to enumerate candidates and generate the final DGD.
187189
- **Requires GPU resources** — the profiler deploys real inference engines on
188190
your cluster during profiling.
189191

190192
## DGDR Detail: AIC Support Matrix
191193

192-
The rapid strategy relies on AIC performance models. AIC currently supports:
194+
The rapid strategy relies on AIC system metadata and performance models. Check
195+
the [AIC support matrix](https://ai-dynamo.github.io/aiconfigurator/support-matrix/)
196+
for the latest support. Measured profiling still needs AIC system and generator
197+
support to enumerate and render candidates; rapid also needs performance support
198+
for the exact model/GPU/backend combination.
193199

194200
### GPU SKUs
195201

196-
| Supported (rapid) | Not Yet Supported (use thorough) |
202+
| Supported (rapid) | Not supported by rapid |
197203
|---|---|
198204
| H100 SXM | V100 (SXM/PCIe) |
199205
| H100 PCIe | T4 |
@@ -208,8 +214,8 @@ The rapid strategy relies on AIC performance models. AIC currently supports:
208214

209215
> [!NOTE]
210216
> Some rapid-mode SKUs use AIC estimate-only data until measured profiles are
211-
> available. Use `searchStrategy: thorough` when you need hardware-measured
212-
> profiling for an estimate-only or unsupported SKU.
217+
> available. Use `searchStrategy: thorough` for measured profiling only when
218+
> DGDR accepts the SKU and AIC has system and generator support for it.
213219

214220
When specifying GPU SKUs manually, use lowercase underscore format (e.g.,
215221
`h100_sxm`, not `H100-SXM5-80GB`). See the

0 commit comments

Comments
 (0)