Skip to content

Commit dd3b71e

Browse files
tanmayv25rmccorm4
andauthored
docs(trtllm): Update the trtllm installation instructions (#11066)
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
1 parent ece7858 commit dd3b71e

3 files changed

Lines changed: 15 additions & 32 deletions

File tree

docs/getting-started/local-installation.md

Lines changed: 5 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,8 @@ versions and backend guides for run instructions: [SGLang](../backends/sglang/RE
5555

5656
### Option B: Install from PyPI
5757

58+
Supported for vLLM and SGLang only. Use Option A for TensorRT-LLM.
59+
5860
```bash
5961
# Install uv (recommended Python package manager)
6062
curl -LsSf https://astral.sh/uv/install.sh | sh
@@ -77,19 +79,6 @@ uv pip install --prerelease=allow "ai-dynamo[sglang]"
7779
For CUDA 13 (B300/GB300), the container is recommended. See
7880
[SGLang install docs](https://docs.sglang.io/get_started/install.html) for details.
7981

80-
**TensorRT-LLM**
81-
82-
```bash
83-
sudo apt install python3-dev
84-
pip install torch==2.9.0 torchvision --index-url https://download.pytorch.org/whl/cu130
85-
pip install --pre --extra-index-url https://pypi.nvidia.com "ai-dynamo[trtllm]"
86-
```
87-
88-
TensorRT-LLM requires `pip` due to a transitive Git URL dependency that
89-
`uv` doesn't resolve. We recommend using the TensorRT-LLM container for
90-
broader compatibility. See the [TRT-LLM backend guide](../backends/trtllm/README.md)
91-
for details.
92-
9382
**vLLM**
9483

9584
```bash
@@ -207,9 +196,10 @@ The default model `Qwen/Qwen3-0.6B` requires ~2GB of GPU memory. Larger models n
207196

208197
Start with a small model and scale up based on your hardware.
209198

210-
**Python 3.11 with TensorRT-LLM**
199+
**TensorRT-LLM**
211200

212-
TensorRT-LLM does not support Python 3.11. If you see installation failures with TensorRT-LLM, check your Python version with `python3 --version`. Use Python 3.10 or 3.12 instead.
201+
TensorRT-LLM is not supported via a local PyPI install. Use the
202+
`tensorrtllm-runtime` container (Option A).
213203

214204
**Container runs but GPU not detected**
215205

docs/getting-started/local-installation.zh-CN.md

Lines changed: 4 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,8 @@ docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/vllm-runt
5757

5858
### 选项 B:从 PyPI 安装
5959

60+
仅支持 vLLM 和 SGLang。TensorRT-LLM 请使用选项 A。
61+
6062
```bash
6163
# Install uv (recommended Python package manager)
6264
curl -LsSf https://astral.sh/uv/install.sh | sh
@@ -79,18 +81,6 @@ uv pip install --prerelease=allow "ai-dynamo[sglang]"
7981
对于 CUDA 13(B300/GB300),推荐使用容器。详情请参阅
8082
[SGLang 安装文档](https://docs.sglang.io/get_started/install.html)
8183

82-
**TensorRT-LLM**
83-
84-
```bash
85-
sudo apt install python3-dev
86-
pip install torch==2.9.0 torchvision --index-url https://download.pytorch.org/whl/cu130
87-
pip install --pre --extra-index-url https://pypi.nvidia.com "ai-dynamo[trtllm]"
88-
```
89-
90-
由于传递性 Git URL 依赖项 `uv` 无法解析,TensorRT-LLM 需要使用 `pip`
91-
为获得更广泛的兼容性,我们建议使用 TensorRT-LLM 容器。
92-
详情请参阅 [TRT-LLM 后端指南](../backends/trtllm/README.md)
93-
9484
**vLLM**
9585

9686
```bash
@@ -209,7 +199,8 @@ curl localhost:8000/v1/chat/completions \
209199

210200
**TensorRT-LLM 与 Python 3.11**
211201

212-
TensorRT-LLM 不支持 Python 3.11。如果你在安装 TensorRT-LLM 时看到失败,请使用 `python3 --version` 检查 Python 版本。请改用 Python 3.10 或 3.12。
202+
TensorRT-LLM 仅支持容器路径。请使用 `tensorrtllm-runtime`
203+
容器(选项 A)。
213204

214205
**容器运行但未检测到 GPU**
215206

docs/reference/release-artifacts.md

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,8 @@ Release history in this document begins at v0.6.0.
3737

3838
### Python Wheels
3939

40-
We recommend using the TensorRT-LLM NGC container instead of the `ai-dynamo[trtllm]` wheel. See the [NGC container collection](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/collections/ai-dynamo) for supported images.
40+
For TensorRT-LLM, use the NGC container — not the `ai-dynamo[trtllm]` PyPI extra.
41+
See the [NGC container collection](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/collections/ai-dynamo).
4142

4243
| Package | Description | Python | Platform | PyPI |
4344
|---------|-------------|--------|----------|------|
@@ -106,11 +107,12 @@ docker pull nvcr.io/nvidia/ai-dynamo/snapshot-agent:1.2.1
106107
> For detailed installation instructions, see the [Quickstart](https://docs.nvidia.com/dynamo/getting-started/quickstart) in the docs.
107108
108109
```bash
109-
# Install Dynamo with a specific backend (Recommended)
110+
# Install Dynamo with a specific backend (vLLM and SGLang)
110111
uv pip install "ai-dynamo[vllm]==1.2.1"
111112
uv pip install --prerelease=allow "ai-dynamo[sglang]==1.2.1"
112-
# TensorRT-LLM requires the NVIDIA PyPI index and pip
113-
pip install --pre --extra-index-url https://pypi.nvidia.com "ai-dynamo[trtllm]==1.2.1"
113+
114+
# TensorRT-LLM: use container (see above)
115+
docker pull nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:1.2.1
114116

115117
# Install Dynamo core only
116118
uv pip install ai-dynamo==1.2.1

0 commit comments

Comments
 (0)