diff --git a/docs/getting-started/local-installation.md b/docs/getting-started/local-installation.md index 4358bfc45681..2432cd2c220c 100644 --- a/docs/getting-started/local-installation.md +++ b/docs/getting-started/local-installation.md @@ -57,6 +57,8 @@ versions and backend guides for run instructions: [SGLang](../backends/sglang/RE ### Option B: Install from PyPI +Supported for vLLM and SGLang only. Use Option A for TensorRT-LLM. + ```bash # Install uv (recommended Python package manager) curl -LsSf https://astral.sh/uv/install.sh | sh @@ -79,19 +81,6 @@ uv pip install --prerelease=allow "ai-dynamo[sglang]" For CUDA 13 (B300/GB300), the container is recommended. See [SGLang install docs](https://docs.sglang.io/get_started/install.html) for details. -**TensorRT-LLM** - -```bash -sudo apt install python3-dev -pip install torch==2.9.0 torchvision --index-url https://download.pytorch.org/whl/cu130 -pip install --pre --extra-index-url https://pypi.nvidia.com "ai-dynamo[trtllm]" -``` - -TensorRT-LLM requires `pip` due to a transitive Git URL dependency that -`uv` doesn't resolve. We recommend using the TensorRT-LLM container for -broader compatibility. See the [TRT-LLM backend guide](../backends/trtllm/README.md) -for details. - **vLLM** ```bash @@ -209,9 +198,10 @@ The default model `Qwen/Qwen3-0.6B` requires ~2GB of GPU memory. Larger models n Start with a small model and scale up based on your hardware. -**Python 3.11 with TensorRT-LLM** +**TensorRT-LLM** -TensorRT-LLM does not support Python 3.11. If you see installation failures with TensorRT-LLM, check your Python version with `python3 --version`. Use Python 3.10 or 3.12 instead. +TensorRT-LLM is not supported via a local PyPI install. Use the +`tensorrtllm-runtime` container (Option A). **Container runs but GPU not detected** diff --git a/docs/getting-started/local-installation.zh-CN.md b/docs/getting-started/local-installation.zh-CN.md index 829ce3c36054..43ed290fe6e3 100644 --- a/docs/getting-started/local-installation.zh-CN.md +++ b/docs/getting-started/local-installation.zh-CN.md @@ -58,6 +58,8 @@ docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/vllm-runt ### 选项 B:从 PyPI 安装 +仅支持 vLLM 和 SGLang。TensorRT-LLM 请使用选项 A。 + ```bash # Install uv (recommended Python package manager) curl -LsSf https://astral.sh/uv/install.sh | sh @@ -80,18 +82,6 @@ uv pip install --prerelease=allow "ai-dynamo[sglang]" 对于 CUDA 13(B300/GB300),推荐使用容器。详情请参阅 [SGLang 安装文档](https://docs.sglang.io/get_started/install.html)。 -**TensorRT-LLM** - -```bash -sudo apt install python3-dev -pip install torch==2.9.0 torchvision --index-url https://download.pytorch.org/whl/cu130 -pip install --pre --extra-index-url https://pypi.nvidia.com "ai-dynamo[trtllm]" -``` - -由于传递性 Git URL 依赖项 `uv` 无法解析,TensorRT-LLM 需要使用 `pip`。 -为获得更广泛的兼容性,我们建议使用 TensorRT-LLM 容器。 -详情请参阅 [TRT-LLM 后端指南](../backends/trtllm/README.md)。 - **vLLM** ```bash @@ -210,7 +200,8 @@ curl localhost:8000/v1/chat/completions \ **TensorRT-LLM 与 Python 3.11** -TensorRT-LLM 不支持 Python 3.11。如果你在安装 TensorRT-LLM 时看到失败,请使用 `python3 --version` 检查 Python 版本。请改用 Python 3.10 或 3.12。 +TensorRT-LLM 仅支持容器路径。请使用 `tensorrtllm-runtime` +容器(选项 A)。 **容器运行但未检测到 GPU** diff --git a/docs/reference/release-artifacts.md b/docs/reference/release-artifacts.md index d2de3308d5ca..674f0cc89391 100644 --- a/docs/reference/release-artifacts.md +++ b/docs/reference/release-artifacts.md @@ -37,7 +37,8 @@ Release history in this document begins at v0.6.0. ### Python Wheels -We recommend using the TensorRT-LLM NGC container instead of the `ai-dynamo[trtllm]` wheel. See the [NGC container collection](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/collections/ai-dynamo) for supported images. +For TensorRT-LLM, use the NGC container — not the `ai-dynamo[trtllm]` PyPI extra. +See the [NGC container collection](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-dynamo/collections/ai-dynamo). | Package | Description | Python | Platform | PyPI | |---------|-------------|--------|----------|------| @@ -106,11 +107,12 @@ docker pull nvcr.io/nvidia/ai-dynamo/snapshot-agent:1.2.1 > For detailed installation instructions, see the [Quickstart](https://docs.nvidia.com/dynamo/getting-started/quickstart) in the docs. ```bash -# Install Dynamo with a specific backend (Recommended) +# Install Dynamo with a specific backend (vLLM and SGLang) uv pip install "ai-dynamo[vllm]==1.2.1" uv pip install --prerelease=allow "ai-dynamo[sglang]==1.2.1" -# TensorRT-LLM requires the NVIDIA PyPI index and pip -pip install --pre --extra-index-url https://pypi.nvidia.com "ai-dynamo[trtllm]==1.2.1" + +# TensorRT-LLM: use container (see above) +docker pull nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:1.2.1 # Install Dynamo core only uv pip install ai-dynamo==1.2.1