feat(trtllm): upgrade TensorRT-LLM to v1.2.1 - #853
Merged
Conversation
mc-nv
marked this pull request as ready for review
April 16, 2026 21:52
3 tasks
whoisj
pushed a commit
that referenced
this pull request
Apr 20, 2026
* docs: add PyTorch (LLMAPI) quick start to README Replaces the broken Steps 2-4 in the PyTorch Backend section with commands that actually work. Key fixes: - Clone TRT-LLM source to /tmp to get the llmapi template (the git submodule in the repo is a placeholder and may be empty) - Run the server from /tmp to avoid Python resolving `tensorrt_llm` to the source tree instead of the installed package, which causes ModuleNotFoundError: No module named 'tensorrt_llm.bindings' - Add a NOTE callout explaining the import conflict pitfall - Add metrics curl example in Step 5 Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com> * docs: make PyTorch/LLMAPI the default quick start, archive engine backend - Move PyTorch (LLM API) to the primary Getting Started section - Replace the 400-line engine backend quick start with a single link to docs/README-engine-backend-archive.md - Simplify PyTorch quick start to: git clone → edit model.yaml → launch - Add [!IMPORTANT] note about running from the clone parent directory (not inside TensorRT-LLM/) to avoid tensorrt_llm.bindings import error Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com> * docs: add engine backend archive for legacy reference Saves the original engine-backend quick start (trtllm-build, inflight_batcher_llm) as docs/README-engine-backend-archive.md so existing users can still find it. The main README now links here from the Legacy section. Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com> * docs: simplify TRT-LLM clone instructions Remove the TRT-LLM version check from the quick start and switch the clone step to use the upstream repository without a pinned branch. Made-with: Cursor Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com> * docs: inline legacy engine backend as a section in README Replace the separate docs/README-engine-backend-archive.md with a "Legacy: TensorRT Engine Backend" section at the bottom of the root README. The archive duplicated most of the current README; the only unique content was the Quick Start (trtllm-build, inflight_batcher_llm model repo setup, config templating, and client examples), which is now preserved inline. TOC and Getting Started link updated to use the in-page anchor. Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com> * docs: add request cancellation example to PyTorch backend quick start Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com> * docs: add LLM API and SamplingParams reference links to PyTorch quick start Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com> * docs: remove redundant SamplingParams link, one sentence per line Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com> * docs: address PR review comments from whoisj - Add missing `> ` to NOTE blockquote continuation line - Indent code blocks under bullet points so they render correctly - Add blank line before "For example one could run:" for readability - Left-align table columns and simplify descriptions for readability Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com> * docs: address PR review comments from whoisj Move inline comments out of bash blocks as prose and split the cancel example into two separate code blocks for easier copy-paste. Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com> * docs: soften llmapi framing, remove "Legacy" label from engine backend Addresses reviewer concern that promoting llmapi as "Recommended" is premature without automated CI coverage. Changes: - Rename section from "— Recommended" to "— Simpler Setup" - Remove "Legacy" label from TensorRT Engine Backend sections - Add note that llmapi CI coverage is tracked in issue #853 - Clarify engine backend note: it has full CI coverage Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com> * docs: fix CI tracking issue number to #854 Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com> * docs: address remaining yinggeh review comments - Add sub-section quick links under PyTorch Backend ToC entry - Fix top-level TensorRT Engine Backend ToC anchor to #tensorrt-engine-backend-1 - Fix inline redirect anchor to #tensorrt-engine-backend-1 - Remove CI tracking note from PyTorch Backend intro Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com> --------- Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>
mc-nv
force-pushed
the
mchornyi/TRI-934/building-the-trtllm-container
branch
from
April 22, 2026 04:31
f854143 to
9b4bcf6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI (internal): http://tritonserver.local/ci/pipelines/48725037
Description
Upgrades the TensorRT-LLM backend to the stable v1.2.1 release. The previous build was pinned to
release/1.2/1.2.0rc8. This change tracks the newrelease/1.2.1tag and advances thetensorrt_llmsubmodule to the matching commit.Changes
Affected Files
dockerfile/Dockerfile.triton.trt_llm_backendtensorrt_llm(submodule)