test(gms): add TensorRT-LLM E2E coverage [GMS 13/18]

[hutm]

Summary

Isolate TensorRT-LLM end-to-end validation after its engine and host-tier integration, while keeping it before optional backends and router D2D transfer.

Scope

Adds local and Kubernetes TensorRT-LLM recipes and recovery assertions. It does not add runtime or backend functionality.

Stack

Position 13/18 in the GMS-managed KV review train. This PR is based on review/gms-v2-latehost-08-3-4-host-tier-trtllm-adapter.

Parent: #10450
Tracking: #10453 #10458

Review migration

This is the one new review entry required by the reordered train; the earlier train did not isolate TensorRT-LLM E2E coverage into its own PR.

The train is rebased on main at aeda23b483831df2f75b697b8ccf65f4ffd9f3d6. The reordered functionality is preserved while each PR boundary is independently buildable and lintable: compatibility call sites and the engine-facing placement adapter now appear at first use; missing type imports and test markers were added; repository-pinned formatting was applied; one unused constant was removed; and every commit carries a DCO sign-off. The complete range passes git diff --check and the full pre-commit suite.

Validation

Covers the dedicated TensorRT-LLM execution path; current-cluster IB, CUDA-graph, throughput, and recovery measurements remain tracked in #10458.

---

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[datadog-official]

⚠️ Warnings

🚦 2 Pipeline jobs failed

Docs link check | lychee     

Pre Merge | pre-merge-status-check     

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: dfad411 | Docs | Give us feedback!}

[devin-ai-integration]

Devin Review found 4 potential issues.

[devin-ai-integration]

🟡 Success log and comment still say "4" after request count was made configurable

The request-count parameterization was applied to the loop, assertions, and error messages but the final success log still hardcodes "All 4 requests" (tests/router/common.py:2391), and the adjacent comment still says "all 4 match" (tests/router/common.py:2364), so when the TRT-LLM test sends 3 requests the log falsely claims 4 were routed.

Impact: Misleading diagnostic output when debugging disaggregated routing tests that use a non-default request count.

Stale references after parameterization of progressive_request_count

The PR added progressive_request_count at tests/router/common.py:2153 and correctly updated the loop at line 2234, the progress log at line 2253, the count assertion at line 2351, the comment at line 2363 (changed "2-4" to "2-N"), and the TCP assertion message at line 2368.

However, two references were missed:

• Line 2364 (comment): # request plane we keep the stronger assertion that all 4 match. — should say "all N match" or similar.
• Line 2391 (log message): f" - All 4 requests routed to same prefill_worker_id={prefill_id} (prefix reuse)\n" — should use {progressive_request_count} instead of 4.

The TRT-LLM disagg test now defaults to progressive_request_count=3 (tests/router/test_router_e2e_with_trtllm.py:443), so the log will print "All 4 requests" even though only 3 were sent.

Prompt for agents

In _test_router_decisions_disagg in tests/router/common.py, two references to the hardcoded count of 4 were missed when progressive_request_count was parameterized:

1. Line 2364 (comment): change 'all 4 match' to reference the variable count, e.g. 'all N match' or 'all requests match'.
2. Line 2391 (log f-string): change 'All 4 requests' to f'All {progressive_request_count} requests'.

Also, the docstring around line 2162 says 'Sends 4 progressive requests' and should be updated to reflect the parameter.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🚩 Default prefill workers reduced from 2 to 1 in disagg test

The TRT-LLM disagg test previously hardcoded num_prefill_workers=2 but now defaults to 1 via os.environ.get("DYNAMO_ROUTER_E2E_NUM_PREFILL_WORKERS", "1") at tests/router/test_router_e2e_with_trtllm.py:428. With only 1 prefill worker, the prefix-reuse routing assertion ("all requests route to the same prefill worker") becomes trivially true since there's only one possible target. The test still validates disagg separation (prefill ≠ decode) but loses coverage of the routing decision quality. The vLLM disagg test at tests/router/test_router_e2e_with_vllm.py:702 keeps its default at 2, so this is an intentional per-backend choice. Worth confirming this weakening is acceptable.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🚩 Aggregate extra-engine-args YAML applied by default for non-disagg tests

Lines 206-209 add a new fallback: when disaggregation_mode is None and no explicit extra_engine_args is provided and DYN_TRTLLM_EXTRA_ENGINE_ARGS is not in the environment, the test now unconditionally passes --extra-engine-args trtllm_agg.yaml which sets attn_backend: VANILLA. This is a behavioral change for all aggregate TRT-LLM tests (basic, decisions, indexers_sync) — they previously ran without any extra engine args. The YAML comment explains this avoids an FMHA NVRTC failure on B200. If the VANILLA backend path differs materially from the default path, this could reduce test coverage for the default configuration.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🚩 Cache transceiver backend changed from DEFAULT to UCX in disagg configs

Both trtllm_disagg_prefill.yaml and trtllm_disagg_decode.yaml changed backend: DEFAULT to backend: UCX and added max_tokens_in_buffer: 8192. This is a concrete behavior change for disaggregated TRT-LLM tests — they'll now explicitly use UCX for KV cache transfer instead of letting TRT-LLM pick a default. Combined with the new UCX environment variable management in the test process, this should work for local TCP-based UCX. Worth confirming this is compatible with all CI environments where these nightly tests run.

---

Was this helpful? React with 👍 or 👎 to provide feedback.