test(gms): add vLLM and SGLang E2E coverage [GMS 10/18]

[hutm]

Summary

Make the vLLM and SGLang GMS behavior reproducible before optional plugins, router D2D transfer, or TensorRT-LLM-specific patches enter the stack.

Scope

Adds local and Kubernetes test recipes for restart, multi-rank failure propagation, CUDA graphs, multi-node operation, and host-tier paths.

Stack

Position 10/18 in the GMS-managed KV review train. This PR is based on review/gms-v2-latehost-08-3-3-host-tier-sglang-adapter.

Parent: #10450
Tracking: #10458

Review migration

This is the current review entry replacing historical merged PR #10484. GitHub does not allow a merged PR to be retargeted; #10484 and all of its comments and reviews remain intact as the historical record.

The train is rebased on main at aeda23b483831df2f75b697b8ccf65f4ffd9f3d6. The reordered functionality is preserved while each PR boundary is independently buildable and lintable: compatibility call sites and the engine-facing placement adapter now appear at first use; missing type imports and test markers were added; repository-pinned formatting was applied; one unused constant was removed; and every commit carries a DCO sign-off. The complete range passes git diff --check and the full pre-commit suite.

Validation

This PR provides the authoritative recipes and assertions. Performance, IB, and sub-second recovery evidence remains tracked in #10458 until current-cluster reruns are recorded.

---

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[datadog-official]

⚠️ Warnings

🚦 2 Pipeline jobs failed

Docs link check | lychee     

Pre Merge | pre-merge-status-check     

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: cd893c9 | Docs | Give us feedback!}

[devin-ai-integration]

Devin Review found 5 potential issues.

View 1 additional finding in Devin Review.

[devin-ai-integration]

🔴 SGLang CUDA-graph disabling flipped from on to off, risking out-of-memory in multi-worker GPU tests

The default for disabling CUDA graphs was changed from True to False (env_bool(…, False) at tests/router/test_router_e2e_with_sglang.py:58-60), so every SGLang test now runs with CUDA graphs enabled and may exceed the 40 % memory budget.

Impact: Multi-worker single-GPU tests can OOM or hang during CUDA graph capture in CI.

Mechanism: the SGLANG_ARGS default no longer disables CUDA graphs

Before this PR, SGLANG_ARGS hardcoded "disable_cuda_graph": True — matching the comment at tests/router/test_router_e2e_with_sglang.py:201-207 which warns that piecewise CUDA graphs consume ~7 GB during capture and "must also be disabled for multi-worker same-GPU tests to avoid OOM." The new code uses env_bool("DYNAMO_ROUTER_E2E_SGLANG_DISABLE_CUDA_GRAPH", False), which defaults to False when the env var is unset, so disable_cuda_graph becomes False and CUDA graphs are now enabled.

All tests that pass SGLANG_ARGS with single_gpu=True and mem_fraction_static=0.4 are affected (e.g. test_sglang_kv_router_basic, test_router_decisions_sglang_multiple_workers, test_sglang_indexers_sync).

Suggested change

	"disable_cuda_graph": env_bool(
	"DYNAMO_ROUTER_E2E_SGLANG_DISABLE_CUDA_GRAPH", False
	),
	"disable_cuda_graph": env_bool(
	"DYNAMO_ROUTER_E2E_SGLANG_DISABLE_CUDA_GRAPH", True
	),

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🔴 vLLM eager-mode default flipped from on to off, risking out-of-memory in multi-worker GPU tests

The default for enforcing eager mode was changed from True to False (env_bool(…, False) at tests/router/test_router_e2e_with_vllm.py:66), so every vLLM test now compiles CUDA graphs and may exceed the 40 % memory budget.

Impact: Multi-worker single-GPU tests can OOM or take much longer to start in CI.

Mechanism: VLLM_ARGS and VLLM_ARGS_NO_BLOCK_SIZE no longer enforce eager mode

Before this PR, both VLLM_ARGS and VLLM_ARGS_NO_BLOCK_SIZE hardcoded "enforce_eager": True with a comment: "Disable CUDA graphs for faster startup & lower memory." The new code at tests/router/test_router_e2e_with_vllm.py:66 and tests/router/test_router_e2e_with_vllm.py:83 uses env_bool("DYNAMO_ROUTER_E2E_VLLM_ENFORCE_EAGER", False), defaulting to False when the env var is unset. This enables CUDA graph compilation, which increases both memory consumption and startup time.

All tests that pass VLLM_ARGS or VLLM_ARGS_NO_BLOCK_SIZE with single_gpu=True and gpu_memory_utilization=0.4 are affected.

Suggested change

	"enforce_eager": env_bool("DYNAMO_ROUTER_E2E_VLLM_ENFORCE_EAGER", False),
	"enforce_eager": env_bool("DYNAMO_ROUTER_E2E_VLLM_ENFORCE_EAGER", True),

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🔴 vLLM secondary args dict also has eager-mode default flipped from on to off

The same default flip applies to VLLM_ARGS_NO_BLOCK_SIZE (env_bool(…, False) at tests/router/test_router_e2e_with_vllm.py:83), which was previously hardcoded to True.

Impact: Tests using the secondary config dict can also OOM or slow down when CUDA graphs are compiled.

Same root cause as the VLLM_ARGS change

VLLM_ARGS_NO_BLOCK_SIZE at tests/router/test_router_e2e_with_vllm.py:83 duplicates the same enforce_eager flip from True to False. This config is used by test_vllm_kv_router_basic_no_block_size, which runs with single_gpu=True and num_workers=2 at 0.4 memory utilization.

Suggested change

	"enforce_eager": env_bool("DYNAMO_ROUTER_E2E_VLLM_ENFORCE_EAGER", False),
	"enforce_eager": env_bool("DYNAMO_ROUTER_E2E_VLLM_ENFORCE_EAGER", True),

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🟡 Duplicate tensor-parallelism flag when both TP and DP env vars are set for SGLang

The tensor-parallelism flag is added twice with conflicting values (--tp-size at tests/router/test_router_e2e_with_sglang.py:244 and again at tests/router/test_router_e2e_with_sglang.py:264) when both TP and DP are configured, so one value is silently ignored.

Impact: Setting the TP environment variable while a test also uses data parallelism produces a misconfigured command line.

Conflicting --tp-size flags in the generated SGLang command

The new code at tests/router/test_router_e2e_with_sglang.py:243-244 adds --tp-size with tensor_parallel_size (from the env var DYNAMO_ROUTER_E2E_SGLANG_TENSOR_PARALLEL_SIZE). The pre-existing DP block at tests/router/test_router_e2e_with_sglang.py:258-268 also adds --tp-size with data_parallel_size. When both are active (TP env var set + DP test config), the command receives two --tp-size flags with different values. Argparse typically uses the last one, silently discarding the TP value.

Fix: skip the standalone --tp-size when data_parallel_size is not None, or remove --tp-size from the DP block in favor of the standalone one.

Suggested change

	if tensor_parallel_size is not None:
	command.extend(["--tp-size", str(tensor_parallel_size)])
	if tensor_parallel_size is not None and data_parallel_size is None:
	command.extend(["--tp-size", str(tensor_parallel_size)])

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🚩 SGLang disagg test default prefill workers reduced from 2 to 1

The default num_prefill_workers for test_router_decisions_sglang_disagg changed from 2 to 1 (tests/router/test_router_e2e_with_sglang.py:453). The test is still marked @pytest.mark.gpu_2, but now only uses 1 prefill worker and 1 decode worker. With a single prefill worker, the prefix-reuse routing assertion at tests/router/common.py:2347-2373 becomes trivially satisfied (only one worker to choose from), reducing test coverage. If this reduction is intentional (e.g. to match a different hardware config), it should be documented; otherwise it weakens the disaggregated routing validation.

---

Was this helpful? React with 👍 or 👎 to provide feedback.