[WIP] Initial work to add llm-d-vllm framework with H200

[ezrasilvera]

WIP - don't merge. Initial testing of llm-d integration

---

Note

Medium Risk
New privileged multi-node Docker/SLURM orchestration and routing stack; limited to benchmark infra but complex failure modes and WIP image pin.

Overview
Adds a new llm-d-vllm multi-node benchmark path on H200: prefill/decode disaggregation with vLLM, llm-d EPP routing, Envoy, and NIXL KV transfer, orchestrated entirely by InferenceX SLURM scripts (similar to the AMD sglang-disagg flow, not srt-slurm).

Benchmark config: Registers dsr1-fp8-h200-llm-d-vllm-simple in nvidia-master.yaml — 1 prefill + 1 decode node, EP=8 dp-attn per side, fixed 1k/1k seq lengths, and CONFIG_FILE=dsr1-fp8-h200-1p1d-simple.yaml (no wide-EP / DeepEP / NVSHMEM ibgda for this “phase 0” shape).

Image & static config: New benchmarks/llm-d/ Dockerfile (llm-d-cuda + EPP + pd-sidecar + Envoy), default epp-config.yaml (file-discovery to /tmp/endpoints.yaml), and envoy.yaml (ext_proc → EPP, ORIGINAL_DST on x-gateway-destination-endpoint).

Runtime: benchmarks/multi_node/llm-d/ (submit.sh, job.slurm, server.sh) allocates nodes, runs one privileged Docker container per rank, starts vLLM + pd-sidecar on leaders, and on the decode leader writes endpoints.yaml, starts EPP/Envoy, runs benchmark_serving concurrency sweeps, and signals job teardown via a done marker. Recipe llm-d-recipes/dsr1-fp8-h200-1p1d-simple.yaml overrides EPP scheduling and per-role vLLM args. Wrapper dsr1_fp8_h200_llm-d-vllm.sh feeds submit.sh.

Runner: launch_h200-dgxc-slurm.sh exports SLURM account/partition, dispatches FRAMEWORK=llm-d-vllm to the wrapper, tails SLURM logs, copies JSON/eval artifacts, and lists llm-d-vllm among supported multinode frameworks.

^{Reviewed by Cursor Bugbot for commit 95b8b2f. Bugbot is set up for automated code reviews on this repo. Configure here.}

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[ezrasilvera]

WIP - don't merge. Initial testing of llm-d integration

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 873fe04. Configure here.}