Add vLLM Llama-70B all-reduce reproducer

[micmelesse]

Motivation

This pr adds a benchmark and tests to iterate on vllm iris integration for llama 70b. It has two dockerfiles that bake a baseline and experimental environment. You can then run two scripts one to check correctness and another to check performance. There is a README attached for instructions.

The main difference is that the baseline pins commits from the main of vllm, aiter and iris. The experimental pins the branches with iris integration work which is enabled by setting VLLM_ROCM_USE_AITER_COMMS=1.

Technical Details

Test Plan

Test Result

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

[aamarnat]

bench.sh runs the measured vllm bench serve cold — --num-warmups defaults to 0, so the only thing before timing is the single "Initial test run" readiness request. The first measured requests then absorb NCCL communicator init and other first-call costs. On my MI300X runs, a cold measured arm was the main driver of an inflated baseline → ~3.9× artifact; adding warmup collapsed it to ~1.05–1.08×.

Since vllm bench serve already has built-in warmup (and excludes those requests from the reported metrics), this is a one-flag fix on the existing command:

timeout 1200 vllm bench serve \
    --host "$HOST" --port "$PORT" --model "$MODEL" \
    --dataset-name random --random-input-len "$INPUT_LEN" --random-output-len "$OUTPUT_LEN" \
    --max-concurrency "$CONCURRENCY" --num-prompts "$NUM_PROMPTS" \
    --num-warmups "$CONCURRENCY" \
    --percentile-metrics ttft,tpot,itl,e2el \
    --ignore-eos \
    --save-result --result-filename "$OUTDIR/result_${ARM}.json" --label "$ARM"

The baked per-arm images already prevent cross-arm cache contamination, so this is purely about removing cold-start from the single measured arm — TTFT is the metric most affected. One nuance: built-in warmup uses a single fixed test input, so it primes the engine/NCCL/comms path but won't touch every cudagraph batch-size bucket at conc=64; if you want those warmed too, a short pre-pass over the random dataset does it. --num-warmups is most of the value for one flag.