Update kimik2.5-fp4-b200-vllm vLLM image to v0.21.0 (#1395)

* $Update kimik2.5-fp4-b200-vllm vLLM image to v0.20.2\n\nRef #1154\n\nCo-authored-by: Klaud Cold <Klaud-Cold@users.noreply.github.com>

* fix(kimik2.5_fp4_b200.sh): raise --gpu-memory-utilization 0.90 -> 0.98

vLLM v0.20.2's CUDA-graph memory profiling subtracts an aggressive
chunk from the requested utilization, leaving negative space for the
KV cache (-39.49 GiB observed). Raising to 0.98 gives the profiler
enough headroom to land KV cache positive while still keeping ~2% as
hard buffer.

Alternative would have been setting VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0,
but raising the cap is the minimum-blast-radius fix and matches what
similar B200 recipes use.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(kimik2.5_fp4_b200.sh): disable CUDA-graph memory estimator + restore 0.90 mem-util

Raising --gpu-memory-utilization to 0.98 wasn't enough — vLLM v0.20.2's
CUDA-graph memory profiler still pre-reserves ~57 GB/GPU upfront, leaving
the effective utilization at ~0.66 and the KV cache at -25 GiB (engine
won't start).

Disable the estimator with VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0
(same pattern as benchmarks/single_node/agentic/kimik2.5_fp4_b200.sh:65)
and revert --gpu-memory-utilization back to 0.90. The 0.90 reservation
already leaves ~18 GB/GPU as the same safety net the estimator was
trying to enforce.

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Bryan Shan <58582368+Oseltamivir@users.noreply.github.com>
Co-authored-by: claude-fix-bot <claude-fix-bot@local>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>

Author: 6 people

.github/configs/nvidia-master.yaml

@@ -2632,7 +2632,7 @@ kimik2.5-int4-h200-vllm-agentic:
       - { tp: 8, offloading: cpu,  conc-list: [6, 7, 8, 9, 10, 11, 12, 13, 14] }
 
 kimik2.5-fp4-b200-vllm:
-  image: vllm/vllm-openai:v0.17.0
+  image: vllm/vllm-openai:v0.20.2
   model: nvidia/Kimi-K2.5-NVFP4
   model-prefix: kimik2.5
   runner: b200

benchmarks/single_node/kimik2.5_fp4_b200.sh

@@ -33,6 +33,13 @@ fi
 # Start GPU monitoring (power, temperature, clocks every second)
 start_gpu_monitor
 
+# vLLM v0.20.2+'s CUDA-graph memory profiler pre-reserves ~57 GB/GPU upfront
+# (~32% of total), which collides with --gpu-memory-utilization=0.90 and
+# leaves negative space for the KV cache. Disable the profiler — our 0.90
+# already leaves ~18 GB/GPU as safety net (same pattern as
+# benchmarks/single_node/agentic/kimik2.5_fp4_b200.sh).
+export VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0
+
 set -x
 vllm serve $MODEL --host 0.0.0.0 --port $PORT \
 --tensor-parallel-size=$TP \

perf-changelog.yaml

@@ -2704,3 +2704,10 @@
   description:
     - "Update vLLM image from v0.19.0-cu130 (25d old) to v0.21.0"
   pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1448
+
+- config-keys:
+    - kimik2.5-fp4-b200-vllm
+  description:
+    - "Update vLLM image from v0.20.2 to v0.21.0"
+    - "Add VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0 to disable aggressive CUDA-graph memory profiler that OOMs the KV cache"
+  pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1395