Skip to content

Commit 417d047

Browse files
mudlerlocalai-org-maint-bot
authored andcommitted
docs(vllm-cpp): correct the MLX-gated figure to 97.6%, from 99.1%
The previous commit quoted 99.1% of MLX-LM for the prefill-gated MLX build. That figure divided by a two-run MLX-LM baseline, 27.135 and 27.744 generation tok/s averaged to 27.44. Re-measured interleaved with ours over four ABBA blocks, MLX-LM's decode is 27.848 with a 0.34% spread across six runs, so the 27.135 was an outlier and averaging it in overstated us by roughly 1.5 points. Corrected: the gated configuration is 24.37 tok/s, or 97.6% of MLX-LM, and the MLX-off build is 23.9 tok/s or 95.9%. Prefill TTFT is unchanged at 524.5 ms against MLX-LM's 532.6, so we remain about 1.5% faster there. Nothing else changes. MLX still wins prefill and loses decode, the shape gate is still the right disposition, and the pin and the flag are still coupled. The gate is worth about 1.7 points over the MLX-off build rather than 2.7. Assisted-by: Claude Code:claude-opus-5 [ClaudeCode] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
1 parent d703fe4 commit 417d047

1 file changed

Lines changed: 7 additions & 2 deletions

File tree

backend/go/vllm-cpp/README.md

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -55,9 +55,14 @@ Apple M4, Qwen3-1.7B-bf16 warm at p=512 g=128:
5555

5656
| configuration | prefill TTFT | warm throughput |
5757
|---|--:|--:|
58-
| MLX **gated to prefill** (pin >= 89c46aeb) | **524.5 ms** | **24.40 tok/s — 99.1% of MLX-LM** |
58+
| MLX **gated to prefill** (pin >= 89c46aeb) | **524.5 ms** | **24.37 tok/s, 97.6% of MLX-LM** |
5959
| MLX ungated (older pins) | 537 ms | 12.7 tok/s |
60-
| MLX off | 602 ms | 23.9 tok/s |
60+
| MLX off | 602 ms | 23.9 tok/s, 95.9% |
61+
62+
Ratios are against an MLX-LM baseline measured INTERLEAVED with ours over four
63+
ABBA blocks (its spread 0.34%, ours 0.12%). An earlier revision of this file
64+
claimed 99.1%; that used a two-run MLX-LM baseline containing an outlier and
65+
overstated us by about 1.5 points.
6166

6267
**`VLLM_CPP_VERSION` and this flag are coupled.** Moving the pin back before
6368
`89c46aeb` while leaving `VLLM_CPP_MLX=on` would take the middle row — roughly

0 commit comments

Comments
 (0)