Skip to content

Commit 466332c

Browse files
apstenku123apstenku123
committed
bench(matrix): assemble full Path-A..E speed campaign + tok/s verdict (review index)
Closes the speed-matrix campaign deliverable. All matrices fail-loud (RULE #1). - docs/TOKPS-DISCREPANCY.md: '75 vs 3000 tok/s' verdict. 75.7 = gb10 path_c muon bf16 16-bit cell; the gap = ~32x token-count (batch1xseq512 vs batch4xseq4096) x muon-vs-adamw tax x eager-MLX-vs-fused-FP8-CUDA substrate; model identical; not a bug. - reports/cppmega_1b_speed_matrix_gb10_fastfused_20260601.{md,html} (gb10, 54 cells, bf16/fp8/nvfp4 x muon/adamw/lion x 8/16bit x path_b/path_c/path_c_chunked): path_c_chunked (flag-ON fast fused) compiles 1.15-2.0x faster than serial path_c, steady-state ~parity at batch=1; path_b blocked on MLX-CUDA; nvfp4 fail-loud. - reports/cppmega_v4_speed_matrix_{metal,gb10}_20260531.{md,html} (paths a-e): Metal a/b/c/e ok + d fail-loud; gb10 only a + GDN-e run (path_c is Metal-target, DLPack fail-loud on CUDA) -> v4 op-level Path-C CUDA wiring is the open item. - reports/local_matrix_latest.{md,csv,json,html} (local Metal 1B, 28 cells). - reports/MATRIX-REVIEW-INDEX.{md,html}: clickable index + headline tables + honest gaps. - scripts/run_gb10_speed_matrix_fastfused_20260601.py: the gb10 fast-fused driver.
1 parent ffb3069 commit 466332c

14 files changed

Lines changed: 3941 additions & 114 deletions

‎docs/TOKPS-DISCREPANCY.md‎

Lines changed: 146 additions & 0 deletions
Large diffs are not rendered by default.

‎reports/MATRIX-REVIEW-INDEX.html‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
<!doctype html><html lang="en"><head><meta charset="utf-8">
2+
<title>Speed-Matrix Campaign — Review Index (2026-06-01)</title>
3+
<style>
4+
body{font-family:-apple-system,Segoe UI,Roboto,sans-serif;margin:28px;color:#1a1a1a;max-width:1080px}
5+
h1{font-size:22px} h2{font-size:17px;margin-top:28px;border-bottom:2px solid #eee;padding-bottom:4px}
6+
.meta{color:#555;font-size:13px;line-height:1.55;margin-bottom:8px}
7+
table{border-collapse:collapse;width:100%;font-size:13px;margin:10px 0}
8+
th,td{border:1px solid #ddd;padding:6px 9px;text-align:left}
9+
th{background:#f3f4f6} td.num{text-align:right;font-variant-numeric:tabular-nums}
10+
a{color:#0a58ca;text-decoration:none} a:hover{text-decoration:underline}
11+
.win{background:#eafaea;font-weight:600} .card{border:1px solid #e3e3e3;border-radius:8px;padding:12px 16px;margin:8px 0;background:#fafafa}
12+
code{background:#f0f0f3;padding:1px 5px;border-radius:4px;font-size:12px}
13+
.blocked{color:#a33} .ok{color:#161}
14+
</style></head><body>
15+
<h1>Speed-Matrix Campaign — Review Index</h1>
16+
<div class="meta">2026-06-01 &middot; local Metal (Apple M4 Max) + gb10 (NVIDIA GB10, CUDA sm_121, MLX-CUDA backend). Every matrix is fail-loud (RULE&nbsp;#1): a cell that cannot run the real selected path reports the real error, never a silent fallback.</div>
17+
18+
<h2>Artifacts (click to open)</h2>
19+
<table>
20+
<tr><th>matrix</th><th>platform</th><th>cells</th><th>HTML</th><th>MD</th></tr>
21+
<tr><td>1B training (full m04 step)</td><td>local Metal</td><td class="num">28</td>
22+
<td><a href="local_matrix_latest.html">open</a></td><td><a href="local_matrix_latest.md">md</a></td></tr>
23+
<tr><td>1B training <b>fast-fused</b> (path_b / path_c / path_c_chunked)</td><td>gb10 CUDA</td><td class="num">54</td>
24+
<td><a href="cppmega_1b_speed_matrix_gb10_fastfused_20260601.html">open</a></td><td><a href="cppmega_1b_speed_matrix_gb10_fastfused_20260601.md">md</a></td></tr>
25+
<tr><td>v4 op-level GDN/KDA (paths a&ndash;e)</td><td>local Metal</td><td class="num">40</td>
26+
<td><a href="cppmega_v4_speed_matrix_metal_20260531.html">open</a></td><td><a href="cppmega_v4_speed_matrix_metal_20260531.md">md</a></td></tr>
27+
<tr><td>v4 op-level GDN/KDA (paths a&ndash;e)</td><td>gb10 CUDA</td><td class="num">40</td>
28+
<td><a href="cppmega_v4_speed_matrix_gb10_20260531.html">open</a></td><td><a href="cppmega_v4_speed_matrix_gb10_20260531.md">md</a></td></tr>
29+
</table>
30+
<div class="meta">Companion analyses: <a href="../docs/TOKPS-DISCREPANCY.md">TOKPS-DISCREPANCY.md</a> (the &ldquo;75 vs 3000 tok/s&rdquo; verdict) &middot; <a href="../docs/NVFP4-TRAINING-KERNELS.md">NVFP4-TRAINING-KERNELS.md</a> (nvfp4 fwd wired / bwd fail-loud).</div>
31+
32+
<h2>Headline 1 — Real Path-C win on CUDA: fast-fused chunked mamba3</h2>
33+
<div class="meta">gb10 1B, flag-OFF (serial <code>path_c</code>) &rarr; flag-ON (<code>path_c_chunked</code>), batch=1 seq=512 steps=10.</div>
34+
<table>
35+
<tr><th>dtype</th><th>opt</th><th>bits</th><th>serial tok/s</th><th>chunked tok/s</th><th>step/s speedup</th><th>compile speedup</th></tr>
36+
<tr><td>bf16</td><td>muon</td><td class="num">16</td><td class="num">75.7</td><td class="num win">78.7</td><td class="num">1.02&times;</td><td class="num win">1.38&times;</td></tr>
37+
<tr><td>bf16</td><td>adamw</td><td class="num">16</td><td class="num">117</td><td class="num win">122</td><td class="num">1.02&times;</td><td class="num win">1.59&times;</td></tr>
38+
<tr><td>bf16</td><td>lion</td><td class="num">16</td><td class="num">121</td><td class="num">121</td><td class="num">0.98&times;</td><td class="num win">1.60&times;</td></tr>
39+
<tr><td>fp8</td><td>muon</td><td class="num">16</td><td class="num">71.7</td><td class="num win">74.7</td><td class="num">1.01&times;</td><td class="num win">1.70&times;</td></tr>
40+
<tr><td>fp8</td><td>adamw</td><td class="num">16</td><td class="num">108</td><td class="num win">114</td><td class="num">1.02&times;</td><td class="num win">2.00&times;</td></tr>
41+
<tr><td>fp8</td><td>lion</td><td class="num">16</td><td class="num">114</td><td class="num win">119</td><td class="num">1.01&times;</td><td class="num win">2.00&times;</td></tr>
42+
</table>
43+
<div class="meta">At batch=1 single-step the steady-state is ~parity (per-call overhead dominates the tiny workload), but the fast-fused chunked path <b>compiles 1.15&ndash;2.0&times; faster</b> (6 small kernels vs one monolithic mega-kernel) and is the only Path-C variant that compiles at all on the long chains. Steady-state advantage is expected to grow with batch &mdash; fair batch=4 run is the open item.</div>
44+
45+
<h2>Headline 2 — &ldquo;75.7 tok/s&rdquo; identified</h2>
46+
<div class="card"><code>75.7 tok/s = gb10 1B &middot; path_c &middot; muon &middot; bf16 &middot; 16-bit &middot; batch=1 &middot; seq=512</code>. It is the <b>muon</b> cell; on the same path adamw=117, lion=121. The Megatron ~3700 tok/s is batch=4&times;seq=4096 (=16,384 tok/step, 32&times; more) on tuned CUDA. Not a bug &mdash; full decomposition in <a href="../docs/TOKPS-DISCREPANCY.md">TOKPS-DISCREPANCY.md</a>.</div>
47+
48+
<h2>Headline 3 — Platform reachability (fail-loud, honest)</h2>
49+
<ul class="meta">
50+
<li><b>gb10 1B:</b> <span class="blocked">path_b blocked</span> (MLX-CUDA can't run the path_b reference); <span class="ok">path_c / path_c_chunked run</span> and pass the loss check (11.3&rarr;6.27, finite+decreasing); <span class="blocked">nvfp4 all fail-loud</span> (backward GEMM miscompiled on sm_121).</li>
51+
<li><b>gb10 v4 (a&ndash;e):</b> only <span class="ok">path_a</span> + <span class="ok">GDN path_e</span> run on CUDA; <span class="blocked">path_b</span> (No Metal back-end), <span class="blocked">path_c</span> (DLPackDeviceError &mdash; TileLang compiled <code>target=metal</code>), <span class="blocked">path_d</span> (Triton disabled), <span class="blocked">KDA path_e</span> (Metal-only) all fail-loud &rarr; v4 op-level Path-C/B/E still Metal-target-only; CUDA-target wiring is the open item.</li>
52+
<li><b>local Metal v4 (a&ndash;e):</b> <span class="ok">path_a/b/c/e run</span>; <span class="blocked">path_d fails loud</span> (Triton disabled). Fastest = path_e (~1&nbsp;ms, 225&ndash;287 Melem/s).</li>
53+
<li><b>local Metal 1B:</b> at batch=1 path_b leads path_c_warm (adamw 457 vs 259) &mdash; same tiny-workload effect; not representative of the fused advantage at scale.</li>
54+
</ul>
55+
56+
<h2>Honest gaps (not papered over)</h2>
57+
<ol class="meta">
58+
<li><b>Fair batch=4 comparison not yet run</b> &mdash; every 1B number is batch=1&times;seq=512=512 tok/step; steady-state verdicts need batch=4&times;seq=4096 to be like-for-like.</li>
59+
<li><b>nvfp4 training is blocked, not done</b> &mdash; fwd NVFP4 GEMM works on gb10 (rel_err 0.147); backward fails loud (enablement = rebuild TE with <code>compute_120f</code>). No fake numbers.</li>
60+
<li><b>v4 op-level Path-C on CUDA</b> needs a <code>target=cuda</code> TileLang compile path; currently Metal-only.</li>
61+
</ol>
62+
</body></html>

‎reports/MATRIX-REVIEW-INDEX.md‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Speed-Matrix Campaign — Review Index (2026-06-01)
2+
3+
One place to review the full Path-A..E / Path-B vs Path-C speed campaign across **local Metal (Apple M4 Max)** and **gb10 (NVIDIA GB10, CUDA sm_121, MLX-CUDA backend)**. Every matrix is fail-loud (RULE #1): a cell that cannot run the *real* selected path reports the real error, never a silent fallback.
4+
5+
## Artifacts
6+
7+
| matrix | platform | cells | MD | HTML |
8+
| --- | --- | --- | --- | --- |
9+
| 1B training (full m04 step) | local Metal | 28 | `local_matrix_latest.md` | `local_matrix_latest.html` |
10+
| 1B training **fast-fused** (path_b / path_c / path_c_chunked) | gb10 CUDA | 54 | `cppmega_1b_speed_matrix_gb10_fastfused_20260601.md` | `cppmega_1b_speed_matrix_gb10_fastfused_20260601.html` |
11+
| v4 op-level GDN/KDA (paths a–e) | local Metal | 40 | `cppmega_v4_speed_matrix_metal_20260531.md` | `cppmega_v4_speed_matrix_metal_20260531.html` |
12+
| v4 op-level GDN/KDA (paths a–e) | gb10 CUDA | 40 | `cppmega_v4_speed_matrix_gb10_20260531.md` | `cppmega_v4_speed_matrix_gb10_20260531.html` |
13+
14+
Companion analyses: `../docs/TOKPS-DISCREPANCY.md` (the "75 vs 3000 tok/s" verdict) and `../docs/NVFP4-TRAINING-KERNELS.md` (nvfp4 fwd wired / bwd fail-loud, gb10 evidence).
15+
16+
---
17+
18+
## Headline 1 — The real Path-C win is on CUDA: fast-fused chunked mamba3
19+
20+
gb10 1B, flag-OFF (serial `path_c`) → flag-ON (`path_c_chunked`, `CPPMEGA_PATH_C_MAMBA3_CHUNKED_SCAN=1`), batch=1 seq=512 steps=10:
21+
22+
| dtype | opt | bits | serial tok/s | chunked tok/s | step/s speedup | compile speedup |
23+
| --- | --- | ---: | ---: | ---: | ---: | ---: |
24+
| bf16 | muon | 16 | 75.7 | **78.7** | 1.02× | **1.38×** |
25+
| bf16 | adamw | 16 | 117 | **122** | 1.02× | **1.59×** |
26+
| bf16 | lion | 16 | 121 | 121 | 0.98× | **1.60×** |
27+
| fp8 | muon | 16 | 71.7 | **74.7** | 1.01× | **1.70×** |
28+
| fp8 | adamw | 16 | 108 | **114** | 1.02× | **2.00×** |
29+
| fp8 | lion | 16 | 114 | **119** | 1.01× | **2.00×** |
30+
31+
**Verdict:** at batch=1 single-step the steady-state is ~parity (≈1.0×, the per-call overhead dominates the tiny workload), but the fast-fused chunked path **compiles 1.15–2.0× faster** (6 small kernels vs one monolithic mega-kernel) and is the only Path-C variant that compiles at all for the long chains (avoids the MTLCompilerService/oversized-kernel crash). The steady-state advantage is expected to grow with batch — see `TOKPS-DISCREPANCY.md` §7 for the fair batch=4 comparison still to run.
32+
33+
## Headline 2 — "75.7 tok/s" identified
34+
35+
`75.7 tok/s = gb10 1B, path_c, muon, bf16, 16-bit, batch=1, seq=512`. It is the **muon** cell; on the same path adamw=117 and lion=121. The Megatron ~3700 tok/s is batch=4 × seq=4096 (=16,384 tok/step, 32× more) on tuned CUDA. Not a bug — full decomposition in `docs/TOKPS-DISCREPANCY.md`.
36+
37+
## Headline 3 — Platform reachability (fail-loud, honest)
38+
39+
- **gb10 1B:** `path_b` is **blocked** (MLX-CUDA cannot run the path_b reference); `path_c` / `path_c_chunked` run and pass the loss check (11.3 → 6.27, finite + decreasing). `nvfp4` cells all **fail-loud** (no nvfp4 training kernels yet; backward GEMM miscompiled on sm_121 — see `NVFP4-TRAINING-KERNELS.md`).
40+
- **gb10 v4 (a–e):** only `path_a` (pure-MLX reference) and `GDN path_e` run on CUDA. `path_b` ("No Metal back-end"), `path_c` (`DLPackDeviceError` — TileLang compiled for `target=metal`, can't take CUDA arrays), `path_d` (Triton disabled), `KDA path_e` (Metal-only kernel) all **fail-loud**. → the v4 op-level Path-C/B/E dispatcher is still **Metal-target-only**; CUDA-target wiring for the v4 GDN/KDA ops is the open item (distinct from the m04-training Path-C, which already works on CUDA per Headline 1).
41+
- **local Metal v4 (a–e):** `path_a/b/c/e` all run; `path_d` fails loud (Triton frontend disabled). Fastest = `path_e` (vendored mlx-lm gated_delta Metal kernel, ~1 ms, 225–287 Melem/s).
42+
- **local Metal 1B:** at batch=1 `path_b` leads `path_c_warm` (e.g. adamw 457 vs 259) — same per-call-overhead-dominates-tiny-workload effect; not representative of the fused advantage at scale.
43+
44+
## Honest gaps (not papered over)
45+
46+
1. **Fair batch=4 comparison not yet run** — every 1B number here is batch=1×seq=512=512 tok/step. The steady-state Path-C-vs-Path-B and MLX-vs-C++ verdicts need batch=4×seq=4096 to be like-for-like (`TOKPS-DISCREPANCY.md` §7).
47+
2. **nvfp4 training is blocked**, not done — fwd NVFP4 GEMM works on gb10 (rel_err 0.147 vs bf16); backward fails loud (TE built `sm_120` plain, FP4-cvt/RHT PTX arch-specific; enablement = rebuild TE with `compute_120f`). Honest blocked cells, no fake numbers.
48+
3. **v4 op-level Path-C on CUDA** needs a `target=cuda` TileLang compile path; currently Metal-only (fails loud on CUDA arrays).

0 commit comments

Comments
 (0)