Skip to content

Commit 9f33d03

Browse files
apstenku123apstenku123
committed
bench(gb10): fair-comparison bs=2/seq=512 — path_b scales, path_c is bs=1-only on CUDA
Memory-safe ramp (peak 60G < 70G cap). bf16/16-bit: - path_b SCALES with batch: muon 92->159, adamw 156->259 tok/s (bs1->bs2), losses 11.3->5.79 PASS. - path_c + path_c_chunked BLOCKED at bs=2: 'direct_fusion_chain_training_runtime_missing' — the Path-C direct-chain training runtime is bs=1-only on CUDA (path_b's CUDA-eager fix is batch-robust). So the steady-state path_c-vs-path_b-at-scale comparison is NOT achievable on gb10 (path_c bs=1-only there); bs=4xseq=4096 OOMs MLX-eager (C++/Megatron does it in ~26G — the MLX memory-inefficiency finding stands).
1 parent 5438cce commit 9f33d03

2 files changed

Lines changed: 85 additions & 0 deletions

File tree

‎reports/cppmega_1b_speed_matrix_gb10_fastfused_20260601_b2s512.html‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
<!DOCTYPE html>
2+
<html lang="en"><head><meta charset="utf-8">
3+
<title>cppmega 1B Speed Matrix — Metal (M4 Max) FAST FUSED Path-C 20260601</title>
4+
<style>
5+
body { font-family: -apple-system, system-ui, sans-serif; margin: 2rem; color: #1a1a1a; }
6+
h1 { font-size: 1.3rem; } h2 { font-size: 1.05rem; margin-top: 1.6rem; }
7+
ul.meta { font-size: 0.82rem; color: #444; line-height: 1.5; }
8+
table { border-collapse: collapse; width: 100%; font-size: 0.82rem; margin-bottom: 1rem; }
9+
th, td { border: 1px solid #ccc; padding: 4px 8px; text-align: left; }
10+
th { background: #2d2d2d; color: #fff; }
11+
td.num { text-align: right; font-variant-numeric: tabular-nums; }
12+
td.fast { color: #0a7d28; font-weight: 700; }
13+
td.slow { color: #b40000; font-weight: 700; }
14+
tr.ok td:nth-child(5) { color: #0a7d28; font-weight: 600; }
15+
tr.bad td:nth-child(5) { color: #b40000; font-weight: 700; }
16+
tr:nth-child(even) { background: #f6f6f6; }
17+
</style></head><body>
18+
<h1>cppmega 1B Speed Matrix — gb10 (NVIDIA GB10, CUDA) — FAST FUSED Path-C</h1>
19+
<ul class="meta">
20+
<li>Date: 20260601 &middot; Host: gb10 (NVIDIA GB10, CUDA sm_121) &middot; Profile: local_gb10_quarter</li>
21+
<li>seq-len 512, batch 2, --steps 10 warm, --grad-checkpoint, dynamic_int8_v1 (tokens/step = 1024)</li>
22+
<li>path_b=reference &middot; path_c=flag-OFF serial mamba3 &middot; path_c_chunked=flag-ON CPPMEGA_PATH_C_MAMBA3_CHUNKED_SCAN=1 (FAST chunked fused mamba3)</li>
23+
<li>loss check = all losses finite AND final&lt;initial (fail-loud) &middot; per-cell bound 1800s</li>
24+
<li>cppmega SHA: <code>a319599</code> &middot; TileLang SHA: <code>unknown</code></li>
25+
</ul>
26+
<table>
27+
<thead><tr><th>dtype</th><th>optimizer</th><th>bits</th><th>path</th><th>status</th><th>tok/s</th><th>step/s</th><th>compile s</th><th>peak GB</th><th>loss check</th><th>first/last loss</th><th>flag</th><th>reason</th></tr></thead>
28+
<tbody>
29+
<tr class="ok"><td>bf16</td><td>muon</td><td class="num">16</td><td>path_b</td><td>ok</td><td class="num">159</td><td class="num">0.165</td><td class="num">13.5</td><td class="num">27</td><td>PASS</td><td class="num">11.3 &rarr; 5.79</td><td></td><td>ok</td></tr>
30+
<tr class="bad"><td>bf16</td><td>muon</td><td class="num">16</td><td>path_c</td><td>blocked</td><td class="num"></td><td class="num"></td><td class="num"></td><td class="num">17.2</td><td>FAIL</td><td class="num"></td><td>0</td><td>direct-chain training runtime not implemented (contract=direct_fusion_chain_training_runtime_missing; runtime_binding=direct_fusion_chain_logical_buffers_missing; model_binding=runtime_activation_owner_missing); split route available=m04_path_c_split_training_route_available</td></tr>
31+
<tr class="bad"><td>bf16</td><td>muon</td><td class="num">16</td><td>path_c_chunked</td><td>blocked</td><td class="num"></td><td class="num"></td><td class="num"></td><td class="num">17.2</td><td>FAIL</td><td class="num"></td><td>1</td><td>direct-chain training runtime not implemented (contract=direct_fusion_chain_training_runtime_missing; runtime_binding=direct_fusion_chain_logical_buffers_missing; model_binding=runtime_activation_owner_missing); split route available=m04_path_c_split_training_route_available</td></tr>
32+
<tr class="ok"><td>bf16</td><td>adamw</td><td class="num">16</td><td>path_b</td><td>ok</td><td class="num">259</td><td class="num">0.27</td><td class="num">9.31</td><td class="num">32</td><td>PASS</td><td class="num">11.3 &rarr; 5.79</td><td></td><td>ok</td></tr>
33+
<tr class="bad"><td>bf16</td><td>adamw</td><td class="num">16</td><td>path_c</td><td>blocked</td><td class="num"></td><td class="num"></td><td class="num"></td><td class="num">22.3</td><td>FAIL</td><td class="num"></td><td>0</td><td>direct-chain training runtime not implemented (contract=direct_fusion_chain_training_runtime_missing; runtime_binding=direct_fusion_chain_logical_buffers_missing; model_binding=runtime_activation_owner_missing); split route available=m04_path_c_split_training_route_available</td></tr>
34+
<tr class="bad"><td>bf16</td><td>adamw</td><td class="num">16</td><td>path_c_chunked</td><td>blocked</td><td class="num"></td><td class="num"></td><td class="num"></td><td class="num">22.3</td><td>FAIL</td><td class="num"></td><td>1</td><td>direct-chain training runtime not implemented (contract=direct_fusion_chain_training_runtime_missing; runtime_binding=direct_fusion_chain_logical_buffers_missing; model_binding=runtime_activation_owner_missing); split route available=m04_path_c_split_training_route_available</td></tr>
35+
</tbody></table>
36+
<h2>Path-C flag-OFF &rarr; flag-ON (chunked fused mamba3) speedup</h2>
37+
<table>
38+
<thead><tr><th>dtype</th><th>optimizer</th><th>bits</th><th>serial step/s</th><th>chunked step/s</th><th>step/s speedup</th><th>serial compile s</th><th>chunked compile s</th><th>compile speedup</th></tr></thead>
39+
<tbody>
40+
<tr><td>bf16</td><td>muon</td><td class='num'>16</td><td class='num'></td><td class='num'></td><td class='num '></td><td class='num'></td><td class='num'></td><td class='num'></td></tr>
41+
<tr><td>bf16</td><td>adamw</td><td class='num'>16</td><td class='num'></td><td class='num'></td><td class='num '></td><td class='num'></td><td class='num'></td><td class='num'></td></tr>
42+
</tbody></table>
43+
</body></html>

‎reports/cppmega_1b_speed_matrix_gb10_fastfused_20260601_b2s512.md‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# cppmega 1B Speed Matrix — gb10 (NVIDIA GB10, CUDA) — FAST FUSED Path-C
2+
3+
- Date: 20260601
4+
- Host: gb10 (NVIDIA GB10, CUDA sm_121)
5+
- Model profile: local_gb10_quarter
6+
- Data: data/parquet_samples/gb10/clang_semantic_4k_v10/val_00000.parquet (parquet, token_ids)
7+
- Settings: seq-len 512, batch 2, --steps 10 warm, --grad-checkpoint, --optimizer-quant-scheme dynamic_int8_v1 (tokens/step = 1024)
8+
- Paths: `path_b` (reference); `path_c` = Path-C flag-OFF (serial mamba3, prior baseline); `path_c_chunked` = Path-C **flag-ON** (CPPMEGA_PATH_C_MAMBA3_CHUNKED_SCAN=1, FAST chunked fused mamba3).
9+
- Path-C route: SPLIT/WARM (Path C fwd + Path B mamba3 bwd, mamba3 bwd=path_b).
10+
- loss check = all losses finite AND final<initial (fail-loud per RULE #1).
11+
- nvfp4: accepted route but full training step fails-LOUD (no nvfp4 training kernels yet) — honest blocked cell.
12+
- CUDA note: the CPPMEGA_PATH_C_MAMBA3_CHUNKED_SCAN flag is target-agnostic (same on Metal+CUDA); per-cell results report whatever CUDA measures.
13+
- Per-cell bound: 1800s OS timeout (fail-loud)
14+
- cppmega SHA: `a319599` · TileLang SHA: `unknown`
15+
16+
| dtype | optimizer | bits | path | status | tok/s | step/s | compile s | peak GB | loss check | first/last loss | flag |
17+
| ----- | --------- | ---: | ---- | ------ | ----: | -----: | --------: | ------: | ---------- | --------------- | ---- |
18+
| bf16 | muon | 16 | path_b | ok | 159 | 0.165 | 13.5 | 27 | PASS | 11.3 → 5.79 | |
19+
| bf16 | muon | 16 | path_c | blocked | | | | 17.2 | FAIL | | 0 |
20+
| bf16 | muon | 16 | path_c_chunked | blocked | | | | 17.2 | FAIL | | 1 |
21+
| bf16 | adamw | 16 | path_b | ok | 259 | 0.27 | 9.31 | 32 | PASS | 11.3 → 5.79 | |
22+
| bf16 | adamw | 16 | path_c | blocked | | | | 22.3 | FAIL | | 0 |
23+
| bf16 | adamw | 16 | path_c_chunked | blocked | | | | 22.3 | FAIL | | 1 |
24+
25+
## Path-C flag-OFF → flag-ON (chunked fused mamba3) speedup
26+
27+
- step/s speedup = chunked step/s ÷ serial step/s (steady-state, >1 = chunked faster).
28+
- compile speedup = serial first-step s ÷ chunked first-step s (>1 = chunked compiles/first-steps faster).
29+
30+
| dtype | optimizer | bits | serial step/s | chunked step/s | step/s speedup | serial compile s | chunked compile s | compile speedup |
31+
| ----- | --------- | ---: | ------------: | -------------: | -------------: | ---------------: | ----------------: | --------------: |
32+
| bf16 | muon | 16 | | | | | | |
33+
| bf16 | adamw | 16 | | | | | | |
34+
35+
## Per-Cell Commands
36+
37+
- `bf16_muon16_path_b` (flag=): `/usr/bin/timeout 1800 /home/dave/cppmega-venv/bin/python scripts/m04_train_step.py --model-profile local_gb10_quarter --data-path data/parquet_samples/gb10/clang_semantic_4k_v10/val_00000.parquet --data-format parquet --token-key token_ids --steps 10 --batch-size 2 --seq-len 512 --dtype bfloat16 --optimizer muon --optimizer-quant-scheme dynamic_int8_v1 --lr 1e-4 --grad-checkpoint --output /tmp/cppmega_1b_speed_matrix_gb10_fastfused_20260601_b2s512_cells/bf16_muon16_path_b.json --json`
38+
- `bf16_muon16_path_c` (flag=0): `/usr/bin/timeout 1800 /home/dave/cppmega-venv/bin/python scripts/m04_train_step.py --model-profile local_gb10_quarter --data-path data/parquet_samples/gb10/clang_semantic_4k_v10/val_00000.parquet --data-format parquet --token-key token_ids --steps 10 --batch-size 2 --seq-len 512 --dtype bfloat16 --optimizer muon --optimizer-quant-scheme dynamic_int8_v1 --lr 1e-4 --grad-checkpoint --output /tmp/cppmega_1b_speed_matrix_gb10_fastfused_20260601_b2s512_cells/bf16_muon16_path_c.json --json`
39+
- `bf16_muon16_path_c_chunked` (flag=1): `/usr/bin/timeout 1800 /home/dave/cppmega-venv/bin/python scripts/m04_train_step.py --model-profile local_gb10_quarter --data-path data/parquet_samples/gb10/clang_semantic_4k_v10/val_00000.parquet --data-format parquet --token-key token_ids --steps 10 --batch-size 2 --seq-len 512 --dtype bfloat16 --optimizer muon --optimizer-quant-scheme dynamic_int8_v1 --lr 1e-4 --grad-checkpoint --output /tmp/cppmega_1b_speed_matrix_gb10_fastfused_20260601_b2s512_cells/bf16_muon16_path_c_chunked.json --json`
40+
- `bf16_adamw16_path_b` (flag=): `/usr/bin/timeout 1800 /home/dave/cppmega-venv/bin/python scripts/m04_train_step.py --model-profile local_gb10_quarter --data-path data/parquet_samples/gb10/clang_semantic_4k_v10/val_00000.parquet --data-format parquet --token-key token_ids --steps 10 --batch-size 2 --seq-len 512 --dtype bfloat16 --optimizer adamw --optimizer-quant-scheme dynamic_int8_v1 --lr 1e-4 --grad-checkpoint --output /tmp/cppmega_1b_speed_matrix_gb10_fastfused_20260601_b2s512_cells/bf16_adamw16_path_b.json --json`
41+
- `bf16_adamw16_path_c` (flag=0): `/usr/bin/timeout 1800 /home/dave/cppmega-venv/bin/python scripts/m04_train_step.py --model-profile local_gb10_quarter --data-path data/parquet_samples/gb10/clang_semantic_4k_v10/val_00000.parquet --data-format parquet --token-key token_ids --steps 10 --batch-size 2 --seq-len 512 --dtype bfloat16 --optimizer adamw --optimizer-quant-scheme dynamic_int8_v1 --lr 1e-4 --grad-checkpoint --output /tmp/cppmega_1b_speed_matrix_gb10_fastfused_20260601_b2s512_cells/bf16_adamw16_path_c.json --json`
42+
- `bf16_adamw16_path_c_chunked` (flag=1): `/usr/bin/timeout 1800 /home/dave/cppmega-venv/bin/python scripts/m04_train_step.py --model-profile local_gb10_quarter --data-path data/parquet_samples/gb10/clang_semantic_4k_v10/val_00000.parquet --data-format parquet --token-key token_ids --steps 10 --batch-size 2 --seq-len 512 --dtype bfloat16 --optimizer adamw --optimizer-quant-scheme dynamic_int8_v1 --lr 1e-4 --grad-checkpoint --output /tmp/cppmega_1b_speed_matrix_gb10_fastfused_20260601_b2s512_cells/bf16_adamw16_path_c_chunked.json --json`

0 commit comments

Comments
 (0)