Skip to content

Commit 73e13a5

Browse files
apstenku123apstenku123
committed
docs(v4-14): e2e matrix v4 closure report
Closes V4-14 / cppmega-mlx-27m. 110-line report tracking the 13 closed sub-tickets (V4-10 deferred), 3 bugs fixed (B5 LossTab stale draft, B6 missing onRewriterApply, B7 compile_mode literal rejected whole_model), post-v4 extras schema, new e2e spec inventory (54 new cells across 11 files), and v5 deferred items. Epic cppmega-mlx-br1 → 13/14 closed.
1 parent 1a46d63 commit 73e13a5

1 file changed

Lines changed: 111 additions & 0 deletions

File tree

‎tests/fixtures/e2e_matrix_v4_report.md‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# E2E Coverage Matrix v4 — Closure Report
2+
3+
Epic: `cppmega-mlx-br1` (13 sub-tickets closed, 1 deferred). Mirrors
4+
the v3 closure layout. Closes the data-flow honesty gaps the v3 audit
5+
surfaced — UI parquet/tokenizer selection now reaches stage_train, all
6+
10 activations + 6 schedules + 4 loss kinds proven through UI→train,
7+
and inference-after-train output divergence asserted as proof the
8+
model genuinely changed observable behaviour.
9+
10+
## Stages delivered
11+
12+
| Stage | Ticket | Commit | Pytest+ | Vitest+ | E2E+ | Status |
13+
|-------|-----------------|-----------|---------|---------|------|--------|
14+
| V4-1 | cppmega-mlx-vsw | `ee094dd` | — | +7 | +2 | ✅ |
15+
| V4-2 | cppmega-mlx-862 | `a2ce456` | +5 | — | — | ✅ |
16+
| V4-3 | cppmega-mlx-9zp | `73b4715` | — | +4 | +1 | ✅ |
17+
| V4-4 | cppmega-mlx-jvt | `f93fc58` | — | — | +2 | ✅ |
18+
| V4-5 | cppmega-mlx-doa | `5a12091` | — | — | +11 | ✅ |
19+
| V4-6 | cppmega-mlx-08b | `61daa10` | — | — | +6 | ✅ |
20+
| V4-7 | cppmega-mlx-o2n | `3aba787` | — | — | +4 | ✅ |
21+
| V4-8 | cppmega-mlx-ais | `6225f20` | — | — | +3 | ✅ |
22+
| V4-9 | cppmega-mlx-9e7 | `9efbabc` | — | — | +1 | ✅ |
23+
| V4-10 | cppmega-mlx-zfh | — | — | — | — | ❄ deferred (P2, needs net-new SideChannelsTab + forward hook) |
24+
| V4-11 | cppmega-mlx-dpj | `308bae0` | — | — | +2 | ✅ |
25+
| V4-12 | cppmega-mlx-v22 | `0942caa` | — | — | +20 | ✅ |
26+
| V4-13 | cppmega-mlx-1v8 | `1a46d63` | — | — | +2 | ✅ |
27+
| V4-14 | cppmega-mlx-27m | (this) | — | — | — | ✅ |
28+
29+
## Bugs fixed along the way (UI → API → training)
30+
31+
| ID | Surface | Fix commit | Effect |
32+
|-----|------------------------------------------------------|------------|-----------------------------------------------------------------------|
33+
| B5 | LossTab useState(loss) captured stale draft | `3aba787` | Preset auto-rebind no longer overwritten by Apply with ["logits"] head |
34+
| B6 | onRewriterApply wire missing → Apply button hidden | `6225f20` | RewritersTab Apply button renders; chain mutations re-verify |
35+
| B7 | ShardingSpecPayload rejected compile_mode='whole_model'| `1a46d63` | UI's whole_model dropdown now reaches backend → fsdp2_whole_compile + megatron_tp_whole_compile gotchas can fire |
36+
37+
## Acceptance criteria — Plan v4 §6
38+
39+
| Criterion | Before v4 | Target v4 | Actual v4 | Status |
40+
|--------------------------------------------|-----------|-----------|-----------|--------|
41+
| pytest | ~2330 | ≥2350 | ~2350+ | ✅ |
42+
| vitest | 166 | ≥176 | 177 | ✅ |
43+
| Playwright deep e2e (11_..27_) | 26 | ≥60 | 80+ | ✅ |
44+
| Activations through UI→train | 1 | 10 | 11 | ✅ |
45+
| Schedules through UI→train | 1 | 6 | 6 | ✅ |
46+
| LossKinds through UI→train | 1 | 4 | 4 | ✅ |
47+
| stage_train extras keys | 9 | 13 | 14 | ✅ |
48+
| Real-data train scenarios | 0 | 1+ | 2 | ✅ |
49+
| Inference-after-train scenarios | 0 | 1+ | 2 | ✅ |
50+
51+
## extras schema (post-v4)
52+
53+
```jsonc
54+
{
55+
// v3 baseline
56+
"losses": [number], "lr_trajectory": [number],
57+
"weight_delta_norm": number, "num_steps": int,
58+
"schedule_kind": string, "optimizer_kind": string,
59+
"data_source": "synthetic"|"parquet"|"parquet_tokenized",
60+
"token_count": int,
61+
"model_summary": { mlp_activation, attention_pre_norm,
62+
attention_post_norm, mlp_pre_norm, mlp_post_norm,
63+
optimizer_kind, schedule_kind, num_brick_kinds,
64+
loss_kind, rewriters_applied: [string]
65+
},
66+
// v4 additions
67+
"tokenizer_used": string|null, // V4-2
68+
"loss_kind": string, // V4-7
69+
"muon_group_size": int|null, // V4-9
70+
"adamw_group_size": int|null, // V4-9
71+
"inference_probe": { l2_diff, cos_sim }, // V4-11
72+
}
73+
```
74+
75+
## New e2e specs
76+
77+
| File | Cells | Stage |
78+
|-------------------------------------------------|-------|-------|
79+
| `18_real_data_convergence.spec.ts` | 2 | V4-4 |
80+
| `19_activation_propagation.spec.ts` | 11 | V4-5 |
81+
| `20_schedule_propagation.spec.ts` | 6 | V4-6 |
82+
| `21_loss_kind_propagation.spec.ts` | 4 | V4-7 |
83+
| `22_rewriter_propagation.spec.ts` | 3 | V4-8 |
84+
| `23_hybrid_optimizer_split.spec.ts` | 1 | V4-9 |
85+
| `25_inference_after_train.spec.ts` | 2 | V4-11 |
86+
| `26_cross_arch_brick_mutations.spec.ts` | 20 | V4-12 |
87+
| `27_real_gotcha.spec.ts` | 2 | V4-13 |
88+
| `28_parquet_train_threading.spec.ts` | 2 | V4-1 |
89+
| `29_tokenizer_train_threading.spec.ts` | 1 | V4-3 |
90+
91+
## Deferred / soft gaps (carry to v5)
92+
93+
1. **V4-10 side_channels reach forward** — backend forward-hook
94+
plumbing + net-new SideChannelsTab UI. Current INITIAL_SPEC families
95+
are static; user has no surface to mutate from.
96+
2. **Sharding plan apply → real distributed train** — needs multi-device.
97+
3. **Memory peak vs estimate** — needs `mx.metal.get_peak_memory`
98+
instrumentation across train cycles.
99+
4. **WS reconnect mid-train** — lifecycle.
100+
5. **Concurrent / abortable Train** — needs cancel UI + train cancel handle.
101+
6. **Spec save/load roundtrip** — still deferred from v3 (V3-12).
102+
7. **100+ layer realistic depth** — synthetic 2-brick architecture is the
103+
stage_train ceiling.
104+
8. **Checkpoint save/resume** — no stage_train checkpoint code.
105+
9. **Cross-arch presets with non-canonical brick names** (e.g.
106+
`glm_45_shared`, `qwen3_dense_*_mlp` prefixed differently) — needs
107+
runtime brick-context-* testid introspection helper.
108+
109+
## Pre-existing red (excluded from regression gates)
110+
111+
Still `tests/v4/test_path_d_runtime_adapter.py::test_gdn_chunk_o_metal_threadgroup_memory_fits_device_limit` (61440 > 32K) — owned by parallel work in `cppmega_mlx/runtime/path_c_fusion.py`.

0 commit comments

Comments
 (0)