Commit 9efbabc
committed
feat(v4-9): hybrid optimizer bucket sizes in extras
Closes V4-9 / cppmega-mlx-9e7. Closes G8 from V4 audit (split was
wire-only proven).
Backend (stage_train):
- When optimizer_kind === 'muon_adamw_hybrid', call
split_param_groups on all_modules.parameters() to count parameter
elements routed to Muon (2-D matmul) vs AdamW (1-D / 3-D+).
- extras.muon_group_size + extras.adamw_group_size populated.
- Both null for non-hybrid optimizers.
E2E 23_hybrid_optimizer_split.spec.ts: UI selects muon_adamw_hybrid,
Train, asserts both bucket sizes > 0 AND muon >= adamw (matmul
weights dominate norm scalars + biases by orders of magnitude).
1/1 green; stage_train pytest 27/27 still passing.1 parent 6225f20 commit 9efbabc
2 files changed
Lines changed: 60 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
457 | 457 | | |
458 | 458 | | |
459 | 459 | | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
| 473 | + | |
| 474 | + | |
| 475 | + | |
| 476 | + | |
460 | 477 | | |
461 | 478 | | |
462 | 479 | | |
| |||
552 | 569 | | |
553 | 570 | | |
554 | 571 | | |
| 572 | + | |
| 573 | + | |
555 | 574 | | |
556 | 575 | | |
557 | 576 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
0 commit comments