Commit 8798113
committed
fix(path-c): seed direct-chain first brick with raw residual, not pre-normed (double-norm)
make_path_c_direct_chain_pre_step_runtime_owner seeded {first_brick}_hidden with
layers[start].norm(prefix_hidden), but the fused chain's entry-RMSNorm re-applies the
norm -> double-norm. The eager layer contract (hybrid_lm.py:687-697) is residual = hidden
(raw), then delta = route_delta(hidden) which norms internally; the chain replays that, so
the seed must be the RAW residual stream.
Fix: boundary_hidden = prefix_hidden (raw) instead of layers[start].norm(prefix).
Metal (chain start_layer=10): direct-chain runtime loss-vs-eager err 1.02e-2 -> 0.0 (exact),
grad err 1.23e-2 -> 0.0. Flag-independent, target-agnostic (corrects CUDA equally).
Tests 58 passed (direct-chain/pre-step-owner/value_and_grad/bridge). Found by the moe-span
executor investigation; extracted minimal.1 parent 299531d commit 8798113
1 file changed
Lines changed: 6 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7626 | 7626 | | |
7627 | 7627 | | |
7628 | 7628 | | |
7629 | | - | |
| 7629 | + | |
| 7630 | + | |
| 7631 | + | |
| 7632 | + | |
| 7633 | + | |
| 7634 | + | |
7630 | 7635 | | |
7631 | | - | |
7632 | | - | |
7633 | 7636 | | |
7634 | 7637 | | |
7635 | 7638 | | |
| |||
0 commit comments