Commit b2b6f29
committed
feat(v5-g23): gradient_clip_norm activation + extras.gradient_clip
Closes V5-G23 / cppmega-mlx-br8. OptimSpec.gradient_clip_norm existed
as decorative field — stage_train never applied it.
Backend (stages.py):
- Read spec.optim.gradient_clip_norm (from rewritten or original spec)
- Per step: compute global L2 grad norm; if > threshold, rescale all
grads by threshold/norm; bump num_clips counter
- extras.gradient_clip = {threshold, max_grad_norm_seen, num_clips}
Wire (schema.py + methods.py + App.tsx):
- OptimSpecPayload.gradient_clip_norm field (default 1.0)
- _make_optim threads it into OptimSpec()
- App.tsx buildVerifyParams + suggest_sharding payload include
gradient_clip_norm: spec.optim.grad_clip_norm (UI used underscore
variant of the name; mapped to camel-free wire form)
Pytest (4 in test_stage_train_clip.py):
- threshold=1.0 populates max_grad_norm_seen > 0
- None disables (no observation)
- 0.001 triggers num_clips > 0
- 1000 large threshold → num_clips=0 but max_seen reported
E2E (1 in 52_clip_norm.spec.ts):
- UI OptimTab clip=0.001 → Apply → Train → extras.gradient_clip
.threshold=0.001 + num_clips>0
Regression: 183 vitest + 49 stage_train pytest unchanged.1 parent a93a8af commit b2b6f29
4 files changed
Lines changed: 41 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
195 | 195 | | |
196 | 196 | | |
197 | 197 | | |
198 | | - | |
| 198 | + | |
| 199 | + | |
199 | 200 | | |
200 | 201 | | |
201 | 202 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
155 | 155 | | |
156 | 156 | | |
157 | 157 | | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
158 | 161 | | |
159 | 162 | | |
160 | 163 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
253 | 253 | | |
254 | 254 | | |
255 | 255 | | |
| 256 | + | |
256 | 257 | | |
257 | 258 | | |
258 | 259 | | |
| |||
481 | 482 | | |
482 | 483 | | |
483 | 484 | | |
| 485 | + | |
484 | 486 | | |
485 | 487 | | |
486 | 488 | | |
| |||
0 commit comments