perf(#1662): bit-identical fused optimizer-in-backward (lever #1)#1664

Merged

ooples merged 9 commits into

perf/1662-optimizer-in-backward

Jun 22, 2026

Commits on Jun 22, 2026

docs(#1662): design spec for backward-pass optimizations (levers #4, #1, #3)

franklinic
and
claude
committed
docs(#1662): correct lever #1 scope — optimizer-in-backward is OOM-only today

franklinic
and
claude
committed
docs(#1662): implementation plan — levers #4, #3, #1 (Tensors-first, PyTorch proof)

franklinic
and
claude
committed
feat(#1662): full-precision streaming Adam (bit-identical fused-in-backward foundation)

franklinic
and
claude
committed
feat(#1662): wire full-precision fused optimizer-in-backward (unclipped) + gate test

franklinic
and
claude
committed
fix(#1662): clipped two-pass streaming path + bit-identical full-precision for clipped

franklinic
and
claude
committed
test(#1662): pytorch cpu comparison - honest result (we do not beat torch on speed)

authored and
ooples
committed
docs(#1662): gap investigation — per-step deficit is small-M GEMM parallel scaling

authored and
ooples
committed
feat(#1662): opt-in single-pass fast approximate grad-clip (§5c)

authored and
ooples
committed