Autograd resident backward and fp16 fix#43

Merged

grillcheese123 merged 56 commits into

mainfrom

autograd-resident-backward

Jun 17, 2026

Commits on Jun 13, 2026

2.0-dev initial commit
grillcheese123
committed

Commits on Jun 14, 2026

2.0 update
grillcheese123
committed
autograd
grillcheese123
committed
autograd: compile bindings_autograd into module, complete TapeContext surface
grillcheese123
committed
autograd: wire node-to-node grad propagation + fan-out accumulation
grillcheese123
committed
autograd: implement resident backward_cross_entropy (training entry point)
grillcheese123
committed
docs: AUTOGRAD_STATE.md — resident autograd working state + gotchas + next steps
grillcheese123
committed
autograd: pointwise activation backward (relu/gelu/silu) + verify fan-out
grillcheese123
committed
docs: update AUTOGRAD_STATE — silu/relu/gelu + fan-out verified; rms-norm-backward is the one missing kernel
grillcheese123
committed
autograd: author rms-norm-backward shader + wire backward_rmsnorm
grillcheese123
committed
docs: rms-norm-backward done — all trunk kernels verified, MinGRU wiring is last piece
grillcheese123
committed
autograd: wire backward_mingru (last trunk piece) — full trunk backward closed
grillcheese123
committed
docs: MinGRU done — trunk backward path CLOSED; next is integration micro-step
grillcheese123
committed
integration: first resident training loop (Linear+CE) — loss descends 1.26->0.02
grillcheese123
committed
integration: train THROUGH the MinGRU recurrence — loss 1.10->0.00
grillcheese123
committed
docs: integration results (Linear+CE, MinGRU+CE both train); SwiGLU wiring is next gap
grillcheese123
committed
autograd: wire backward_swiglu — gated FFN activation verified
grillcheese123
committed
autograd: fix two backward-pass sync races; full Cubby block trains end-to-end
grillcheese123
committed
docs: full block trains; document the two backward sync-barrier requirements; next = resident forward
grillcheese123
committed
autograd: resident FORWARD pass (forward_linear) feeding the backward tape
grillcheese123
committed
docs: resident forward opened for Linear; next = forward_rmsnorm/swiglu/mingru + fully-resident block
grillcheese123
committed
autograd: resident forward_rmsnorm + forward_swiglu
grillcheese123
committed
Full training milestone: resident backward trains composed Cubby block to generalize
grillcheese123
committed
State doc: full-training generalization milestone + resident-forward crash root-cause & VMA-canary fix path
grillcheese123
committed
Keep two repros for the open resident-forward crash: ttr3 (minimal crash), ttr8 (backward-clean proof)
grillcheese123
committed
Fix resident-forward 0xC0000005: BufferRegistry vector -> deque (stable resolve() refs)
grillcheese123
committed
State doc: resident-forward crash ROOT-CAUSED (vector resolve() dangling ref) + FIXED (deque); fully-resident training works; VMA-margin was a red herring
grillcheese123
committed
Create bisect_crash.py
grillcheese123
committed
Resident forward: add forward_mingru + forward_embedding (parity-verified)
grillcheese123
committed
CLAUDE.md: resident-autograd section for Claude Code continuation
grillcheese123
committed
CLAUDE.md: resident forward set complete (mingru+embedding); instrumentation removed; integration is next
grillcheese123
committed
Add TODO.md: cross-repo work board (resident trunk integration -> cubby link -> ladder)
grillcheese123
committed
Fix conv2d: backward-weight dispatch grid + forward batch de-interleave; route nn.Conv2d/1d through C++ bridge
grillcheese123
committed
TODO: record conv2d/1d fix (done) + conv3d add (pending) + test-infra note
grillcheese123
committed
TODO: record conv2d/1d fix (done) + conv3d add (pending) + test-infra note
grillcheese123
committed
Resident single-tape trunk integration (P0 steps 1-4): trains on TinyStories

grillcheese123
and
claude
committed
Update train_trunk_lm.py
grillcheese123
committed
Update TODO.md
grillcheese123
committed
docs+probe: steps 4 (L=18 capacity + TinyStories) & 5 (cubby-link parity gates)

grillcheese123
and
claude
committed
docs: P0 resident-trunk integration COMPLETE (step 5 cubby link + flip)

grillcheese123
and
claude
committed
Fix deep-trunk (L>=7) grad corruption: descriptor-set cache + resident grad clip

grillcheese123
and
claude
committed
v3.3 throughput 0.23->0.77 it/s: GPU grad-norm reduction + resident embedding bwd

grillcheese123
and
claude
committed
backward grad_input via tiled GEMM: v3.3 0.77->0.97 it/s (4.2x total)

grillcheese123
and
claude
committed
backward grad_weight via transpose + tiled GEMM: v3.3 0.97->1.25 it/s (5.4x total)

grillcheese123
and
claude
committed
device: query cooperative-matrix configs (vkGetPhysicalDeviceCooperativeMatrixPropertiesKHR)

grillcheese123
and
claude
committed
bench_gemm: coopmat fp16 vs fp32 tiled -- measured, not assumed

grillcheese123
and
claude
committed

Commits on Jun 15, 2026

Commits on Jun 16, 2026

Commits on Jun 17, 2026

fp16 fixes
grillcheese123
committed