Skip to content

Commit 0db3e88

Browse files
committed
fix(path_c bwd): dseg block — T.Parallel(L*L) instead of lane-strided T.serial (CUDA scope bug)
The B2 cuda prim FAILED to compile on gb10 sm_121: 'cb_v' undefined (line 134) + 'dseg' undefined (line 139). Root cause: the dseg segsum-VJP loop used the lane-strided 'for ls0 in T.serial(0, L*L, threads)' idiom with a masked 'if lsi<L*L: if ll>ss:' body that (a) reads cb from global and (b) atomic_adds into SHARED dAcs_acc. TileLang's shared-mem barrier-insertion pass put __syncthreads() between the cb load, the dseg compute, and the atomic scatter, re-emitting the if-guard around each fragment WITHOUT hoisting the local declarations — so cb_v (declared in fragment 1) and dseg (declared in fragment 2) were out of scope at their uses in later fragments. Fix: map the L*L (ll,ss) grid over T.Parallel(L*L) — the SAME un-fragmented thread-parallel idiom the dC apply block (which also atomic_adds into dAcs_acc) and the DYX build already use and which compiled cleanly. dseg lands in a thread-local; the +/- scatter keeps the ll>ss mask (dseg=0 off the strict lower-tri -> no-op atomic adds). Math IDENTICAL to the Metal prim / the prior lane-strided form. Metal prim untouched. RULE #1: still the ONE CUDA path.
1 parent 7995ea6 commit 0db3e88

1 file changed

Lines changed: 21 additions & 20 deletions

File tree

cppmega_mlx/nn/_tilelang/mamba3_chunked_backward_core.py

Lines changed: 21 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -661,26 +661,27 @@ def main(
661661
# dLmat[l,s] = sum_p dY[l,p]*x[s,p] == DYX[l,s] (reuse, zero recompute)
662662
# M[l,s] = cb*lmat*dt_s ; dseg = DYX[l,s]*M[l,s]
663663
# segsum-vjp: +dAcs_acc[l] over l>=s, -dAcs_acc[s] over s<l (strictly l>s).
664-
# Lane-strided over the L*L (ll,ss) grid with the ll>ss mask (mirrors the
665-
# dinp lane-strided pattern); the +/- scatter races on dAcs_acc -> atomic.
666-
for ls0 in T.serial(0, L * L, threads):
667-
lane = T.get_thread_binding(0)
668-
lsi = ls0 + lane
669-
if lsi < L * L:
670-
ll = lsi // L
671-
ss = lsi % L
672-
if ll > ss:
673-
cb_v = T.Cast(
674-
accum_dtype, cb[batch_idx, chunk_idx, group_idx, ll, ss]
675-
)
676-
lmat = T.exp2((dacs[ll] - dacs[ss]) * p)
677-
dt_s = T.Cast(
678-
accum_dtype, dt[batch_idx, head_idx, chunk_idx, ss]
679-
)
680-
m_ls = cb_v * lmat * dt_s
681-
dseg = DYX[ll, ss] * m_ls
682-
T.atomic_add(dAcs_acc[ll], dseg)
683-
T.atomic_add(dAcs_acc[ss], -dseg)
664+
# Map the L*L (ll,ss) grid over T.Parallel (the SAME un-fragmented
665+
# thread-parallel idiom the dC apply / DYX build above use — a
666+
# lane-strided T.serial here gets its masked body split across the
667+
# shared-mem barriers TileLang inserts between the global cb load and the
668+
# atomic scatter, hoisting the locals out of scope). The dseg compute
669+
# lands in a thread-local; the +/- scatter (ll>ss mask) races on shared
670+
# dAcs_acc -> T.atomic_add. dseg=0 off the strict lower-tri (no-op adds).
671+
for ls in T.Parallel(L * L):
672+
ll = ls // L
673+
ss = ls % L
674+
dseg = T.alloc_local((1,), accum_dtype)
675+
dseg[0] = T.Cast(accum_dtype, 0)
676+
if ss < ll:
677+
cb_v = T.Cast(
678+
accum_dtype, cb[batch_idx, chunk_idx, group_idx, ll, ss]
679+
)
680+
lmat = T.exp2((dacs[ll] - dacs[ss]) * p)
681+
dt_s = T.Cast(accum_dtype, dt[batch_idx, head_idx, chunk_idx, ss])
682+
dseg[0] = DYX[ll, ss] * cb_v * lmat * dt_s
683+
T.atomic_add(dAcs_acc[ll], dseg[0])
684+
T.atomic_add(dAcs_acc[ss], -dseg[0])
684685
T.sync_threads()
685686

686687
for l in T.Parallel(L):

0 commit comments

Comments
 (0)