Commit 14d4810
committed
Fix tq4_sdpa split-K dispatch for export/AOTI tracing
The split-K decode path was branching on runtime kv_len value via
kv_len_t.item() >= threshold, which is data-dependent control flow
that breaks torch.export/AOTI tracing.
Fix: Make dispatch static by using N_KV (buffer size) instead of
runtime kv_len value. The kv_len tensor is still used inside the
kernel for bounds checking via tl.load on device (CUDA-graph safe).
- Dispatch now based on static N_KV >= 256 (buffer size)
- Removes incorrect guard_or_false usage with .item()
- Preserves kv_len on-device usage in kernel for bounds
- Maintains byte-identical behavior for qwen (kv_len fallback)
- Eager correctness preserved1 parent f786724 commit 14d4810
1 file changed
Lines changed: 4 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
800 | 800 | | |
801 | 801 | | |
802 | 802 | | |
| 803 | + | |
| 804 | + | |
| 805 | + | |
803 | 806 | | |
804 | 807 | | |
805 | 808 | | |
806 | 809 | | |
807 | 810 | | |
808 | | - | |
| 811 | + | |
809 | 812 | | |
810 | 813 | | |
811 | 814 | | |
| |||
0 commit comments