Commit b37e5f0
feat: Metal kernels for TurboQuant KV cache (turbo3, turbo4) PrismML-Eng#21
Added Metal shader implementations:
- quantize_turbo3_0 / quantize_turbo4_0 (per-block quantization)
- dequantize_turbo3_0 / dequantize_turbo4_0 (type4x4 and type4 variants)
- kernel_set_rows_turbo template (128-element block size)
- Flash attention instantiations for all dk/dv variants
Added TURBO3_0/TURBO4_0 to Metal device SET_ROWS validation.
Builds successfully. Testing with Qwen 3.5 35B-A3B MoE on M5 Max.
Note: Initial version uses simplified quantization (no rotation matrix)
for Metal compatibility. Full rotation requires custom kernel with extra
buffer bindings — tracked for follow-up.
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>1 parent 0b37448 commit b37e5f0
2 files changed
Lines changed: 282 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1266 | 1266 | | |
1267 | 1267 | | |
1268 | 1268 | | |
| 1269 | + | |
| 1270 | + | |
1269 | 1271 | | |
1270 | 1272 | | |
1271 | 1273 | | |
| |||
0 commit comments