Commit b7b8dd0
turbo Vulkan port: wire turbo KV cache into flash attention (Phase B)
Completes the turbo Vulkan port. turbo KV cache now runs end-to-end on Vulkan.
Phase B wiring:
- ggml-vulkan supports_op FLASH_ATTN_EXT: accept TURBO2/3/4_0 K/V (coopmat2-only,
like Q1_0). fa_block_bytes already generic (ggml_type_size); TURBO_WHT op +
SET_ROWS for turbo + cm2 dequantFuncTURBO3_0 survived earlier merges.
- common/arg.cpp: re-added turbo2/3/4 to kv_cache_types (-ctk/-ctv turbo3).
Status (AMD Radeon RX 9070 XT / RADV GFX1201):
- turbo3 KV cache RUNS via llama-bench (tg64 @ d2048 = 12.2 t/s).
- CORRECTNESS NOT VALIDATED on this GPU: RADV flash-attention is broken on
RDNA4/GFX1201 for ALL types — test-backend-ops FLASH_ATTN_EXT 864/8453 pass,
f16 KV perplexity = 9.4M with -fa vs 1.0092 without. Base (non-FA) Vulkan
inference is correct (PPL 1.0092 == CPU); turbo weights validated. turbo KV
correctness depends on FA, which this driver miscomputes for f16 too.
-> Validate turbo KV on conformant-FA hardware (NVIDIA/CUDA or fixed RADV).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>1 parent 1a859aa commit b7b8dd0
2 files changed
Lines changed: 8 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
397 | 397 | | |
398 | 398 | | |
399 | 399 | | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
400 | 403 | | |
401 | 404 | | |
402 | 405 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16531 | 16531 | | |
16532 | 16532 | | |
16533 | 16533 | | |
| 16534 | + | |
| 16535 | + | |
| 16536 | + | |
| 16537 | + | |
| 16538 | + | |
16534 | 16539 | | |
16535 | 16540 | | |
16536 | 16541 | | |
| |||
0 commit comments