Add TurboQuant turbo4 Vulkan goal prompt

Author: romgenie

docs/development/turboquant-turbo4-vulkan-goal.md

@@ -0,0 +1,43 @@
+/goal
+Solve TurboQuant `turbo4` KV-cache support in llama.cpp Vulkan with code.
+
+Context:
+CompleteTech's fork is now synced to ggml-org/llama.cpp `master` through `b0df4c0cf`, with local fixes retained:
+- `fix Gemma 4 thinking generation prompt`
+- `Add audio transcription fallback for chat input`
+
+Observed failure:
+Running Gemma4 on the B60 Vulkan install with `--cache-type-k q8_0 --cache-type-v turbo4` aborts during graph scheduling with:
+
+`pre-allocated tensor (cache_v_l0 (view)) in a buffer (Vulkan0) that cannot run the operation (SET_ROWS)`
+
+Root cause:
+The V KV cache tensor is preallocated in a Vulkan buffer with type `GGML_TYPE_TURBO4_0`. During decode, llama.cpp uses `GGML_OP_SET_ROWS` to update cache rows. The current Vulkan backend wires `SET_ROWS` for supported quantized destination types, but TurboQuant's `GGML_TYPE_TURBO4_0` path is missing or incomplete. Older TurboQuant wiring covers turbo3-style support, but turbo4 needs its own Vulkan copy/dequant/packing path and backend support checks.
+
+Goal:
+Implement and verify Vulkan `GGML_OP_SET_ROWS` support for `GGML_TYPE_TURBO4_0`, preserving existing turbo3 behavior and avoiding regressions in CPU/CUDA/Metal.
+
+Suggested code scope:
+1. Inspect `ggml/src/ggml-vulkan/` shader generation, type declarations, copy/dequant helpers, pipeline registration, and `op_supports_*` checks for existing `SET_ROWS` quantized destinations.
+2. Add or port Vulkan shader type definitions needed for TurboQuant blocks, including `block_turbo4_0` and any dependent `block_turbo2_0` support if turbo4 packing uses it.
+3. Extend the Vulkan `copy_to_quant.comp` / generated SET_ROWS shader flow for turbo4:
+   - support i32 and i64 row-index variants
+   - apply turbo4's transform/quantization requirements correctly
+   - pack 4-bit centroid indices into the expected GGML/TurboQuant memory layout
+   - use the same workgroup/dispatch conventions as neighboring quantized SET_ROWS kernels
+4. Register `set_rows_turbo4_0_i32` and `set_rows_turbo4_0_i64` pipelines, and ensure `pipeline_set_rows_i32/i64[GGML_TYPE_TURBO4_0]` is populated.
+5. Include `GGML_TYPE_TURBO4_0` in Vulkan SET_ROWS support checks and element-count dispatch logic so scheduler placement accepts Vulkan buffers for this operation.
+6. Keep turbo3 paths unchanged except where shared helper code is required and covered by tests.
+
+Validation:
+- Build Vulkan artifacts and at minimum: `cmake --build build --target test-chat llama-server -j$(nproc)`.
+- Run `./build/bin/test-chat`.
+- On the B60 Vulkan host, start Gemma4 with the production-like command using:
+  `--cache-type-k q8_0 --cache-type-v turbo4 -ngl 999 -ngld 999 -ncmoe 38 -ncmoed 38 -c 65537 -b 512 -ub 512 -fa on --reasoning-budget 0 -np 1`
+- Confirm the server starts without the Vulkan scheduler abort, `/health` is available, and a short completion succeeds.
+- Compare a short smoke prompt against `--cache-type-v q8_0` or `turbo3` to catch obvious corruption.
+
+Constraints:
+- Keep the patch backend-local and narrowly scoped.
+- Do not switch production to turbo4 until runtime validation passes.
+- Preserve the deployed audio transcription fallback and Gemma4 thinking-template fixes.