Skip to content

Commit 9670eae

Browse files
iancrisclaude
andcommitted
fix: QVAC-21320 keep clip FA on backends without the efficient-FA query (Metal/CUDA)
The FA-gate defaulted efficient_fa=false, so on non-Vulkan GPU backends (Metal, CUDA) — where ggml_backend_supports_efficient_fa is not implemented — flash attention was wrongly disabled. The resulting explicit-attention clip path overflows the pre-sized compute buffer at high n_pos (image_tile_mode=disabled + image_max_tokens=4096), hitting GGML_ASSERT in ggml-backend.cpp:2043 (SIGABRT) on darwin/iOS Metal integration tests. Fix: default efficient_fa=true and disable only when a backend affirmatively reports non-efficient FA. Only ggml-vulkan implements the query (returns false for VK_VENDOR_ID_ARM / non-coopmat), so the Mali FA-disable win is unchanged; Metal/CUDA/CPU keep their efficient FA. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 22da21f commit 9670eae

1 file changed

Lines changed: 6 additions & 3 deletions

File tree

tools/mtmd/clip.cpp

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2521,11 +2521,14 @@ struct clip_model_loader {
25212521
// keep FA enabled. Resolved at runtime via proc_address — no compile-time backend dep.
25222522
// Acts on AUTO *and* ENABLED (the addon enables FA by default) — an inefficient
25232523
// scalar-FA GPU should never be forced into FA; only explicit DISABLED is left alone.
2524-
// Default when the backend can't confirm efficient FA = DISABLE (the safe original
2525-
// behaviour): efficient FA only exists on coopmat GPUs, which DO answer the query.
2524+
// Default when the backend can't confirm efficient FA = KEEP it. Only ggml-vulkan
2525+
// implements the query (returning false for Mali/non-coopmat); backends that don't
2526+
// answer (Metal, CUDA, …) have efficient FA and must keep it — disabling there forces
2527+
// explicit attention whose QK^T overflows the clip compute buffer at high n_pos
2528+
// (GGML_ASSERT in ggml-backend, e.g. image_tile_mode=disabled + large image_max_tokens).
25262529
if (ctx_clip.flash_attn_type != CLIP_FLASH_ATTN_TYPE_DISABLED &&
25272530
ctx_clip.backend && ctx_clip.backend != ctx_clip.backend_cpu) {
2528-
bool efficient_fa = false;
2531+
bool efficient_fa = true;
25292532
ggml_backend_dev_t dev = ggml_backend_get_device(ctx_clip.backend);
25302533
ggml_backend_reg_t reg = dev ? ggml_backend_dev_backend_reg(dev) : nullptr;
25312534
if (reg) {

0 commit comments

Comments
 (0)