Skip to content

Commit f963db0

Browse files
committed
ggml-opencl: treat null attention mask as bidirectional, not causal
The flash-attention dispatch inferred causal masking from shape with `is_causal = (mask == NULL && n_q > 1 && n_q == n_kv)`. A null mask means no masking, i.e. bidirectional attention (the SigLIP vision and embedding encoders), while causal attention always supplies an explicit causal mask in this codebase (llama-graph.cpp build_attn passes a kq_mask filled with -INFINITY). The heuristic therefore wrongly made the bidirectional Qwen3-VL vision tower attend causally, so each patch only saw earlier patches and the image embedding was corrupted. Set is_causal = 0 unconditionally; causality is always expressed via the explicit mask. This cannot regress the LLM, which already passes a real causal mask (is_causal was already 0 for it) and relies on that mask.
1 parent d781a13 commit f963db0

1 file changed

Lines changed: 8 additions & 1 deletion

File tree

ggml/src/ggml-opencl/ggml-opencl.cpp

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9853,7 +9853,14 @@ static void ggml_cl_flash_attn(ggml_backend_t backend, const ggml_tensor * q, co
98539853
max_bias = params[1];
98549854
logit_softcap = params[2];
98559855

9856-
const int is_causal = (mask == NULL && n_q > 1 && n_q == n_kv);
9856+
// A null mask means no masking, i.e. bidirectional attention (e.g. the
9857+
// SigLIP vision / embedding encoders). Causal attention always supplies an
9858+
// explicit causal mask in this codebase (llama-graph.cpp build_attn passes
9859+
// kq_mask filled with -INFINITY), so a null mask must NOT be inferred as
9860+
// causal. The previous `mask == NULL && n_q == n_kv` heuristic wrongly made
9861+
// the bidirectional Qwen3-VL vision tower attend causally, corrupting the
9862+
// image embedding (each patch only saw earlier patches).
9863+
const int is_causal = 0;
98579864

98589865
const int n_head_log2_val = n_head > 0 ? 1u << (int)floorf(log2f((float)n_head)) : 0;
98599866
const float n_head_log2_f = n_head_log2_val > 0 ? (float)n_head_log2_val : 1.0f;

0 commit comments

Comments
 (0)