ggml-opencl: harden flash-attn finite-math strip and document is_causal invariant

The flash-attn compile-flag strip removed only the first occurrence of each
unsafe flag and silently tolerated a miss. If compile_opts ever changes its
spelling/spacing or repeats a flag, a surviving -cl-finite-math-only /
-cl-fast-relaxed-math would rebuild the FA kernels with Inf-assuming math and
silently reintroduce the -INFINITY online-softmax/masking miscompile this
strip exists to prevent.

Erase every occurrence of each flag and add a GGML_ASSERT that no
finite-math/fast-math/unsafe-math flag survived, so a future regression fails
loudly at load time instead of producing silently wrong attention output.

Also document the is_causal invariant in ggml_cl_flash_attn: a null mask is
treated as bidirectional, so any caller needing causal masking must supply an
explicit causal mask rather than relying on shape inference.

Author: olyasir

ggml/src/ggml-opencl/ggml-opencl.cpp

@@ -2149,11 +2149,23 @@ static void load_cl_kernels(ggml_backend_opencl_context *backend_ctx, ggml_cl_ve
             for (const char* unsafe_flag : { " -cl-fast-relaxed-math",
                                              " -cl-finite-math-only",
                                              " -cl-unsafe-math-optimizations" }) {
-                const size_t pos = fa_compile_opts.find(unsafe_flag);
-                if (pos != std::string::npos) {
+                // Erase every occurrence: the strip must not depend on the flag
+                // appearing exactly once or in a particular position. A surviving
+                // copy would silently rebuild FA with finite-math and reintroduce
+                // the -INFINITY miscompile this strip exists to prevent.
+                for (size_t pos = fa_compile_opts.find(unsafe_flag);
+                     pos != std::string::npos;
+                     pos = fa_compile_opts.find(unsafe_flag)) {
                     fa_compile_opts.erase(pos, std::string(unsafe_flag).size());
                 }
             }
+            // Fail loudly if any Inf-assuming flag survived (e.g. a future change
+            // to compile_opts spelling/spacing that the strip above misses),
+            // rather than shipping a silently miscompiled flash-attention kernel.
+            GGML_ASSERT(fa_compile_opts.find("finite-math")  == std::string::npos &&
+                        fa_compile_opts.find("fast-relaxed") == std::string::npos &&
+                        fa_compile_opts.find("unsafe-math")  == std::string::npos &&
+                        "flash-attn kernels must not be built with finite-math/fast-math flags");
 
             for (size_t i = 0; i < sizeof(fa_dims)/sizeof(fa_dims[0]); ++i) {
                 const int dk = fa_dims[i].dk;
@@ -9860,6 +9872,10 @@ static void ggml_cl_flash_attn(ggml_backend_t backend, const ggml_tensor * q, co
     // causal. The previous `mask == NULL && n_q == n_kv` heuristic wrongly made
     // the bidirectional Qwen3-VL vision tower attend causally, corrupting the
     // image embedding (each patch only saw earlier patches).
+    //
+    // INVARIANT: this backend treats a null mask as bidirectional. Any caller
+    // that needs causal masking MUST supply an explicit causal mask; relying on
+    // shape inference here will silently produce bidirectional (wrong) output.
     const int is_causal = 0;
 
     const int n_head_log2_val = n_head > 0 ? 1u << (int)floorf(log2f((float)n_head)) : 0;