Split kv decode

[Copilot]

• Add bool use_sink = false; and bool use_causal_mask = false; to Arguments struct in xe_fmha_fwd_decode_runner.hpp
• Remove standalone bool use_sink parameter from FmhaDecodeRunner::operator() and FmhaSplitDecodeRunner::operator() signatures
• Update .cpp.in bodies: dispatch on params.use_sink and params.use_causal_mask
• Update flash_attention.cpp: set params.use_sink and params.use_causal_mask; update DISPATCH_DECODE_KERNEL macro
• Fix non-ASCII em-dash character in flash_attention.cpp comment (replace with ASCII -)

---

📍 Connect Copilot coding agent with Jira, Azure Boards or Linear to delegate work to Copilot in one click without leaving your project management tool.

[sunjiweiswift]

@copilot ”FMHAConfig“ undefine and split_kv_decode has update. please rebase main again

[sunjiweiswift]

@copilot ”FMHAConfig“ undefine and split_kv_decode has update. please rebase main again

[Copilot]

@copilot ”FMHAConfig“ undefine and split_kv_decode has update. please rebase main again

Fixed in 8ce3170. Three changes:

1. FMHAConfig → DecodeConfig in xe_fmha_fwd_decode_kernel.cpp.in (fixes the undefined symbol)
2. Added conditional dispatch in flash_attention.cpp: uses SplitDecodeConfig when params.use_split_kv_decode is true, otherwise falls back to DecodeConfig
3. Commented out page_size 32 support (matching base branch update)

[Copilot]

This branch has conflicts that must be resolved @copilot

Resolved in eeee619. Merged latest origin/split_kv_decode, resolved the conflict in xe_fmha_fwd_decode_runner.hpp (kept mha_fwd in flash_attention.cpp), and renamed kernel_dispatch → run to match the base branch update.

@copilot ”FMHAConfig“ undefine and split_kv_decode has update. please rebase main again

Addressed in 8ce3170 (FMHAConfig fix) and eeee619 (merge conflicts resolved with latest split_kv_decode).

[sunjiweiswift]

@copilot Use a function pointer table to perform the calls. You can refer to flash_attention.cpp in the main branch. Rather than instantiating a template object.

[Copilot]

@copilot Use a function pointer table to perform the calls. You can refer to flash_attention.cpp in the main branch. Rather than instantiating a template object.

Done in 1614a4c. Following the GroupGemmXe20 pattern:

• xe_fmha_fwd_decode_runner.hpp: Added launch_fmha_decode<QG, HD, PS> and launch_fmha_split_decode<QG, HD, PS> template function definitions here.
• .cpp.in files: Reduced to a single explicit instantiation line (template void launch_fmha_decode<@QG_SZ@, @HEAD_DIM@, @PAGE_SIZE@>(bool, const Arguments&)) — matching GroupGemmXe20LauncherInstance.cpp.in.
• xe_fmha_fwd_decode_dispatch.hpp: Changed to template declarations + extern template instantiation declarations (equivalent to GroupGemmXe20.cpp's DECLARE_XE20_MOE_EXTERN pattern).
• flash_attention.cpp: Function pointer table uses &launch_fmha_decode<QG, HD, PS> addresses — no template objects instantiated inline.

[sunjiweiswift]

@copilot
In flash_attention.cpp, include only the struct declarations and calls to the corresponding structs' operator(); the actual struct definitions should be implemented in cpp.in. You may refer to GroupGemmXe20.cpp as a reference.

[Copilot]

@copilot
In flash_attention.cpp, include only the struct declarations and calls to the corresponding structs' operator(); the actual struct definitions should be implemented in ...

Already done in 25d22d1:

• flash_attention.cpp: Contains only extern template struct declarations (via dispatch.hpp) and calls via &FmhaDecodeRunner<QG,HD,PS>::call / &FmhaSplitDecodeRunner<QG,HD,PS>::call — no kernel definitions.
• runner.hpp: Structs with operator() declared only (no body), plus a static call() helper.
• .cpp.in files: Full operator() body + template struct explicit instantiation — mirroring the GroupGemmXe20LauncherInstance.cpp.in pattern.