Qualcomm AI Engine Direct - Fix sliding window attention (#20736)

DannyYuyang-quic · web-flow · commit 4db56439a1de · 2026-07-06T22:41:13.000-05:00
### Summary
When `n_past + n_update` exceeds the `available_cache_len`
it should mask only the remain part.

### Test plan
gemma3 CI in `TestExampleLLMScript.test_static_llm_model`
diff --git a/examples/qualcomm/oss_scripts/llama/runner/kv_manager.cpp b/examples/qualcomm/oss_scripts/llama/runner/kv_manager.cpp
@@ -260,7 +260,7 @@ void KVManager::update_attention_mask(
       fill_mask(
           attention_mask_dtype_,
           cur_ptr - n_past * getDtypeSize(attention_mask_dtype_),
-          n_past + n_update,
+          n_past + n_update - available_cache_len,
           /*use_pos_value=*/false);
     }
     cur_ptr += metadata_.context_len * getDtypeSize(attention_mask_dtype_);

Original file line number	Diff line number	Diff line change
`@@ -260,7 +260,7 @@ void KVManager::update_attention_mask(`
`260`	`260`	`fill_mask(`
`261`	`261`	`attention_mask_dtype_,`
`262`	`262`	`cur_ptr - n_past * getDtypeSize(attention_mask_dtype_),`
`263`		`- n_past + n_update,`
	`263`	`+ n_past + n_update - available_cache_len,`
`264`	`264`	`/use_pos_value=/false);`
`265`	`265`	`}`
`266`	`266`	`cur_ptr += metadata_.context_len * getDtypeSize(attention_mask_dtype_);`