fix naive mha out-of-bound read#6812
Conversation
|
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #6812 +/- ##
==========================================
- Coverage 95.13% 94.70% -0.43%
==========================================
Files 947 947
Lines 416258 416884 +626
==========================================
- Hits 396004 394808 -1196
- Misses 20254 22076 +1822 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
Fixes an out-of-bounds read in the MultiHeadAttention KV-cache path when decoding with cur_seqlen > 1, and extends the corresponding tests to cover multi-token decode.
Changes:
- Copy only
past_seqlenelements from cached K/V into the newk_affine/v_affinebuffers (instead ofdst_seqlen). - Update KV-cache decode test helpers to accept a configurable
cur_seqlen. - Add decode test cases with
cur_seqlen = 32for both fp32 and int8 paths.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
tests/test_multiheadattention_kvcache.cpp |
Extends decode tests to cover multi-token decode (cur_seqlen > 1) for fp32 and int8. |
src/layer/multiheadattention.cpp |
Fixes KV-cache memcpy length to prevent reading past cached K/V buffers during decode. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
No description provided.