Skip to content

fix naive mha out-of-bound read#6812

Merged
nihui merged 1 commit into
Tencent:masterfrom
nihui:fix-mha-oob
Jul 8, 2026
Merged

fix naive mha out-of-bound read#6812
nihui merged 1 commit into
Tencent:masterfrom
nihui:fix-mha-oob

Conversation

@nihui

@nihui nihui commented Jul 8, 2026

Copy link
Copy Markdown
Member

No description provided.

@tencent-adm

Copy link
Copy Markdown
Member

CLA assistant check
Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@codecov-commenter

codecov-commenter commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.70%. Comparing base (bd58421) to head (81db1d5).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6812      +/-   ##
==========================================
- Coverage   95.13%   94.70%   -0.43%     
==========================================
  Files         947      947              
  Lines      416258   416884     +626     
==========================================
- Hits       396004   394808    -1196     
- Misses      20254    22076    +1822     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes an out-of-bounds read in the MultiHeadAttention KV-cache path when decoding with cur_seqlen > 1, and extends the corresponding tests to cover multi-token decode.

Changes:

  • Copy only past_seqlen elements from cached K/V into the new k_affine/v_affine buffers (instead of dst_seqlen).
  • Update KV-cache decode test helpers to accept a configurable cur_seqlen.
  • Add decode test cases with cur_seqlen = 32 for both fp32 and int8 paths.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
tests/test_multiheadattention_kvcache.cpp Extends decode tests to cover multi-token decode (cur_seqlen > 1) for fp32 and int8.
src/layer/multiheadattention.cpp Fixes KV-cache memcpy length to prevent reading past cached K/V buffers during decode.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@nihui
nihui merged commit 13b6d53 into Tencent:master Jul 8, 2026
127 of 133 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants