Commit 8e24fc5
gemma4: full feature mirror resync after prefix cache restore
After restoring KV from a snapshot, do_prefill only syncs the feature mirror
for the delta tokens [snap_pos..committed). The positions [0..snap_pos) in
the mirror retain stale data from the previous request's decode phase (which
may have diverged from the current prompt context after the ring buffer
wraps).
Fix: call draft_feature_mirror_sync_tail after restore to resync the entire
[0..committed) feature range from cache_.target_feat to the mirror. This
ensures the draft model sees consistent features and maintains high
acceptance rate (AL) during speculative decoding.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>1 parent fc360e5 commit 8e24fc5
1 file changed
Lines changed: 8 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
671 | 671 | | |
672 | 672 | | |
673 | 673 | | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
| 679 | + | |
| 680 | + | |
| 681 | + | |
674 | 682 | | |
675 | 683 | | |
676 | 684 | | |
| |||
0 commit comments