dflash: fix VL/multimodal crash by clearing draft on position gap

[dblmca]

Summary

DFlash crashes when processing requests containing images (VL/multimodal) with:

failed to process speculative batch

Root cause: When the server processes image embeddings through the target model, the draft context's KV cache misses those positions (embedding-only batches are correctly skipped in process()). This creates a non-consecutive position gap — the draft has positions 0..X, but the next text batch starts at Y > X+1. llama_decode(ctx_dft) enforces consecutive positions and fails.

Fix: Detect position gaps between llama_memory_seq_pos_max(ctx_dft) and the incoming batch position. When a gap is found, clear the draft sequence with llama_memory_seq_rm() and let processing rebuild from the current text position. Spec decode gracefully degrades for the first few tokens after an image, then resumes normally.

Same class of bug as EAGLE3 + VL (ggml-org#24816) and MTP + VL (ggml-org#22867).

Benchmark Results

Tested on Qwen3.6-27B VL (Q4_K + mmproj-f16) with DFlash drafter on RTX 8000:

Metric	Baseline (no DFlash)	DFlash + VL fix
BrightPath Score	0.965	0.976
Total Time (34 cases)	581s	264s (2.2x faster)
Text draft acceptance	—	37-64%
VL image processing	✅ works	✅ works (no crash)

The benchmark covers all 11 BrightPath LLM call types including vision grading (math work, spelling, vocab, sentence combining/expansion) with synthetic handwriting images.

The patch

22 lines added to common_speculative_impl_draft_dflash::process() — a position gap detection loop inserted before the main per-seq processing loop. No changes to any other files.

Reviewed by Codex (gpt-5.5) and GLM 5.2 before submission — both approved the core mechanism and confirmed llama_memory_seq_rm(mem, seq_id, -1, -1) is the correct API for full-sequence KV clear.

Test plan

• Text-only generation: DFlash spec decode works (~26 tok/s, 37-64% acceptance)
• VL with image: face recognition pipeline processes images without crash
• BrightPath benchmark: 34/34 cases pass, score 0.976, 2.2x faster than baseline
• Multi-slot: gap detection is per-sequence, doesn't affect text-only slots

[ruixiang63]

Thanks. But need to close this per ggml-org#22105 (comment) for now.

[dblmca]

No worries. Thanks for the great work.

…

On Sun, Jun 28, 2026 at 6:56 AM Ruixiang Wang ***@***.***> wrote: *ruixiang63* left a comment (ruixiang63/llama.cpp#7) <#7 (comment)> Thanks. But need to close this per ggml-org#22105 (comment) <ggml-org#22105 (comment)> for now. — Reply to this email directly, view it on GitHub <#7?email_source=notifications&email_token=AGJFPXFSKFVCSACBY5PPCQL5CEPXHA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIOBSGYZTCMJRGA22M4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2KYZTPN52GK4S7MNWGSY3L#issuecomment-4826311105>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGJFPXBJHUTTVUATN4RMA535CEPXHAVCNFSNUABGKJSXA33TNF2G64TZHMYTAOBWGYYTCNZWGU5US43TOVSTWNBXGYYTKMJSGQ2DTILWAI> . Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS <https://github.com/notifications/mobile/ios/AGJFPXA7RYZKHQI2FRFULNL5CEPXHA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIOBSGYZTCMJRGA22M4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2KUZTPN52GK4S7NFXXG> and Android <https://github.com/notifications/mobile/android/AGJFPXH6AKJWITJHHHYJNMD5CEPXHA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIOBSGYZTCMJRGA22M4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2K4ZTPN52GK4S7MFXGI4TPNFSA>. Download it today! You are receiving this because you authored the thread.Message ID: ***@***.***>