Commit 8d078ea
fix(kv-paged): size paged K/V from first attention layer for hybrid models
The paged KV cache read n_head_kv/head_dim from layer 0, but hybrid
models (e.g. LFM2.5 lfm2moe) have conv/recurrent layers (n_head_kv==0)
interleaved with attention layers and layer 0 may be non-attention,
tripping GGML_ASSERT(n_kv_heads > 0). Derive both from the first
attention layer (n_head_kv > 0) instead.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>1 parent 1f671b8 commit 8d078ea
1 file changed
Lines changed: 11 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
108 | 108 | | |
109 | 109 | | |
110 | 110 | | |
111 | | - | |
112 | | - | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
113 | 122 | | |
114 | 123 | | |
115 | 124 | | |
| |||
0 commit comments