Commit 0b1bbc1
committed
server: use full logical ctx size for slot.n_ctx when tiered KV is enabled
Hot tier shrinks n_ctx passed to llama init, so llama_n_ctx_seq() returns
the hot tier size (~60% of --ctx-size). With tiering, slot.n_ctx must equal
kv_tier_total_ctx so requests up to the full context window are accepted.1 parent 1c4e38c commit 0b1bbc1
1 file changed
Lines changed: 3 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
877 | 877 | | |
878 | 878 | | |
879 | 879 | | |
880 | | - | |
| 880 | + | |
| 881 | + | |
| 882 | + | |
881 | 883 | | |
882 | 884 | | |
883 | 885 | | |
| |||
0 commit comments