Skip to content

Commit 9e8bc6b

Browse files
committed
Re-enable draft-mtp,ngram-mod on Qwen3.6 24GB tiers (RS-rollback fix #23269)
Combined-mode RS-rollback regression (#23184) was fixed upstream in #23269, which is present in the pinned llama.cpp. Re-add ngram-mod fallback to both Qwen3.6 24GB entries, re-enable mmproj-offload, and drop the now-stale 'combined mode broken' warning comment.
1 parent ca80e80 commit 9e8bc6b

2 files changed

Lines changed: 12 additions & 14 deletions

File tree

CHANGELOG.md

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,9 +8,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
88
## [Unreleased]
99

1010
### Changed
11-
- [Presets] Switch Qwen3.6-35B-A3B 24 GB tier from ngram-mod to draft-mtp
12-
- [Presets] Drop ngram-mod from Qwen3.6-27B 24 GB tier (combined mode broken, #23184)
13-
- [Presets] Disable mmproj-offload on Qwen3.6-27B 24 GB tier
11+
- [Presets] Combine draft-mtp with ngram-mod on Qwen3.6 24 GB tiers (RS-rollback regression fixed upstream, #23269)
12+
- [Presets] Re-enable mmproj-offload on Qwen3.6 24 GB tiers
1413
- [Presets] Raise Qwen3.6-27B 24 GB ctx-size from 172032 to 184320
1514

1615

presets/models_24GB_VRAM.ini

Lines changed: 10 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -25,18 +25,11 @@ spec-ngram-mod-n-match = 24
2525
spec-ngram-mod-n-min = 48
2626
spec-ngram-mod-n-max = 64
2727

28-
# WARNING: spec-type draft-mtp,ngram-mod is broken on hybrid Qwen 3.6
29-
#
30-
# Speculators run independently (https://github.com/ggml-org/llama.cpp/issues/23184)
31-
# and any non-MTP speculator unconditionally disables MTP partial rollback on hybrid
32-
# models (https://github.com/ggml-org/llama.cpp/blob/255582687/common/common.cpp#L1258).
33-
# This causes ~30% TG regression vs draft-mtp alone. Use either solo, never combined!
34-
3528
[Qwen3.6-27B.IQ4_XS.gguf]
3629
alias = Qwen3.6-27B.IQ4_XS.gguf
3730
model = D:\AI\LLM\gguf\Qwen3.6-27B\Qwen3.6-27B.IQ4_XS.gguf
3831
mmproj = D:\AI\LLM\gguf\Qwen3.6-27B\mmproj.Qwen3.6-27B.BF16.gguf
39-
no-mmproj-offload = true
32+
mmproj-offload = true
4033
ctx-size = 262144
4134
threads = 24
4235
parallel = 2
@@ -57,15 +50,18 @@ top-p = 0.95
5750
top-k = 20
5851
temp = 0.6
5952
presence-penalty = 1.5
60-
spec-type = draft-mtp
53+
spec-type = draft-mtp,ngram-mod
6154
spec-draft-n-max = 4
55+
spec-ngram-mod-n-match = 24
56+
spec-ngram-mod-n-min = 48
57+
spec-ngram-mod-n-max = 64
6258
fit = off
6359

6460
[Qwen3.6-35B-A3B.IQ4_XS.gguf]
6561
alias = Qwen3.6-35B-A3B.IQ4_XS.gguf
6662
model = D:\AI\LLM\gguf\Qwen3.6-35B-A3B\Qwen3.6-35B-A3B.IQ4_XS.gguf
6763
mmproj = D:\AI\LLM\gguf\Qwen3.6-35B-A3B\mmproj.Qwen3.6-35B-A3B.BF16.gguf
68-
no-mmproj-offload = true
64+
mmproj-offload = true
6965
ctx-size = 262144
7066
threads = 24
7167
parallel = 2
@@ -86,6 +82,9 @@ top-p = 0.95
8682
top-k = 20
8783
temp = 0.6
8884
presence-penalty = 1.5
89-
spec-type = draft-mtp
85+
spec-type = draft-mtp,ngram-mod
9086
spec-draft-n-max = 3
87+
spec-ngram-mod-n-match = 24
88+
spec-ngram-mod-n-min = 48
89+
spec-ngram-mod-n-max = 64
9190
fit = off

0 commit comments

Comments
 (0)