Skip to content

Commit 40294a4

Browse files
committed
Bump Qwen-Fixed-Chat-Templates to v19 and adopt unified chat_template.jinja
Upstream restructured the repository: the qwen3.5/ and qwen3.6/ subdirectories are gone, replaced by a single chat_template.jinja at the root that handles both variants. v19 also flips preserve_thinking to default true, so the explicit chat-template-kwargs override is no longer needed and is dropped from all presets.
1 parent a52c6db commit 40294a4

6 files changed

Lines changed: 30 additions & 23 deletions

File tree

CHANGELOG.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
### Changed
11+
- [Vendor] Bump Qwen-Fixed-Chat-Templates submodule to v19
12+
- [Presets] Repoint chat-template-file to unified chat_template.jinja
13+
- [Presets] Drop chat-template-kwargs preserve_thinking override
14+
1015
### Removed
1116
- [Build] Drop webui-download.cmake npm-resolver patch (fixed upstream in #23064)
1217

CLAUDE.md

Lines changed: 16 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -44,20 +44,25 @@ See `presets/README.md` for the user-facing quick-start; notes below are for edi
4444
compute buffer OOMs but the server keeps running — only image requests error at generation
4545
time. Set `false` on tiers where LLM + KV already saturate VRAM.
4646

47-
- **All Qwen 3.6 entries pin `chat-template-file = vendor\Qwen-Fixed-Chat-Templates\qwen3.6\chat_template-v13.jinja`.**
47+
- **All Qwen 3.6 entries pin `chat-template-file = vendor\Qwen-Fixed-Chat-Templates\chat_template.jinja`.**
4848
Required, *not* redundant with `jinja = true``chat-template-file` *replaces*
4949
the GGUF-embedded template entirely (`vendor/llama.cpp/common/arg.cpp:3142`,
5050
`params.chat_template = read_file(value)`). The upstream embedded template has
51-
fourteen documented bugs affecting tool calls, role handling, `<think>` block
52-
rendering, agentic loops, and llama.cpp KV-prefix cache stability; the vendored
53-
template fixes all of them (full list in `vendor/Qwen-Fixed-Chat-Templates/README-v13.md`).
54-
The template also adds a `<|think_on|>` / `<|think_off|>` toggle that coexists
55-
with `chat-template-kwargs = {"preserve_thinking":true}` (default strips past
56-
`<think>` blocks from history; set the kwarg to keep them for agentic reasoning).
57-
Path is repo-relative, so `llama-server` must be launched from the repo root —
58-
`read_file()` resolves against the process CWD, not the INI file's directory.
59-
Gemma and `Qwen3-Coder-Next` entries deliberately keep their GGUF-embedded
60-
templates; upstream's README only claims compatibility for Qwen 3.5 / 3.6 variants.
51+
documented issues with tool calls, role handling, `<think>` block rendering,
52+
agentic loops, and llama.cpp KV-prefix cache stability; the vendored template
53+
fixes all of them (full list in `vendor/Qwen-Fixed-Chat-Templates/README.md`).
54+
Since v19 the template is a single unified file covering both Qwen 3.5 and 3.6
55+
variants (the old `qwen3.5/` and `qwen3.6/` subdirectories now live under
56+
`archive/`). The template adds a `<|think_on|>` / `<|think_off|>` toggle, and
57+
v19 defaults `preserve_thinking` to `true` (past `<think>` blocks are kept
58+
chronologically for 100% KV prefix cache stability and agentic reasoning
59+
continuity). To strip past `<think>` blocks instead, set
60+
`chat-template-kwargs = {"preserve_thinking":false}` — at the cost of a lower
61+
KV cache hit rate. Path is repo-relative, so `llama-server` must be launched
62+
from the repo root — `read_file()` resolves against the process CWD, not the
63+
INI file's directory. Gemma and `Qwen3-Coder-Next` entries deliberately keep
64+
their GGUF-embedded templates; upstream's README only claims compatibility for
65+
Qwen 3.5 / 3.6 variants.
6166

6267
**ngram-mod speculative decoding** (`--spec-type ngram-mod`): model-agnostic, works on any model.
6368
- All models: `spec-ngram-mod-n-match = 24`, `spec-ngram-mod-n-min = 48`, `spec-ngram-mod-n-max = 64`

presets/README.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,8 +39,9 @@ The section header (e.g. `[gemma-4-31B-it.IQ4_XS.gguf]`) is the model name clien
3939
> See `llama-server --help` for all flags.
4040
4141
> [!IMPORTANT]
42-
> All `Qwen3.6-*` entries set `chat-template-file = vendor\Qwen-Fixed-Chat-Templates\qwen3.6\chat_template-v13.jinja`,
43-
> overriding the buggy template embedded in the GGUF. The path is repo-relative,
42+
> All `Qwen3.6-*` entries set `chat-template-file = vendor\Qwen-Fixed-Chat-Templates\chat_template.jinja`,
43+
> overriding the buggy template embedded in the GGUF. The vendored template is a
44+
> single unified file that handles both Qwen 3.5 and 3.6 variants. The path is repo-relative,
4445
> so launch `llama-server` from the repository root (as the examples above do).
4546
> If you cloned without `--recurse-submodules`, run `git submodule update --init`
4647
> first — otherwise startup fails with a missing-file error. Gemma and

presets/models_16GB_VRAM.ini

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -91,7 +91,7 @@ cache-type-v = q8_0
9191
flash-attn = true
9292
reasoning = on
9393
jinja = true
94-
chat-template-file = vendor\Qwen-Fixed-Chat-Templates\qwen3.6\chat_template-v13.jinja
94+
chat-template-file = vendor\Qwen-Fixed-Chat-Templates\chat_template.jinja
9595
kv-unified = true
9696
ctx-checkpoints = 32
9797
cache-ram = 51200
@@ -120,8 +120,7 @@ cache-type-v = q8_0
120120
flash-attn = true
121121
reasoning = on
122122
jinja = true
123-
chat-template-file = vendor\Qwen-Fixed-Chat-Templates\qwen3.6\chat_template-v13.jinja
124-
chat-template-kwargs = {"preserve_thinking":true}
123+
chat-template-file = vendor\Qwen-Fixed-Chat-Templates\chat_template.jinja
125124
kv-unified = true
126125
ctx-checkpoints = 32
127126
cache-ram = 51200
@@ -150,8 +149,7 @@ cache-type-v = q8_0
150149
flash-attn = true
151150
reasoning = on
152151
jinja = true
153-
chat-template-file = vendor\Qwen-Fixed-Chat-Templates\qwen3.6\chat_template-v13.jinja
154-
chat-template-kwargs = {"preserve_thinking":true}
152+
chat-template-file = vendor\Qwen-Fixed-Chat-Templates\chat_template.jinja
155153
kv-unified = true
156154
ctx-checkpoints = 32
157155
cache-ram = 51200

presets/models_24GB_VRAM.ini

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -64,8 +64,7 @@ cache-type-v = q8_0
6464
flash-attn = true
6565
reasoning = on
6666
jinja = true
67-
chat-template-file = vendor\Qwen-Fixed-Chat-Templates\qwen3.6\chat_template-v13.jinja
68-
chat-template-kwargs = {"preserve_thinking":true}
67+
chat-template-file = vendor\Qwen-Fixed-Chat-Templates\chat_template.jinja
6968
kv-unified = true
7069
ctx-checkpoints = 32
7170
cache-ram = 51200
@@ -95,8 +94,7 @@ cache-type-v = q8_0
9594
flash-attn = true
9695
reasoning = on
9796
jinja = true
98-
chat-template-file = vendor\Qwen-Fixed-Chat-Templates\qwen3.6\chat_template-v13.jinja
99-
chat-template-kwargs = {"preserve_thinking":true}
97+
chat-template-file = vendor\Qwen-Fixed-Chat-Templates\chat_template.jinja
10098
kv-unified = true
10199
ctx-checkpoints = 32
102100
cache-ram = 51200

vendor/Qwen-Fixed-Chat-Templates

Submodule Qwen-Fixed-Chat-Templates updated from 5983684 to c31fd39

0 commit comments

Comments
 (0)