Gemma4 think tags not detected

[LukeyGuy47]

Description:
Currently, when using Gemma 4 models with native thinking enabled, frontends like SillyTavern fail to stream the output correctly because the raw <|channel>thought tags are passed directly into the text stream, causing the frontend parsers to buffer the entire generation before rendering.

Upstream llama.cpp recently added support for the peg-gemma4 chat template, which natively translates Gemma 4's channel tags into the standard OpenAI reasoning_content field over the API. Porting this upstream handling to KoboldCpp would fix the streaming lock-up in compatible frontends.

Steps to Reproduce:

1. Load a Gemma 4 model (e.g., google_gemma-4-31B-it) in KoboldCpp v1.111.
2. Launch with the argument: --jinja_kwargs='{"enable_thinking":true}'
3. Connect a frontend like SillyTavern using the OpenAI-compatible API endpoint (/v1/chat/completions).
4. Generate a response.
5. Result: The backend generates tokens normally (streaming is active in the console), but the frontend UI freezes/buffers until the generation is 100% complete due to the custom <|channel> tags in the raw text block.

Expected Behavior:
The thoughts should be stripped from the main text stream and sent as reasoning_content in the API payload, allowing frontends to stream both the reasoning and the response seamlessly.

Additional Context:
Testing the exact same model and prompt directly in standard llama.cpp server results in flawless streaming. The llama.cpp logs show it correctly identifying the template and activating the reasoning budget, which KoboldCpp does not currently do:

Plaintext

srv params_from_: Chat format: peg-gemma4
init: chat template, thinking = 1
reasoning-budget: activated, budget=2147483647 tokens

Environment:

• OS: Windows 11
• GPU: RTX 4080 + RTX 3090 (Tensor Split)
• KoboldCpp Version: 1.111

[henk717]

We do not make use of their reasoning system so there is nothing to sync.

[henk717]

Re-opened since you also mentioned our current detect might not be working.
We still can't sync anything, but we can at least keep track of the tag issue on our side.

[LostRuins]

yup, looks like i just need to shift the reasoning_content for chat completions users. I'm assuming text completions is still pretty hosed due to the complex formatting required, correct?

[LostRuins]

Made a little guide for ST users. Note: This requires v1.111.1 patch release!

[henk717]

We should test / add instructions for Text Completions with the KoboldAI format as well, since thats a more popular use case for tavern and our chat adapters should ideally work (Unless they screw with the formatting to much).

[askmyteapot]

I added 2 text completion templates on the Sillytavern discord under https://discord.com/channels/1100685673633153084/1490291891672059944

[LukeyGuy47]

Amazing, thank you!

I actually haven't tested Text Completions myself since I exclusively use Chat Completions (for function calls). But your assumption sounds correct, because it just sends the raw string, the frontends trip over the <|channel> syntax and break the formatting.

Really appreciate you looking into the Chat Completions fix so quickly!

[LostRuins]

Hi all, the patch release 1.111.1 is out. Most issues should be fixed. Enjoy.

[Topsy9]

I tried the sillytavern guide, but it doesnt put the thinking in a thinking block like your picture.

Also it still tries to think when i leave the kwargs block empty.

Is there any way i can get those text completion templates? I usually use text completion anyway and I dont have access to the discord.

[askmyteapot]

Gemma4-31b-think-template.json
Gemma4-31b-nothink.json
@Topsy9 hope that helps. :) I forgot i can attach json to my comments (i forgot how to brain and was suffering from teh dum)

Some additional notes. You have to set the include names to never, and the reasoning format always has to be selected as Gemma4 always outputs its thinking tags.

[Topsy9]

@askmyteapot Thank you! they work perfectly.

[johnlabor]

Hi @LostRuins @henk717 , sorry to hijack this thread because I have so much confusions.

1. with 1.111.1 does that mean text completion would work, just buggy, so we are suggested to use the chat completion method (openai compatible v1) picture suggested by @LostRuins for now on silly tavern?
2. I assume jinja is only for chat completion, in my setup, I didnt use jinja, and simply write a system prompt ask gemma4 to write thoughts within <think></think> and use text completion, and somehow it did start thinking within <think> instead of <|channel>thought, is this expected?
3. Is there a general benefits of using text completion vs chat completion in ST? I have tried both but I am a new user, so didnt see much differences.

Appreciate if you can help with these.
Thanks!.