Skip to content

Commit 8bf4f4d

Browse files
linuxid10tclaude
andcommitted
chat: fix Laguna thinking=off false reasoning and </assistant> in output
Two bugs in the Laguna autoparser workaround: 1. With reasoning.start="" (delimiter-style), the streaming PEG parser speculatively captures all generated content as reasoning while waiting for </think>. When thinking is disabled and </think> never arrives, the stream ends with everything misclassified as reasoning, causing [Start thinking] to display for ordinary responses. Fix: check ctx.inputs.enable_thinking in analyze_reasoning::build_parser for the empty-start case. When thinking is off, return p.eps() so the parser skips reasoning entirely and treats all output as content. 2. </assistant> (the Laguna EOT token) appears as raw text in streaming output because the stop-word erase logic runs only on antiprompt entries. preserved_tokens only controls special-token decode; it does not strip text from the stream. Fix: add an additional_stops field to autoparser, populate it from the Laguna workaround with "</assistant>", propagate it through generate_parser into common_chat_params::additional_stops (which the server already routes into antiprompt via the "stop" JSON key), and apply it to task.params.antiprompt in cli.cpp so the CLI path also benefits. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent b4ae008 commit 8bf4f4d

4 files changed

Lines changed: 29 additions & 4 deletions

File tree

common/chat-auto-parser-generator.cpp

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,8 @@ common_chat_params peg_generator::generate_parser(const common_chat_template &
4747
data.generation_prompt = common_chat_template_generation_prompt(tmpl, inputs);
4848
data.format = COMMON_CHAT_FORMAT_PEG_NATIVE;
4949
data.preserved_tokens = autoparser.preserved_tokens;
50+
data.additional_stops.insert(data.additional_stops.end(),
51+
autoparser.additional_stops.begin(), autoparser.additional_stops.end());
5052

5153
std::string parser_generation_prompt = data.generation_prompt;
5254

@@ -164,7 +166,12 @@ common_peg_parser analyze_reasoning::build_parser(parser_build_context & ctx) co
164166
// Standard tag-based: optional(<think>reasoning</think>)
165167
return p.optional(p.optspace(start) + p.reasoning(p.until(trim_whitespace(end))) + p.optspace(end));
166168
}
167-
// Delimiter-style (empty start)
169+
// Delimiter-style (empty start): reasoning begins immediately at generation start.
170+
// Only applies when thinking is enabled; with thinking disabled the generated stream
171+
// contains no reasoning at all, so return eps to avoid misclassifying plain content.
172+
if (!ctx.inputs.enable_thinking) {
173+
return p.eps();
174+
}
168175
return p.optional(p.reasoning(p.until(trim_whitespace(end))) + p.optspace(end));
169176
}
170177
}

common/chat-auto-parser.h

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -388,6 +388,9 @@ struct autoparser {
388388
// Preserved tokens for tokenizer (union of all non-empty markers)
389389
std::vector<std::string> preserved_tokens;
390390

391+
// Extra stop sequences added to sampling antiprompt (strip from streaming output)
392+
std::vector<std::string> additional_stops;
393+
391394
autoparser() = default;
392395

393396
// Find the starting marker for the user message and assistant message

common/chat-diff-analyzer.cpp

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -173,11 +173,21 @@ static std::vector<std::function<void(const common_chat_template & tmpl, autopar
173173
LOG_DBG(ANSI_ORANGE "[Patch: JSON name/parameters tool instruction]\n" ANSI_RESET);
174174
}
175175
},
176-
// Laguna (poolside) - </assistant> is both the EOT token and the role-closing tag;
177-
// auto-detection leaves content.end empty so it never lands in preserved_tokens.
176+
// Laguna (poolside): <think> is prefilled in the generation prompt, so the model's
177+
// generated output starts with thinking content directly (no <think> marker in stream).
178+
// Auto-detection picks up <think> as reasoning.start from multi-turn history diffs,
179+
// which breaks the live parser since <think> is never in the generated token stream.
180+
// Fix: clear reasoning.start so the parser treats generation-start as reasoning-start
181+
// when enable_thinking=true; analyze_reasoning::build_parser skips reasoning entirely
182+
// when enable_thinking=false to avoid misclassifying plain content as reasoning.
183+
// Also: </assistant> is the EOT token and role-closing tag — add it to additional_stops
184+
// so it is erased from streaming output (both server API and CLI) before being sent.
178185
[](const common_chat_template & tmpl, autoparser & analysis) -> void {
179186
if (tmpl.src.find("laguna_glm_thinking_v5") != std::string::npos) {
180-
analysis.preserved_tokens.push_back("</assistant>");
187+
analysis.reasoning.start = "";
188+
analysis.reasoning.end = "</think>";
189+
analysis.reasoning.mode = reasoning_mode::TAG_BASED;
190+
analysis.additional_stops.push_back("</assistant>");
181191
LOG_DBG(ANSI_ORANGE "[Patch: Laguna]\n" ANSI_RESET);
182192
}
183193
},

tools/cli/cli.cpp

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -122,6 +122,11 @@ struct cli_context {
122122
common_tokenize(vocab, defaults.sampling.reasoning_budget_message + chat_params.thinking_end_tag, false, true);
123123
}
124124

125+
// Apply template-derived stop sequences (e.g. </assistant> for Laguna)
126+
for (const auto & stop : chat_params.additional_stops) {
127+
task.params.antiprompt.push_back(stop);
128+
}
129+
125130
rd.post_task({std::move(task)});
126131
}
127132

0 commit comments

Comments
 (0)