Skip to content

Commit 52e5f0a

Browse files
authored
common : re-arm reasoning budget after DONE on new <think> (ggml-org#22323)
DONE state absorbs all tokens including a new start tag, causing any think blocks after the first to run unbudgeted. Observed on unsloth/Qwen3.6-27B-GGUF which interleaves multiple <think> blocks per response. Fixed by advancing start_matcher in DONE branch and re-arming to COUNTING with a fresh budget on match. Adds regression test (test-reasoning-budget: test 6).
1 parent f9f3365 commit 52e5f0a

2 files changed

Lines changed: 38 additions & 1 deletion

File tree

common/reasoning-budget.cpp

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -122,6 +122,20 @@ static void common_reasoning_budget_accept(struct llama_sampler * smpl, llama_to
122122
}
123123
break;
124124
case REASONING_BUDGET_DONE:
125+
// Re-arm on a new start tag: some models emit multiple <think> blocks
126+
// per response, and each should get a fresh budget window.
127+
if (ctx->start_matcher.advance(token)) {
128+
ctx->state = REASONING_BUDGET_COUNTING;
129+
ctx->remaining = ctx->budget;
130+
ctx->end_matcher.reset();
131+
LOG_INF("reasoning-budget: re-activated on new start tag, budget=%d tokens\n", ctx->budget);
132+
133+
if (ctx->remaining <= 0) {
134+
ctx->state = REASONING_BUDGET_FORCING;
135+
ctx->force_pos = 0;
136+
LOG_INF("reasoning-budget: budget=0, forcing immediately\n");
137+
}
138+
}
125139
break;
126140
}
127141
}

tests/test-reasoning-budget.cpp

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -227,7 +227,30 @@ int main(void) {
227227
3); // forcing continues through i=3
228228
}
229229

230-
printf("OK (5 tests passed)\n");
230+
// Test 6: Multi-block thinking. First block ends naturally at i=2, second
231+
// start tag at i=3 re-arms the budget, which then exhausts at i=5.
232+
// Regression: before this fix, DONE absorbed all subsequent tokens and a
233+
// second <think> block ran unbudgeted.
234+
// Flow: i=0 accept(100)->COUNTING rem=2; i=1 accept(50)->rem=1;
235+
// i=2 accept(101)->end_matcher matches, DONE;
236+
// i=3 accept(100)->re-arm, COUNTING rem=2;
237+
// i=4 accept(60)->rem=1; i=5 accept(61)->rem=0->FORCING;
238+
// i=6 apply()->forces token[0]=102, accept(62)->force_pos=1, stay FORCING;
239+
// i=7 apply()->forces token[1]=101, accept(63)->force_pos=2->DONE.
240+
{
241+
const std::vector<llama_token> start = {100};
242+
const std::vector<llama_token> end = {101};
243+
const std::vector<llama_token> forced = {102, 101};
244+
const std::vector<llama_token> sequence = {100, 50, 101, 100, 60, 61, 62, 63};
245+
246+
test_reasoning_budget("multi-block re-arms budget after DONE", sequence, start, end, forced,
247+
2, // budget of 2 tokens (per block)
248+
REASONING_BUDGET_IDLE,
249+
6, // forcing starts at i=6 (after second block exhausts at i=5)
250+
7); // forcing continues through i=7
251+
}
252+
253+
printf("OK (6 tests passed)\n");
231254

232255
printf("Testing UTF-8 boundary detection... ");
233256
test_utf8_boundary_detection();

0 commit comments

Comments
 (0)