Skip to content

Commit 37159cd

Browse files
committed
Fix BOS zeroing in TextLLMRunner::prefill for skipped inputs
Only zero num_bos/num_eos after actually processing a text input, not after skipping a non-text input. Previously, if a non-text input preceded a text input, the BOS would be lost. This PR was authored with the assistance of Claude.
1 parent e0ee4dc commit 37159cd

1 file changed

Lines changed: 2 additions & 2 deletions

File tree

extension/llm/runner/text_llm_runner.cpp

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -260,10 +260,10 @@ Error TextLLMRunner::prefill(
260260
std::vector<uint64_t> tokens = encode_res.get();
261261
auto prefill_res = text_prefiller_->prefill(tokens, pos_);
262262
ET_CHECK_OK_OR_RETURN_ERROR(prefill_res.error());
263+
num_bos = 0;
264+
num_eos = 0;
263265
}
264266
// Skip non-text inputs — text-only runner
265-
num_bos = 0;
266-
num_eos = 0;
267267
}
268268

269269
return Error::Ok;

0 commit comments

Comments
 (0)