Skip to content

Commit 159d093

Browse files
ngxsonggerganov
andauthored
server: fix non-bound n_discard value (ctx shifting) (#24786)
* server: fix non-bound n_discard value * Update tools/server/server-context.cpp Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
1 parent 5fd2dc2 commit 159d093

1 file changed

Lines changed: 4 additions & 1 deletion

File tree

tools/server/server-context.cpp

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2565,7 +2565,10 @@ struct server_context_impl {
25652565
n_keep = std::min(slot.n_ctx - 4, n_keep);
25662566

25672567
const int n_left = slot.prompt.n_tokens() - n_keep;
2568-
const int n_discard = slot.task->params.n_discard ? slot.task->params.n_discard : (n_left / 2);
2568+
int n_discard = slot.task->params.n_discard ? slot.task->params.n_discard : (n_left / 2);
2569+
2570+
// ref: https://github.com/ggml-org/llama.cpp/pull/24786
2571+
n_discard = std::clamp(n_discard, 0, std::max(0, n_left - 1));
25692572

25702573
SLT_WRN(slot, "slot context shift, n_keep = %d, n_left = %d, n_discard = %d\n", n_keep, n_left, n_discard);
25712574

0 commit comments

Comments
 (0)