Skip to content

Commit b25eddd

Browse files
committed
server: clear checkpoints on context shift to prevent stale restore faults
After context shift, slot.prompt.checkpoints retained pre-shift positions. Subsequent checkpoint restore via llama_state_seq_set_data_ext used stale positions causing GPU memory access faults on multi-GPU ROCm setups.
1 parent 16d66aa commit b25eddd

1 file changed

Lines changed: 1 addition & 0 deletions

File tree

tools/server/server-context.cpp

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2199,6 +2199,7 @@ struct server_context_impl {
21992199
slot.prompt.tokens.insert(new_tokens);
22002200
}
22012201

2202+
slot.prompt.checkpoints.clear();
22022203
slot.truncated = true;
22032204
}
22042205
}

0 commit comments

Comments
 (0)