Skip to content

Commit d22c215

Browse files
committed
server: clear checkpoints on context shift to prevent stale restore faults
After context shift, slot.prompt.checkpoints retained pre-shift positions. Subsequent checkpoint restore via llama_state_seq_set_data_ext used stale positions causing GPU memory access faults on multi-GPU ROCm setups.
1 parent 98cf181 commit d22c215

1 file changed

Lines changed: 1 addition & 0 deletions

File tree

tools/server/server-context.cpp

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2257,6 +2257,7 @@ struct server_context_impl {
22572257
slot.prompt.tokens.insert(new_tokens);
22582258
}
22592259

2260+
slot.prompt.checkpoints.clear();
22602261
slot.truncated = true;
22612262
}
22622263
}

0 commit comments

Comments
 (0)