Skip to content

Commit ed93530

Browse files
authored
[BugFix] fix mtp reset bugs in rl (#7957)
1 parent de1c8bd commit ed93530

1 file changed

Lines changed: 4 additions & 0 deletions

File tree

fastdeploy/worker/input_batch.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1029,6 +1029,10 @@ def reset_model_inputs(self) -> None:
10291029
# NOTE(fix): These tensors are dynamically resized during runtime inference.
10301030
# Must recreate at full initial size to avoid CUDAGraph replay OOB access.
10311031
max_num_seqs = self.scheduler_config.max_num_seqs
1032+
if self.enable_mm and self.model_config.mm_max_tokens_per_item is None:
1033+
self.max_chunk_tokens = self.model_config.max_model_len
1034+
else:
1035+
self.max_chunk_tokens = self.fd_config.get_max_chunk_tokens(self.model_config.mm_max_tokens_per_item)
10321036
self.ids_remove_padding = paddle.full([max_num_seqs * self.max_chunk_tokens], 0, dtype="int64")
10331037
self.batch_id_per_token = paddle.full([max_num_seqs * self.max_chunk_tokens, 1], 0, dtype="int32")
10341038
self.cu_seqlens_q = paddle.full([max_num_seqs + 1], 0, dtype="int32")

0 commit comments

Comments
 (0)