Skip to content

Commit 0720f38

Browse files
[TRTLLM-12669][chore] simplify populate_sampling_params_for_one_model docstring
Signed-off-by: ZhaoyangWang <zhaoyangw@nvidia.com>
1 parent 969ae2b commit 0720f38

1 file changed

Lines changed: 3 additions & 11 deletions

File tree

tensorrt_llm/_torch/speculative/interface.py

Lines changed: 3 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -522,17 +522,9 @@ def populate_sampling_params_for_one_model(
522522
"""
523523
Set up topp/topk/temperatures for 1-model sampler.
524524
525-
Two-phase implementation:
526-
Phase 1: scan request sampling configs to compute skip_*/has_greedy
527-
flags and decide whether the batch is all-greedy. No per-token
528-
list construction.
529-
Phase 2 (only when is_all_greedy_sample=False): build per-token /
530-
per-request lists and issue 6 H->D copies into the sampling
531-
parameter buffers.
532-
533-
For all-greedy batches we skip phase 2 entirely (no list building, no
534-
H->D copies). This is the common case when callers do not pass
535-
temperature/top_k/top_p.
525+
Scans sampling configs to set skip_*/is_all_greedy_sample flags. When
526+
any request needs sampling, also builds per-token/per-request lists
527+
and copies them to GPU buffers; all-greedy batches skip this entirely.
536528
"""
537529
from tensorrt_llm._torch.pyexecutor.llm_request import LlmRequestState
538530
from tensorrt_llm.sampling_params import SamplingParams

0 commit comments

Comments
 (0)