You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# Presence penalty coefficient, reducing the probability of generating new topics (unseen topics) (`>1.0` suppresses new topics, `<1.0` encourages new topics).
383
+
384
+
echo: Optional[bool] =False
385
+
# Whether to include the input prompt in the generated output (default: `False`, i.e., exclude the prompt).
386
+
387
+
n: Optional[int] =1
388
+
# Number of candidate outputs to generate (i.e., return multiple independent text completions). Default 1 (return only one result).
# Stop generation conditions - can be a single string or list of strings.
396
+
# Generation terminates when any stop string is produced (default empty list means disabled).
397
+
398
+
stream: Optional[bool] =False
399
+
# Whether to enable streaming output (return results token by token), default `False` (returns complete results at once).
400
+
401
+
stream_options: Optional[StreamOptions] =None
402
+
# Additional configurations for streaming output (such as chunk size, timeout, etc.), refer to the specific definition of `StreamOptions`.
403
+
404
+
temperature: Optional[float] =None
405
+
# Temperature coefficient, controlling generation randomness (`0.0` for deterministic generation, `>1.0` for more randomness, default `None` uses model default).
406
+
407
+
top_p: Optional[float] =None
408
+
# Nucleus sampling threshold, only retaining tokens whose cumulative probability exceeds `top_p` (default `None` disables).
# Directly passes the token ID list of the prompt, skipping the text encoding step (default None means using text input).
471
+
472
+
temp_scaled_logprobs: Optional[bool] =False
473
+
# Whether to divide the logits by the temperature coefficient when calculating logprobs (default is False, meaning the logits are not divided by the temperature coefficient).
474
+
475
+
top_p_normalized_logprobs: Optional[bool] =False
476
+
# Whether to perform top-p normalization when calculating logprobs (default is False, indicating that top-p normalization is not performed).
477
+
478
+
include_draft_logprobs: Optional[bool] =False
479
+
# Whether to return log probabilities during draft stages (e.g., pre-generation or intermediate steps)
480
+
# for debugging or analysis of the generation process (default False means not returned).
481
+
482
+
logits_processors_args: Optional[Dict] =None
483
+
# Additional arguments for logits processors, enabling customization of generation logic
484
+
# (e.g., dynamically adjusting probability distributions).
485
+
486
+
mm_hashes: Optional[list] =None
487
+
# Hash values for multimodal (e.g., image/audio) inputs, used for verification or tracking.
488
+
# Default None indicates no multimodal input or hash validation required.
0 commit comments