Skip to content

Commit 0c75e65

Browse files
committed
add cloud vllm disable thinking (#908)
disable thinking for vllm_amd and vllm_cloud
1 parent 79ccef5 commit 0c75e65

1 file changed

Lines changed: 5 additions & 1 deletion

File tree

  • services/api/src/owl/utils

services/api/src/owl/utils/lm.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -774,7 +774,11 @@ def _prepare_hyperparams(
774774
"summary": reasoning_summary,
775775
}
776776
return
777-
elif ctx.inference_provider == OnPremProvider.VLLM:
777+
elif ctx.inference_provider in (
778+
OnPremProvider.VLLM,
779+
OnPremProvider.VLLM_AMD,
780+
CloudProvider.VLLM_CLOUD,
781+
):
778782
hyperparams["extra_body"] = {"chat_template_kwargs": {"enable_thinking": False}}
779783
return
780784
logger.warning(

0 commit comments

Comments
 (0)