Skip to content

Commit 7bcb62c

Browse files
committed
fix(vllm): structured outputs silently ignored on vLLM >= 0.23
vLLM >= 0.23 removed GuidedDecodingParams (now StructuredOutputsParams) and renamed the SamplingParams field guided_decoding -> structured_outputs. The import failed, HAS_GUIDED_DECODING became False, and the whole guided-decoding block was skipped, so response_format / grammar constraints were silently ignored. Adapt the existing request.Grammar path to the new class/field. Signed-off-by: pos-ei-don <1822533+pos-ei-don@users.noreply.github.com>
1 parent f648f07 commit 7bcb62c

1 file changed

Lines changed: 6 additions & 4 deletions

File tree

backend/python/vllm/backend.py

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -48,8 +48,10 @@
4848
except ImportError:
4949
HAS_REASONING_PARSERS = False
5050

51+
# vLLM >= 0.23 renamed GuidedDecodingParams -> StructuredOutputsParams and the
52+
# SamplingParams field guided_decoding -> structured_outputs.
5153
try:
52-
from vllm.sampling_params import GuidedDecodingParams
54+
from vllm.sampling_params import StructuredOutputsParams
5355
HAS_GUIDED_DECODING = True
5456
except ImportError:
5557
HAS_GUIDED_DECODING = False
@@ -536,13 +538,13 @@ async def _predict(self, request, context, streaming=False):
536538
if value not in (None, 0, [], False):
537539
setattr(sampling_params, param_field, value)
538540

539-
# Guided decoding: use Grammar field to pass JSON schema or BNF
541+
# Structured-output decoding: use Grammar field to pass JSON schema or BNF
540542
if HAS_GUIDED_DECODING and request.Grammar:
541543
try:
542544
json.loads(request.Grammar) # valid JSON = JSON schema
543-
sampling_params.guided_decoding = GuidedDecodingParams(json=request.Grammar)
545+
sampling_params.structured_outputs = StructuredOutputsParams(json=request.Grammar)
544546
except json.JSONDecodeError:
545-
sampling_params.guided_decoding = GuidedDecodingParams(grammar=request.Grammar)
547+
sampling_params.structured_outputs = StructuredOutputsParams(grammar=request.Grammar)
546548

547549
# Extract image paths and process images
548550
prompt = request.Prompt

0 commit comments

Comments
 (0)