Skip to content

Commit 587bff8

Browse files
windreamerCopilot
andauthored
Update lmdeploy/serve/openai/api_server.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent 1f6d748 commit 587bff8

1 file changed

Lines changed: 4 additions & 3 deletions

File tree

lmdeploy/serve/openai/api_server.py

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -409,10 +409,11 @@ async def chat_completions_v1(request: ChatCompletionRequest, raw_request: Reque
409409
schema_json = json.dumps(response_format, ensure_ascii=False)
410410
format_section = f'\n\n# Response Formats\n\n{schema_json}'
411411
try:
412-
messages = request.messages
413-
if isinstance(messages, str):
414-
messages += format_section
412+
if isinstance(request.messages, str):
413+
# For string prompts, append the format section directly to request.messages
414+
request.messages += format_section
415415
else:
416+
messages = request.messages
416417
for msg in messages:
417418
if msg['role'] == 'system':
418419
msg['content'] += format_section

0 commit comments

Comments
 (0)