Skip to content

fix: emit final response after stream parse fallback#8367

Closed
zouyonghe wants to merge 1 commit into
AstrBotDevs:masterfrom
zouyonghe:fix/openai-empty-stream-completion
Closed

fix: emit final response after stream parse fallback#8367
zouyonghe wants to merge 1 commit into
AstrBotDevs:masterfrom
zouyonghe:fix/openai-empty-stream-completion

Conversation

@zouyonghe

@zouyonghe zouyonghe commented May 27, 2026

Copy link
Copy Markdown
Member

Summary

  • preserve streamed text, reasoning, and usage while reading OpenAI-compatible streaming chunks
  • emit a final non-chunk assistant response from streamed content if final completion parsing fails
  • re-raise the final parsing error when no usable streamed output exists

Fixes #8366

Tests

  • uv run pytest tests/test_openai_source.py
  • uv run ruff check astrbot/core/provider/sources/openai_source.py tests/test_openai_source.py

Summary by Sourcery

Ensure OpenAI-compatible streaming queries emit a usable final assistant response when final completion parsing fails while preserving streamed output.

Bug Fixes:

  • Return a synthesized final assistant response from accumulated streaming chunks when final completion parsing fails but partial output exists.
  • Propagate final completion parsing errors when no usable streamed content was produced to avoid silent failures.

Enhancements:

  • Capture and aggregate streamed text, reasoning content, and usage metadata during OpenAI-compatible streaming responses for reuse in the final response.

Tests:

  • Add a streaming query test that verifies a final non-chunk assistant response is emitted when final completion parsing raises an empty model output error.

@dosubot dosubot Bot added size:S This PR changes 10-29 lines, ignoring generated files. area:provider The bug / feature is about AI Provider, Models, LLM Agent, LLM Agent Runner. labels May 27, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the OpenAI stream query logic to accumulate streamed text, reasoning parts, and usage. If an exception occurs during stream processing, the method now yields a final fallback LLMResponse containing the accumulated content instead of silently returning. A unit test has been added to verify this behavior. The reviewer suggests initializing result_chain with a MessageChain in the fallback LLMResponse to avoid potential inconsistencies or errors in downstream handlers that expect a valid MessageChain object.

Comment on lines +723 to +728
yield LLMResponse(
"assistant",
completion_text="".join(streamed_text_parts),
reasoning_content="".join(streamed_reasoning_parts) or None,
usage=latest_usage,
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In AstrBot, LLMResponse objects typically expect result_chain to be populated with a MessageChain when text content is present (as seen in _parse_openai_completion and the chunk-yielding logic). Yielding an LLMResponse with result_chain=None and only completion_text set can cause inconsistency or potential AttributeErrors in downstream handlers that expect a valid MessageChain object.

It is highly recommended to initialize result_chain using MessageChain().message(...) when yielding the fallback response.

Suggested change
yield LLMResponse(
"assistant",
completion_text="".join(streamed_text_parts),
reasoning_content="".join(streamed_reasoning_parts) or None,
usage=latest_usage,
)
yield LLMResponse(
"assistant",
result_chain=MessageChain().message("".join(streamed_text_parts)) if streamed_text_parts else None,
reasoning_content="".join(streamed_reasoning_parts) or None,
usage=latest_usage,
)

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • In the get_final_completion error handler, consider using logger.exception instead of logger.error so the traceback is preserved when re-raising the parsing error and debugging downstream failures is easier.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In the `get_final_completion` error handler, consider using `logger.exception` instead of `logger.error` so the traceback is preserved when re-raising the parsing error and debugging downstream failures is easier.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@zouyonghe

Copy link
Copy Markdown
Member Author

Closing this PR because the root cause was misattributed. The observed failure is in the Multica runtime completion/comment-delivery contract, not in AstrBot's OpenAI provider.

@zouyonghe zouyonghe closed this May 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:provider The bug / feature is about AI Provider, Models, LLM Agent, LLM Agent Runner. size:S This PR changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OpenAI-compatible streaming can finish without final assistant response

1 participant