Skip to content

fix: handle GPT-5 empty content response gracefully#7410

Open
he-yufeng wants to merge 3 commits intoAstrBotDevs:masterfrom
he-yufeng:fix/gpt5-empty-content-handling
Open

fix: handle GPT-5 empty content response gracefully#7410
he-yufeng wants to merge 3 commits intoAstrBotDevs:masterfrom
he-yufeng:fix/gpt5-empty-content-handling

Conversation

@he-yufeng
Copy link
Copy Markdown
Contributor

@he-yufeng he-yufeng commented Apr 7, 2026

Fixes #7409

问题

GPT-5 系列模型(gpt-5.3-codex、gpt-5.4)通过 Chat Completions API 返回的响应中 content=Nonereasoning_content=None,但 finish_reason="stop" 且有 completion_tokens > 0。这些模型将 token 用于内部推理,不会在常规字段中暴露内容。

之前所有空输出响应都会触发 EmptyModelOutputError,导致重试循环反复失败。

修复

finish_reason == "stop" 时,空输出不再视为错误。改为记录 warning 并返回空 result chain,避免无意义的重试。只有 finish_reason"stop"(异常终止)时才抛出 EmptyModelOutputError

改动

  • openai_source.py: 在空输出检查中区分正常完成(finish_reason="stop")和异常情况

Summary by Sourcery

Handle OpenAI chat completions that have empty content but a normal stop finish_reason without treating them as errors.

Bug Fixes:

  • Avoid raising EmptyModelOutputError when GPT-5-style models return empty content with finish_reason=stop by treating these responses as successful but contentless completions.

Tests:

  • Add tests ensuring empty outputs with non-stop finish_reason still raise EmptyModelOutputError and that empty outputs with finish_reason=stop are handled gracefully and produce an assistant response.

…tputError

GPT-5 series models (gpt-5.3-codex, gpt-5.4) can return responses with
content=None and reasoning_content=None while still finishing normally
(finish_reason="stop"). This happens because these models perform
internal reasoning that doesn't surface in the Chat Completions response
fields.

Previously, all empty-output responses raised EmptyModelOutputError,
triggering retry loops that always fail for this model behavior. Now we
only raise the error when finish_reason is not "stop" -- a legitimate
abnormal termination. Normal completions with no visible content log a
warning and return an empty result chain instead.

Fixes AstrBotDevs#7409
@dosubot dosubot bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Apr 7, 2026
Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue, and left some high level feedback:

  • The special-casing of choice.finish_reason == "stop" for empty outputs is reasonable, but it might be safer to centralize/constant-ize the set of allowed 'normal completion' reasons so future finish_reason variants (or capitalization changes) don’t accidentally fall back into the error path.
  • When creating a MessageChain().message("") for the empty visible-content case, consider documenting or encapsulating this as a helper (e.g. MessageChain.empty()), so downstream consumers clearly understand that this represents a deliberate, model-driven empty output rather than a bug.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The special-casing of `choice.finish_reason == "stop"` for empty outputs is reasonable, but it might be safer to centralize/constant-ize the set of allowed 'normal completion' reasons so future finish_reason variants (or capitalization changes) don’t accidentally fall back into the error path.
- When creating a `MessageChain().message("")` for the empty visible-content case, consider documenting or encapsulating this as a helper (e.g. `MessageChain.empty()`), so downstream consumers clearly understand that this represents a deliberate, model-driven empty output rather than a bug.

## Individual Comments

### Comment 1
<location path="astrbot/core/provider/sources/openai_source.py" line_range="896-900" />
<code_context>
+            # tokens into internal reasoning, leaving content/reasoning_content
+            # both None.  When finish_reason is "stop" this is not an error.
+            if choice.finish_reason == "stop":
+                logger.warning(
+                    f"OpenAI completion returned no visible content "
+                    f"(response_id={completion.id}, model={completion.model}). "
+                    f"The model may have used internal reasoning only."
+                )
+                if not llm_response.result_chain:
</code_context>
<issue_to_address>
**suggestion:** Expand the warning metadata to help diagnose empty responses in production.

Given how unexpected this state is, consider adding `choice.index`, `choice.finish_reason`, and a fixed marker (e.g., "internal_reasoning_only") to the warning. This will make production log analysis and correlation with multi-choice/multi-model issues much easier.

```suggestion
                logger.warning(
                    "OpenAI completion returned no visible content "
                    f"(response_id={completion.id}, model={completion.model}, "
                    f"choice_index={choice.index}, finish_reason={choice.finish_reason}, "
                    "marker=internal_reasoning_only). "
                    "The model may have used internal reasoning only."
                )
```
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@dosubot dosubot bot added the area:provider The bug / feature is about AI Provider, Models, LLM Agent, LLM Agent Runner. label Apr 7, 2026
@dosubot
Copy link
Copy Markdown

dosubot bot commented Apr 7, 2026

Related Documentation

1 document(s) may need updating based on files changed in this PR:

AstrBotTeam's Space

pr4697的改动
View Suggested Changes
@@ -981,6 +981,7 @@
 新增 `EmptyModelOutputError` 异常(位于 `astrbot/core/exceptions.py`),用于显式标识提供商返回的无效空响应:
 
 - **触发条件**:当提供商返回的响应既无文本内容、又无推理内容(reasoning content)、也无工具调用时抛出
+- **OpenAI 提供商行为增强(PR #7410)**:[PR #7410](https://github.com/AstrBotDevs/AstrBot/pull/7410) 优化了 OpenAI 提供商的空输出检测逻辑。现在仅在 `finish_reason` 不为 `"stop"` 时才抛出 `EmptyModelOutputError`。当 `finish_reason="stop"` 但输出为空时(如 GPT-5 系列模型将 token 用于内部推理而不暴露 content 或 reasoning_content),系统记录 warning 并返回空结果链,避免触发无意义的重试循环
 - **适用范围**:所有提供商(OpenAI、Anthropic、Gemini)均已集成此异常
 - **语义说明**:区分于网络错误、超时等其他错误类型,专门用于标识模型输出为空的情况
 

[Accept] [Decline]

Note: You must be authenticated to accept/decline updates.

How did I do? Any feedback?  Join Discord

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request modifies the OpenAI completion parsing logic to prevent EmptyModelOutputError when a model returns no visible content but has a 'stop' finish reason. In such cases, the system now logs a warning and ensures an empty message chain is initialized, accommodating models that may use internal reasoning without producing external output. I have no feedback to provide.

The existing test used finish_reason="stop" which now returns gracefully.
Change it to finish_reason="length" to keep testing the error path, and
add a new test verifying that finish_reason="stop" with empty content
returns a valid response (GPT-5 behavior).
@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:S This PR changes 10-29 lines, ignoring generated files. labels Apr 7, 2026
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
@zouyonghe
Copy link
Copy Markdown
Member

@sourcery-ai review

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue, and left some high level feedback:

  • When synthesizing an empty MessageChain for the finish_reason == 'stop' case, consider explicitly setting the role (e.g. MessageChain().message('', role='assistant')) so the fabricated message structurally matches standard assistant outputs and avoids any downstream role assumptions.
  • You might want to also gate the 'internal reasoning only' path on completion.usage.completion_tokens > 0 (or equivalent) so that truly degenerate responses with no tokens and finish_reason='stop' are still treated as errors rather than silently accepted.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- When synthesizing an empty `MessageChain` for the `finish_reason == 'stop'` case, consider explicitly setting the role (e.g. `MessageChain().message('', role='assistant')`) so the fabricated message structurally matches standard assistant outputs and avoids any downstream role assumptions.
- You might want to also gate the 'internal reasoning only' path on `completion.usage.completion_tokens > 0` (or equivalent) so that truly degenerate responses with no tokens and `finish_reason='stop'` are still treated as errors rather than silently accepted.

## Individual Comments

### Comment 1
<location path="astrbot/core/provider/sources/openai_source.py" line_range="892-895" />
<code_context>
             and not has_reasoning_output
             and not llm_response.tools_call_args
         ):
-            logger.error(f"OpenAI completion has no usable output: {completion}.")
-            raise EmptyModelOutputError(
-                "OpenAI completion has no usable output. "
-                f"response_id={completion.id}, finish_reason={choice.finish_reason}"
-            )
+            # Some models (e.g. GPT-5 series) complete normally but put all
</code_context>
<issue_to_address>
**🚨 suggestion (security):** Logging the full `completion` object in the error path may be heavy and could expose more data than necessary.

In the error branch, the full `completion` object is logged, which increases log volume and risks exposing sensitive payloads. Consider logging only key fields (e.g., response id, model, choice index, finish reason), and reserve full-object dumps for debug-level logs when strictly necessary.

Suggested implementation:

```python
            # Some models (e.g. GPT-5 series) complete normally but put all
            # tokens into internal reasoning, leaving content/reasoning_content
            # both None.  When finish_reason is "stop" this is not an error.
            if choice.finish_reason == "stop":
                logger.warning(
                    "OpenAI completion returned no visible content "
                    f"(response_id={completion.id}, model={completion.model}, "
                    f"choice_index={choice.index}, finish_reason={choice.finish_reason}, "
                    "marker=internal_reasoning_only). "
                    "The model may have used internal reasoning only."
                )
            else:
                # Log only key metadata at error level to avoid dumping the full completion.
                logger.error(
                    "OpenAI completion has no usable output. "
                    f"(response_id={completion.id}, model={completion.model}, "
                    f"choice_index={choice.index}, finish_reason={choice.finish_reason})"
                )
                # Full completion object is logged at debug level only, to limit log volume
                # and exposure of sensitive payload data.
                logger.debug("OpenAI completion with no usable output: %r", completion)
                raise EmptyModelOutputError(
                    "OpenAI completion has no usable output. "
                    f"response_id={completion.id}, finish_reason={choice.finish_reason}"
                )

```

If the surrounding code uses a different logging style (e.g., `logger.error("...", extra={...})` or structured logging helpers), you may want to adapt the new `logger.error` and `logger.debug` calls to match that convention. Also ensure `EmptyModelOutputError` is imported in this module if it is not already.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment on lines +892 to +895
# Some models (e.g. GPT-5 series) complete normally but put all
# tokens into internal reasoning, leaving content/reasoning_content
# both None. When finish_reason is "stop" this is not an error.
if choice.finish_reason == "stop":
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚨 suggestion (security): Logging the full completion object in the error path may be heavy and could expose more data than necessary.

In the error branch, the full completion object is logged, which increases log volume and risks exposing sensitive payloads. Consider logging only key fields (e.g., response id, model, choice index, finish reason), and reserve full-object dumps for debug-level logs when strictly necessary.

Suggested implementation:

            # Some models (e.g. GPT-5 series) complete normally but put all
            # tokens into internal reasoning, leaving content/reasoning_content
            # both None.  When finish_reason is "stop" this is not an error.
            if choice.finish_reason == "stop":
                logger.warning(
                    "OpenAI completion returned no visible content "
                    f"(response_id={completion.id}, model={completion.model}, "
                    f"choice_index={choice.index}, finish_reason={choice.finish_reason}, "
                    "marker=internal_reasoning_only). "
                    "The model may have used internal reasoning only."
                )
            else:
                # Log only key metadata at error level to avoid dumping the full completion.
                logger.error(
                    "OpenAI completion has no usable output. "
                    f"(response_id={completion.id}, model={completion.model}, "
                    f"choice_index={choice.index}, finish_reason={choice.finish_reason})"
                )
                # Full completion object is logged at debug level only, to limit log volume
                # and exposure of sensitive payload data.
                logger.debug("OpenAI completion with no usable output: %r", completion)
                raise EmptyModelOutputError(
                    "OpenAI completion has no usable output. "
                    f"response_id={completion.id}, finish_reason={choice.finish_reason}"
                )

If the surrounding code uses a different logging style (e.g., logger.error("...", extra={...}) or structured logging helpers), you may want to adapt the new logger.error and logger.debug calls to match that convention. Also ensure EmptyModelOutputError is imported in this module if it is not already.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:provider The bug / feature is about AI Provider, Models, LLM Agent, LLM Agent Runner. size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]v4.22.3 使用 GPT-5 系列推理模型时出现 "OpenAI completion has no usable output"

3 participants