Skip to content

fix: close Claude stream when final usage is estimated#4943

Open
mercy719 wants to merge 1 commit into
QuantumNous:mainfrom
mercy719:codex/fix-anthropic-stream-end
Open

fix: close Claude stream when final usage is estimated#4943
mercy719 wants to merge 1 commit into
QuantumNous:mainfrom
mercy719:codex/fix-anthropic-stream-end

Conversation

@mercy719
Copy link
Copy Markdown

@mercy719 mercy719 commented May 18, 2026

📝 变更描述 / Description

修复 OpenAI-compatible stream 转 Anthropic /v1/messages 时的结束事件丢失问题。

此前当上游最后一个 chunk 只有 finish_reason: "stop"、没有 usage 时,转换器会提前返回等待 usage-only chunk。对于已经在最终处理阶段写入本地估算 usage 的请求,这会跳过 content_block_stopmessage_deltamessage_stop,导致 Claude Code 等客户端一直等待流结束。

本次改动只在当前 chunk 和 ClaudeConvertInfo.Usage 都没有 usage 时继续等待;如果已有估算 usage,则正常关闭当前 content block 并输出 Anthropic 结束事件。

🚀 变更类型 / Type of change

  • 🐛 Bug 修复 (Bug fix) - 请关联对应 Issue,避免将设计取舍、理解偏差或预期不一致直接归类为 bug
  • ✨ 新功能 (New feature) - 重大特性建议先通过 Issue 沟通
  • ⚡ 性能优化 / 重构 (Refactor)
  • 📝 文档更新 (Documentation)

🔗 关联任务 / Related Issue

✅ 提交前检查项 / Checklist

  • 人工确认: 我已亲自整理并撰写此描述,没有直接粘贴未经处理的 AI 输出。
  • 非重复提交: 我已搜索现有的 IssuesPRs,确认不是重复提交。
  • Bug fix 说明: 若此 PR 标记为 Bug fix,我已提交或关联对应 Issue,且不会将设计取舍、预期不一致或理解偏差直接归类为 bug。
  • 变更理解: 我已理解这些更改的工作原理及可能影响。
  • 范围聚焦: 本 PR 未包含任何与当前任务无关的代码改动。
  • 本地验证: 已在本地运行并通过测试或手动验证,维护者可以据此复核结果。
  • 安全合规: 代码中无敏感凭据,且符合项目代码规范。

📸 运行证明 / Proof of Work

docker run --rm -v /Users/yimingshi/Documents/new-api:/workspace -w /workspace golang:1.25 go test ./service -run TestStreamResponseOpenAI2ClaudeClosesStopChunkWithoutUpstreamUsage -count=1
docker run --rm -v /Users/yimingshi/Documents/new-api:/workspace -w /workspace golang:1.25 go test ./service -count=1
docker run --rm -v /Users/yimingshi/Documents/new-api:/workspace -w /workspace golang:1.25 go test ./relay/channel/openai -run TestNonexistent -count=1

Summary by CodeRabbit

  • Bug Fixes

    • Prevented incorrect completion/usage emission when upstream usage data is missing, ensuring stop reasons and final message signals are handled correctly during stream conversion.
  • Tests

    • Added unit tests covering streaming behavior across reasoning, content, and finish-reason scenarios to validate correct stop-reason mapping, usage propagation, and finalization.

Review Change Stack

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 18, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 089fc861-fda9-411d-8118-7930127509fc

📥 Commits

Reviewing files that changed from the base of the PR and between 29d2466 and acb5574.

📒 Files selected for processing (2)
  • service/convert.go
  • service/convert_test.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • service/convert_test.go
  • service/convert.go

Walkthrough

Adds a nil-usage guard in StreamResponseOpenAI2Claude's done-chunk path to avoid emitting a final message_delta when no usage exists, and adds a unit test that verifies chunk ordering, stop-reason mapping, usage propagation, and Claude conversion completion state.

Changes

Stream Response Completion Handling

Layer / File(s) Summary
Guard for nil usage in done-chunk handling and validation test
service/convert.go, service/convert_test.go
Adds a conditional in the doneChunk path of StreamResponseOpenAI2Claude that detects when both openAIResponse.Usage and info.ClaudeConvertInfo.Usage are nil and prevents emitting the final message_delta/stop-reason. Adds TestStreamResponseOpenAI2ClaudeClosesStopChunkWithoutUpstreamUsage verifying chunk ordering (reasoning/content/stop), index expectations, mapping of FinishReasonStop"end_turn", propagation of usage from ClaudeConvertInfo, and setting info.ClaudeConvertInfo.Done=true on completion.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • QuantumNous/new-api#2887: Addresses nil-usage and patching missing usage fields before downstream message_delta processing.
  • QuantumNous/new-api#4090: Modifies the same StreamResponseOpenAI2Claude done-chunk handling and adds tests for missing upstream usage.
  • QuantumNous/new-api#1522: Adjusts finish-stop handling and stop-reason propagation in StreamResponseOpenAI2Claude.

Poem

🐰 I bounced through streams to find the end,
Guarded the gap where usage might bend,
Mapped stop to "end_turn" with a careful hop,
Now chunks close cleanly — no more drop-stop! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: fixing the stream closure by handling estimated usage in the OpenAI-to-Claude conversion logic when final chunks lack upstream usage data.
Linked Issues check ✅ Passed The PR implements the solution to issue #4697 by adding a guard that ensures the converter continues processing and emits required Anthropic SSE end events (content_block_stop, message_delta, message_stop) when estimated usage exists, preventing clients from hanging.
Out of Scope Changes check ✅ Passed All changes are directly scoped to fixing the identified issue: the production code change adds a usage check guard in the conversion logic, and the test case validates the specific scenario of closing a stop chunk without upstream usage.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@mercy719 mercy719 force-pushed the codex/fix-anthropic-stream-end branch from 29d2466 to acb5574 Compare May 18, 2026 11:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: /v1/messages streaming for qwen3.6-plus lacks content_block_stop/message_delta/message_stop, causing Claude Code to hang

1 participant