Claude streaming: no terminal event on clean upstream EOF → clients report "Connection closed mid-response"

[sweetcornna]

Summary

On a streaming request whose Claude upstream closes the SSE connection cleanly (TCP/EOF, no error) without sending a terminal message_stop event, ClaudeExecutor.ExecuteStream forwards/translates whatever it received and then simply returns — emitting no terminal event. Downstream clients (Claude Code, Cherry Studio / AI SDK, etc.) then surface:

API Error: Connection closed mid-response. The response above may be incomplete.

Every other streaming executor guards against this; the Claude executor is the only one that does not.

Version

• CLIProxyAPI v7.2.36 (7712ffe).
• Also present on origin/main — the only commit after v7.2.36 is the pluginhost refactor (b53d1e9), which does not touch the streaming code paths.

Root cause

internal/runtime/executor/claude_executor.go, ExecuteStream:

• Claude→Claude passthrough branch (responseFormat == to, ~L545–573): scans upstream SSE and forwards each line. On clean EOF (scanner.Err() == nil) it just returns — no terminal event is synthesized.
• Claude→other translation branch (~L577–613): same — on clean EOF it returns without feeding any terminal marker to the translator.

Contrast with internal/runtime/executor/openai_compat_executor.go (~L442–454), which on clean EOF explicitly feeds a synthetic data: [DONE] through the translator:

} else {
    // In case the upstream close the stream without a terminal [DONE] marker.
    // Feed a synthetic done marker through the translator so pending
    // response.completed events are still emitted exactly once.
    chunks := sdktranslator.TranslateStream(ctx, to, responseFormat, req.Model, opts.OriginalRequest, translated, []byte("data: [DONE]"), &param)
    ...
}

The Claude executor has no equivalent guard, so a Claude upstream that ends a 200 SSE stream without message_stop (e.g. an intermediary/relay severing the connection, or an upstream that omits the terminal event) yields a silently-truncated stream rather than a clean terminator or an explicit error.

This is distinct from #3995 (request hangs forever with no timeout): here the stream does end, but without a terminal event.

Minimal reproduction (no credentials needed)

A unit test driving ExecuteStream against an httptest upstream that returns a 200 SSE body without message_stop:

func TestClaudeExecutor_ExecuteStreamEmitsMessageStopOnCleanEOFWithoutTerminal(t *testing.T) {
	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		w.Header().Set("Content-Type", "text/event-stream")
		// Upstream closes cleanly WITHOUT a terminal message_stop event.
		_, _ = w.Write([]byte("event: message_start\ndata: {\"type\":\"message_start\",\"message\":{\"id\":\"msg_1\",\"model\":\"claude-3-5-sonnet-20241022\"}}\n\n"))
		_, _ = w.Write([]byte("event: content_block_delta\ndata: {\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"hi\"}}\n\n"))
	}))
	defer server.Close()

	executor := NewClaudeExecutor(&config.Config{})
	auth := &cliproxyauth.Auth{Attributes: map[string]string{"api_key": "key-123", "base_url": server.URL}}
	payload := []byte(`{"messages":[{"role":"user","content":[{"type":"text","text":"hi"}]}]}`)

	result, err := executor.ExecuteStream(context.Background(), auth, cliproxyexecutor.Request{
		Model:   "claude-3-5-sonnet-20241022",
		Payload: payload,
	}, cliproxyexecutor.Options{SourceFormat: sdktranslator.FromString("claude")})
	if err != nil {
		t.Fatalf("ExecuteStream() error = %v", err)
	}
	var buf []byte
	for chunk := range result.Chunks {
		if chunk.Err != nil {
			t.Fatalf("unexpected chunk error: %v", chunk.Err)
		}
		buf = append(buf, chunk.Payload...)
	}
	// FAILS on v7.2.36: no message_stop is ever emitted.
	if !strings.Contains(string(buf), `"type":"message_stop"`) {
		t.Fatalf("expected a synthetic message_stop terminal event on clean EOF, got:\n%s", string(buf))
	}
}

Proposed fix

Mirror the openai_compat_executor guard in both branches of ClaudeExecutor.ExecuteStream, gated by a sawMessageStop flag so an upstream that already sent message_stop is never duplicated:

• Passthrough branch: on clean EOF without message_stop, emit a synthetic event: message_stop\ndata: {"type":"message_stop"}\n\n (leading blank line to guarantee SSE event separation if the upstream truncated mid-event).
• Translation branch: on clean EOF without message_stop, feed a synthetic Claude data: {"type":"message_stop"} line through sdktranslator.TranslateStream so the translator emits its proper terminal exactly once.

I have a working patch + passing tests against v7.2.36 and am happy to open a PR if the approach looks right.

[luispater]

Thanks for the detailed report and reproduction.

After reviewing the behavior, we do not want to synthesize a message_stop when the upstream Claude SSE stream reaches EOF without a proper terminal event. In that situation the upstream did not complete the response according to the protocol, so forwarding a successful terminal message_stop would incorrectly tell downstream clients that the response completed cleanly and could hide a truncated response.

Downstream clients reporting the stream as closed mid-response is therefore the safer and more accurate outcome for this case. If the upstream or an intermediary closes the stream before sending message_stop, that should remain visible as an incomplete stream rather than being converted into a successful completion.

Closing this as not planned.