Skip to content

[bot] OpenAI Responses API streaming tracer ignores response.failed and response.incomplete terminal events #115

@braintrust-bot

Description

@braintrust-bot

Summary

The OpenAI Responses API streaming tracer only processes response.completed events. The two other terminal event types — response.failed and response.incomplete — are silently ignored. When a streaming response ends with failure or truncation, the span captures no output, no error status, and no usage metrics (only time_to_first_token).

What is missing

The OpenAI Responses API defines three terminal streaming events (docs):

Event When it fires Currently handled?
response.completed Generation succeeded Yes
response.failed Generation failed (content filter, server error, etc.) No
response.incomplete Generation truncated (e.g. max_output_tokens reached) No

In trace/contrib/openai/responses.go lines 132–147, only response.completed is matched:

if msgType == "response.completed" {
    // ...
    completedMsg = msg
}

When the stream ends with response.failed:

  • The response object in the event contains an error field with code and message
  • The status field is "failed"
  • None of this is captured — the span ends with no output, no error status, and no error details

When the stream ends with response.incomplete:

  • The response object contains incomplete_details explaining why (e.g. max_output_tokens)
  • The output array may contain partial results
  • The usage object is present with final token counts
  • None of this is captured — partial output and usage metrics are lost

By contrast, the non-streaming path (parseResponse) handles the full response body regardless of status, so it correctly captures failed/incomplete responses.

Braintrust docs status

not_found — The Braintrust OpenAI integration docs do not mention handling of response.failed or response.incomplete streaming events specifically.

Upstream sources

Local repo files inspected

  • trace/contrib/openai/responses.goparseStreamingResponse() at lines 109–162: only response.completed event type is handled
  • trace/contrib/openai/responses.gohandleResponseCompletedMessage() at lines 177–229: shared handler that would work for failed/incomplete if called
  • trace/contrib/openai/traceopenai.go — router and shared utilities
  • trace/internal/middleware.go — generic middleware handles HTTP-level errors but not stream-level API failures (streaming responses return HTTP 200)
  • trace/contrib/openai/chatcompletions.go — Chat Completions streaming for comparison (aggregates deltas, so partial output is always captured)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions