[bot] OpenAI Chat Completions streaming tracer does not aggregate `refusal` content from deltas

[braintrust-bot]

Summary

The OpenAI Chat Completions streaming tracer aggregates delta.content and delta.tool_calls from streamed chunks, but does not aggregate delta.refusal. When a model refuses to generate content (e.g. due to content policy), the refusal text is streamed via delta.refusal instead of delta.content. The current tracer drops this text entirely, producing an output with empty content and no refusal field.

The non-streaming path captures refusal correctly because it passes through the full choices array as-is.

What is missing

In trace/contrib/openai/chatcompletions.go, postprocessStreamingResults() (lines 170–275) processes these delta fields:

• delta.role — aggregated (line 191)
• delta.content — aggregated (line 202)
• delta.tool_calls — aggregated (lines 207–248)
• delta.refusal — not handled

When the model refuses, OpenAI sends chunks like:

{"choices": [{"delta": {"refusal": "I'm unable to help with that request."}}]}

Since delta.refusal is not accumulated, the final output at lines 263–274 contains "content": "" with no refusal field. The reason the model stopped (finish_reason: "stop") gives no indication that a refusal occurred.

The fix would be to accumulate delta.refusal the same way delta.content is accumulated, and include it in the output message object:

"refusal": refusalContent,  // alongside "content" and "tool_calls"

Braintrust docs status

not_found — The Braintrust OpenAI integration docs do not mention the refusal field specifically.

Upstream sources

• OpenAI Chat Completions streaming chunk object: platform.openai.com/docs/api-reference/chat/streaming — documents delta.refusal as a string field
• OpenAI Chat Completions create: platform.openai.com/docs/api-reference/chat/create — documents message.refusal in the response
• OpenAI Refusals documentation: platform.openai.com/docs/guides/structured-outputs#refusals — explains refusal behavior

Local repo files inspected

• trace/contrib/openai/chatcompletions.go — postprocessStreamingResults() lines 170–275: aggregates content and tool_calls but not refusal
• trace/contrib/openai/chatcompletions.go — handleChatCompletionResponse() lines 290–328: non-streaming path passes choices as-is (correctly includes refusal)
• trace/contrib/openai/chatcompletions_test.go — no test case for streaming refusal
• trace/contrib/openai/traceopenai.go — shared middleware and token parsing