[bot] OpenAI Chat Completions tracer does not capture `web_search_options` or aggregate streaming URL citation annotations

[braintrust-bot]

Summary

The OpenAI Chat Completions tracer does not capture the web_search_options request parameter in metadata and does not aggregate URL citation annotations from streaming response chunks. When using web-search-enabled models (e.g. gpt-4o-search-preview), the trace is missing the search configuration (including user location context) and citation annotations are dropped from streaming output.

The non-streaming path preserves citation annotations correctly because it passes the full choices array as-is. The web_search_options parameter is missing from metadata in both paths.

What is missing

1. web_search_options not captured in request metadata

In trace/contrib/openai/chatcompletions.go (lines 51–76), the metadata fields list does not include web_search_options:

metadataFields := []string{
    "model",
    "frequency_penalty",
    "logit_bias",
    // ... other fields ...
    "tools",
    "tool_choice",
    "parallel_tool_calls",
    "user",
    "functions",
    "function_call",
}

When a user enables web search grounding with web_search_options: { user_location: { type: "approximate", country: "US", city: "San Francisco" } }, this configuration is silently dropped from the span metadata. Users cannot see whether web search was enabled or what location context was provided.

2. Streaming aggregation drops annotations

In postprocessStreamingResults() (lines 263–274), the aggregated output only includes role, content, tool_calls, logprobs, and finish_reason:

return []map[string]interface{}{
    {
        "index": 0,
        "message": map[string]interface{}{
            "role":       finalRole,
            "content":    content,
            "tool_calls": finalToolCalls,
        },
        "logprobs":      nil,
        "finish_reason": finishReason,
    },
}

URL citation annotations (containing url_citation objects with url, title, start_index, end_index) are not aggregated from streaming chunks and not included in the final output.

3. Non-streaming path works correctly

handleChatCompletionResponse() (lines 290–328) passes the full choices array to braintrust.output_json, preserving message.annotations including URL citations.

4. Why this matters

Web search grounding is a significant generative capability that fundamentally changes model output — responses are grounded in real-time web search results rather than training data alone. Not capturing web_search_options means:

• Users can't see whether a response was web-grounded or not
• Location context that affects search results is invisible
• Citation annotations (which OpenAI requires to be "clearly visible and clickable in your user interface") are lost in streaming traces, making it impossible to verify citation compliance

5. prediction parameter also not captured

The prediction request parameter (for Predicted Outputs) is similarly absent from the metadata fields list. While less impactful than web search, it controls generative behavior by providing a prediction hint that speeds up responses when most output tokens are known.

Braintrust docs status

Braintrust's OpenAI integration documentation does not mention Chat Completions web search or the web_search_options parameter. The Responses API supports web search via built-in tools, but the Chat Completions web search is a separate mechanism with different models. Status: not_found.

Upstream sources

• OpenAI web search in Chat Completions: https://developers.openai.com/api/docs/guides/tools-web-search — documents web_search_options parameter and url_citation annotations
• OpenAI Chat Completions API reference: documents web_search_options parameter with user_location (type, country, city, region)
• OpenAI Predicted Outputs guide: https://developers.openai.com/api/docs/guides/predicted-outputs — documents prediction parameter
• Web search models: gpt-4o-search-preview, gpt-4o-mini-search-preview
• openai-go SDK defines ChatCompletionNewParams.WebSearchOptions and ChatCompletionMessage.Annotations

Braintrust docs sources

• https://www.braintrust.dev/docs/integrations/ai-providers/openai (no mention of Chat Completions web search)
• https://www.braintrust.dev/docs/instrument/trace-llm-calls (general tracing, no web search specifics)

Local repo files inspected

• trace/contrib/openai/chatcompletions.go — metadata fields (lines 51–76): web_search_options and prediction absent; postprocessStreamingResults() (lines 263–274): does not include annotations in aggregated output
• trace/contrib/openai/chatcompletions.go — handleChatCompletionResponse() (lines 290–328): non-streaming path passes full choices preserving annotations
• trace/contrib/openai/traceopenai_test.go — no tests for web search models
• examples/internal/openai-v2/main.go — no web search example