fix(openai-v2): Ports the fix from upstream opentelemetry-python-contrib PR #4184 (fixes upstream issue #4113) (#331)

adityamehra · cursoragent · web-flow · commit f8269ada02d2 · 2026-05-19T13:24:00.000-07:00
* feat(langchain): Release version 0.1.14

* fix(openai-v2): add __getattr__ to StreamWrapper to proxy unknown attributes

StreamWrapper was missing __getattr__, causing AttributeError when
LiteLLM (and other clients) access raw_response.headers after calling
with_raw_response.create(stream=True).

Ports the fix from upstream opentelemetry-python-contrib PR #4184
(fixes issue #4113).

Co-authored-by: Cursor &lt;cursoragent@cursor.com&gt;

* docs(openai-v2): add changelog entry for StreamWrapper __getattr__ fix

Co-authored-by: Cursor &lt;cursoragent@cursor.com&gt;

* fix(openai-v2): preserve LegacyAPIResponse headers on StreamWrapper for with_raw_response streaming

When LiteLLM calls with_raw_response.create(stream=True), the OpenAI SDK
returns a LegacyAPIResponse (with .headers). SDOT's _parse_response was
calling .parse() on it to get the AsyncStream before wrapping in StreamWrapper,
discarding the headers. LiteLLM then accesses raw_response.headers, which
failed with AttributeError since AsyncStream has no .headers attribute.

Fix:
- Capture LegacyAPIResponse.headers before _parse_response discards it
- Store captured headers as StreamWrapper.headers (direct attribute, not proxied)
- Add parse() returning self so callers can treat StreamWrapper as a raw response
- Keep __getattr__ for any other unknown attribute proxying to the underlying stream

Co-authored-by: Cursor &lt;cursoragent@cursor.com&gt;

* style: apply ruff formatting to test_patch_unit.py

Co-authored-by: Cursor &lt;cursoragent@cursor.com&gt;

* test(openai-v2): add standalone reproducer for with_raw_response streaming headers bug

Reproduces the production error reported in lab0:
  AttributeError: 'StreamWrapper' object has no attribute 'headers'

Mirrors the pattern from upstream issues:
  #4032 - StreamWrapper missing .parse()
  #4113 - StreamWrapper missing .headers

Co-authored-by: Cursor &lt;cursoragent@cursor.com&gt;

* test(openai-v2): use make_azure_openai_chat_completion_request verbatim in reproducer

Co-authored-by: Cursor &lt;cursoragent@cursor.com&gt;

* chore(openai-v2): move reproducer script to examples/scripts/

Co-authored-by: Cursor &lt;cursoragent@cursor.com&gt;

* fix(openai-v2): remove unused time import from reproducer

Co-authored-by: Cursor &lt;cursoragent@cursor.com&gt;

---------

Co-authored-by: Cursor &lt;cursoragent@cursor.com&gt;
diff --git a/instrumentation-genai/opentelemetry-instrumentation-openai-v2/CHANGELOG.md b/instrumentation-genai/opentelemetry-instrumentation-openai-v2/CHANGELOG.md
@@ -7,6 +7,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## Unreleased
 
+### Fixed
+
+- Fix `AttributeError: 'StreamWrapper' object has no attribute 'headers'` when
+  using `with_raw_response.create(stream=True)` (e.g. via LiteLLM's Azure provider).
+  `_parse_response` was calling `.parse()` on the `LegacyAPIResponse` before wrapping
+  in `StreamWrapper`, discarding the raw HTTP headers. `StreamWrapper` now captures
+  headers from the `LegacyAPIResponse` before it is parsed and exposes them directly,
+  and adds a `parse()` method returning `self` so callers can treat the wrapper as
+  a drop-in for the raw response. Also adds `__getattr__` to proxy any other unknown
+  attributes to the underlying stream. Inspired by upstream fix
+  ([opentelemetry-python-contrib#4184](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/4184),
+  fixes [#4113](https://github.com/open-telemetry/opentelemetry-python-contrib/issues/4113)).
+
 ### Added
 
 - Add `gen_ai.tool.definitions` attribute on LLM spans when
diff --git a/instrumentation-genai/opentelemetry-instrumentation-openai-v2/examples/scripts/reproduce_raw_response_streaming.py b/instrumentation-genai/opentelemetry-instrumentation-openai-v2/examples/scripts/reproduce_raw_response_streaming.py
@@ -0,0 +1,168 @@
+"""
+Reproducer for: StreamWrapper missing .headers when LiteLLM calls
+with_raw_response.create(stream=True) on an SDOT-instrumented Azure OpenAI client.
+
+Production traceback (lab0, 2026-05-15):
+  File "litellm/llms/azure/azure.py", line 619, in async_streaming
+    headers, response = await self.make_azure_openai_chat_completion_request(...)
+  File "litellm/llms/azure/azure.py", line 176, in make_azure_openai_chat_completion_request
+    headers = dict(raw_response.headers)
+  AttributeError: 'StreamWrapper' object has no attribute 'headers'
+
+This reproducer calls make_azure_openai_chat_completion_request verbatim to
+confirm the fix in SDOT's StreamWrapper resolves the crash.
+
+Related upstream issues:
+  #4032 - StreamWrapper missing .parse()  (fixed upstream)
+  #4113 - StreamWrapper missing .headers  (fixed upstream via __getattr__,
+           but SDOT needed a deeper fix: preserve LegacyAPIResponse.headers
+           before _parse_response discards it)
+
+Run:
+  pip install openai litellm opentelemetry-sdk splunk-otel-instrumentation-openai
+  python reproduce_raw_response_streaming.py
+"""
+
+import asyncio
+from unittest.mock import AsyncMock, MagicMock, patch
+
+import httpx
+from openai import AsyncAzureOpenAI
+
+from opentelemetry.instrumentation.openai_v2 import OpenAIInstrumentor
+from opentelemetry.sdk.trace import TracerProvider
+from opentelemetry.sdk.trace.export import SimpleSpanProcessor
+from opentelemetry.sdk.trace.export.in_memory_span_exporter import (
+    InMemorySpanExporter,
+)
+
+# ---------------------------------------------------------------------------
+# Minimal SSE streaming chunks that mimic Azure OpenAI
+# ---------------------------------------------------------------------------
+SSE_CHUNKS = [
+    b'data: {"id":"chatcmpl-123","object":"chat.completion.chunk","model":"gpt-4o","choices":[{"index":0,"delta":{"role":"assistant","content":"Hello"},"finish_reason":null}]}\n\n',
+    b'data: {"id":"chatcmpl-123","object":"chat.completion.chunk","model":"gpt-4o","choices":[{"index":0,"delta":{"content":"!"},"finish_reason":"stop"}]}\n\n',
+    b"data: [DONE]\n\n",
+]
+
+
+def _make_mock_httpx_response() -> httpx.Response:
+    """Build a fake httpx.Response that the OpenAI SDK treats as a raw streaming response.
+
+    The request must carry X-Stainless-Raw-Response: true so the OpenAI SDK
+    returns LegacyAPIResponse (sync .parse()) rather than AsyncAPIResponse
+    (async .parse()). SDOT's _parse_response calls .parse() synchronously.
+    """
+    response_headers = {
+        "content-type": "text/event-stream",
+        "x-request-id": "test-request-id-abc123",
+        "openai-model": "gpt-4o",
+        "ms-azureml-model-session": "d0",
+    }
+
+    async def aiter_bytes(_chunk_size=None):
+        for chunk in SSE_CHUNKS:
+            yield chunk
+
+    mock_request = MagicMock(spec=httpx.Request)
+    # RAW_RESPONSE_HEADER value that async_to_raw_response_wrapper injects
+    mock_request.headers = httpx.Headers({"X-Stainless-Raw-Response": "true"})
+
+    mock_response = MagicMock(spec=httpx.Response)
+    mock_response.status_code = 200
+    mock_response.headers = httpx.Headers(response_headers)
+    mock_response.aiter_bytes = aiter_bytes
+    mock_response.aclose = AsyncMock()
+    mock_response.request = mock_request
+    mock_response.http_version = "HTTP/1.1"
+    mock_response.elapsed = MagicMock()
+    return mock_response
+
+
+# ---------------------------------------------------------------------------
+# Verbatim copy of LiteLLM's make_azure_openai_chat_completion_request
+# (litellm/llms/azure/azure.py lines 154-179)
+# ---------------------------------------------------------------------------
+async def make_azure_openai_chat_completion_request(
+    azure_client, data, timeout
+):
+    """
+    Helper to:
+    - call chat.completions.create.with_raw_response when litellm.return_response_headers is True
+    - call chat.completions.create by default
+    """
+    try:
+        raw_response = (
+            await azure_client.chat.completions.with_raw_response.create(
+                **data, timeout=timeout
+            )
+        )
+
+        headers = dict(raw_response.headers)
+        response = raw_response.parse()
+        return headers, response
+    except Exception as e:
+        raise e
+
+
+# ---------------------------------------------------------------------------
+# Reproducer
+# ---------------------------------------------------------------------------
+async def reproducer():
+    exporter = InMemorySpanExporter()
+    provider = TracerProvider()
+    provider.add_span_processor(SimpleSpanProcessor(exporter))
+
+    instrumentor = OpenAIInstrumentor()
+    instrumentor.instrument(tracer_provider=provider)
+
+    azure_client = AsyncAzureOpenAI(
+        api_key="test-key",
+        azure_endpoint="https://test.openai.azure.com",
+        api_version="2024-02-15-preview",
+    )
+
+    data = {
+        "model": "gpt-4o",
+        "messages": [{"role": "user", "content": "Say hello"}],
+        "max_tokens": 10,
+        "stream": True,
+    }
+
+    mock_httpx_response = _make_mock_httpx_response()
+
+    with patch.object(
+        azure_client._client,
+        "send",
+        new_callable=AsyncMock,
+        return_value=mock_httpx_response,
+    ):
+        # This is the exact call that crashed in production
+        headers, response = await make_azure_openai_chat_completion_request(
+            azure_client=azure_client,
+            data=data,
+            timeout=60.0,
+        )
+
+    print(f"✓ headers accessible: {dict(headers)}")
+    print(f"✓ response type: {type(response).__name__}")
+
+    collected = []
+    async for chunk in response:
+        for choice in chunk.choices:
+            if choice.delta.content:
+                collected.append(choice.delta.content)
+
+    print(f"✓ streamed content: {''.join(collected)!r}")
+
+    spans = exporter.get_finished_spans()
+    print(f"✓ OTel spans: {[s.name for s in spans]}")
+
+    instrumentor.uninstrument()
+    print(
+        "\nReproducer passed — 'StreamWrapper' has no attribute 'headers' is fixed."
+    )
+
+
+if __name__ == "__main__":
+    asyncio.run(reproducer())
diff --git a/instrumentation-genai/opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/patch.py b/instrumentation-genai/opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/patch.py
@@ -418,12 +418,14 @@ def traced_method(wrapped, instance, args, kwargs):
             raise
 
         try:
+            raw_headers = getattr(result, "headers", None)
             parsed_result = _parse_response(result)
             if is_streaming(kwargs):
                 return StreamWrapper(
                     parsed_result,
                     invocation,
                     handler,
+                    raw_headers=raw_headers,
                 )
 
             if span and span.is_recording():
@@ -471,12 +473,14 @@ async def traced_method(wrapped, instance, args, kwargs):
             raise
 
         try:
+            raw_headers = getattr(result, "headers", None)
             parsed_result = _parse_response(result)
             if is_streaming(kwargs):
                 return StreamWrapper(
                     parsed_result,
                     invocation,
                     handler,
+                    raw_headers=raw_headers,
                 )
 
             if span and span.is_recording():
@@ -755,11 +759,13 @@ def __init__(
         stream: Stream,
         invocation: LLMInvocation,
         handler,
+        raw_headers=None,
     ):
         self.stream = stream
         self.invocation = invocation
         self.span = getattr(invocation, "span", None)
         self.handler = handler
+        self.headers = raw_headers
         self.choice_buffers = []
         self.finish_reasons = []  # Instance-level to avoid cross-request contamination
         self._span_started = False
@@ -768,6 +774,15 @@ def __init__(
         self._first_chunk_processed = False
         self.setup()
 
+    def __getattr__(self, name: str):
+        return getattr(self.stream, name)
+
+    def parse(self):
+        """Proxy for with_raw_response callers (e.g. LiteLLM) that call
+        .parse() on the result of with_raw_response.create(stream=True).
+        Returns self so the caller can iterate the stream directly."""
+        return self
+
     def setup(self):
         if not self._span_started:
             self._span_started = True
diff --git a/instrumentation-genai/opentelemetry-instrumentation-openai-v2/tests/test_patch_unit.py b/instrumentation-genai/opentelemetry-instrumentation-openai-v2/tests/test_patch_unit.py
@@ -202,6 +202,55 @@ def test_time_to_first_chunk_only_captured_once(self):
             "time_to_first_chunk should not change after first chunk"
         )
 
+    def test_raw_response_headers_stored_from_legacy_api_response(self):
+        """Test the real production scenario: LiteLLM calls with_raw_response.create(stream=True),
+        which makes the OpenAI SDK return a LegacyAPIResponse. SDOT's _parse_response calls
+        .parse() on it to get the AsyncStream — discarding .headers. StreamWrapper must
+        preserve the headers captured before _parse_response so LiteLLM can access them.
+
+        Regression test for the AttributeError seen in production:
+          File "litellm/llms/azure/azure.py", line 176
+            headers = dict(raw_response.headers)
+          AttributeError: 'StreamWrapper' object has no attribute 'headers'
+        """
+        invocation = LLMInvocation(request_model="gpt-4o")
+        mock_stream = MagicMock(spec=[])  # AsyncStream has no .headers
+        mock_handler = MagicMock()
+        raw_headers = {
+            "content-type": "text/event-stream",
+            "x-request-id": "abc123",
+        }
+
+        wrapper = StreamWrapper(
+            stream=mock_stream,
+            invocation=invocation,
+            handler=mock_handler,
+            raw_headers=raw_headers,
+        )
+
+        # LiteLLM: headers = dict(raw_response.headers)
+        assert wrapper.headers == raw_headers
+        # LiteLLM: response = raw_response.parse()
+        assert wrapper.parse() is wrapper
+
+    def test_getattr_proxies_unknown_attributes_to_stream(self):
+        """Test that unknown attributes on StreamWrapper are proxied to the
+        underlying stream object.
+        Regression test for https://github.com/open-telemetry/opentelemetry-python-contrib/issues/4113
+        """
+        invocation = LLMInvocation(request_model="gpt-4o")
+        mock_stream = MagicMock()
+        mock_stream.some_custom_attr = "value"
+        mock_handler = MagicMock()
+
+        wrapper = StreamWrapper(
+            stream=mock_stream,
+            invocation=invocation,
+            handler=mock_handler,
+        )
+
+        assert wrapper.some_custom_attr == "value"
+
     def test_time_to_first_chunk_not_captured_without_start_time(self):
         """Test that time_to_first_chunk is not captured without _start_time."""
         invocation = LLMInvocation(request_model="gpt-4o")
