Instrument OpenAI structured outputs

[alexmojaki]

What problem do you want to solve?

Instrument https://platform.openai.com/docs/guides/structured-outputs?api-mode=chat

Currently this records nothing even when exporters are configured:

import os

import openai
from opentelemetry.instrumentation.openai_v2 import OpenAIInstrumentor
from pydantic import BaseModel

os.environ['OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT'] = 'true'

OpenAIInstrumentor().instrument()


class CalendarEvent(BaseModel):
    name: str
    date: str
    participants: list[str]


completion = openai.beta.chat.completions.parse(
    model='gpt-4o-2024-08-06',
    messages=[
        {'role': 'system', 'content': 'Extract the event information.'},
        {'role': 'user', 'content': 'Alice and Bob are going to a science fair on Friday.'},
    ],
    response_format=CalendarEvent,
)

print(completion.choices[0].message.parsed)

Describe the solution you'd like

openai.beta.chat.completions.parse should be instrumented just like openai.chat.completions.create is.

Describe alternatives you've considered

No response

Additional Context

No response

Would you like to implement a fix?

No

[savvaki]

This would be much appreciated. It is a VERY common practice now.

[dehanjl]

Bump. I feel like this would be especially useful considering it's not a beta feature anymore, so openai.chat.completions.parse instrumentation would be awesome.

[Nik-Reddy]

Hi @alexmojaki, I've opened a PR to address this: #4416.

As @dehanjl noted, parse() has graduated out of beta, openai.chat.completions.parse now works alongside openai.beta.chat.completions.parse, and both route through the same Completions/AsyncCompletions class. The PR patches parse() on that class, so both paths are covered.

What's implemented:

• Wraps both sync and async Completions.parse / AsyncCompletions.parse, reusing the existing chat completion instrumentation logic
• Version-guarded via hasattr check (requires openai >= 1.40.0, where parse() was introduced)
• Handles response_format being a Pydantic model class (Python type), records json_schema as the response format attribute instead of attempting to serialize the class
• 8 new tests (sync + async x with/without content capture x experimental/legacy semconv) with VCR cassettes
• All 84 existing chat completion tests continue to pass

Would appreciate a review. @savvaki @dehanjl, this should cover your use cases as well.