Skip to content

Instrument OpenAI structured outputs (chat completions parse) #12

@Nik-Reddy

Description

@Nik-Reddy

Problem

The OpenAI v2 instrumentation wraps chat.completions.create() but not the parse helper used for structured outputs. When developers use parse for structured responses, no telemetry is generated even when instrumentation is enabled.

Originally tracked in opentelemetry-python-contrib#3449 with a previous implementation in opentelemetry-python-contrib#4416 (approved by @lzchen, @lmolkova). That PR was closed when GenAI instrumentations moved to this repository.

Context

The parse helper returns ParsedChatCompletion, which carries the same response fields the existing chat completion instrumentation reads (model, usage, choices, finish_reason). The additional handling needed is detecting when response_format is a Python type rather than a string or dict, since structured outputs typically pass a Pydantic model class directly.

Proposed solution

  1. Add a version guard (_is_parse_supported) since parse was introduced in openai >= 1.40.0, while the instrumentation supports >= 1.26.0.
  2. Wrap the parse helper for both supported SDK layouts (beta.chat.completions.parse in older versions, chat.completions.parse in current versions), along with the async equivalents.
  3. Handle response_format being a Python type (Pydantic model class) by recording gen_ai.output.type = json via the generated semconv constant.
  4. Add tests covering sync and async parse calls across supported semconv paths, with and without content capture.

Acceptance criteria

  • Sync and async parse calls produce spans with the same attributes as create() calls
  • gen_ai.output.type is set to json when response_format is a Pydantic model class
  • No telemetry regression for existing create() calls
  • Tests pass on both old and new semconv paths

cc @lmolkova

References

I have a previous implementation from opentelemetry-python-contrib#4416 ready to port.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions