Skip to content

docs: clarify OpenAI Python parse vs response_format guidance #2884

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jannikmaierhoefer wants to merge 1 commit into
base: main
Choose a base branch
from
+430 −423
Open

docs: clarify OpenAI Python parse vs response_format guidance #2884

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions content/guides/cookbook/integration_openai_structured_output.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ This setup is useful for applications where each step needs to be displayed sepa

(Example taken from [OpenAI cookbook](https://cookbook.openai.com/examples/structured_outputs_intro))

**Note:** While OpenAI also offer structured output parsing via its beta API (`client.beta.chat.completions.parse`), this approach currently does not allow setting Langfuse specific attributes such as `name`, `metadata`, `userId` etc. Please use the approach using `response_format` with the standard `client.chat.completions.create` as described below.
**Note on `parse` vs `response_format`:** The Langfuse OpenAI integration instruments both the stable `client.chat.completions.parse(...)` (available in `openai>=1.92.0`) and the legacy `client.beta.chat.completions.parse(...)` (for older SDK versions, where `parse` is re-routed to the stable method on newer SDKs). Both paths support Langfuse-specific attributes such as `name`, `metadata`, `langfuse_session_id`, etc. The `response_format` approach with `client.chat.completions.create` shown below works on all SDK versions and is useful if you cannot upgrade `openai`. For an example using the `parse` helper with a Pydantic model, see the section further down.


```python
Expand Down Expand Up @@ -190,7 +190,7 @@ You can now see the trace and the JSON schema in Langfuse.

## Alternative: Using the SDK `parse` helper

The new SDK version adds a `parse` helper, allowing you to use your own Pydantic model without defining a JSON schema.
OpenAI also offers a `parse` helper that lets you pass a Pydantic model directly via `response_format`. As of [`openai-python` v1.92.0](https://github.com/openai/openai-python/releases/tag/v1.92.0), `parse` is part of the stable API at `client.chat.completions.parse(...)` (it previously lived under `client.beta.chat.completions.parse`). Langfuse instruments both paths, so you can set Langfuse attributes (`name`, `metadata`, `langfuse_session_id`, …) on either one. Prefer the stable path in new code:


```python
Expand All @@ -205,13 +205,15 @@ class MathReasoning(BaseModel):
final_answer: str

def get_math_solution(question: str):
response = client.beta.chat.completions.parse(
# Stable API on openai>=1.92.0. On older SDKs, use client.beta.chat.completions.parse instead.
response = client.chat.completions.parse(
model=openai_model,
messages=[
{"role": "system", "content": math_tutor_prompt},
{"role": "user", "content": question},
],
response_format=MathReasoning,
name="math-tutor-parse",
)

return response.choices[0].message
Expand Down
51 changes: 42 additions & 9 deletions content/integrations/model-providers/openai-py.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -381,22 +381,28 @@

#### Structured Output

For **structured output parsing**, please use the `response_format` argument to `openai.chat.completions.create()` instead of the Beta API. This will allow you to set Langfuse attributes and metadata.
For **structured output parsing**, you have two fully instrumented options depending on your `openai` Python SDK version:

If you rely on parsing Pydantic defintions for your `response_format`, you may leverage the `type_to_response_format_param` utility function from the OpenAI Python SDK to convert the Pydantic definition to a `response_format` dictionary. This is the same function the OpenAI Beta API uses to convert Pydantic definitions to `response_format` dictionaries.
- **`openai>=1.92.0` (recommended):** use `client.chat.completions.parse(...)`. OpenAI graduated `parse` and `stream` out of beta in [v1.92.0](https://github.com/openai/openai-python/releases/tag/v1.92.0), and Langfuse wraps the stable `openai.resources.chat.completions.Completions.parse` (and the async variant). You can pass a Pydantic model directly via `response_format` and still set Langfuse attributes such as `name`, `metadata`, `langfuse_session_id`, etc.
- **`openai<1.92.0` (legacy):** the parse helper is only available under `client.beta.chat.completions.parse(...)`. Langfuse also wraps the beta path on these older versions, so attributes like `name` and `metadata` work there too.

Check warning on line 387 in content/integrations/model-providers/openai-py.mdx

View check run for this annotation

Claude / Claude Code Review

langfuse_session_id wording implies it is a direct kwarg 

The new note groups `name`, `metadata`, and `langfuse_session_id` together as 'Langfuse attributes' (line 386 and again in the cookbook .md/.ipynb), implying all three are direct kwargs of `chat.completions.parse(...)`. However, the same MDX file's 'Custom trace properties' table (lines 206-211) lists only `name`/`metadata`/`trace_id`/`parent_observation_id` as direct kwargs, and the 'Setting trace attributes' section (line 230) shows that `langfuse_session_id`, `langfuse_user_id`, and `langfuse
Comment on lines +386 to +387
Copy link
Copy Markdown

@claude claude Bot Apr 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 The new note groups name, metadata, and langfuse_session_id together as 'Langfuse attributes' (line 386 and again in the cookbook .md/.ipynb), implying all three are direct kwargs of chat.completions.parse(...). However, the same MDX file's 'Custom trace properties' table (lines 206-211) lists only name/metadata/trace_id/parent_observation_id as direct kwargs, and the 'Setting trace attributes' section (line 230) shows that langfuse_session_id, langfuse_user_id, and langfuse_tags must live inside the metadata dict. A reader could try parse(..., langfuse_session_id='x') and have it silently dropped. Suggest rewording to something like 'attributes such as name and metadata (with langfuse_session_id, langfuse_user_id, langfuse_tags etc. nested inside metadata)' — note the existing example at line 416 already demonstrates the correct pattern with metadata={"langfuse_tags": [...]}.

Extended reasoning...

The bug. The new prose at content/integrations/model-providers/openai-py.mdx lines 386-387 (and the same wording duplicated in content/guides/cookbook/integration_openai_structured_output.md at lines 58 and 193, plus the matching .ipynb cells) reads:

You can pass a Pydantic model directly via response_format and still set Langfuse attributes such as name, metadata, langfuse_session_id, etc.

This sentence flattens two different things into one comma-separated list: name and metadata are real keyword arguments accepted by the Langfuse-wrapped OpenAI call, but langfuse_session_id is not — it is a key inside the metadata dict.

Why this contradicts the rest of the file. The same MDX has a 'Custom trace properties' table at lines 204-211 that lists exactly four direct kwargs: name, metadata, trace_id, parent_observation_id. langfuse_session_id is deliberately absent from that table. The 'Setting trace attributes (session_id, user_id, tags)' section that follows (lines 213-236) makes the correct usage explicit:

metadata={
    "langfuse_session_id": "session_123",
    "langfuse_user_id": "user_456",
    "langfuse_tags": ["calculator"],
    ...
}

So the new prose contradicts the canonical documentation just 150 lines above it.

The new prose also contradicts its own example. The very next code block (line 416 in the MDX) does the right thing:

completion = openai.chat.completions.parse(
    ...,
    name="extract-calendar-event",
    metadata={"langfuse_tags": ["structured-output"]},
)

Here langfuse_tags is correctly nested inside metadata, not passed as a direct kwarg. The example demonstrates the correct pattern; the prose above it does not.

Step-by-step proof of how a reader gets misled.

  1. Reader lands on the new 'Structured Output' section because they want session/user attribution on a parse call.

  2. They read the bullet: "You can pass a Pydantic model directly via response_format and still set Langfuse attributes such as name, metadata, langfuse_session_id, etc."

  3. By analogy with name="..." (a direct kwarg shown in the example), they write:

    completion = openai.chat.completions.parse(
    model="gpt-4o-2024-08-06",
    messages=[...],
    response_format=CalendarEvent,
    name="extract-calendar-event",
    langfuse_session_id="session_abc",   # ← reader infers this from the prose
)

  4. The wrapper recognizes name and consumes it. langfuse_session_id is not a recognized wrapper kwarg, so it is either passed through to the underlying OpenAI client (which will reject it as an unknown parameter, or silently ignore depending on version), or simply dropped — in any case, it never makes it onto the trace as a session id.

  5. The reader's traces have no session id and they don't know why; the docs that would tell them (lines 213-236) are now contradicted by the new bullet they trusted.

Cross-file impact. The cookbook .md at lines 58 and 193 has the same sentence, but in that file the disambiguating 'Custom trace properties' / 'Setting trace attributes' sections do not exist nearby — so a cookbook reader has even less chance of catching the issue. The .ipynb carries the same wording.

Fix. Reword along the lines of:

...Both paths support Langfuse attributes such as name and metadata. To attach a session id, user id, or tags, set them as keys inside metadata (langfuse_session_id, langfuse_user_id, langfuse_tags).

This keeps the reassurance that the stable parse path is fully instrumented, while no longer presenting metadata-keys as if they were direct kwargs. The change is text-only and applies to: the new note in the MDX (around line 386), the same note in the cookbook .md (line 58 and 193), and the corresponding cells in the .ipynb.


Check warning on line 388 in content/integrations/model-providers/openai-py.mdx

View check run for this annotation

Claude / Claude Code Review

Structured Output content contradicts parent "OpenAI Beta APIs" section intro 

The parent **OpenAI Beta APIs** section intro (just above line 381) still says "we fully support only the stable APIs in the OpenAI SDK. If you are using a beta API, you can still use the Langfuse SDK by wrapping the OpenAI SDK manually with the @observe() decorator", but the new Structured Output subsection placed under it now explicitly states that Langfuse instruments the legacy beta parse path too. Readers following the parent intro would unnecessarily wrap with `@observe()`. Consider updati
Comment on lines +384 to +388
Copy link
Copy Markdown

@claude claude Bot Apr 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 The parent OpenAI Beta APIs section intro (just above line 381) still says "we fully support only the stable APIs in the OpenAI SDK. If you are using a beta API, you can still use the Langfuse SDK by wrapping the OpenAI SDK manually with the @observe() decorator", but the new Structured Output subsection placed under it now explicitly states that Langfuse instruments the legacy beta parse path too. Readers following the parent intro would unnecessarily wrap with @observe(). Consider updating the intro paragraph or moving Structured Output out from under the OpenAI Beta APIs header (since parse is no longer beta on openai>=1.92.0).

Extended reasoning...

What the bug is. This PR rewrites the Structured Output subsection of content/integrations/model-providers/openai-py.mdx to explain that Langfuse instruments both the stable client.chat.completions.parse(...) (on openai>=1.92.0) and the legacy client.beta.chat.completions.parse(...) (on older SDKs). However, that subsection still lives under the parent ### OpenAI Beta APIs header, whose intro paragraph (untouched by this PR) reads:

Since OpenAI beta APIs are changing frequently across versions, we fully support only the stable APIs in the OpenAI SDK. If you are using a beta API, you can still use the Langfuse SDK by wrapping the OpenAI SDK manually with the @observe() decorator.

The new subsection directly contradicts that intro by stating:

openai<1.92.0 (legacy): the parse helper is only available under client.beta.chat.completions.parse(...). Langfuse also wraps the beta path on these older versions, so attributes like name and metadata work there too.

Why it's a problem. A reader who scans the page top-down hits the section header ("OpenAI Beta APIs"), reads the intro that says beta APIs need manual @observe() wrapping, and may stop reading there or carry that mental model forward. Even readers who continue then see the opposite claim a few lines later. Both messages can't be true at once.

Section structure is also stale. Since parse graduated out of beta in openai-python v1.92.0 (the very point this PR is making), the Structured Output content no longer belongs under an "OpenAI Beta APIs" header at all. The recommended path (client.chat.completions.parse(...)) is stable, not beta.

Step-by-step proof.

  1. Open the rendered page /integrations/model-providers/openai-py after this PR.
  2. Scroll to the ### OpenAI Beta APIs heading and read the intro paragraph: it tells the user that only stable APIs are fully supported and that beta APIs require manual @observe() wrapping.
  3. Continue into the #### Structured Output subsection just below: bullet two explicitly says Langfuse "also wraps the beta path on these older versions, so attributes like name and metadata work there too."
  4. The two statements are mutually exclusive — either the legacy beta parse path is automatically instrumented (so no @observe() is needed) or it isn't. The PR makes the new claim true but leaves the contradicting intro in place.

How to fix. Either (a) update the intro paragraph under ### OpenAI Beta APIs to clarify that some beta endpoints are now instrumented (and call out Structured Output as an example), or (b) promote #### Structured Output to its own top-level section (e.g., under "Advanced usage") since parse is no longer a beta API on supported SDK versions. Option (b) probably ages better given that parse is the recommended stable path.

<Callout type="info">
On `openai>=1.92.0`, calls to `client.beta.chat.completions.parse(...)` are
re-routed to the stable `parse` method by the OpenAI SDK, so they remain
instrumented. Prefer `client.chat.completions.parse(...)` in new code.
</Callout>

```python
from langfuse import get_client
from langfuse.openai import openai
from openai.lib._parsing._completions import type_to_response_format_param
from pydantic import BaseModel

class CalendarEvent(BaseModel):
name: str
date: str
participants: list[str]
name: str
date: str
participants: list[str]

completion = openai.chat.completions.create(
completion = openai.chat.completions.parse(
model="gpt-4o-2024-08-06",
messages=[
{"role": "system", "content": "Extract the event information."},
Expand All @@ -405,15 +411,42 @@
"content": "Alice and Bob are going to a science fair on Friday.",
},
],
response_format=type_to_response_format_param(CalendarEvent),
response_format=CalendarEvent,
name="extract-calendar-event",
metadata={"langfuse_tags": ["structured-output"]},
)

print(completion)
print(completion.choices[0].message.parsed)

# Flush via global client
get_client().flush()
```

If you cannot upgrade the OpenAI SDK and want to stay on `chat.completions.create`, you can still pass a Pydantic model by converting it with `type_to_response_format_param` from the OpenAI SDK:

```python
from langfuse.openai import openai
from openai.lib._parsing._completions import type_to_response_format_param
from pydantic import BaseModel

class CalendarEvent(BaseModel):
name: str
date: str
participants: list[str]

completion = openai.chat.completions.create(
model="gpt-4o-2024-08-06",
messages=[
{"role": "system", "content": "Extract the event information."},
{
"role": "user",
"content": "Alice and Bob are going to a science fair on Friday.",
},
],
response_format=type_to_response_format_param(CalendarEvent),
)
```

#### Assistants API

Tracing of the assistants api is not supported by this integration as OpenAI Assistants have server-side state that cannot easily be captured without additional api requests. Check out this [notebook](/integrations/model-providers/openai-assistants-api) for an end-to-end example on how to best track usage of the assistants api in Langfuse.
Expand Down
Loading
Loading