From 6452eef819490119863fbf4b19e21780b5dedefb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jannik=20Maierh=C3=B6fer?= Date: Thu, 30 Apr 2026 15:52:53 +0900 Subject: [PATCH] docs: clarify OpenAI Python parse vs response_format guidance openai-python>=1.92.0 graduated parse/stream out of beta. The Langfuse SDK already instruments both client.chat.completions.parse (stable) and client.beta.chat.completions.parse (legacy), so the previous "use response_format with chat.completions.create instead of the Beta API" guidance was outdated. Update the OpenAI Python integration page and the structured output cookbook to recommend the stable parse helper and scope the beta caveat to older SDKs. Reported by David Traina (Ramp) via support. Co-Authored-By: Claude Opus 4.7 (1M context) Langfuse-Session: https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/sessions/b55454d8-1e5e-477d-a6ba-f81f65fe959c --- .../integration_openai_structured_output.md | 8 +- .../model-providers/openai-py.mdx | 51 +- ...integration_openai_structured_output.ipynb | 794 +++++++++--------- 3 files changed, 430 insertions(+), 423 deletions(-) diff --git a/content/guides/cookbook/integration_openai_structured_output.md b/content/guides/cookbook/integration_openai_structured_output.md index ce19bbcad..a384d4645 100644 --- a/content/guides/cookbook/integration_openai_structured_output.md +++ b/content/guides/cookbook/integration_openai_structured_output.md @@ -55,7 +55,7 @@ This setup is useful for applications where each step needs to be displayed sepa (Example taken from [OpenAI cookbook](https://cookbook.openai.com/examples/structured_outputs_intro)) -**Note:** While OpenAI also offer structured output parsing via its beta API (`client.beta.chat.completions.parse`), this approach currently does not allow setting Langfuse specific attributes such as `name`, `metadata`, `userId` etc. Please use the approach using `response_format` with the standard `client.chat.completions.create` as described below. +**Note on `parse` vs `response_format`:** The Langfuse OpenAI integration instruments both the stable `client.chat.completions.parse(...)` (available in `openai>=1.92.0`) and the legacy `client.beta.chat.completions.parse(...)` (for older SDK versions, where `parse` is re-routed to the stable method on newer SDKs). Both paths support Langfuse-specific attributes such as `name`, `metadata`, `langfuse_session_id`, etc. The `response_format` approach with `client.chat.completions.create` shown below works on all SDK versions and is useful if you cannot upgrade `openai`. For an example using the `parse` helper with a Pydantic model, see the section further down. ```python @@ -190,7 +190,7 @@ You can now see the trace and the JSON schema in Langfuse. ## Alternative: Using the SDK `parse` helper -The new SDK version adds a `parse` helper, allowing you to use your own Pydantic model without defining a JSON schema. +OpenAI also offers a `parse` helper that lets you pass a Pydantic model directly via `response_format`. As of [`openai-python` v1.92.0](https://github.com/openai/openai-python/releases/tag/v1.92.0), `parse` is part of the stable API at `client.chat.completions.parse(...)` (it previously lived under `client.beta.chat.completions.parse`). Langfuse instruments both paths, so you can set Langfuse attributes (`name`, `metadata`, `langfuse_session_id`, …) on either one. Prefer the stable path in new code: ```python @@ -205,13 +205,15 @@ class MathReasoning(BaseModel): final_answer: str def get_math_solution(question: str): - response = client.beta.chat.completions.parse( + # Stable API on openai>=1.92.0. On older SDKs, use client.beta.chat.completions.parse instead. + response = client.chat.completions.parse( model=openai_model, messages=[ {"role": "system", "content": math_tutor_prompt}, {"role": "user", "content": question}, ], response_format=MathReasoning, + name="math-tutor-parse", ) return response.choices[0].message diff --git a/content/integrations/model-providers/openai-py.mdx b/content/integrations/model-providers/openai-py.mdx index 96ae402ef..510590d77 100644 --- a/content/integrations/model-providers/openai-py.mdx +++ b/content/integrations/model-providers/openai-py.mdx @@ -381,22 +381,28 @@ Since OpenAI beta APIs are changing frequently across versions, we fully support #### Structured Output -For **structured output parsing**, please use the `response_format` argument to `openai.chat.completions.create()` instead of the Beta API. This will allow you to set Langfuse attributes and metadata. +For **structured output parsing**, you have two fully instrumented options depending on your `openai` Python SDK version: -If you rely on parsing Pydantic defintions for your `response_format`, you may leverage the `type_to_response_format_param` utility function from the OpenAI Python SDK to convert the Pydantic definition to a `response_format` dictionary. This is the same function the OpenAI Beta API uses to convert Pydantic definitions to `response_format` dictionaries. +- **`openai>=1.92.0` (recommended):** use `client.chat.completions.parse(...)`. OpenAI graduated `parse` and `stream` out of beta in [v1.92.0](https://github.com/openai/openai-python/releases/tag/v1.92.0), and Langfuse wraps the stable `openai.resources.chat.completions.Completions.parse` (and the async variant). You can pass a Pydantic model directly via `response_format` and still set Langfuse attributes such as `name`, `metadata`, `langfuse_session_id`, etc. +- **`openai<1.92.0` (legacy):** the parse helper is only available under `client.beta.chat.completions.parse(...)`. Langfuse also wraps the beta path on these older versions, so attributes like `name` and `metadata` work there too. + + + On `openai>=1.92.0`, calls to `client.beta.chat.completions.parse(...)` are + re-routed to the stable `parse` method by the OpenAI SDK, so they remain + instrumented. Prefer `client.chat.completions.parse(...)` in new code. + ```python from langfuse import get_client from langfuse.openai import openai -from openai.lib._parsing._completions import type_to_response_format_param from pydantic import BaseModel class CalendarEvent(BaseModel): - name: str - date: str - participants: list[str] + name: str + date: str + participants: list[str] -completion = openai.chat.completions.create( +completion = openai.chat.completions.parse( model="gpt-4o-2024-08-06", messages=[ {"role": "system", "content": "Extract the event information."}, @@ -405,15 +411,42 @@ completion = openai.chat.completions.create( "content": "Alice and Bob are going to a science fair on Friday.", }, ], - response_format=type_to_response_format_param(CalendarEvent), + response_format=CalendarEvent, + name="extract-calendar-event", + metadata={"langfuse_tags": ["structured-output"]}, ) -print(completion) +print(completion.choices[0].message.parsed) # Flush via global client get_client().flush() ``` +If you cannot upgrade the OpenAI SDK and want to stay on `chat.completions.create`, you can still pass a Pydantic model by converting it with `type_to_response_format_param` from the OpenAI SDK: + +```python +from langfuse.openai import openai +from openai.lib._parsing._completions import type_to_response_format_param +from pydantic import BaseModel + +class CalendarEvent(BaseModel): + name: str + date: str + participants: list[str] + +completion = openai.chat.completions.create( + model="gpt-4o-2024-08-06", + messages=[ + {"role": "system", "content": "Extract the event information."}, + { + "role": "user", + "content": "Alice and Bob are going to a science fair on Friday.", + }, + ], + response_format=type_to_response_format_param(CalendarEvent), +) +``` + #### Assistants API Tracing of the assistants api is not supported by this integration as OpenAI Assistants have server-side state that cannot easily be captured without additional api requests. Check out this [notebook](/integrations/model-providers/openai-assistants-api) for an end-to-end example on how to best track usage of the assistants api in Langfuse. diff --git a/cookbook/integration_openai_structured_output.ipynb b/cookbook/integration_openai_structured_output.ipynb index 076c2fb34..5cbfa7bea 100644 --- a/cookbook/integration_openai_structured_output.ipynb +++ b/cookbook/integration_openai_structured_output.ipynb @@ -1,420 +1,392 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "id": "m13Z9rJHCicu" - }, - "source": [ - "---\n", - "title: Observe OpenAI Structured Outputs with Langfuse\n", - "description: Learn how to use Langfuse to monitor OpenAI Structured Outputs\n", - "category: Integrations\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "yIyq5cb2GsrX" - }, - "source": [ - "# Cookbook: Trace OpenAI Structured Outputs with Langfuse\n", - "\n", - "In this cookbook you will learn how to use Langfuse to monitor OpenAI Structured Outputs.\n", - "\n", - "## What are structured outputs?\n", - "Generating structured data from unstructured inputs is a core AI use case today. Structured outputs make especially chained LLM calls, UI component generation, and model-based evaluation more reliable. [Structured Outputs](https://openai.com/index/introducing-structured-outputs-in-the-api/) is a new capability of the OpenAI API that builds upon JSON mode and function calling to enforce a strict schema in a model output.\n", - "\n", - "## How to trace structured output in Langfuse?\n", - "If you use the OpenAI Python SDK, you can use the [Langfuse drop-in replacement](https://langfuse.com/integrations/model-providers/openai-py) to get full logging by changing only the import. With that, you can monitor the structured output generated by OpenAI in Langfuse.\n", - "\n", - "```diff\n", - "- import openai\n", - "+ from langfuse.openai import openai\n", - "\n", - "Alternative imports:\n", - "+ from langfuse.openai import OpenAI, AsyncOpenAI, AzureOpenAI, AsyncAzureOpenAI\n", - "```\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "hH5klZnTTzuq" - }, - "source": [ - "## Step 1: Initialize Langfuse\n", - "Initialize the Langfuse client with your [API keys](https://langfuse.com/faq/all/where-are-langfuse-api-keys) from the project settings in the Langfuse UI and add them to your environment." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "collapsed": true, - "id": "n11_vChnEdrg", - "outputId": "9c5c1aad-380d-4950-ae30-7d033deef0a9" - }, - "outputs": [], - "source": [ - "%pip install langfuse openai --upgrade" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": { - "id": "79Cu3eLlEOCA" - }, - "outputs": [], - "source": [ - "import os\n", - "\n", - "# Get keys for your project from the project settings page\n", - "# https://cloud.langfuse.com\n", - "os.environ[\"LANGFUSE_PUBLIC_KEY\"] = \"\"\n", - "os.environ[\"LANGFUSE_SECRET_KEY\"] = \"\"\n", - "os.environ[\"LANGFUSE_BASE_URL\"] = \"https://cloud.langfuse.com\" # πŸ‡ͺπŸ‡Ί EU region\n", - "# Other Langfuse data regions include πŸ‡ΊπŸ‡Έ US: https://us.cloud.langfuse.com, πŸ‡―πŸ‡΅ Japan: https://jp.cloud.langfuse.com and βš•οΈ HIPAA: https://hipaa.cloud.langfuse.com\n", - "\n", - "# Your openai key\n", - "os.environ[\"OPENAI_API_KEY\"] = \"\"" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "eK8oTvWHMaTx" - }, - "source": [ - "## Step 2: Math tutor example\n", - "\n", - "In this example, we'll build a math tutoring tool that outputs steps to solve a math problem as an array of structured objects.\n", - "\n", - "This setup is useful for applications where each step needs to be displayed separately, allowing users to progress through the solution at their own pace.\n", - "\n", - "(Example taken from [OpenAI cookbook](https://cookbook.openai.com/examples/structured_outputs_intro))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "**Note:** While OpenAI also offer structured output parsing via its beta API (`client.beta.chat.completions.parse`), this approach currently does not allow setting Langfuse specific attributes such as `name`, `metadata`, `userId` etc. Please use the approach using `response_format` with the standard `client.chat.completions.create` as described below." - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": { - "id": "8IZjc893Ef1c" - }, - "outputs": [], - "source": [ - "# Use the Langfuse drop-in replacement to get full logging by changing only the import.\n", - "# With that, you can monitor the structured output generated by OpenAI in Langfuse.\n", - "from langfuse.openai import OpenAI\n", - "import json\n", - "\n", - "openai_model = \"gpt-4o-2024-08-06\"\n", - "client = OpenAI()" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "YV7nfCKqV-FS" - }, - "source": [ - "In the `response_format` parameter you can now supply a JSON Schema via `json_schema`. When using `response_format` with `strict: true`, the model's output will adhere to the provided schema.\n", - "\n", - "Function calling remains similar, but with the new parameter `strict: true`, you can now ensure that the schema provided for the functions is strictly followed." - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": { - "id": "iKWRHCjVF55-" - }, - "outputs": [], - "source": [ - "math_tutor_prompt = '''\n", - " You are a helpful math tutor. You will be provided with a math problem,\n", - " and your goal will be to output a step by step solution, along with a final answer.\n", - " For each step, just provide the output as an equation use the explanation field to detail the reasoning.\n", - "'''\n", - "\n", - "def get_math_solution(question):\n", - " response = client.chat.completions.create(\n", - " model = openai_model,\n", - " messages=[\n", - " {\n", - " \"role\": \"system\",\n", - " \"content\": math_tutor_prompt\n", - " },\n", - " {\n", - " \"role\": \"user\",\n", - " \"content\": question\n", - " }\n", - " ],\n", - " response_format={\n", - " \"type\": \"json_schema\",\n", - " \"json_schema\": {\n", - " \"name\": \"math_reasoning\",\n", - " \"schema\": {\n", - " \"type\": \"object\",\n", - " \"properties\": {\n", - " \"steps\": {\n", - " \"type\": \"array\",\n", - " \"items\": {\n", - " \"type\": \"object\",\n", - " \"properties\": {\n", - " \"explanation\": {\"type\": \"string\"},\n", - " \"output\": {\"type\": \"string\"}\n", - " },\n", - " \"required\": [\"explanation\", \"output\"],\n", - " \"additionalProperties\": False\n", - " }\n", - " },\n", - " \"final_answer\": {\"type\": \"string\"}\n", - " },\n", - " \"required\": [\"steps\", \"final_answer\"],\n", - " \"additionalProperties\": False\n", - " },\n", - " \"strict\": True\n", - " }\n", - " }\n", - " )\n", - "\n", - " return response.choices[0].message" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "Qn9N-utqINy_", - "outputId": "2510cbb4-1784-4fcf-c7f5-ead1de58abfe" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "{\"steps\":[{\"explanation\":\"We need to isolate the term with the variable, 8x. So, we start by subtracting 7 from both sides to remove the constant term on the left side.\",\"output\":\"8x + 7 - 7 = -23 - 7\"},{\"explanation\":\"The +7 and -7 on the left side cancel each other out, leaving us with 8x. The right side simplifies to -30.\",\"output\":\"8x = -30\"},{\"explanation\":\"To solve for x, divide both sides of the equation by 8, which is the coefficient of x.\",\"output\":\"x = -30 / 8\"},{\"explanation\":\"Simplify the fraction -30/8 by finding the greatest common divisor, which is 2.\",\"output\":\"x = -15 / 4\"}],\"final_answer\":\"x = -15/4\"}\n" - ] - } - ], - "source": [ - "# Testing with an example question\n", - "question = \"how can I solve 8x + 7 = -23\"\n", - "\n", - "result = get_math_solution(question)\n", - "\n", - "print(result.content)" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 456 - }, - "id": "mAwEts-hCA73", - "outputId": "9101c8ff-7b83-4ae6-f6c0-0da6a45ab251" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Step 1: We need to isolate the term with the variable, 8x. So, we start by subtracting 7 from both sides to remove the constant term on the left side.\n", - "\n", - "8x + 7 - 7 = -23 - 7\n", - "\n", - "\n", - "Step 2: The +7 and -7 on the left side cancel each other out, leaving us with 8x. The right side simplifies to -30.\n", - "\n", - "8x = -30\n", - "\n", - "\n", - "Step 3: To solve for x, divide both sides of the equation by 8, which is the coefficient of x.\n", - "\n", - "x = -30 / 8\n", - "\n", - "\n", - "Step 4: Simplify the fraction -30/8 by finding the greatest common divisor, which is 2.\n", - "\n", - "x = -15 / 4\n", - "\n", - "\n", - "Final answer:\n", - "\n", - "\n", - "x = -15/4\n" - ] - } - ], - "source": [ - "# Print results step by step\n", - "\n", - "result = json.loads(result.content)\n", - "steps = result['steps']\n", - "final_answer = result['final_answer']\n", - "for i in range(len(steps)):\n", - " print(f\"Step {i+1}: {steps[i]['explanation']}\\n\")\n", - " print(steps[i]['output'])\n", - " print(\"\\n\")\n", - "\n", - "print(\"Final answer:\\n\\n\")\n", - "print(final_answer)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "KBoP_eEfLGMb" - }, - "source": [ - "## Step 3: See your trace in Langfuse\n", - "\n", - "You can now see the trace and the JSON schema in Langfuse.\n", - "\n", - "[Example trace in Langfuse](https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/traces/3ecc3849-66c9-4eaf-b26b-bde26b7eebed)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "xhY0etZVU33V" - }, - "source": [ - "![View example trace in the Langfuse UI](https://langfuse.com/images/cookbook/integration-openai-structured-outputs-tracing.png)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Alternative: Using the SDK `parse` helper\n", - "\n", - "The new SDK version adds a `parse` helper, allowing you to use your own Pydantic model without defining a JSON schema." - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": {}, - "outputs": [], - "source": [ - "from pydantic import BaseModel\n", - "\n", - "class MathReasoning(BaseModel):\n", - " class Step(BaseModel):\n", - " explanation: str\n", - " output: str\n", - "\n", - " steps: list[Step]\n", - " final_answer: str\n", - "\n", - "def get_math_solution(question: str):\n", - " response = client.beta.chat.completions.parse(\n", - " model=openai_model,\n", - " messages=[\n", - " {\"role\": \"system\", \"content\": math_tutor_prompt},\n", - " {\"role\": \"user\", \"content\": question},\n", - " ],\n", - " response_format=MathReasoning,\n", - " )\n", - "\n", - " return response.choices[0].message" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "[Step(explanation='To isolate the term with the variable on one side of the equation, start by subtracting 7 from both sides.', output='8x = -23 - 7'), Step(explanation='Combine like terms on the right side to simplify the equation.', output='8x = -30'), Step(explanation='Divide both sides by 8 to solve for x.', output='x = -30 / 8'), Step(explanation='Simplify the fraction by dividing both the numerator and the denominator by their greatest common divisor, which is 2.', output='x = -15 / 4')]\n", - "Final answer:\n", - "x = -15/4\n" - ] - } - ], - "source": [ - "result = get_math_solution(question).parsed\n", - "\n", - "print(result.steps)\n", - "print(\"Final answer:\")\n", - "print(result.final_answer)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## See your trace in Langfuse\n", - "\n", - "You can now see the trace and your supplied Pydantic model in Langfuse.\n", - "\n", - "[Example trace in Langfuse](https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/traces/59c4376a-c8eb-4ecb-8780-2f028b87e7eb)" - ] + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "m13Z9rJHCicu" + }, + "source": [ + "---\n", + "title: Observe OpenAI Structured Outputs with Langfuse\n", + "description: Learn how to use Langfuse to monitor OpenAI Structured Outputs\n", + "category: Integrations\n", + "---" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "yIyq5cb2GsrX" + }, + "source": [ + "# Cookbook: Trace OpenAI Structured Outputs with Langfuse\n", + "\n", + "In this cookbook you will learn how to use Langfuse to monitor OpenAI Structured Outputs.\n", + "\n", + "## What are structured outputs?\n", + "Generating structured data from unstructured inputs is a core AI use case today. Structured outputs make especially chained LLM calls, UI component generation, and model-based evaluation more reliable. [Structured Outputs](https://openai.com/index/introducing-structured-outputs-in-the-api/) is a new capability of the OpenAI API that builds upon JSON mode and function calling to enforce a strict schema in a model output.\n", + "\n", + "## How to trace structured output in Langfuse?\n", + "If you use the OpenAI Python SDK, you can use the [Langfuse drop-in replacement](https://langfuse.com/integrations/model-providers/openai-py) to get full logging by changing only the import. With that, you can monitor the structured output generated by OpenAI in Langfuse.\n", + "\n", + "```diff\n", + "- import openai\n", + "+ from langfuse.openai import openai\n", + "\n", + "Alternative imports:\n", + "+ from langfuse.openai import OpenAI, AsyncOpenAI, AzureOpenAI, AsyncAzureOpenAI\n", + "```\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "hH5klZnTTzuq" + }, + "source": [ + "## Step 1: Initialize Langfuse\n", + "Initialize the Langfuse client with your [API keys](https://langfuse.com/faq/all/where-are-langfuse-api-keys) from the project settings in the Langfuse UI and add them to your environment." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "![View example trace in the Langfuse UI](https://langfuse.com/images/cookbook/integration_openai_structured_outputs_tracing_parse.png)" - ] + "collapsed": true, + "id": "n11_vChnEdrg", + "outputId": "9c5c1aad-380d-4950-ae30-7d033deef0a9" + }, + "outputs": [], + "source": [ + "%pip install langfuse openai --upgrade" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "id": "79Cu3eLlEOCA" + }, + "outputs": [], + "source": [ + "import os\n", + "\n", + "# Get keys for your project from the project settings page\n", + "# https://cloud.langfuse.com\n", + "os.environ[\"LANGFUSE_PUBLIC_KEY\"] = \"\"\n", + "os.environ[\"LANGFUSE_SECRET_KEY\"] = \"\"\n", + "os.environ[\"LANGFUSE_BASE_URL\"] = \"https://cloud.langfuse.com\" # πŸ‡ͺπŸ‡Ί EU region\n", + "# Other Langfuse data regions include πŸ‡ΊπŸ‡Έ US: https://us.cloud.langfuse.com, πŸ‡―πŸ‡΅ Japan: https://jp.cloud.langfuse.com and βš•οΈ HIPAA: https://hipaa.cloud.langfuse.com\n", + "\n", + "# Your openai key\n", + "os.environ[\"OPENAI_API_KEY\"] = \"\"" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "eK8oTvWHMaTx" + }, + "source": [ + "## Step 2: Math tutor example\n", + "\n", + "In this example, we'll build a math tutoring tool that outputs steps to solve a math problem as an array of structured objects.\n", + "\n", + "This setup is useful for applications where each step needs to be displayed separately, allowing users to progress through the solution at their own pace.\n", + "\n", + "(Example taken from [OpenAI cookbook](https://cookbook.openai.com/examples/structured_outputs_intro))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "**Note on `parse` vs `response_format`:** The Langfuse OpenAI integration instruments both the stable `client.chat.completions.parse(...)` (available in `openai>=1.92.0`) and the legacy `client.beta.chat.completions.parse(...)` (for older SDK versions, where `parse` is re-routed to the stable method on newer SDKs). Both paths support Langfuse-specific attributes such as `name`, `metadata`, `langfuse_session_id`, etc. The `response_format` approach with `client.chat.completions.create` shown below works on all SDK versions and is useful if you cannot upgrade `openai`. For an example using the `parse` helper with a Pydantic model, see the section further down." + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "id": "8IZjc893Ef1c" + }, + "outputs": [], + "source": [ + "# Use the Langfuse drop-in replacement to get full logging by changing only the import.\n", + "# With that, you can monitor the structured output generated by OpenAI in Langfuse.\n", + "from langfuse.openai import OpenAI\n", + "import json\n", + "\n", + "openai_model = \"gpt-4o-2024-08-06\"\n", + "client = OpenAI()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "YV7nfCKqV-FS" + }, + "source": [ + "In the `response_format` parameter you can now supply a JSON Schema via `json_schema`. When using `response_format` with `strict: true`, the model's output will adhere to the provided schema.\n", + "\n", + "Function calling remains similar, but with the new parameter `strict: true`, you can now ensure that the schema provided for the functions is strictly followed." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "id": "iKWRHCjVF55-" + }, + "outputs": [], + "source": [ + "math_tutor_prompt = '''\n", + " You are a helpful math tutor. You will be provided with a math problem,\n", + " and your goal will be to output a step by step solution, along with a final answer.\n", + " For each step, just provide the output as an equation use the explanation field to detail the reasoning.\n", + "'''\n", + "\n", + "def get_math_solution(question):\n", + " response = client.chat.completions.create(\n", + " model = openai_model,\n", + " messages=[\n", + " {\n", + " \"role\": \"system\",\n", + " \"content\": math_tutor_prompt\n", + " },\n", + " {\n", + " \"role\": \"user\",\n", + " \"content\": question\n", + " }\n", + " ],\n", + " response_format={\n", + " \"type\": \"json_schema\",\n", + " \"json_schema\": {\n", + " \"name\": \"math_reasoning\",\n", + " \"schema\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"steps\": {\n", + " \"type\": \"array\",\n", + " \"items\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"explanation\": {\"type\": \"string\"},\n", + " \"output\": {\"type\": \"string\"}\n", + " },\n", + " \"required\": [\"explanation\", \"output\"],\n", + " \"additionalProperties\": False\n", + " }\n", + " },\n", + " \"final_answer\": {\"type\": \"string\"}\n", + " },\n", + " \"required\": [\"steps\", \"final_answer\"],\n", + " \"additionalProperties\": False\n", + " },\n", + " \"strict\": True\n", + " }\n", + " }\n", + " )\n", + "\n", + " return response.choices[0].message" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" }, + "id": "Qn9N-utqINy_", + "outputId": "2510cbb4-1784-4fcf-c7f5-ead1de58abfe" + }, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "UFWiSfqtEdJO" - }, - "source": [ - "## Feedback\n", - "\n", - "If you have any feedback or requests, please create a GitHub [Issue](https://langfuse.com/issue) or share your idea with the community on [Discord](https://langfuse.com/discord)." - ] + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"steps\":[{\"explanation\":\"We need to isolate the term with the variable, 8x. So, we start by subtracting 7 from both sides to remove the constant term on the left side.\",\"output\":\"8x + 7 - 7 = -23 - 7\"},{\"explanation\":\"The +7 and -7 on the left side cancel each other out, leaving us with 8x. The right side simplifies to -30.\",\"output\":\"8x = -30\"},{\"explanation\":\"To solve for x, divide both sides of the equation by 8, which is the coefficient of x.\",\"output\":\"x = -30 / 8\"},{\"explanation\":\"Simplify the fraction -30/8 by finding the greatest common divisor, which is 2.\",\"output\":\"x = -15 / 4\"}],\"final_answer\":\"x = -15/4\"}\n" + ] } - ], - "metadata": { + ], + "source": [ + "# Testing with an example question\n", + "question = \"how can I solve 8x + 7 = -23\"\n", + "\n", + "result = get_math_solution(question)\n", + "\n", + "print(result.content)" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { "colab": { - "provenance": [] + "base_uri": "https://localhost:8080/", + "height": 456 }, - "kernelspec": { - "display_name": "Python 3", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.18" + "id": "mAwEts-hCA73", + "outputId": "9101c8ff-7b83-4ae6-f6c0-0da6a45ab251" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Step 1: We need to isolate the term with the variable, 8x. So, we start by subtracting 7 from both sides to remove the constant term on the left side.\n", + "\n", + "8x + 7 - 7 = -23 - 7\n", + "\n", + "\n", + "Step 2: The +7 and -7 on the left side cancel each other out, leaving us with 8x. The right side simplifies to -30.\n", + "\n", + "8x = -30\n", + "\n", + "\n", + "Step 3: To solve for x, divide both sides of the equation by 8, which is the coefficient of x.\n", + "\n", + "x = -30 / 8\n", + "\n", + "\n", + "Step 4: Simplify the fraction -30/8 by finding the greatest common divisor, which is 2.\n", + "\n", + "x = -15 / 4\n", + "\n", + "\n", + "Final answer:\n", + "\n", + "\n", + "x = -15/4\n" + ] } + ], + "source": [ + "# Print results step by step\n", + "\n", + "result = json.loads(result.content)\n", + "steps = result['steps']\n", + "final_answer = result['final_answer']\n", + "for i in range(len(steps)):\n", + " print(f\"Step {i+1}: {steps[i]['explanation']}\\n\")\n", + " print(steps[i]['output'])\n", + " print(\"\\n\")\n", + "\n", + "print(\"Final answer:\\n\\n\")\n", + "print(final_answer)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "KBoP_eEfLGMb" + }, + "source": [ + "## Step 3: See your trace in Langfuse\n", + "\n", + "You can now see the trace and the JSON schema in Langfuse.\n", + "\n", + "[Example trace in Langfuse](https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/traces/3ecc3849-66c9-4eaf-b26b-bde26b7eebed)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "xhY0etZVU33V" + }, + "source": [ + "![View example trace in the Langfuse UI](https://langfuse.com/images/cookbook/integration-openai-structured-outputs-tracing.png)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "## Alternative: Using the SDK `parse` helper\n\nOpenAI also offers a `parse` helper that lets you pass a Pydantic model directly via `response_format`. As of [`openai-python` v1.92.0](https://github.com/openai/openai-python/releases/tag/v1.92.0), `parse` is part of the stable API at `client.chat.completions.parse(...)` (it previously lived under `client.beta.chat.completions.parse`). Langfuse instruments both paths, so you can set Langfuse attributes (`name`, `metadata`, `langfuse_session_id`, …) on either one. Prefer the stable path in new code:" + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": "from pydantic import BaseModel\n\nclass MathReasoning(BaseModel):\n class Step(BaseModel):\n explanation: str\n output: str\n\n steps: list[Step]\n final_answer: str\n\ndef get_math_solution(question: str):\n # Stable API on openai>=1.92.0. On older SDKs, use client.beta.chat.completions.parse instead.\n response = client.chat.completions.parse(\n model=openai_model,\n messages=[\n {\"role\": \"system\", \"content\": math_tutor_prompt},\n {\"role\": \"user\", \"content\": question},\n ],\n response_format=MathReasoning,\n name=\"math-tutor-parse\",\n )\n\n return response.choices[0].message" + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[Step(explanation='To isolate the term with the variable on one side of the equation, start by subtracting 7 from both sides.', output='8x = -23 - 7'), Step(explanation='Combine like terms on the right side to simplify the equation.', output='8x = -30'), Step(explanation='Divide both sides by 8 to solve for x.', output='x = -30 / 8'), Step(explanation='Simplify the fraction by dividing both the numerator and the denominator by their greatest common divisor, which is 2.', output='x = -15 / 4')]\n", + "Final answer:\n", + "x = -15/4\n" + ] + } + ], + "source": [ + "result = get_math_solution(question).parsed\n", + "\n", + "print(result.steps)\n", + "print(\"Final answer:\")\n", + "print(result.final_answer)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## See your trace in Langfuse\n", + "\n", + "You can now see the trace and your supplied Pydantic model in Langfuse.\n", + "\n", + "[Example trace in Langfuse](https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/traces/59c4376a-c8eb-4ecb-8780-2f028b87e7eb)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![View example trace in the Langfuse UI](https://langfuse.com/images/cookbook/integration_openai_structured_outputs_tracing_parse.png)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "UFWiSfqtEdJO" + }, + "source": [ + "## Feedback\n", + "\n", + "If you have any feedback or requests, please create a GitHub [Issue](https://langfuse.com/issue) or share your idea with the community on [Discord](https://langfuse.com/discord)." + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" }, - "nbformat": 4, - "nbformat_minor": 0 -} + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.18" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} \ No newline at end of file