diff --git a/content/guides/cookbook/integration_openai_structured_output.md b/content/guides/cookbook/integration_openai_structured_output.md index ce19bbcad..a384d4645 100644 --- a/content/guides/cookbook/integration_openai_structured_output.md +++ b/content/guides/cookbook/integration_openai_structured_output.md @@ -55,7 +55,7 @@ This setup is useful for applications where each step needs to be displayed sepa (Example taken from [OpenAI cookbook](https://cookbook.openai.com/examples/structured_outputs_intro)) -**Note:** While OpenAI also offer structured output parsing via its beta API (`client.beta.chat.completions.parse`), this approach currently does not allow setting Langfuse specific attributes such as `name`, `metadata`, `userId` etc. Please use the approach using `response_format` with the standard `client.chat.completions.create` as described below. +**Note on `parse` vs `response_format`:** The Langfuse OpenAI integration instruments both the stable `client.chat.completions.parse(...)` (available in `openai>=1.92.0`) and the legacy `client.beta.chat.completions.parse(...)` (for older SDK versions, where `parse` is re-routed to the stable method on newer SDKs). Both paths support Langfuse-specific attributes such as `name`, `metadata`, `langfuse_session_id`, etc. The `response_format` approach with `client.chat.completions.create` shown below works on all SDK versions and is useful if you cannot upgrade `openai`. For an example using the `parse` helper with a Pydantic model, see the section further down. ```python @@ -190,7 +190,7 @@ You can now see the trace and the JSON schema in Langfuse. ## Alternative: Using the SDK `parse` helper -The new SDK version adds a `parse` helper, allowing you to use your own Pydantic model without defining a JSON schema. +OpenAI also offers a `parse` helper that lets you pass a Pydantic model directly via `response_format`. As of [`openai-python` v1.92.0](https://github.com/openai/openai-python/releases/tag/v1.92.0), `parse` is part of the stable API at `client.chat.completions.parse(...)` (it previously lived under `client.beta.chat.completions.parse`). Langfuse instruments both paths, so you can set Langfuse attributes (`name`, `metadata`, `langfuse_session_id`, …) on either one. Prefer the stable path in new code: ```python @@ -205,13 +205,15 @@ class MathReasoning(BaseModel): final_answer: str def get_math_solution(question: str): - response = client.beta.chat.completions.parse( + # Stable API on openai>=1.92.0. On older SDKs, use client.beta.chat.completions.parse instead. + response = client.chat.completions.parse( model=openai_model, messages=[ {"role": "system", "content": math_tutor_prompt}, {"role": "user", "content": question}, ], response_format=MathReasoning, + name="math-tutor-parse", ) return response.choices[0].message diff --git a/content/integrations/model-providers/openai-py.mdx b/content/integrations/model-providers/openai-py.mdx index 96ae402ef..510590d77 100644 --- a/content/integrations/model-providers/openai-py.mdx +++ b/content/integrations/model-providers/openai-py.mdx @@ -381,22 +381,28 @@ Since OpenAI beta APIs are changing frequently across versions, we fully support #### Structured Output -For **structured output parsing**, please use the `response_format` argument to `openai.chat.completions.create()` instead of the Beta API. This will allow you to set Langfuse attributes and metadata. +For **structured output parsing**, you have two fully instrumented options depending on your `openai` Python SDK version: -If you rely on parsing Pydantic defintions for your `response_format`, you may leverage the `type_to_response_format_param` utility function from the OpenAI Python SDK to convert the Pydantic definition to a `response_format` dictionary. This is the same function the OpenAI Beta API uses to convert Pydantic definitions to `response_format` dictionaries. +- **`openai>=1.92.0` (recommended):** use `client.chat.completions.parse(...)`. OpenAI graduated `parse` and `stream` out of beta in [v1.92.0](https://github.com/openai/openai-python/releases/tag/v1.92.0), and Langfuse wraps the stable `openai.resources.chat.completions.Completions.parse` (and the async variant). You can pass a Pydantic model directly via `response_format` and still set Langfuse attributes such as `name`, `metadata`, `langfuse_session_id`, etc. +- **`openai<1.92.0` (legacy):** the parse helper is only available under `client.beta.chat.completions.parse(...)`. Langfuse also wraps the beta path on these older versions, so attributes like `name` and `metadata` work there too. + + + On `openai>=1.92.0`, calls to `client.beta.chat.completions.parse(...)` are + re-routed to the stable `parse` method by the OpenAI SDK, so they remain + instrumented. Prefer `client.chat.completions.parse(...)` in new code. + ```python from langfuse import get_client from langfuse.openai import openai -from openai.lib._parsing._completions import type_to_response_format_param from pydantic import BaseModel class CalendarEvent(BaseModel): - name: str - date: str - participants: list[str] + name: str + date: str + participants: list[str] -completion = openai.chat.completions.create( +completion = openai.chat.completions.parse( model="gpt-4o-2024-08-06", messages=[ {"role": "system", "content": "Extract the event information."}, @@ -405,15 +411,42 @@ completion = openai.chat.completions.create( "content": "Alice and Bob are going to a science fair on Friday.", }, ], - response_format=type_to_response_format_param(CalendarEvent), + response_format=CalendarEvent, + name="extract-calendar-event", + metadata={"langfuse_tags": ["structured-output"]}, ) -print(completion) +print(completion.choices[0].message.parsed) # Flush via global client get_client().flush() ``` +If you cannot upgrade the OpenAI SDK and want to stay on `chat.completions.create`, you can still pass a Pydantic model by converting it with `type_to_response_format_param` from the OpenAI SDK: + +```python +from langfuse.openai import openai +from openai.lib._parsing._completions import type_to_response_format_param +from pydantic import BaseModel + +class CalendarEvent(BaseModel): + name: str + date: str + participants: list[str] + +completion = openai.chat.completions.create( + model="gpt-4o-2024-08-06", + messages=[ + {"role": "system", "content": "Extract the event information."}, + { + "role": "user", + "content": "Alice and Bob are going to a science fair on Friday.", + }, + ], + response_format=type_to_response_format_param(CalendarEvent), +) +``` + #### Assistants API Tracing of the assistants api is not supported by this integration as OpenAI Assistants have server-side state that cannot easily be captured without additional api requests. Check out this [notebook](/integrations/model-providers/openai-assistants-api) for an end-to-end example on how to best track usage of the assistants api in Langfuse. diff --git a/cookbook/integration_openai_structured_output.ipynb b/cookbook/integration_openai_structured_output.ipynb index 076c2fb34..5cbfa7bea 100644 --- a/cookbook/integration_openai_structured_output.ipynb +++ b/cookbook/integration_openai_structured_output.ipynb @@ -1,420 +1,392 @@ { - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "id": "m13Z9rJHCicu" - }, - "source": [ - "---\n", - "title: Observe OpenAI Structured Outputs with Langfuse\n", - "description: Learn how to use Langfuse to monitor OpenAI Structured Outputs\n", - "category: Integrations\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "yIyq5cb2GsrX" - }, - "source": [ - "# Cookbook: Trace OpenAI Structured Outputs with Langfuse\n", - "\n", - "In this cookbook you will learn how to use Langfuse to monitor OpenAI Structured Outputs.\n", - "\n", - "## What are structured outputs?\n", - "Generating structured data from unstructured inputs is a core AI use case today. Structured outputs make especially chained LLM calls, UI component generation, and model-based evaluation more reliable. [Structured Outputs](https://openai.com/index/introducing-structured-outputs-in-the-api/) is a new capability of the OpenAI API that builds upon JSON mode and function calling to enforce a strict schema in a model output.\n", - "\n", - "## How to trace structured output in Langfuse?\n", - "If you use the OpenAI Python SDK, you can use the [Langfuse drop-in replacement](https://langfuse.com/integrations/model-providers/openai-py) to get full logging by changing only the import. With that, you can monitor the structured output generated by OpenAI in Langfuse.\n", - "\n", - "```diff\n", - "- import openai\n", - "+ from langfuse.openai import openai\n", - "\n", - "Alternative imports:\n", - "+ from langfuse.openai import OpenAI, AsyncOpenAI, AzureOpenAI, AsyncAzureOpenAI\n", - "```\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "hH5klZnTTzuq" - }, - "source": [ - "## Step 1: Initialize Langfuse\n", - "Initialize the Langfuse client with your [API keys](https://langfuse.com/faq/all/where-are-langfuse-api-keys) from the project settings in the Langfuse UI and add them to your environment." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "collapsed": true, - "id": "n11_vChnEdrg", - "outputId": "9c5c1aad-380d-4950-ae30-7d033deef0a9" - }, - "outputs": [], - "source": [ - "%pip install langfuse openai --upgrade" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": { - "id": "79Cu3eLlEOCA" - }, - "outputs": [], - "source": [ - "import os\n", - "\n", - "# Get keys for your project from the project settings page\n", - "# https://cloud.langfuse.com\n", - "os.environ[\"LANGFUSE_PUBLIC_KEY\"] = \"\"\n", - "os.environ[\"LANGFUSE_SECRET_KEY\"] = \"\"\n", - "os.environ[\"LANGFUSE_BASE_URL\"] = \"https://cloud.langfuse.com\" # πŸ‡ͺπŸ‡Ί EU region\n", - "# Other Langfuse data regions include πŸ‡ΊπŸ‡Έ US: https://us.cloud.langfuse.com, πŸ‡―πŸ‡΅ Japan: https://jp.cloud.langfuse.com and βš•οΈ HIPAA: https://hipaa.cloud.langfuse.com\n", - "\n", - "# Your openai key\n", - "os.environ[\"OPENAI_API_KEY\"] = \"\"" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "eK8oTvWHMaTx" - }, - "source": [ - "## Step 2: Math tutor example\n", - "\n", - "In this example, we'll build a math tutoring tool that outputs steps to solve a math problem as an array of structured objects.\n", - "\n", - "This setup is useful for applications where each step needs to be displayed separately, allowing users to progress through the solution at their own pace.\n", - "\n", - "(Example taken from [OpenAI cookbook](https://cookbook.openai.com/examples/structured_outputs_intro))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "**Note:** While OpenAI also offer structured output parsing via its beta API (`client.beta.chat.completions.parse`), this approach currently does not allow setting Langfuse specific attributes such as `name`, `metadata`, `userId` etc. Please use the approach using `response_format` with the standard `client.chat.completions.create` as described below." - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": { - "id": "8IZjc893Ef1c" - }, - "outputs": [], - "source": [ - "# Use the Langfuse drop-in replacement to get full logging by changing only the import.\n", - "# With that, you can monitor the structured output generated by OpenAI in Langfuse.\n", - "from langfuse.openai import OpenAI\n", - "import json\n", - "\n", - "openai_model = \"gpt-4o-2024-08-06\"\n", - "client = OpenAI()" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "YV7nfCKqV-FS" - }, - "source": [ - "In the `response_format` parameter you can now supply a JSON Schema via `json_schema`. When using `response_format` with `strict: true`, the model's output will adhere to the provided schema.\n", - "\n", - "Function calling remains similar, but with the new parameter `strict: true`, you can now ensure that the schema provided for the functions is strictly followed." - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": { - "id": "iKWRHCjVF55-" - }, - "outputs": [], - "source": [ - "math_tutor_prompt = '''\n", - " You are a helpful math tutor. You will be provided with a math problem,\n", - " and your goal will be to output a step by step solution, along with a final answer.\n", - " For each step, just provide the output as an equation use the explanation field to detail the reasoning.\n", - "'''\n", - "\n", - "def get_math_solution(question):\n", - " response = client.chat.completions.create(\n", - " model = openai_model,\n", - " messages=[\n", - " {\n", - " \"role\": \"system\",\n", - " \"content\": math_tutor_prompt\n", - " },\n", - " {\n", - " \"role\": \"user\",\n", - " \"content\": question\n", - " }\n", - " ],\n", - " response_format={\n", - " \"type\": \"json_schema\",\n", - " \"json_schema\": {\n", - " \"name\": \"math_reasoning\",\n", - " \"schema\": {\n", - " \"type\": \"object\",\n", - " \"properties\": {\n", - " \"steps\": {\n", - " \"type\": \"array\",\n", - " \"items\": {\n", - " \"type\": \"object\",\n", - " \"properties\": {\n", - " \"explanation\": {\"type\": \"string\"},\n", - " \"output\": {\"type\": \"string\"}\n", - " },\n", - " \"required\": [\"explanation\", \"output\"],\n", - " \"additionalProperties\": False\n", - " }\n", - " },\n", - " \"final_answer\": {\"type\": \"string\"}\n", - " },\n", - " \"required\": [\"steps\", \"final_answer\"],\n", - " \"additionalProperties\": False\n", - " },\n", - " \"strict\": True\n", - " }\n", - " }\n", - " )\n", - "\n", - " return response.choices[0].message" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "Qn9N-utqINy_", - "outputId": "2510cbb4-1784-4fcf-c7f5-ead1de58abfe" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "{\"steps\":[{\"explanation\":\"We need to isolate the term with the variable, 8x. So, we start by subtracting 7 from both sides to remove the constant term on the left side.\",\"output\":\"8x + 7 - 7 = -23 - 7\"},{\"explanation\":\"The +7 and -7 on the left side cancel each other out, leaving us with 8x. The right side simplifies to -30.\",\"output\":\"8x = -30\"},{\"explanation\":\"To solve for x, divide both sides of the equation by 8, which is the coefficient of x.\",\"output\":\"x = -30 / 8\"},{\"explanation\":\"Simplify the fraction -30/8 by finding the greatest common divisor, which is 2.\",\"output\":\"x = -15 / 4\"}],\"final_answer\":\"x = -15/4\"}\n" - ] - } - ], - "source": [ - "# Testing with an example question\n", - "question = \"how can I solve 8x + 7 = -23\"\n", - "\n", - "result = get_math_solution(question)\n", - "\n", - "print(result.content)" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 456 - }, - "id": "mAwEts-hCA73", - "outputId": "9101c8ff-7b83-4ae6-f6c0-0da6a45ab251" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Step 1: We need to isolate the term with the variable, 8x. So, we start by subtracting 7 from both sides to remove the constant term on the left side.\n", - "\n", - "8x + 7 - 7 = -23 - 7\n", - "\n", - "\n", - "Step 2: The +7 and -7 on the left side cancel each other out, leaving us with 8x. The right side simplifies to -30.\n", - "\n", - "8x = -30\n", - "\n", - "\n", - "Step 3: To solve for x, divide both sides of the equation by 8, which is the coefficient of x.\n", - "\n", - "x = -30 / 8\n", - "\n", - "\n", - "Step 4: Simplify the fraction -30/8 by finding the greatest common divisor, which is 2.\n", - "\n", - "x = -15 / 4\n", - "\n", - "\n", - "Final answer:\n", - "\n", - "\n", - "x = -15/4\n" - ] - } - ], - "source": [ - "# Print results step by step\n", - "\n", - "result = json.loads(result.content)\n", - "steps = result['steps']\n", - "final_answer = result['final_answer']\n", - "for i in range(len(steps)):\n", - " print(f\"Step {i+1}: {steps[i]['explanation']}\\n\")\n", - " print(steps[i]['output'])\n", - " print(\"\\n\")\n", - "\n", - "print(\"Final answer:\\n\\n\")\n", - "print(final_answer)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "KBoP_eEfLGMb" - }, - "source": [ - "## Step 3: See your trace in Langfuse\n", - "\n", - "You can now see the trace and the JSON schema in Langfuse.\n", - "\n", - "[Example trace in Langfuse](https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/traces/3ecc3849-66c9-4eaf-b26b-bde26b7eebed)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "xhY0etZVU33V" - }, - "source": [ - "![View example trace in the Langfuse UI](https://langfuse.com/images/cookbook/integration-openai-structured-outputs-tracing.png)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Alternative: Using the SDK `parse` helper\n", - "\n", - "The new SDK version adds a `parse` helper, allowing you to use your own Pydantic model without defining a JSON schema." - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": {}, - "outputs": [], - "source": [ - "from pydantic import BaseModel\n", - "\n", - "class MathReasoning(BaseModel):\n", - " class Step(BaseModel):\n", - " explanation: str\n", - " output: str\n", - "\n", - " steps: list[Step]\n", - " final_answer: str\n", - "\n", - "def get_math_solution(question: str):\n", - " response = client.beta.chat.completions.parse(\n", - " model=openai_model,\n", - " messages=[\n", - " {\"role\": \"system\", \"content\": math_tutor_prompt},\n", - " {\"role\": \"user\", \"content\": question},\n", - " ],\n", - " response_format=MathReasoning,\n", - " )\n", - "\n", - " return response.choices[0].message" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "[Step(explanation='To isolate the term with the variable on one side of the equation, start by subtracting 7 from both sides.', output='8x = -23 - 7'), Step(explanation='Combine like terms on the right side to simplify the equation.', output='8x = -30'), Step(explanation='Divide both sides by 8 to solve for x.', output='x = -30 / 8'), Step(explanation='Simplify the fraction by dividing both the numerator and the denominator by their greatest common divisor, which is 2.', output='x = -15 / 4')]\n", - "Final answer:\n", - "x = -15/4\n" - ] - } - ], - "source": [ - "result = get_math_solution(question).parsed\n", - "\n", - "print(result.steps)\n", - "print(\"Final answer:\")\n", - "print(result.final_answer)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## See your trace in Langfuse\n", - "\n", - "You can now see the trace and your supplied Pydantic model in Langfuse.\n", - "\n", - "[Example trace in Langfuse](https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/traces/59c4376a-c8eb-4ecb-8780-2f028b87e7eb)" - ] + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "m13Z9rJHCicu" + }, + "source": [ + "---\n", + "title: Observe OpenAI Structured Outputs with Langfuse\n", + "description: Learn how to use Langfuse to monitor OpenAI Structured Outputs\n", + "category: Integrations\n", + "---" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "yIyq5cb2GsrX" + }, + "source": [ + "# Cookbook: Trace OpenAI Structured Outputs with Langfuse\n", + "\n", + "In this cookbook you will learn how to use Langfuse to monitor OpenAI Structured Outputs.\n", + "\n", + "## What are structured outputs?\n", + "Generating structured data from unstructured inputs is a core AI use case today. Structured outputs make especially chained LLM calls, UI component generation, and model-based evaluation more reliable. [Structured Outputs](https://openai.com/index/introducing-structured-outputs-in-the-api/) is a new capability of the OpenAI API that builds upon JSON mode and function calling to enforce a strict schema in a model output.\n", + "\n", + "## How to trace structured output in Langfuse?\n", + "If you use the OpenAI Python SDK, you can use the [Langfuse drop-in replacement](https://langfuse.com/integrations/model-providers/openai-py) to get full logging by changing only the import. With that, you can monitor the structured output generated by OpenAI in Langfuse.\n", + "\n", + "```diff\n", + "- import openai\n", + "+ from langfuse.openai import openai\n", + "\n", + "Alternative imports:\n", + "+ from langfuse.openai import OpenAI, AsyncOpenAI, AzureOpenAI, AsyncAzureOpenAI\n", + "```\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "hH5klZnTTzuq" + }, + "source": [ + "## Step 1: Initialize Langfuse\n", + "Initialize the Langfuse client with your [API keys](https://langfuse.com/faq/all/where-are-langfuse-api-keys) from the project settings in the Langfuse UI and add them to your environment." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "![View example trace in the Langfuse UI](https://langfuse.com/images/cookbook/integration_openai_structured_outputs_tracing_parse.png)" - ] + "collapsed": true, + "id": "n11_vChnEdrg", + "outputId": "9c5c1aad-380d-4950-ae30-7d033deef0a9" + }, + "outputs": [], + "source": [ + "%pip install langfuse openai --upgrade" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "id": "79Cu3eLlEOCA" + }, + "outputs": [], + "source": [ + "import os\n", + "\n", + "# Get keys for your project from the project settings page\n", + "# https://cloud.langfuse.com\n", + "os.environ[\"LANGFUSE_PUBLIC_KEY\"] = \"\"\n", + "os.environ[\"LANGFUSE_SECRET_KEY\"] = \"\"\n", + "os.environ[\"LANGFUSE_BASE_URL\"] = \"https://cloud.langfuse.com\" # πŸ‡ͺπŸ‡Ί EU region\n", + "# Other Langfuse data regions include πŸ‡ΊπŸ‡Έ US: https://us.cloud.langfuse.com, πŸ‡―πŸ‡΅ Japan: https://jp.cloud.langfuse.com and βš•οΈ HIPAA: https://hipaa.cloud.langfuse.com\n", + "\n", + "# Your openai key\n", + "os.environ[\"OPENAI_API_KEY\"] = \"\"" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "eK8oTvWHMaTx" + }, + "source": [ + "## Step 2: Math tutor example\n", + "\n", + "In this example, we'll build a math tutoring tool that outputs steps to solve a math problem as an array of structured objects.\n", + "\n", + "This setup is useful for applications where each step needs to be displayed separately, allowing users to progress through the solution at their own pace.\n", + "\n", + "(Example taken from [OpenAI cookbook](https://cookbook.openai.com/examples/structured_outputs_intro))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "**Note on `parse` vs `response_format`:** The Langfuse OpenAI integration instruments both the stable `client.chat.completions.parse(...)` (available in `openai>=1.92.0`) and the legacy `client.beta.chat.completions.parse(...)` (for older SDK versions, where `parse` is re-routed to the stable method on newer SDKs). Both paths support Langfuse-specific attributes such as `name`, `metadata`, `langfuse_session_id`, etc. The `response_format` approach with `client.chat.completions.create` shown below works on all SDK versions and is useful if you cannot upgrade `openai`. For an example using the `parse` helper with a Pydantic model, see the section further down." + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "id": "8IZjc893Ef1c" + }, + "outputs": [], + "source": [ + "# Use the Langfuse drop-in replacement to get full logging by changing only the import.\n", + "# With that, you can monitor the structured output generated by OpenAI in Langfuse.\n", + "from langfuse.openai import OpenAI\n", + "import json\n", + "\n", + "openai_model = \"gpt-4o-2024-08-06\"\n", + "client = OpenAI()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "YV7nfCKqV-FS" + }, + "source": [ + "In the `response_format` parameter you can now supply a JSON Schema via `json_schema`. When using `response_format` with `strict: true`, the model's output will adhere to the provided schema.\n", + "\n", + "Function calling remains similar, but with the new parameter `strict: true`, you can now ensure that the schema provided for the functions is strictly followed." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "id": "iKWRHCjVF55-" + }, + "outputs": [], + "source": [ + "math_tutor_prompt = '''\n", + " You are a helpful math tutor. You will be provided with a math problem,\n", + " and your goal will be to output a step by step solution, along with a final answer.\n", + " For each step, just provide the output as an equation use the explanation field to detail the reasoning.\n", + "'''\n", + "\n", + "def get_math_solution(question):\n", + " response = client.chat.completions.create(\n", + " model = openai_model,\n", + " messages=[\n", + " {\n", + " \"role\": \"system\",\n", + " \"content\": math_tutor_prompt\n", + " },\n", + " {\n", + " \"role\": \"user\",\n", + " \"content\": question\n", + " }\n", + " ],\n", + " response_format={\n", + " \"type\": \"json_schema\",\n", + " \"json_schema\": {\n", + " \"name\": \"math_reasoning\",\n", + " \"schema\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"steps\": {\n", + " \"type\": \"array\",\n", + " \"items\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"explanation\": {\"type\": \"string\"},\n", + " \"output\": {\"type\": \"string\"}\n", + " },\n", + " \"required\": [\"explanation\", \"output\"],\n", + " \"additionalProperties\": False\n", + " }\n", + " },\n", + " \"final_answer\": {\"type\": \"string\"}\n", + " },\n", + " \"required\": [\"steps\", \"final_answer\"],\n", + " \"additionalProperties\": False\n", + " },\n", + " \"strict\": True\n", + " }\n", + " }\n", + " )\n", + "\n", + " return response.choices[0].message" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" }, + "id": "Qn9N-utqINy_", + "outputId": "2510cbb4-1784-4fcf-c7f5-ead1de58abfe" + }, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "UFWiSfqtEdJO" - }, - "source": [ - "## Feedback\n", - "\n", - "If you have any feedback or requests, please create a GitHub [Issue](https://langfuse.com/issue) or share your idea with the community on [Discord](https://langfuse.com/discord)." - ] + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"steps\":[{\"explanation\":\"We need to isolate the term with the variable, 8x. So, we start by subtracting 7 from both sides to remove the constant term on the left side.\",\"output\":\"8x + 7 - 7 = -23 - 7\"},{\"explanation\":\"The +7 and -7 on the left side cancel each other out, leaving us with 8x. The right side simplifies to -30.\",\"output\":\"8x = -30\"},{\"explanation\":\"To solve for x, divide both sides of the equation by 8, which is the coefficient of x.\",\"output\":\"x = -30 / 8\"},{\"explanation\":\"Simplify the fraction -30/8 by finding the greatest common divisor, which is 2.\",\"output\":\"x = -15 / 4\"}],\"final_answer\":\"x = -15/4\"}\n" + ] } - ], - "metadata": { + ], + "source": [ + "# Testing with an example question\n", + "question = \"how can I solve 8x + 7 = -23\"\n", + "\n", + "result = get_math_solution(question)\n", + "\n", + "print(result.content)" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { "colab": { - "provenance": [] + "base_uri": "https://localhost:8080/", + "height": 456 }, - "kernelspec": { - "display_name": "Python 3", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.18" + "id": "mAwEts-hCA73", + "outputId": "9101c8ff-7b83-4ae6-f6c0-0da6a45ab251" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Step 1: We need to isolate the term with the variable, 8x. So, we start by subtracting 7 from both sides to remove the constant term on the left side.\n", + "\n", + "8x + 7 - 7 = -23 - 7\n", + "\n", + "\n", + "Step 2: The +7 and -7 on the left side cancel each other out, leaving us with 8x. The right side simplifies to -30.\n", + "\n", + "8x = -30\n", + "\n", + "\n", + "Step 3: To solve for x, divide both sides of the equation by 8, which is the coefficient of x.\n", + "\n", + "x = -30 / 8\n", + "\n", + "\n", + "Step 4: Simplify the fraction -30/8 by finding the greatest common divisor, which is 2.\n", + "\n", + "x = -15 / 4\n", + "\n", + "\n", + "Final answer:\n", + "\n", + "\n", + "x = -15/4\n" + ] } + ], + "source": [ + "# Print results step by step\n", + "\n", + "result = json.loads(result.content)\n", + "steps = result['steps']\n", + "final_answer = result['final_answer']\n", + "for i in range(len(steps)):\n", + " print(f\"Step {i+1}: {steps[i]['explanation']}\\n\")\n", + " print(steps[i]['output'])\n", + " print(\"\\n\")\n", + "\n", + "print(\"Final answer:\\n\\n\")\n", + "print(final_answer)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "KBoP_eEfLGMb" + }, + "source": [ + "## Step 3: See your trace in Langfuse\n", + "\n", + "You can now see the trace and the JSON schema in Langfuse.\n", + "\n", + "[Example trace in Langfuse](https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/traces/3ecc3849-66c9-4eaf-b26b-bde26b7eebed)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "xhY0etZVU33V" + }, + "source": [ + "![View example trace in the Langfuse UI](https://langfuse.com/images/cookbook/integration-openai-structured-outputs-tracing.png)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "## Alternative: Using the SDK `parse` helper\n\nOpenAI also offers a `parse` helper that lets you pass a Pydantic model directly via `response_format`. As of [`openai-python` v1.92.0](https://github.com/openai/openai-python/releases/tag/v1.92.0), `parse` is part of the stable API at `client.chat.completions.parse(...)` (it previously lived under `client.beta.chat.completions.parse`). Langfuse instruments both paths, so you can set Langfuse attributes (`name`, `metadata`, `langfuse_session_id`, …) on either one. Prefer the stable path in new code:" + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": "from pydantic import BaseModel\n\nclass MathReasoning(BaseModel):\n class Step(BaseModel):\n explanation: str\n output: str\n\n steps: list[Step]\n final_answer: str\n\ndef get_math_solution(question: str):\n # Stable API on openai>=1.92.0. On older SDKs, use client.beta.chat.completions.parse instead.\n response = client.chat.completions.parse(\n model=openai_model,\n messages=[\n {\"role\": \"system\", \"content\": math_tutor_prompt},\n {\"role\": \"user\", \"content\": question},\n ],\n response_format=MathReasoning,\n name=\"math-tutor-parse\",\n )\n\n return response.choices[0].message" + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[Step(explanation='To isolate the term with the variable on one side of the equation, start by subtracting 7 from both sides.', output='8x = -23 - 7'), Step(explanation='Combine like terms on the right side to simplify the equation.', output='8x = -30'), Step(explanation='Divide both sides by 8 to solve for x.', output='x = -30 / 8'), Step(explanation='Simplify the fraction by dividing both the numerator and the denominator by their greatest common divisor, which is 2.', output='x = -15 / 4')]\n", + "Final answer:\n", + "x = -15/4\n" + ] + } + ], + "source": [ + "result = get_math_solution(question).parsed\n", + "\n", + "print(result.steps)\n", + "print(\"Final answer:\")\n", + "print(result.final_answer)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## See your trace in Langfuse\n", + "\n", + "You can now see the trace and your supplied Pydantic model in Langfuse.\n", + "\n", + "[Example trace in Langfuse](https://cloud.langfuse.com/project/cloramnkj0002jz088vzn1ja4/traces/59c4376a-c8eb-4ecb-8780-2f028b87e7eb)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![View example trace in the Langfuse UI](https://langfuse.com/images/cookbook/integration_openai_structured_outputs_tracing_parse.png)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "UFWiSfqtEdJO" + }, + "source": [ + "## Feedback\n", + "\n", + "If you have any feedback or requests, please create a GitHub [Issue](https://langfuse.com/issue) or share your idea with the community on [Discord](https://langfuse.com/discord)." + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" }, - "nbformat": 4, - "nbformat_minor": 0 -} + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.18" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} \ No newline at end of file