[INFERENCE PROVIDERS] guide on structured output#1824
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| - local: guides/building-first-app | ||
| title: Building Your First AI App | ||
| - local: guides/structured-output | ||
| title: Structured Outputs with LLMs |
There was a problem hiding this comment.
I would align the toctree title with the doc title (or vice versa) to avoid confusion for users when referencing the doc
There was a problem hiding this comment.
shouldn't the guides be under the provider list?
i think we want to focus on providers more
There was a problem hiding this comment.
(forgot to post this on the previous PR @burtenshaw sorry =)
There was a problem hiding this comment.
No worries. I'll make another PR with the menu change.
| @@ -0,0 +1,467 @@ | |||
| # Structured Outputs with Inference Providers | |||
|
|
|||
| In this guide, we'll show you how to use Inference Providers to generate structured outputs that follow a specific JSON schema. This is incredibly useful for building reliable AI applications that need predictable, parseable responses. | |||
There was a problem hiding this comment.
| In this guide, we'll show you how to use Inference Providers to generate structured outputs that follow a specific JSON schema. This is incredibly useful for building reliable AI applications that need predictable, parseable responses. | |
| In this guide, we'll show you how to use Inference Providers to generate structured outputs that follow a specific JSON schema. This is incredibly useful for building reliable AI applications that need predictable, parsable responses. |
|
|
||
| We'll create a simple schema that captures the most essential elements: the paper's title and a summary of its abstract. The easiest way to do this is to use Pydantic, a library that allows you to define Python classes that represent JSON schemas (among other things). | ||
|
|
||
| <hfoptions id="json-pydantic"> |
There was a problem hiding this comment.
I wouldn't use the toggles here since it sets up as a kind of "choose one or the other". After showing the Pydantic code example, maybe transition with:
"The Pydantic represented JSON schema is shown below."
|
|
||
| </hfoption> | ||
|
|
||
| The OpenAI client returns a `ChatCompletion` object, which contains the response from the model as a Python object. You can then access the structured data using the `title` and `abstract_summary` attributes of the `PaperAnalysis` class. |
There was a problem hiding this comment.
This should go under the <hfoption id="openai"> toggle
julien-c
left a comment
julien-c
left a comment
There was a problem hiding this comment.
cool doc page, is there one about tool calling coming up as well? :)
Note, @SBrandeis has been building some validation matrix of model/provider support for structured output & function calling, we could dynamically display it here in this page (or link to it from this page)
cc @gary149 too for viz
Yep. It's coming tomorrow.
👍 |
|
|
||
| ## Step 2: Set up your inference client | ||
|
|
||
| Now that we have our schema defined, let's set up the client to communicate with the inference providers. We'll show you two approaches: the Hugging Face Hub client (which gives you direct access to all Inference Providers) and the OpenAI client (which works through OpenAI-compatible endpoints). |
There was a problem hiding this comment.
Sadge - it's only Python 🥲
There was a problem hiding this comment.
true. I'll come back to that on all new guides.
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
…ace/hub-docs into guide-structured-output
5 participants
This PR adds a guide on structured outputs with inference providers. It uses hf client, openai client, cerebras, qwen3 32b