Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions docs/llms-full.txt
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,7 @@ The Agents SDK delivers a focused set of Python primitives—agents, tools, guar
- [Realtime guide](https://openai.github.io/openai-agents-python/realtime/guide/): Deep dive into realtime session lifecycle, structured input, approvals, interruptions, and low-level transport control.

## Models and Provider Integrations
- [Model catalog](https://openai.github.io/openai-agents-python/models/): Lists supported OpenAI and partner models with guidance on selecting capabilities for different workloads.
- [LiteLLM integration](https://openai.github.io/openai-agents-python/models/litellm/): Configure LiteLLM as a provider, map model aliases, and route requests across heterogeneous backends.
- [Model catalog](https://openai.github.io/openai-agents-python/models/): Covers OpenAI model selection, non-OpenAI provider patterns, websocket transport, and the SDK's best-effort LiteLLM guidance in one place.

## API Reference – Agents SDK Core
- [API index](https://openai.github.io/openai-agents-python/ref/index/): Directory of all documented modules, classes, and functions in the SDK.
Expand Down
3 changes: 1 addition & 2 deletions docs/llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,7 @@ The SDK focuses on a concise set of primitives so you can orchestrate multi-agen
- [Extensions](https://openai.github.io/openai-agents-python/ref/extensions/handoff_filters/): Extend the SDK with custom handoff filters, prompts, LiteLLM integration, and SQLAlchemy session memory.

## Models and Providers
- [Model catalog](https://openai.github.io/openai-agents-python/models/): Overview of supported model families and configuration guidance.
- [LiteLLM integration](https://openai.github.io/openai-agents-python/models/litellm/): Configure LiteLLM as a provider to fan out across multiple model backends.
- [Model catalog](https://openai.github.io/openai-agents-python/models/): Overview of OpenAI models, non-OpenAI provider patterns, websocket transport, and the SDK's best-effort LiteLLM guidance.

## Optional
- [Release notes](https://openai.github.io/openai-agents-python/release/): Track SDK changes, migration notes, and deprecations.
Expand Down
143 changes: 84 additions & 59 deletions docs/models/index.md

Large diffs are not rendered by default.

103 changes: 6 additions & 97 deletions docs/models/litellm.md
Original file line number Diff line number Diff line change
@@ -1,100 +1,9 @@
# Using any model via LiteLLM
# LiteLLM

!!! note
<script>
window.location.replace("../#litellm");
</script>

The LiteLLM integration is in beta. You may run into issues with some model providers, especially smaller ones. Please report any issues via [Github issues](https://github.com/openai/openai-agents-python/issues) and we'll fix quickly.
This page moved to the [LiteLLM section in Models](index.md#litellm).

[LiteLLM](https://docs.litellm.ai/docs/) is a library that allows you to use 100+ models via a single interface. We've added a LiteLLM integration to allow you to use any AI model in the Agents SDK.

## Setup

You'll need to ensure `litellm` is available. You can do this by installing the optional `litellm` dependency group:

```bash
pip install "openai-agents[litellm]"
```

Once done, you can use [`LitellmModel`][agents.extensions.models.litellm_model.LitellmModel] in any agent.

## Example

This is a fully working example. When you run it, you'll be prompted for a model name and API key. For example, you could enter:

- `openai/gpt-4.1` for the model, and your OpenAI API key
- `anthropic/claude-3-5-sonnet-20240620` for the model, and your Anthropic API key
- etc

For a full list of models supported in LiteLLM, see the [litellm providers docs](https://docs.litellm.ai/docs/providers).

```python
from __future__ import annotations

import asyncio

from agents import Agent, Runner, function_tool, set_tracing_disabled
from agents.extensions.models.litellm_model import LitellmModel

@function_tool
def get_weather(city: str):
print(f"[debug] getting weather for {city}")
return f"The weather in {city} is sunny."


async def main(model: str, api_key: str):
agent = Agent(
name="Assistant",
instructions="You only respond in haikus.",
model=LitellmModel(model=model, api_key=api_key),
tools=[get_weather],
)

result = await Runner.run(agent, "What's the weather in Tokyo?")
print(result.final_output)


if __name__ == "__main__":
# First try to get model/api key from args
import argparse

parser = argparse.ArgumentParser()
parser.add_argument("--model", type=str, required=False)
parser.add_argument("--api-key", type=str, required=False)
args = parser.parse_args()

model = args.model
if not model:
model = input("Enter a model name for Litellm: ")

api_key = args.api_key
if not api_key:
api_key = input("Enter an API key for Litellm: ")

asyncio.run(main(model, api_key))
```

## Tracking usage data

If you want LiteLLM responses to populate the Agents SDK usage metrics, pass `ModelSettings(include_usage=True)` when creating your agent.

```python
from agents import Agent, ModelSettings
from agents.extensions.models.litellm_model import LitellmModel

agent = Agent(
name="Assistant",
model=LitellmModel(model="your/model", api_key="..."),
model_settings=ModelSettings(include_usage=True),
)
```

With `include_usage=True`, LiteLLM requests report token and request counts through `result.context_wrapper.usage` just like the built-in OpenAI models.

## Troubleshooting

If you see Pydantic serializer warnings from LiteLLM responses, enable a small compatibility patch by setting:

```bash
export OPENAI_AGENTS_ENABLE_LITELLM_SERIALIZER_PATCH=true
```

This opt-in flag suppresses known LiteLLM serializer warnings while preserving normal behavior. Turn it off (unset or `false`) if you do not need it.
If you are not redirected automatically, use the link above.
33 changes: 33 additions & 0 deletions docs/stylesheets/extra.css
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,32 @@
font-size: 14px;
}

.md-typeset .field-table {
overflow-x: auto;
}

.md-typeset .field-table table:not([class]) {
display: table;
table-layout: fixed;
width: 100%;
}

.md-typeset .field-table table:not([class]) th:first-child,
.md-typeset .field-table table:not([class]) td:first-child {
width: 11rem;
}

.md-typeset .field-table table:not([class]) th:nth-child(2),
.md-typeset .field-table table:not([class]) td:nth-child(2) {
width: 18rem;
}

.md-typeset .field-table table:not([class]) th:first-child code,
.md-typeset .field-table table:not([class]) td:first-child code {
white-space: nowrap;
word-break: normal;
}

/* Custom link styling */
.md-content a {
text-decoration: none;
Expand Down Expand Up @@ -203,3 +229,10 @@
.md-sidebar__scrollwrap {
scrollbar-color: auto !important;
}

/* Let the docs layout use more of large viewports without becoming fully fluid. */
@media screen and (min-width: 76.25em) {
.md-grid {
max-width: clamp(76rem, 92vw, 92rem);
}
}
2 changes: 1 addition & 1 deletion docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Usage is aggregated across all model calls during the run (including tool calls

### Enabling usage with LiteLLM models

LiteLLM providers do not report usage metrics by default. When you are using [`LitellmModel`](models/litellm.md), pass `ModelSettings(include_usage=True)` to your agent so that LiteLLM responses populate `result.context_wrapper.usage`.
LiteLLM providers do not report usage metrics by default. When you are using [`LitellmModel`][agents.extensions.models.litellm_model.LitellmModel], pass `ModelSettings(include_usage=True)` to your agent so that LiteLLM responses populate `result.context_wrapper.usage`. See the [LiteLLM note](models/index.md#litellm) in the Models guide for setup guidance and examples.

```python
from agents import Agent, ModelSettings, Runner
Expand Down
16 changes: 4 additions & 12 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,9 +56,7 @@ plugins:
- Configuration: config.md
- Documentation:
- agents.md
- Models:
- models/index.md
- models/litellm.md
- Models: models/index.md
- tools.md
- guardrails.md
- running_agents.md
Expand Down Expand Up @@ -177,9 +175,7 @@ plugins:
- config.md
- ドキュメント:
- agents.md
- モデル:
- models/index.md
- models/litellm.md
- モデル: models/index.md
- tools.md
- guardrails.md
- running_agents.md
Expand Down Expand Up @@ -217,9 +213,7 @@ plugins:
- config.md
- 문서:
- agents.md
- 모델:
- models/index.md
- models/litellm.md
- 모델: models/index.md
- tools.md
- guardrails.md
- running_agents.md
Expand Down Expand Up @@ -257,9 +251,7 @@ plugins:
- config.md
- 文档:
- agents.md
- 模型:
- models/index.md
- models/litellm.md
- 模型: models/index.md
- tools.md
- guardrails.md
- running_agents.md
Expand Down
Loading