|
| 1 | +<p align="center"> |
| 2 | + <picture> |
| 3 | + <img src="https://raw.githubusercontent.com/mozilla-ai/otari/refs/heads/main/docs/public/images/otari-logo-mark.png" width="20%" alt="Project logo"/> |
| 4 | + </picture> |
| 5 | +</p> |
| 6 | + |
| 7 | +<div align="center"> |
| 8 | + |
1 | 9 | # otari (Python) |
2 | 10 |
|
3 | | -Python client for the [otari gateway](https://github.com/mozilla-ai/otari). |
| 11 | + |
| 12 | +[](https://pypi.org/project/otari/) |
| 13 | +<a href="https://discord.gg/4gf3zXrQUc"> |
| 14 | + <img src="https://img.shields.io/static/v1?label=Chat%20on&message=Discord&color=blue&logo=Discord&style=flat-square" alt="Discord"> |
| 15 | +</a> |
| 16 | + |
| 17 | +**Python client for [otari-gateway](https://github.com/mozilla-ai/otari).** |
| 18 | +Communicate with any LLM provider through the gateway using a single, typed interface. |
| 19 | + |
| 20 | +[TypeScript SDK](https://github.com/mozilla-ai/otari-sdk-ts) | [Documentation](https://mozilla-ai.github.io/otari/) | [Platform (Beta)](https://otari.ai/) |
| 21 | + |
| 22 | +</div> |
| 23 | + |
| 24 | +## Quickstart |
| 25 | + |
| 26 | +```python |
| 27 | +from otari import OtariClient |
| 28 | + |
| 29 | +client = OtariClient( |
| 30 | + api_base="http://localhost:8000", |
| 31 | + platform_token="your-token-here", |
| 32 | +) |
| 33 | + |
| 34 | +response = await client.completion( |
| 35 | + model="openai:gpt-4o-mini", |
| 36 | + messages=[{"role": "user", "content": "Hello!"}], |
| 37 | +) |
| 38 | + |
| 39 | +print(response.choices[0].message.content) |
| 40 | +``` |
| 41 | + |
| 42 | +**That's it!** Change the model string to switch between LLM providers through the gateway. |
| 43 | + |
| 44 | +## Installation |
| 45 | + |
| 46 | +### Requirements |
| 47 | + |
| 48 | +- Python 3.11 or newer |
| 49 | +- A running [otari-gateway](https://mozilla-ai.github.io/otari/gateway/overview/) instance |
| 50 | + |
| 51 | +### Install |
| 52 | + |
| 53 | +```bash |
| 54 | +pip install otari |
| 55 | +``` |
| 56 | + |
| 57 | +### Setting Up Credentials |
| 58 | + |
| 59 | +Set environment variables for your gateway: |
| 60 | + |
| 61 | +```bash |
| 62 | +export GATEWAY_API_BASE="http://localhost:8000" |
| 63 | +export GATEWAY_PLATFORM_TOKEN="your-token-here" |
| 64 | +# or for non-platform mode: |
| 65 | +export GATEWAY_API_KEY="your-key-here" |
| 66 | +``` |
| 67 | + |
| 68 | +Alternatively, pass credentials directly when creating the client (see [Usage](#usage) examples). |
| 69 | + |
| 70 | +## otari-gateway |
| 71 | + |
| 72 | +This Python SDK is a client for [otari-gateway](https://github.com/mozilla-ai/otari), an **optional** FastAPI-based proxy server that adds enterprise-grade features on top of the core library: |
| 73 | + |
| 74 | +- **Budget Management** - Enforce spending limits with automatic daily, weekly, or monthly resets |
| 75 | +- **API Key Management** - Issue, revoke, and monitor virtual API keys without exposing provider credentials |
| 76 | +- **Usage Analytics** - Track every request with full token counts, costs, and metadata |
| 77 | +- **Multi-tenant Support** - Manage access and budgets across users and teams |
| 78 | + |
| 79 | +The gateway sits between your applications and LLM providers, exposing an OpenAI-compatible API that works with any supported provider. |
| 80 | + |
| 81 | +### Quick Start |
| 82 | + |
| 83 | +```bash |
| 84 | +docker run \ |
| 85 | + -e GATEWAY_MASTER_KEY="your-secure-master-key" \ |
| 86 | + -e OPENAI_API_KEY="your-api-key" \ |
| 87 | + -p 8000:8000 \ |
| 88 | + ghcr.io/mozilla-ai/otari/gateway:latest |
| 89 | +``` |
| 90 | + |
| 91 | +> **Note:** You can use a specific release version instead of `latest` (e.g., `1.2.0`). See [available versions](https://github.com/orgs/mozilla-ai/packages/container/package/otari%2Fgateway). |
| 92 | +
|
| 93 | +### Managed Platform (Beta) |
| 94 | + |
| 95 | +Prefer a hosted experience? The [otari platform](https://otari.ai/) provides a managed control plane for keys, usage tracking, and cost visibility across providers, while still building on the same `otari` interfaces. |
| 96 | + |
| 97 | +## Usage |
| 98 | + |
| 99 | +### Authentication Modes |
| 100 | + |
| 101 | +The client supports two authentication modes, matching the TypeScript SDK: |
| 102 | + |
| 103 | +#### Platform Mode (Recommended) |
| 104 | + |
| 105 | +Uses a Bearer token in the standard Authorization header: |
| 106 | + |
| 107 | +```python |
| 108 | +client = OtariClient( |
| 109 | + api_base="http://localhost:8000", |
| 110 | + platform_token="tk_your_platform_token", |
| 111 | +) |
| 112 | +``` |
| 113 | + |
| 114 | +#### Non-Platform Mode |
| 115 | + |
| 116 | +Sends the API key via a custom `Otari-Key` header: |
| 117 | + |
| 118 | +```python |
| 119 | +client = OtariClient( |
| 120 | + api_base="http://localhost:8000", |
| 121 | + api_key="your-api-key", |
| 122 | +) |
| 123 | +``` |
| 124 | + |
| 125 | +#### Auto-Detection from Environment Variables |
| 126 | + |
| 127 | +When no explicit credentials are provided, the client reads from environment variables: |
| 128 | + |
| 129 | +```python |
| 130 | +# Uses GATEWAY_API_BASE, GATEWAY_PLATFORM_TOKEN, or GATEWAY_API_KEY |
| 131 | +client = OtariClient() |
| 132 | +``` |
| 133 | + |
| 134 | +### Chat Completions |
| 135 | + |
| 136 | +```python |
| 137 | +response = await client.completion( |
| 138 | + model="openai:gpt-4o-mini", |
| 139 | + messages=[{"role": "user", "content": "Hello!"}], |
| 140 | +) |
| 141 | + |
| 142 | +print(response.choices[0].message.content) |
| 143 | +``` |
| 144 | + |
| 145 | +### Streaming |
| 146 | + |
| 147 | +```python |
| 148 | +stream = await client.completion( |
| 149 | + model="openai:gpt-4o-mini", |
| 150 | + messages=[{"role": "user", "content": "Tell me a story."}], |
| 151 | + stream=True, |
| 152 | +) |
| 153 | + |
| 154 | +async for chunk in stream: |
| 155 | + content = chunk.choices[0].delta.content |
| 156 | + if content: |
| 157 | + print(content, end="", flush=True) |
| 158 | +``` |
| 159 | + |
| 160 | +### Responses API |
| 161 | + |
| 162 | +```python |
| 163 | +response = await client.response( |
| 164 | + model="openai:gpt-4o-mini", |
| 165 | + input="Summarize this in one sentence.", |
| 166 | +) |
| 167 | + |
| 168 | +print(response.output_text) |
| 169 | +``` |
| 170 | + |
| 171 | +### Embeddings |
| 172 | + |
| 173 | +```python |
| 174 | +result = await client.embedding( |
| 175 | + model="openai:text-embedding-3-small", |
| 176 | + input="Hello world", |
| 177 | +) |
| 178 | + |
| 179 | +print(result.data[0].embedding) |
| 180 | +``` |
| 181 | + |
| 182 | +### Listing Models |
| 183 | + |
| 184 | +```python |
| 185 | +models = await client.list_models() |
| 186 | +for model in models: |
| 187 | + print(model.id) |
| 188 | +``` |
| 189 | + |
| 190 | +### Error Handling |
| 191 | + |
| 192 | +In platform mode, HTTP errors are mapped to typed exceptions: |
| 193 | + |
| 194 | +```python |
| 195 | +from otari import OtariClient, AuthenticationError, RateLimitError |
| 196 | + |
| 197 | +try: |
| 198 | + response = await client.completion( |
| 199 | + model="openai:gpt-4o-mini", |
| 200 | + messages=[{"role": "user", "content": "Hello!"}], |
| 201 | + ) |
| 202 | +except AuthenticationError as e: |
| 203 | + print(f"Invalid credentials: {e.message}") |
| 204 | +except RateLimitError as e: |
| 205 | + print(f"Rate limited, retry after: {e.retry_after}") |
| 206 | +``` |
| 207 | + |
| 208 | +| HTTP Status | Error Class | Description | |
| 209 | +|------------|-------------|-------------| |
| 210 | +| 400 (capability) | `UnsupportedCapabilityError` | Selected provider does not support the requested capability | |
| 211 | +| 401, 403 | `AuthenticationError` | Invalid or missing credentials | |
| 212 | +| 402 | `InsufficientFundsError` | Budget or credits exhausted | |
| 213 | +| 404 | `ModelNotFoundError` | Model not found or unavailable | |
| 214 | +| 429 | `RateLimitError` | Rate limit exceeded (includes `retry_after`) | |
| 215 | +| 502 | `UpstreamProviderError` | Upstream provider unreachable | |
| 216 | +| 504 | `GatewayTimeoutError` | Gateway timed out waiting for provider | |
| 217 | + |
| 218 | +`UnsupportedCapabilityError` surfaces in both platform and non-platform modes; the other mappings are platform-mode only. |
| 219 | + |
| 220 | +### Context Manager |
| 221 | + |
| 222 | +The client supports async context manager for automatic cleanup: |
| 223 | + |
| 224 | +```python |
| 225 | +async with OtariClient(api_base="http://localhost:8000") as client: |
| 226 | + response = await client.completion( |
| 227 | + model="openai:gpt-4o-mini", |
| 228 | + messages=[{"role": "user", "content": "Hello!"}], |
| 229 | + ) |
| 230 | +``` |
| 231 | + |
| 232 | +## Why choose `otari`? |
| 233 | + |
| 234 | +- **Simple, unified interface** - Single client for all providers through the gateway, switch models with just a string change |
| 235 | +- **Developer friendly** - Full type hints for better IDE support and clear, actionable error messages |
| 236 | +- **Leverages the OpenAI SDK** - Built on the official OpenAI Python SDK for maximum compatibility |
| 237 | +- **Async-first** - Built on `AsyncOpenAI` for modern async Python applications |
| 238 | +- **Stays framework-agnostic** so it can be used across different projects and use cases |
| 239 | +- **Battle-tested** - Powers our own production tools ([any-agent](https://github.com/mozilla-ai/any-agent)) |
| 240 | + |
| 241 | +## Development |
| 242 | + |
| 243 | +```bash |
| 244 | +# Create a virtual environment |
| 245 | +python -m venv .venv |
| 246 | +source .venv/bin/activate |
| 247 | + |
| 248 | +# Install with dev dependencies |
| 249 | +pip install -e ".[dev]" |
| 250 | + |
| 251 | +# Run unit tests |
| 252 | +pytest tests/ |
| 253 | + |
| 254 | +# Lint |
| 255 | +ruff check src/ tests/ |
| 256 | + |
| 257 | +# Type-check |
| 258 | +mypy src/ |
| 259 | +``` |
| 260 | + |
| 261 | +## Documentation |
| 262 | + |
| 263 | +- **[Full Documentation](https://mozilla-ai.github.io/otari/)** - Complete guides and API reference |
| 264 | +- **[Supported Providers](https://mozilla-ai.github.io/otari/providers/)** - List of all supported LLM providers |
| 265 | +- **[Gateway Documentation](https://mozilla-ai.github.io/otari/gateway/overview/)** - Gateway setup and deployment |
| 266 | +- **[TypeScript SDK](https://github.com/mozilla-ai/otari-sdk-ts)** - The TypeScript SDK for Node.js applications |
| 267 | +- **[otari Platform (Beta)](https://otari.ai/)** - Hosted control plane for key management, usage tracking, and cost visibility |
| 268 | + |
| 269 | +## Contributing |
| 270 | + |
| 271 | +We welcome contributions from developers of all skill levels! Please see the [Contributing Guide](https://github.com/mozilla-ai/otari/blob/main/CONTRIBUTING.md) or open an issue to discuss changes. |
| 272 | + |
| 273 | +## License |
| 274 | + |
| 275 | +This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details. |
0 commit comments