Skip to content

Commit 00b1413

Browse files
committed
Add comprehensive README matching TypeScript SDK structure
1 parent aebf27f commit 00b1413

1 file changed

Lines changed: 273 additions & 1 deletion

File tree

README.md

Lines changed: 273 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,275 @@
1+
<p align="center">
2+
<picture>
3+
<img src="https://raw.githubusercontent.com/mozilla-ai/otari/refs/heads/main/docs/public/images/otari-logo-mark.png" width="20%" alt="Project logo"/>
4+
</picture>
5+
</p>
6+
7+
<div align="center">
8+
19
# otari (Python)
210

3-
Python client for the [otari gateway](https://github.com/mozilla-ai/otari).
11+
![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue.svg)
12+
[![PyPI](https://img.shields.io/pypi/v/otari)](https://pypi.org/project/otari/)
13+
<a href="https://discord.gg/4gf3zXrQUc">
14+
<img src="https://img.shields.io/static/v1?label=Chat%20on&message=Discord&color=blue&logo=Discord&style=flat-square" alt="Discord">
15+
</a>
16+
17+
**Python client for [otari-gateway](https://github.com/mozilla-ai/otari).**
18+
Communicate with any LLM provider through the gateway using a single, typed interface.
19+
20+
[TypeScript SDK](https://github.com/mozilla-ai/otari-sdk-ts) | [Documentation](https://mozilla-ai.github.io/otari/) | [Platform (Beta)](https://otari.ai/)
21+
22+
</div>
23+
24+
## Quickstart
25+
26+
```python
27+
from otari import OtariClient
28+
29+
client = OtariClient(
30+
api_base="http://localhost:8000",
31+
platform_token="your-token-here",
32+
)
33+
34+
response = await client.completion(
35+
model="openai:gpt-4o-mini",
36+
messages=[{"role": "user", "content": "Hello!"}],
37+
)
38+
39+
print(response.choices[0].message.content)
40+
```
41+
42+
**That's it!** Change the model string to switch between LLM providers through the gateway.
43+
44+
## Installation
45+
46+
### Requirements
47+
48+
- Python 3.11 or newer
49+
- A running [otari-gateway](https://mozilla-ai.github.io/otari/gateway/overview/) instance
50+
51+
### Install
52+
53+
```bash
54+
pip install otari
55+
```
56+
57+
### Setting Up Credentials
58+
59+
Set environment variables for your gateway:
60+
61+
```bash
62+
export GATEWAY_API_BASE="http://localhost:8000"
63+
export GATEWAY_PLATFORM_TOKEN="your-token-here"
64+
# or for non-platform mode:
65+
export GATEWAY_API_KEY="your-key-here"
66+
```
67+
68+
Alternatively, pass credentials directly when creating the client (see [Usage](#usage) examples).
69+
70+
## otari-gateway
71+
72+
This Python SDK is a client for [otari-gateway](https://github.com/mozilla-ai/otari), an **optional** FastAPI-based proxy server that adds enterprise-grade features on top of the core library:
73+
74+
- **Budget Management** - Enforce spending limits with automatic daily, weekly, or monthly resets
75+
- **API Key Management** - Issue, revoke, and monitor virtual API keys without exposing provider credentials
76+
- **Usage Analytics** - Track every request with full token counts, costs, and metadata
77+
- **Multi-tenant Support** - Manage access and budgets across users and teams
78+
79+
The gateway sits between your applications and LLM providers, exposing an OpenAI-compatible API that works with any supported provider.
80+
81+
### Quick Start
82+
83+
```bash
84+
docker run \
85+
-e GATEWAY_MASTER_KEY="your-secure-master-key" \
86+
-e OPENAI_API_KEY="your-api-key" \
87+
-p 8000:8000 \
88+
ghcr.io/mozilla-ai/otari/gateway:latest
89+
```
90+
91+
> **Note:** You can use a specific release version instead of `latest` (e.g., `1.2.0`). See [available versions](https://github.com/orgs/mozilla-ai/packages/container/package/otari%2Fgateway).
92+
93+
### Managed Platform (Beta)
94+
95+
Prefer a hosted experience? The [otari platform](https://otari.ai/) provides a managed control plane for keys, usage tracking, and cost visibility across providers, while still building on the same `otari` interfaces.
96+
97+
## Usage
98+
99+
### Authentication Modes
100+
101+
The client supports two authentication modes, matching the TypeScript SDK:
102+
103+
#### Platform Mode (Recommended)
104+
105+
Uses a Bearer token in the standard Authorization header:
106+
107+
```python
108+
client = OtariClient(
109+
api_base="http://localhost:8000",
110+
platform_token="tk_your_platform_token",
111+
)
112+
```
113+
114+
#### Non-Platform Mode
115+
116+
Sends the API key via a custom `Otari-Key` header:
117+
118+
```python
119+
client = OtariClient(
120+
api_base="http://localhost:8000",
121+
api_key="your-api-key",
122+
)
123+
```
124+
125+
#### Auto-Detection from Environment Variables
126+
127+
When no explicit credentials are provided, the client reads from environment variables:
128+
129+
```python
130+
# Uses GATEWAY_API_BASE, GATEWAY_PLATFORM_TOKEN, or GATEWAY_API_KEY
131+
client = OtariClient()
132+
```
133+
134+
### Chat Completions
135+
136+
```python
137+
response = await client.completion(
138+
model="openai:gpt-4o-mini",
139+
messages=[{"role": "user", "content": "Hello!"}],
140+
)
141+
142+
print(response.choices[0].message.content)
143+
```
144+
145+
### Streaming
146+
147+
```python
148+
stream = await client.completion(
149+
model="openai:gpt-4o-mini",
150+
messages=[{"role": "user", "content": "Tell me a story."}],
151+
stream=True,
152+
)
153+
154+
async for chunk in stream:
155+
content = chunk.choices[0].delta.content
156+
if content:
157+
print(content, end="", flush=True)
158+
```
159+
160+
### Responses API
161+
162+
```python
163+
response = await client.response(
164+
model="openai:gpt-4o-mini",
165+
input="Summarize this in one sentence.",
166+
)
167+
168+
print(response.output_text)
169+
```
170+
171+
### Embeddings
172+
173+
```python
174+
result = await client.embedding(
175+
model="openai:text-embedding-3-small",
176+
input="Hello world",
177+
)
178+
179+
print(result.data[0].embedding)
180+
```
181+
182+
### Listing Models
183+
184+
```python
185+
models = await client.list_models()
186+
for model in models:
187+
print(model.id)
188+
```
189+
190+
### Error Handling
191+
192+
In platform mode, HTTP errors are mapped to typed exceptions:
193+
194+
```python
195+
from otari import OtariClient, AuthenticationError, RateLimitError
196+
197+
try:
198+
response = await client.completion(
199+
model="openai:gpt-4o-mini",
200+
messages=[{"role": "user", "content": "Hello!"}],
201+
)
202+
except AuthenticationError as e:
203+
print(f"Invalid credentials: {e.message}")
204+
except RateLimitError as e:
205+
print(f"Rate limited, retry after: {e.retry_after}")
206+
```
207+
208+
| HTTP Status | Error Class | Description |
209+
|------------|-------------|-------------|
210+
| 400 (capability) | `UnsupportedCapabilityError` | Selected provider does not support the requested capability |
211+
| 401, 403 | `AuthenticationError` | Invalid or missing credentials |
212+
| 402 | `InsufficientFundsError` | Budget or credits exhausted |
213+
| 404 | `ModelNotFoundError` | Model not found or unavailable |
214+
| 429 | `RateLimitError` | Rate limit exceeded (includes `retry_after`) |
215+
| 502 | `UpstreamProviderError` | Upstream provider unreachable |
216+
| 504 | `GatewayTimeoutError` | Gateway timed out waiting for provider |
217+
218+
`UnsupportedCapabilityError` surfaces in both platform and non-platform modes; the other mappings are platform-mode only.
219+
220+
### Context Manager
221+
222+
The client supports async context manager for automatic cleanup:
223+
224+
```python
225+
async with OtariClient(api_base="http://localhost:8000") as client:
226+
response = await client.completion(
227+
model="openai:gpt-4o-mini",
228+
messages=[{"role": "user", "content": "Hello!"}],
229+
)
230+
```
231+
232+
## Why choose `otari`?
233+
234+
- **Simple, unified interface** - Single client for all providers through the gateway, switch models with just a string change
235+
- **Developer friendly** - Full type hints for better IDE support and clear, actionable error messages
236+
- **Leverages the OpenAI SDK** - Built on the official OpenAI Python SDK for maximum compatibility
237+
- **Async-first** - Built on `AsyncOpenAI` for modern async Python applications
238+
- **Stays framework-agnostic** so it can be used across different projects and use cases
239+
- **Battle-tested** - Powers our own production tools ([any-agent](https://github.com/mozilla-ai/any-agent))
240+
241+
## Development
242+
243+
```bash
244+
# Create a virtual environment
245+
python -m venv .venv
246+
source .venv/bin/activate
247+
248+
# Install with dev dependencies
249+
pip install -e ".[dev]"
250+
251+
# Run unit tests
252+
pytest tests/
253+
254+
# Lint
255+
ruff check src/ tests/
256+
257+
# Type-check
258+
mypy src/
259+
```
260+
261+
## Documentation
262+
263+
- **[Full Documentation](https://mozilla-ai.github.io/otari/)** - Complete guides and API reference
264+
- **[Supported Providers](https://mozilla-ai.github.io/otari/providers/)** - List of all supported LLM providers
265+
- **[Gateway Documentation](https://mozilla-ai.github.io/otari/gateway/overview/)** - Gateway setup and deployment
266+
- **[TypeScript SDK](https://github.com/mozilla-ai/otari-sdk-ts)** - The TypeScript SDK for Node.js applications
267+
- **[otari Platform (Beta)](https://otari.ai/)** - Hosted control plane for key management, usage tracking, and cost visibility
268+
269+
## Contributing
270+
271+
We welcome contributions from developers of all skill levels! Please see the [Contributing Guide](https://github.com/mozilla-ai/otari/blob/main/CONTRIBUTING.md) or open an issue to discuss changes.
272+
273+
## License
274+
275+
This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.

0 commit comments

Comments
 (0)