Skip to content

Commit 97c75c2

Browse files
starfolkai[bot]Nova (SFK)claude
authored
docs(examples): convert to per-integration uv projects using auto_instrument (#363)
## Summary - New `examples/` directory at the repo root — each integration gets a self-contained `uv` project with its own `pyproject.toml` (pointing `braintrust` at the local `py/` checkout), `README.md`, and example script(s). Lockfiles are gitignored so each clone resolves fresh against current PyPI. - Every provider example uses `braintrust.auto_instrument()` + `init_logger()` as the canonical setup, instead of integration-specific `setup_*()` / `wrap_*()` helpers. - Add 13 missing examples: `agentscope`, `autogen`, `claude_agent_sdk`, `cohere`, `crewai`, `google_genai`, `langchain`, `litellm`, `llamaindex`, `mistral`, `openai_agents`, `openrouter`, `strands`. ### What changed - **Layout:** examples live at `examples/` (repo root), not `py/examples/`. The repo-root README points at them. - **Existing top-level scripts** moved into their own dirs (history preserved): `openai_example.py` → `openai/`, `anthropic_*.py` → `anthropic/`, `pydantic_ai_example.py` → `pydantic_ai/`. - **`temporal/`:** `requirements.txt` → `pyproject.toml`; `Procfile` and `mise.toml` updated to use `uv sync` / `uv run`. - **`langsmith/`:** drop committed `uv.lock` for consistency with the new per-example convention. - **`adk/`:** drop `manual_patching.py` since `auto_instrument()` covers the same case; rename `auto.py` → `example.py`. - **Drop the dedicated `auto_instrument/` showcase** since every provider example now demonstrates it by default. - **`examples/README.md`** explains the layout, the canonical 2-line pattern, and lists every example. ### CI lint coverage - Extend the `pylint` nox session in `py/noxfile.py` to also enumerate `../examples/**/*.py` and lint them in the same invocation. Reuses the existing matrix (Python 3.10–3.14), reuses the existing `lint` dependency-group, keeps it as one CI job — no new workflow surface area. - Add `claude-agent-sdk` and `strands-agents` to the `lint` dependency-group so pylint can resolve the new examples' imports. - Skip `examples/crewai/` on Python 3.14 to mirror the existing conditional crewai pin (its transitive `pydantic-core` has no 3.14 wheel yet). ### Modernized model names Replaced stale model identifiers in examples with current ones, matching what the SDK's own integration tests use: | Example | Before | After | | --- | --- | --- | | `anthropic/sync.py` + `anthropic/async_example.py` | `claude-3-haiku-20240307` / `claude-3-5-sonnet-latest` | `claude-haiku-4-5` | | `openai/example.py` + `pydantic_ai/example.py` | `gpt-4o` | `gpt-4o-mini` | | `otel/basic_otel_example.py` + `otel/filtered_otel_example.py` | `gpt-3.5-turbo` | `gpt-4o-mini` | Already-current models (`gemini-2.0-flash`, `command-r-plus-08-2024`, `mistral-small-latest`, `claude-haiku-4-5-20251001`, `gpt-4o-mini`) left alone. ### Why no committed lockfiles Examples are meant to demonstrate the **current** SDK against **current** providers, not snapshot a moment in time. Committing ~24 `uv.lock` files would create constant churn on every transitive bump and would actively work against the demo goal. `pyproject.toml` already constrains `requires-python` and direct deps. (See `examples/README.md` for the rationale.) ## Test plan - [x] `uv sync` succeeds in every example dir - [x] `braintrust.auto_instrument()` returns `instrumented: True` for the relevant integration in each example's venv (19/19 verified) - [x] Imports succeed in every example with the correct symbols - [x] `nox -s pylint` runs locally on Python 3.14 and lints all SDK source plus all moved examples in one pass (with `crewai` correctly skipped) - [ ] Reviewer manually runs one or two examples end-to-end with real API keys 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Nova (SFK) <nova@starfolk.ai> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
1 parent cf07dab commit 97c75c2

99 files changed

Lines changed: 1173 additions & 1250 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,8 @@ This repository contains Braintrust's Python SDKs and integrations, including:
66

77
- The main `braintrust` SDK package in [`./py`](./py)
88
- Built-in integrations under [`py/src/braintrust/integrations`](py/src/braintrust/integrations) and related compatibility packages under [`./integrations`](./integrations)
9-
- Examples, tests, and local development tooling for Python SDK development
9+
- Self-contained `uv`-project examples for every integration in [`./examples`](./examples)
10+
- Tests and local development tooling for Python SDK development
1011

1112
## Quickstart
1213

examples/.gitignore

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
run.sh
2+
3+
# Examples install fresh against current PyPI; locks aren't checked in.
4+
uv.lock
5+
.venv/
6+
__pycache__/
7+
*.pyc
8+
.env

examples/README.md

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
# Braintrust Python SDK Examples
2+
3+
Each subdirectory in this folder is a self-contained [`uv`](https://docs.astral.sh/uv/) project demonstrating one Braintrust integration or feature. They are designed to be cloned, copied, or run as-is without affecting the rest of the repository.
4+
5+
## Layout
6+
7+
Every example has the same shape:
8+
9+
```
10+
examples/<name>/
11+
├── pyproject.toml # declares deps + a local `path` source for braintrust
12+
├── README.md # what it shows + how to run it
13+
└── *.py # the example script(s)
14+
```
15+
16+
The `pyproject.toml` in each example pins `braintrust` to the local SDK checkout in `py/` via:
17+
18+
```toml
19+
[tool.uv.sources]
20+
braintrust = { path = "../../py", editable = true }
21+
```
22+
23+
That means `uv sync` inside any example installs the version of the SDK currently on disk — edits to `py/src/braintrust/` are picked up immediately.
24+
25+
## Running an example
26+
27+
From the repo root:
28+
29+
```bash
30+
cd examples/<name>
31+
uv sync
32+
uv run python <script>.py
33+
```
34+
35+
You will need a `BRAINTRUST_API_KEY` in your environment, plus whatever provider key the example uses (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, etc.). Each example's README lists its requirements.
36+
37+
## The default pattern: `auto_instrument()`
38+
39+
Most provider/framework examples follow the same two-line setup:
40+
41+
```python
42+
import braintrust
43+
44+
braintrust.auto_instrument()
45+
braintrust.init_logger(project="example-<name>")
46+
47+
# now import and use the library normally
48+
```
49+
50+
`braintrust.auto_instrument()` patches every supported integration that's installed in the current environment — there's no need to call individual `setup_*()` or `wrap_*()` helpers. The exceptions are examples that demonstrate non-tracing concerns (`evals/`, `langsmith/`, `otel/`, `temporal/`).
51+
52+
If you want to see `auto_instrument()` light up multiple providers in one trace, the `openai/`, `anthropic/`, and any other provider example can be combined trivially — just install both packages in the same environment, call `auto_instrument()` once, and use both clients.
53+
54+
## Available examples
55+
56+
Unless noted otherwise, every example below uses `braintrust.auto_instrument()`.
57+
58+
| Directory | What it shows |
59+
| --- | --- |
60+
| `adk/` | Google ADK weather agent with one tool |
61+
| `agentscope/` | AgentScope `ReActAgent` against `gpt-4o-mini` |
62+
| `agno/` | Agno agents and teams, sync + async, streaming + non-streaming |
63+
| `anthropic/` | Sync and async Anthropic clients |
64+
| `autogen/` | AutoGen `AssistantAgent` backed by `OpenAIChatCompletionClient` |
65+
| `claude_agent_sdk/` | Claude Agent SDK subprocess query |
66+
| `cohere/` | Cohere `ClientV2` chat call |
67+
| `crewai/` | CrewAI `Agent` + `Task` + `Crew` end-to-end |
68+
| `dspy/` | DSPy `ReAct` agent with two tools (LiteLLM token metrics propagate) |
69+
| `evals/` | The `Eval` framework — does **not** use `auto_instrument()` |
70+
| `google_genai/` | Google GenAI `generate_content` against Gemini |
71+
| `langchain/` | LangChain `prompt | model` chain — global handler installed by `auto_instrument()` |
72+
| `langsmith/` | Migration helper for projects coming from LangSmith — uses `setup_langsmith()` |
73+
| `litellm/` | LiteLLM `completion` |
74+
| `llamaindex/` | LlamaIndex LLM completion |
75+
| `mistral/` | Mistral chat completion |
76+
| `openai/` | OpenAI chat completion plus `@braintrust.traced` for the surrounding function |
77+
| `openai_agents/` | OpenAI Agents SDK `Runner` running an agent |
78+
| `openrouter/` | OpenRouter chat completion routed to OpenAI |
79+
| `otel/` | OpenTelemetry interop — `BraintrustSpanProcessor`, filtering, distributed tracing |
80+
| `pydantic_ai/` | Pydantic AI agent run inside a `start_span` for a permalink |
81+
| `strands/` | Strands `Agent` against `gpt-4o-mini` |
82+
| `temporal/` | Distributed Temporal workflow tracing via `BraintrustPlugin` |
83+
84+
## Why no committed `uv.lock`?
85+
86+
Examples are meant to demonstrate the **current** SDK against **current** provider releases, not snapshot a specific point in time. Committing lockfiles would create constant churn (a new lockfile for every transitive bump on PyPI) and would actively work against the demo goal — someone running an example next quarter wants it to work with whatever's on PyPI today, not a 3-month-old pin. Lockfiles are gitignored.
87+
88+
If you want a reproducible snapshot for your own use, run `uv lock` locally; it just won't be checked in.
89+
90+
## Adding a new example
91+
92+
1. Create `examples/<name>/` with the layout above.
93+
2. In `pyproject.toml`, declare deps under `[project] dependencies` and add the local `braintrust` source under `[tool.uv.sources]` (see any existing example).
94+
3. Write a short README explaining required env vars and how to run it.
95+
4. Verify with `uv sync && uv run python <script>.py` from inside the example dir.
96+
5. The `pylint` nox session in `py/noxfile.py` automatically lints `examples/`, so any imports your example needs must be in the `lint` dependency group of `py/pyproject.toml`.

examples/adk/README.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Google ADK + Braintrust
2+
3+
Calls `braintrust.auto_instrument()` so Google ADK is patched automatically, then runs a weather agent against `gemini-2.0-flash` with one tool. The trace shows the runner span, the agent span, the LLM span, and the tool call.
4+
5+
## Run
6+
7+
```bash
8+
export BRAINTRUST_API_KEY=...
9+
export GOOGLE_API_KEY=...
10+
11+
uv sync
12+
uv run python example.py
13+
```
Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,12 @@
1-
"""Auto-instrument Google ADK with Braintrust tracing.
2-
3-
Usage:
4-
export BRAINTRUST_API_KEY="your-api-key"
5-
export GOOGLE_API_KEY="your-google-api-key"
6-
python auto.py
7-
"""
1+
"""Google ADK weather agent traced via braintrust.auto_instrument()."""
82

93
import asyncio
104

115
import braintrust
126

137

14-
# Auto-instrument all supported libraries including Google ADK
158
braintrust.auto_instrument()
9+
braintrust.init_logger(project="example-adk")
1610

1711
from google.adk import Agent
1812
from google.adk.runners import Runner

examples/adk/pyproject.toml

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
[project]
2+
name = "braintrust-adk-example"
3+
version = "0.1.0"
4+
description = "Google ADK agents traced via Braintrust"
5+
requires-python = ">=3.10"
6+
dependencies = [
7+
"braintrust",
8+
"google-adk",
9+
"google-genai",
10+
]
11+
12+
[tool.uv.sources]
13+
braintrust = { path = "../../py", editable = true }

examples/agentscope/README.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# AgentScope + Braintrust
2+
3+
Calls `braintrust.auto_instrument()` to register the AgentScope tracing hooks, then runs a `ReActAgent` against `gpt-4o-mini`. The trace shows the agent's `*.reply` task span and the underlying LLM span.
4+
5+
## Run
6+
7+
```bash
8+
export BRAINTRUST_API_KEY=...
9+
export OPENAI_API_KEY=...
10+
11+
uv sync
12+
uv run python example.py
13+
```

examples/agentscope/example.py

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
#!/usr/bin/env python
2+
"""AgentScope ReAct agent traced via braintrust.auto_instrument()."""
3+
4+
import asyncio
5+
6+
import braintrust
7+
8+
9+
braintrust.auto_instrument()
10+
braintrust.init_logger(project="example-agentscope")
11+
12+
from agentscope.agent import ReActAgent
13+
from agentscope.formatter import OpenAIChatFormatter
14+
from agentscope.memory import InMemoryMemory
15+
from agentscope.message import Msg
16+
from agentscope.model import OpenAIChatModel
17+
from agentscope.tool import Toolkit
18+
19+
20+
async def main() -> None:
21+
agent = ReActAgent(
22+
name="Friday",
23+
sys_prompt="You are a concise assistant. Answer in one sentence.",
24+
model=OpenAIChatModel(model_name="gpt-4o-mini", stream=False),
25+
formatter=OpenAIChatFormatter(),
26+
toolkit=Toolkit(),
27+
memory=InMemoryMemory(),
28+
)
29+
30+
response = await agent(Msg(name="user", content="Say hello in exactly two words.", role="user"))
31+
print(response.content if response is not None else "(no response)")
32+
33+
34+
if __name__ == "__main__":
35+
asyncio.run(main())

examples/agentscope/pyproject.toml

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
[project]
2+
name = "braintrust-agentscope-example"
3+
version = "0.1.0"
4+
description = "AgentScope ReAct agent traced with Braintrust"
5+
requires-python = ">=3.10"
6+
dependencies = [
7+
"braintrust",
8+
"agentscope",
9+
"openai",
10+
]
11+
12+
[tool.uv.sources]
13+
braintrust = { path = "../../py", editable = true }

examples/agno/README.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# Agno + Braintrust
2+
3+
Each script calls `braintrust.auto_instrument()` before importing `agno`, so all subsequent agent/team activity is traced. The agents use `YFinanceTools` so they actually exercise tool calls.
4+
5+
| Script | Shape |
6+
| --- | --- |
7+
| `simple_agent.py` | one agent, one question |
8+
| `simple_agent_stream.py` | one agent, streamed |
9+
| `async_simple_agent_stream.py` | one agent, async + streamed |
10+
| `team_agent.py` | research + advisor team, sync |
11+
| `async_team_agent.py` | research + advisor team, async + streamed |
12+
13+
## Run
14+
15+
```bash
16+
export BRAINTRUST_API_KEY=...
17+
export OPENAI_API_KEY=...
18+
19+
uv sync
20+
uv run python simple_agent.py
21+
```

0 commit comments

Comments
 (0)