Skip to content

Commit 784ad87

Browse files
committed
Add AI Gateway integration docs
1 parent 55abff0 commit 784ad87

18 files changed

Lines changed: 1032 additions & 1 deletion

docs/nav.json

Lines changed: 28 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -230,7 +230,34 @@
230230
{ "label": "Export Data", "slug": "how-to-guides/query-api" },
231231
{ "label": "Feature Flags (OFREP)", "slug": "how-to-guides/client-side-feature-flags" },
232232
{ "label": "Convert to Organization", "slug": "how-to-guides/convert-to-organization" },
233-
{ "label": "AI Gateway", "slug": "reference/advanced/gateway/index" }
233+
{
234+
"label": "AI Gateway",
235+
"items": [
236+
{ "label": "Overview", "slug": "reference/advanced/gateway/index" },
237+
{
238+
"label": "Integrations",
239+
"items": [
240+
{ "label": "Overview", "slug": "reference/advanced/gateway/agent-frameworks" },
241+
{ "label": "Agno", "slug": "reference/advanced/gateway/integrations/agno" },
242+
{ "label": "Genkit", "slug": "reference/advanced/gateway/integrations/genkit" },
243+
{ "label": "Google ADK", "slug": "reference/advanced/gateway/integrations/google-adk" },
244+
{ "label": "Haystack", "slug": "reference/advanced/gateway/integrations/haystack" },
245+
{ "label": "Instructor", "slug": "reference/advanced/gateway/integrations/instructor" },
246+
{ "label": "LangChain", "slug": "reference/advanced/gateway/integrations/langchain" },
247+
{ "label": "LlamaIndex", "slug": "reference/advanced/gateway/integrations/llamaindex" },
248+
{ "label": "Mastra", "slug": "reference/advanced/gateway/integrations/mastra" },
249+
{ "label": "Microsoft Agent Framework", "slug": "reference/advanced/gateway/integrations/microsoft-agent-framework" },
250+
{ "label": "OpenAI Agents SDK", "slug": "reference/advanced/gateway/integrations/openai-agents-sdk" },
251+
{ "label": "OpenAI SDK", "slug": "reference/advanced/gateway/integrations/openai-sdk" },
252+
{ "label": "Semantic Kernel", "slug": "reference/advanced/gateway/integrations/semantic-kernel" },
253+
{ "label": "Smolagents", "slug": "reference/advanced/gateway/integrations/smolagents" },
254+
{ "label": "Strands Agents", "slug": "reference/advanced/gateway/integrations/strands-agents" },
255+
{ "label": "Vercel AI SDK", "slug": "reference/advanced/gateway/integrations/vercel-ai-sdk" },
256+
{ "label": "VoltAgent", "slug": "reference/advanced/gateway/integrations/voltagent" }
257+
]
258+
}
259+
]
260+
}
234261
]
235262
},
236263
{
Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
---
2+
title: "AI Gateway integrations"
3+
description: "Connect frameworks and SDKs to the Logfire AI Gateway."
4+
---
5+
6+
# AI Gateway integrations
7+
8+
These pages show how to point each framework or SDK at the Logfire AI Gateway so that all model calls flow through one place for usage tracking, spending limits, and provider fallback.
9+
10+
## Before you start
11+
12+
1. In Logfire, open your organization, then go to **AI Engineering** > **Gateway**.
13+
2. Create or copy a gateway API key from the Gateway **API Keys** tab.
14+
3. Set the key in your terminal:
15+
16+
```bash
17+
export LOGFIRE_GATEWAY_API_KEY="..."
18+
```
19+
20+
Each example uses a gateway URL like `https://gateway-us.pydantic.dev/proxy/openai`, where the last part picks which model provider (OpenAI, Anthropic, Google, and so on) handles the call. To switch to a different provider — or to a routing group that combines several providers with fallback — copy the URL you want from the Gateway **Providers** or **Routing** tab.
21+
22+
## Examples
23+
24+
| Framework or SDK | Available in |
25+
| --- | --- |
26+
| Agno | [Python](/logfire/manage/ai-gateway/integrations/agno/) |
27+
| Genkit | [Go](/logfire/manage/ai-gateway/integrations/genkit/#go), [JavaScript](/logfire/manage/ai-gateway/integrations/genkit/#javascript) |
28+
| Google ADK | [Go](/logfire/manage/ai-gateway/integrations/google-adk/#go), [Python](/logfire/manage/ai-gateway/integrations/google-adk/#python) |
29+
| Haystack | [Python](/logfire/manage/ai-gateway/integrations/haystack/) |
30+
| Instructor | [Python](/logfire/manage/ai-gateway/integrations/instructor/) |
31+
| LangChain | [TypeScript](/logfire/manage/ai-gateway/integrations/langchain/#typescript), [Python](/logfire/manage/ai-gateway/integrations/langchain/#python) |
32+
| LlamaIndex | [Python](/logfire/manage/ai-gateway/integrations/llamaindex/) |
33+
| Mastra | [JavaScript](/logfire/manage/ai-gateway/integrations/mastra/) |
34+
| Microsoft Agent Framework | [.NET](/logfire/manage/ai-gateway/integrations/microsoft-agent-framework/#dotnet), [Python](/logfire/manage/ai-gateway/integrations/microsoft-agent-framework/#python) |
35+
| OpenAI Agents SDK | [TypeScript](/logfire/manage/ai-gateway/integrations/openai-agents-sdk/#typescript), [Python](/logfire/manage/ai-gateway/integrations/openai-agents-sdk/#python) |
36+
| OpenAI SDK | [.NET](/logfire/manage/ai-gateway/integrations/openai-sdk/#dotnet), [Go](/logfire/manage/ai-gateway/integrations/openai-sdk/#go), [TypeScript](/logfire/manage/ai-gateway/integrations/openai-sdk/#typescript), [Python](/logfire/manage/ai-gateway/integrations/openai-sdk/#python) |
37+
| Semantic Kernel | [.NET](/logfire/manage/ai-gateway/integrations/semantic-kernel/#dotnet), [Python](/logfire/manage/ai-gateway/integrations/semantic-kernel/#python) |
38+
| Smolagents | [Python](/logfire/manage/ai-gateway/integrations/smolagents/) |
39+
| Strands Agents | [TypeScript](/logfire/manage/ai-gateway/integrations/strands-agents/#typescript), [Python](/logfire/manage/ai-gateway/integrations/strands-agents/#python) |
40+
| Vercel AI SDK | [JavaScript](/logfire/manage/ai-gateway/integrations/vercel-ai-sdk/) |
41+
| VoltAgent | [JavaScript](/logfire/manage/ai-gateway/integrations/voltagent/) |
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
---
2+
title: "AI Gateway: Agno"
3+
description: "Route Agno model calls through the Logfire AI Gateway."
4+
---
5+
6+
# Agno
7+
8+
[Agno](https://docs.agno.com/) is a Python framework for building multi-modal, multi-agent AI systems. To route its model calls through the Logfire AI Gateway, configure `OpenAIChat` with the gateway URL and set `LOGFIRE_GATEWAY_API_KEY` to a key from the Gateway **API Keys** tab.
9+
10+
```python title="agno-gateway.py" skip-run="true" skip-reason="external-connection"
11+
import os
12+
13+
from agno.agent import Agent
14+
from agno.models.openai import OpenAIChat
15+
16+
agent = Agent(
17+
name="Weather Agent",
18+
model=OpenAIChat(
19+
id="gpt-5.4-mini",
20+
api_key=os.environ["LOGFIRE_GATEWAY_API_KEY"],
21+
base_url="https://gateway-us.pydantic.dev/proxy/openai",
22+
),
23+
)
24+
25+
agent.print_response("What is the weather in London?")
26+
```
Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
---
2+
title: "AI Gateway: Genkit"
3+
description: "Route Genkit model calls through the Logfire AI Gateway."
4+
---
5+
6+
# Genkit
7+
8+
[Genkit](https://genkit.dev/) is Google's open-source framework for building AI-powered applications. To route its model calls through the Logfire AI Gateway, use Genkit's OpenAI-compatible plugin pointed at the gateway URL and set `LOGFIRE_GATEWAY_API_KEY` to a key from the Gateway **API Keys** tab.
9+
10+
## JavaScript
11+
12+
```typescript title="genkit-gateway.mts" skip-run="true" skip-reason="external-connection"
13+
import { openAICompatible } from '@genkit-ai/compat-oai';
14+
import { genkit } from 'genkit';
15+
import { z } from 'zod';
16+
17+
const envSchema = z.object({
18+
LOGFIRE_GATEWAY_API_KEY: z.string(),
19+
});
20+
21+
const env = envSchema.parse(process.env);
22+
23+
const ai = genkit({
24+
plugins: [
25+
openAICompatible({
26+
name: 'openai',
27+
apiKey: env.LOGFIRE_GATEWAY_API_KEY,
28+
baseURL: 'https://gateway-us.pydantic.dev/proxy/openai',
29+
}),
30+
],
31+
});
32+
33+
const { text } = await ai.generate({
34+
model: 'openai/gpt-5.4-mini',
35+
prompt: 'What is the weather in London?',
36+
});
37+
38+
console.log(text);
39+
```
40+
41+
## Go
42+
43+
```go title="genkit-gateway.go" skip-run="true" skip-reason="external-connection"
44+
package main
45+
46+
import (
47+
"context"
48+
"log"
49+
"os"
50+
"time"
51+
52+
"github.com/firebase/genkit/go/ai"
53+
"github.com/firebase/genkit/go/genkit"
54+
"github.com/firebase/genkit/go/plugins/compat_oai/openai"
55+
"github.com/openai/openai-go/option"
56+
)
57+
58+
func main() {
59+
apiKey := os.Getenv("LOGFIRE_GATEWAY_API_KEY")
60+
if apiKey == "" {
61+
log.Fatal("LOGFIRE_GATEWAY_API_KEY is required")
62+
}
63+
64+
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
65+
defer cancel()
66+
67+
g := genkit.Init(ctx, genkit.WithPlugins(&openai.OpenAI{
68+
APIKey: apiKey,
69+
Opts: []option.RequestOption{
70+
option.WithBaseURL("https://gateway-us.pydantic.dev/proxy/openai"),
71+
},
72+
}))
73+
74+
response, err := genkit.Generate(ctx, g,
75+
ai.WithPrompt("What is the weather in London?"),
76+
ai.WithModelName("openai/gpt-5.4-mini"),
77+
)
78+
if err != nil {
79+
log.Fatal(err)
80+
}
81+
82+
log.Println(response.Text())
83+
}
84+
```
Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
---
2+
title: "AI Gateway: Google ADK"
3+
description: "Route Google ADK model calls through the Logfire AI Gateway."
4+
---
5+
6+
# Google ADK
7+
8+
[Google Agent Development Kit (ADK)](https://adk.dev/) is Google's framework for building multi-step, multi-agent AI systems. To route its model calls through the Logfire AI Gateway, set `LOGFIRE_GATEWAY_API_KEY` to a key from the Gateway **API Keys** tab. The Python example routes calls via LiteLLM (a library that translates between LLM provider APIs); the Go example connects directly to the gateway's Google Vertex proxy.
9+
10+
## Python
11+
12+
```python title="google-adk-gateway.py" skip-run="true" skip-reason="external-connection"
13+
import asyncio
14+
import os
15+
16+
from google.adk.agents import LlmAgent
17+
from google.adk.models.lite_llm import LiteLlm
18+
from google.adk.runners import InMemoryRunner
19+
from google.genai import types
20+
21+
os.environ["OPENAI_API_KEY"] = os.environ["LOGFIRE_GATEWAY_API_KEY"]
22+
os.environ["OPENAI_BASE_URL"] = "https://gateway-us.pydantic.dev/proxy/openai"
23+
24+
agent = LlmAgent(
25+
model=LiteLlm(model="openai/gpt-5.4-mini"),
26+
name="weather_agent",
27+
instruction="You are a concise weather assistant.",
28+
)
29+
30+
31+
async def main() -> None:
32+
runner = InMemoryRunner(agent=agent)
33+
session = await runner.session_service.create_session(
34+
app_name=runner.app_name,
35+
user_id="gateway-example-user",
36+
)
37+
content = types.Content(
38+
role="user",
39+
parts=[types.Part.from_text(text="What is the weather in London?")],
40+
)
41+
async for event in runner.run_async(
42+
user_id=session.user_id,
43+
session_id=session.id,
44+
new_message=content,
45+
):
46+
if event.is_final_response() and event.content and event.content.parts:
47+
print(event.content.parts[0].text)
48+
49+
50+
if __name__ == "__main__":
51+
asyncio.run(main())
52+
```
53+
54+
## Go
55+
56+
The Go example routes Google Vertex model calls through `https://gateway-us.pydantic.dev/proxy/google-vertex`
57+
and uses `gemini-2.5-flash`.
58+
59+
```go title="google-adk-gateway.go" skip-run="true" skip-reason="external-connection"
60+
package main
61+
62+
import (
63+
"context"
64+
"fmt"
65+
"log"
66+
"net/http"
67+
"os"
68+
"time"
69+
70+
adkmodel "google.golang.org/adk/v2/model"
71+
"google.golang.org/adk/v2/model/gemini"
72+
"google.golang.org/genai"
73+
)
74+
75+
func main() {
76+
apiKey := os.Getenv("LOGFIRE_GATEWAY_API_KEY")
77+
if apiKey == "" {
78+
log.Fatal("LOGFIRE_GATEWAY_API_KEY is required")
79+
}
80+
81+
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
82+
defer cancel()
83+
84+
model, err := gemini.NewModel(ctx, "gemini-2.5-flash", &genai.ClientConfig{
85+
Backend: genai.BackendVertexAI,
86+
HTTPOptions: genai.HTTPOptions{
87+
BaseURL: "https://gateway-us.pydantic.dev/proxy/google-vertex",
88+
Headers: http.Header{
89+
"Authorization": []string{"Bearer " + apiKey},
90+
},
91+
},
92+
})
93+
if err != nil {
94+
panic(err)
95+
}
96+
97+
request := &adkmodel.LLMRequest{
98+
Contents: []*genai.Content{
99+
genai.NewContentFromText("What is the weather in London?", genai.RoleUser),
100+
},
101+
}
102+
103+
for response, err := range model.GenerateContent(ctx, request, false) {
104+
if err != nil {
105+
panic(err)
106+
}
107+
if response.Content == nil {
108+
continue
109+
}
110+
for _, part := range response.Content.Parts {
111+
if part.Text != "" {
112+
fmt.Println(part.Text)
113+
}
114+
}
115+
}
116+
}
117+
```
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
---
2+
title: "AI Gateway: Haystack"
3+
description: "Route Haystack model calls through the Logfire AI Gateway."
4+
---
5+
6+
# Haystack
7+
8+
[Haystack](https://haystack.deepset.ai/) is an open-source Python framework for building AI search and document processing pipelines. To route its model calls through the Logfire AI Gateway, configure `OpenAIChatGenerator` with the gateway URL and set `LOGFIRE_GATEWAY_API_KEY` to a key from the Gateway **API Keys** tab.
9+
10+
```python title="haystack-gateway.py" skip-run="true" skip-reason="external-connection"
11+
from haystack.components.generators.chat import OpenAIChatGenerator
12+
from haystack.dataclasses import ChatMessage
13+
from haystack.utils import Secret
14+
15+
generator = OpenAIChatGenerator(
16+
model="gpt-5.4-mini",
17+
api_key=Secret.from_env_var("LOGFIRE_GATEWAY_API_KEY"),
18+
api_base_url="https://gateway-us.pydantic.dev/proxy/openai",
19+
)
20+
21+
response = generator.run([ChatMessage.from_user("What is the weather in London?")])
22+
23+
print(response["replies"][0].text)
24+
```
Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
---
2+
title: "AI Gateway: Instructor"
3+
description: "Route Instructor model calls through the Logfire AI Gateway."
4+
---
5+
6+
# Instructor
7+
8+
[Instructor](https://python.useinstructor.com/) is a Python library for extracting structured, typed data from large language model (LLM) responses — you describe the shape of the data you want and Instructor handles the rest. Because Instructor wraps a standard OpenAI client, routing through the gateway is just a matter of configuring that underlying client with the gateway URL. Set `LOGFIRE_GATEWAY_API_KEY` to a key from the Gateway **API Keys** tab before running this example.
9+
10+
```python title="instructor-gateway.py" skip-run="true" skip-reason="external-connection"
11+
import os
12+
13+
import instructor
14+
from openai import OpenAI
15+
from pydantic import BaseModel
16+
17+
18+
class Weather(BaseModel):
19+
city: str
20+
condition: str
21+
22+
23+
client = instructor.from_openai(
24+
OpenAI(
25+
api_key=os.environ["LOGFIRE_GATEWAY_API_KEY"],
26+
base_url="https://gateway-us.pydantic.dev/proxy/openai",
27+
)
28+
)
29+
30+
weather = client.create(
31+
model="gpt-5.4-mini",
32+
response_model=Weather,
33+
messages=[{"role": "user", "content": "What is the weather in London?"}],
34+
)
35+
36+
print(weather)
37+
```

0 commit comments

Comments
 (0)