Skip to content

Commit c4a5a71

Browse files
HaystackBotjulian-risch
HaystackBot
and
julian-risch
authored
docs: sync Core Integrations API reference (litellm) on Docusaurus (#11435)
Co-authored-by: julian-risch <4181769+julian-risch@users.noreply.github.com>
1 parent 169ce93 commit c4a5a71

13 files changed

Lines changed: 1625 additions & 0 deletions

File tree

‎docs-website/reference/integrations-api/litellm.md‎

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
---
2+
title: "LiteLLM"
3+
id: integrations-litellm
4+
description: "LiteLLM integration for Haystack"
5+
slug: "/integrations-litellm"
6+
---
7+
8+
9+
## haystack_integrations.components.generators.litellm.chat.chat_generator
10+
11+
### LiteLLMChatGenerator
12+
13+
Completes chats using any of 100+ LLM providers via LiteLLM.
14+
15+
LiteLLM routes to OpenAI, Anthropic, Google, AWS Bedrock, Azure, Cohere,
16+
Mistral, Groq, and many more through a single unified interface.
17+
18+
Model names use LiteLLM format: `provider/model-name`, e.g.
19+
`anthropic/claude-sonnet-4-20250514`, `openai/gpt-4o`,
20+
`bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0`.
21+
22+
See https://docs.litellm.ai/docs/providers for the full list.
23+
24+
Usage example:
25+
26+
```python
27+
from haystack_integrations.components.generators.litellm import LiteLLMChatGenerator
28+
from haystack.dataclasses import ChatMessage
29+
30+
generator = LiteLLMChatGenerator(
31+
model="anthropic/claude-sonnet-4-20250514",
32+
generation_kwargs={"max_tokens": 1024, "temperature": 0.7},
33+
)
34+
35+
messages = [
36+
ChatMessage.from_system("You are a helpful assistant"),
37+
ChatMessage.from_user("What's Natural Language Processing?"),
38+
]
39+
result = generator.run(messages=messages)
40+
print(result["replies"][0].text)
41+
```
42+
43+
#### __init__
44+
45+
```python
46+
__init__(
47+
*,
48+
api_key: Secret | None = None,
49+
model: str = "openai/gpt-4o",
50+
streaming_callback: StreamingCallbackT | None = None,
51+
api_base_url: str | None = None,
52+
generation_kwargs: dict[str, Any] | None = None,
53+
tools: ToolsType | None = None
54+
) -> None
55+
```
56+
57+
Create a LiteLLMChatGenerator instance.
58+
59+
**Parameters:**
60+
61+
- **api_key** (<code>Secret | None</code>) – The API key for the provider. Optional: when not set, LiteLLM resolves
62+
credentials itself from the provider's standard environment variable
63+
(e.g. `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`). Pass a `Secret` only
64+
when you want Haystack to manage and serialize the key explicitly.
65+
- **model** (<code>str</code>) – The model name in LiteLLM format (provider/model-name).
66+
- **streaming_callback** (<code>StreamingCallbackT | None</code>) – A callback function invoked with each new StreamingChunk.
67+
- **api_base_url** (<code>str | None</code>) – Custom API base URL (e.g. for a self-hosted LiteLLM proxy).
68+
- **generation_kwargs** (<code>dict\[str, Any\] | None</code>) – Additional parameters passed to litellm.completion().
69+
See https://docs.litellm.ai/docs/completion/input for details.
70+
- **tools** (<code>ToolsType | None</code>) – A list of Tool / Toolset objects the model can prepare calls for.
71+
72+
#### run
73+
74+
```python
75+
run(
76+
messages: list[ChatMessage],
77+
streaming_callback: StreamingCallbackT | None = None,
78+
generation_kwargs: dict[str, Any] | None = None,
79+
*,
80+
tools: ToolsType | None = None
81+
) -> dict[str, list[ChatMessage]]
82+
```
83+
84+
Invoke chat completion via LiteLLM.
85+
86+
**Parameters:**
87+
88+
- **messages** (<code>list\[ChatMessage\]</code>) – Input messages as ChatMessage instances.
89+
- **streaming_callback** (<code>StreamingCallbackT | None</code>) – Override the streaming callback for this call.
90+
- **generation_kwargs** (<code>dict\[str, Any\] | None</code>) – Override generation parameters for this call.
91+
- **tools** (<code>ToolsType | None</code>) – Override tools for this call.
92+
93+
**Returns:**
94+
95+
- <code>dict\[str, list\[ChatMessage\]\]</code> – A dict with key `replies` containing ChatMessage instances.
96+
97+
#### run_async
98+
99+
```python
100+
run_async(
101+
messages: list[ChatMessage],
102+
streaming_callback: StreamingCallbackT | None = None,
103+
generation_kwargs: dict[str, Any] | None = None,
104+
*,
105+
tools: ToolsType | None = None
106+
) -> dict[str, list[ChatMessage]]
107+
```
108+
109+
Async version of run().
110+
111+
#### to_dict
112+
113+
```python
114+
to_dict() -> dict[str, Any]
115+
```
116+
117+
Serialize this component to a dictionary.
118+
119+
#### from_dict
120+
121+
```python
122+
from_dict(data: dict[str, Any]) -> LiteLLMChatGenerator
123+
```
124+
125+
Deserialize a component from a dictionary.

‎docs-website/reference_versioned_docs/version-2.18/integrations-api/litellm.md‎

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
---
2+
title: "LiteLLM"
3+
id: integrations-litellm
4+
description: "LiteLLM integration for Haystack"
5+
slug: "/integrations-litellm"
6+
---
7+
8+
9+
## haystack_integrations.components.generators.litellm.chat.chat_generator
10+
11+
### LiteLLMChatGenerator
12+
13+
Completes chats using any of 100+ LLM providers via LiteLLM.
14+
15+
LiteLLM routes to OpenAI, Anthropic, Google, AWS Bedrock, Azure, Cohere,
16+
Mistral, Groq, and many more through a single unified interface.
17+
18+
Model names use LiteLLM format: `provider/model-name`, e.g.
19+
`anthropic/claude-sonnet-4-20250514`, `openai/gpt-4o`,
20+
`bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0`.
21+
22+
See https://docs.litellm.ai/docs/providers for the full list.
23+
24+
Usage example:
25+
26+
```python
27+
from haystack_integrations.components.generators.litellm import LiteLLMChatGenerator
28+
from haystack.dataclasses import ChatMessage
29+
30+
generator = LiteLLMChatGenerator(
31+
model="anthropic/claude-sonnet-4-20250514",
32+
generation_kwargs={"max_tokens": 1024, "temperature": 0.7},
33+
)
34+
35+
messages = [
36+
ChatMessage.from_system("You are a helpful assistant"),
37+
ChatMessage.from_user("What's Natural Language Processing?"),
38+
]
39+
result = generator.run(messages=messages)
40+
print(result["replies"][0].text)
41+
```
42+
43+
#### __init__
44+
45+
```python
46+
__init__(
47+
*,
48+
api_key: Secret | None = None,
49+
model: str = "openai/gpt-4o",
50+
streaming_callback: StreamingCallbackT | None = None,
51+
api_base_url: str | None = None,
52+
generation_kwargs: dict[str, Any] | None = None,
53+
tools: ToolsType | None = None
54+
) -> None
55+
```
56+
57+
Create a LiteLLMChatGenerator instance.
58+
59+
**Parameters:**
60+
61+
- **api_key** (<code>Secret | None</code>) – The API key for the provider. Optional: when not set, LiteLLM resolves
62+
credentials itself from the provider's standard environment variable
63+
(e.g. `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`). Pass a `Secret` only
64+
when you want Haystack to manage and serialize the key explicitly.
65+
- **model** (<code>str</code>) – The model name in LiteLLM format (provider/model-name).
66+
- **streaming_callback** (<code>StreamingCallbackT | None</code>) – A callback function invoked with each new StreamingChunk.
67+
- **api_base_url** (<code>str | None</code>) – Custom API base URL (e.g. for a self-hosted LiteLLM proxy).
68+
- **generation_kwargs** (<code>dict\[str, Any\] | None</code>) – Additional parameters passed to litellm.completion().
69+
See https://docs.litellm.ai/docs/completion/input for details.
70+
- **tools** (<code>ToolsType | None</code>) – A list of Tool / Toolset objects the model can prepare calls for.
71+
72+
#### run
73+
74+
```python
75+
run(
76+
messages: list[ChatMessage],
77+
streaming_callback: StreamingCallbackT | None = None,
78+
generation_kwargs: dict[str, Any] | None = None,
79+
*,
80+
tools: ToolsType | None = None
81+
) -> dict[str, list[ChatMessage]]
82+
```
83+
84+
Invoke chat completion via LiteLLM.
85+
86+
**Parameters:**
87+
88+
- **messages** (<code>list\[ChatMessage\]</code>) – Input messages as ChatMessage instances.
89+
- **streaming_callback** (<code>StreamingCallbackT | None</code>) – Override the streaming callback for this call.
90+
- **generation_kwargs** (<code>dict\[str, Any\] | None</code>) – Override generation parameters for this call.
91+
- **tools** (<code>ToolsType | None</code>) – Override tools for this call.
92+
93+
**Returns:**
94+
95+
- <code>dict\[str, list\[ChatMessage\]\]</code> – A dict with key `replies` containing ChatMessage instances.
96+
97+
#### run_async
98+
99+
```python
100+
run_async(
101+
messages: list[ChatMessage],
102+
streaming_callback: StreamingCallbackT | None = None,
103+
generation_kwargs: dict[str, Any] | None = None,
104+
*,
105+
tools: ToolsType | None = None
106+
) -> dict[str, list[ChatMessage]]
107+
```
108+
109+
Async version of run().
110+
111+
#### to_dict
112+
113+
```python
114+
to_dict() -> dict[str, Any]
115+
```
116+
117+
Serialize this component to a dictionary.
118+
119+
#### from_dict
120+
121+
```python
122+
from_dict(data: dict[str, Any]) -> LiteLLMChatGenerator
123+
```
124+
125+
Deserialize a component from a dictionary.

‎docs-website/reference_versioned_docs/version-2.19/integrations-api/litellm.md‎

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
---
2+
title: "LiteLLM"
3+
id: integrations-litellm
4+
description: "LiteLLM integration for Haystack"
5+
slug: "/integrations-litellm"
6+
---
7+
8+
9+
## haystack_integrations.components.generators.litellm.chat.chat_generator
10+
11+
### LiteLLMChatGenerator
12+
13+
Completes chats using any of 100+ LLM providers via LiteLLM.
14+
15+
LiteLLM routes to OpenAI, Anthropic, Google, AWS Bedrock, Azure, Cohere,
16+
Mistral, Groq, and many more through a single unified interface.
17+
18+
Model names use LiteLLM format: `provider/model-name`, e.g.
19+
`anthropic/claude-sonnet-4-20250514`, `openai/gpt-4o`,
20+
`bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0`.
21+
22+
See https://docs.litellm.ai/docs/providers for the full list.
23+
24+
Usage example:
25+
26+
```python
27+
from haystack_integrations.components.generators.litellm import LiteLLMChatGenerator
28+
from haystack.dataclasses import ChatMessage
29+
30+
generator = LiteLLMChatGenerator(
31+
model="anthropic/claude-sonnet-4-20250514",
32+
generation_kwargs={"max_tokens": 1024, "temperature": 0.7},
33+
)
34+
35+
messages = [
36+
ChatMessage.from_system("You are a helpful assistant"),
37+
ChatMessage.from_user("What's Natural Language Processing?"),
38+
]
39+
result = generator.run(messages=messages)
40+
print(result["replies"][0].text)
41+
```
42+
43+
#### __init__
44+
45+
```python
46+
__init__(
47+
*,
48+
api_key: Secret | None = None,
49+
model: str = "openai/gpt-4o",
50+
streaming_callback: StreamingCallbackT | None = None,
51+
api_base_url: str | None = None,
52+
generation_kwargs: dict[str, Any] | None = None,
53+
tools: ToolsType | None = None
54+
) -> None
55+
```
56+
57+
Create a LiteLLMChatGenerator instance.
58+
59+
**Parameters:**
60+
61+
- **api_key** (<code>Secret | None</code>) – The API key for the provider. Optional: when not set, LiteLLM resolves
62+
credentials itself from the provider's standard environment variable
63+
(e.g. `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`). Pass a `Secret` only
64+
when you want Haystack to manage and serialize the key explicitly.
65+
- **model** (<code>str</code>) – The model name in LiteLLM format (provider/model-name).
66+
- **streaming_callback** (<code>StreamingCallbackT | None</code>) – A callback function invoked with each new StreamingChunk.
67+
- **api_base_url** (<code>str | None</code>) – Custom API base URL (e.g. for a self-hosted LiteLLM proxy).
68+
- **generation_kwargs** (<code>dict\[str, Any\] | None</code>) – Additional parameters passed to litellm.completion().
69+
See https://docs.litellm.ai/docs/completion/input for details.
70+
- **tools** (<code>ToolsType | None</code>) – A list of Tool / Toolset objects the model can prepare calls for.
71+
72+
#### run
73+
74+
```python
75+
run(
76+
messages: list[ChatMessage],
77+
streaming_callback: StreamingCallbackT | None = None,
78+
generation_kwargs: dict[str, Any] | None = None,
79+
*,
80+
tools: ToolsType | None = None
81+
) -> dict[str, list[ChatMessage]]
82+
```
83+
84+
Invoke chat completion via LiteLLM.
85+
86+
**Parameters:**
87+
88+
- **messages** (<code>list\[ChatMessage\]</code>) – Input messages as ChatMessage instances.
89+
- **streaming_callback** (<code>StreamingCallbackT | None</code>) – Override the streaming callback for this call.
90+
- **generation_kwargs** (<code>dict\[str, Any\] | None</code>) – Override generation parameters for this call.
91+
- **tools** (<code>ToolsType | None</code>) – Override tools for this call.
92+
93+
**Returns:**
94+
95+
- <code>dict\[str, list\[ChatMessage\]\]</code> – A dict with key `replies` containing ChatMessage instances.
96+
97+
#### run_async
98+
99+
```python
100+
run_async(
101+
messages: list[ChatMessage],
102+
streaming_callback: StreamingCallbackT | None = None,
103+
generation_kwargs: dict[str, Any] | None = None,
104+
*,
105+
tools: ToolsType | None = None
106+
) -> dict[str, list[ChatMessage]]
107+
```
108+
109+
Async version of run().
110+
111+
#### to_dict
112+
113+
```python
114+
to_dict() -> dict[str, Any]
115+
```
116+
117+
Serialize this component to a dictionary.
118+
119+
#### from_dict
120+
121+
```python
122+
from_dict(data: dict[str, Any]) -> LiteLLMChatGenerator
123+
```
124+
125+
Deserialize a component from a dictionary.

0 commit comments

Comments
 (0)