Skip to content

Commit a913f01

Browse files
Aarkin7Aarkin7
authored
docs: add guide page for AnthropicFoundryChatGenerator (#11790) (#11812)
1 parent 7fbcff1 commit a913f01

6 files changed

Lines changed: 350 additions & 0 deletions

File tree

‎docs-website/docs/pipeline-components/generators.mdx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ Generators are responsible for generating text after you give them a prompt. The
1515
| [AmazonBedrockGenerator](generators/amazonbedrockgenerator.mdx) | Enables text generation using models through Amazon Bedrock service. | ✅ |
1616
| [AIMLAPIChatGenerator](generators/aimllapichatgenerator.mdx) | Enables chat completion using AI models through the AIMLAPI. | ✅ |
1717
| [AnthropicChatGenerator](generators/anthropicchatgenerator.mdx) | This component enables chat completions using Anthropic large language models (LLMs). | ✅ |
18+
| [AnthropicFoundryChatGenerator](generators/anthropicfoundrychatgenerator.mdx) | This component enables chat completions using Anthropic models served through Azure Foundry. | ✅ |
1819
| [AnthropicVertexChatGenerator](generators/anthropicvertexchatgenerator.mdx) | This component enables chat completions using AnthropicVertex API. | ✅ |
1920
| [AnthropicGenerator](generators/anthropicgenerator.mdx) | This component enables text completions using Anthropic large language models (LLMs). | ✅ |
2021
| [AzureOpenAIChatGenerator](generators/azureopenaichatgenerator.mdx) | Enables chat completion using OpenAI's LLMs through Azure services. | ✅ |

‎docs-website/docs/pipeline-components/generators/anthropicfoundrychatgenerator.mdx‎

Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
---
2+
title: "AnthropicFoundryChatGenerator"
3+
id: anthropicfoundrychatgenerator
4+
slug: "/anthropicfoundrychatgenerator"
5+
description: "This component enables chat completions using Anthropic models served through Azure Foundry."
6+
---
7+
8+
# AnthropicFoundryChatGenerator
9+
10+
This component enables chat completions using Anthropic models served through Azure Foundry.
11+
12+
<div className="key-value-table">
13+
14+
| | |
15+
| --- | --- |
16+
| **Most common position in a pipeline** | After a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) |
17+
| **Mandatory init variables** | `api_key`: Your Azure Foundry API key. Can be set with the `ANTHROPIC_FOUNDRY_API_KEY` env var. Alternatively, pass an `azure_ad_token_provider` callable. <br /> <br />`resource`: Your Azure Foundry resource name. Can be set with the `ANTHROPIC_FOUNDRY_RESOURCE` env var. Alternatively, pass a full `endpoint` URL. |
18+
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects |
19+
| **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects <br /> <br />`meta`: A dictionary on each reply with metadata such as the model name, finish reason, and token usage |
20+
| **API reference** | [Anthropic](/reference/integrations-anthropic) |
21+
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/anthropic |
22+
| **Package name** | `anthropic-haystack` |
23+
24+
</div>
25+
26+
## Overview
27+
28+
`AnthropicFoundryChatGenerator` lets you call Anthropic's Claude models through an [Azure Foundry](https://learn.microsoft.com/en-us/azure/ai-foundry/) deployment. It is a thin subclass of [`AnthropicChatGenerator`](anthropicchatgenerator.mdx) — the request and response shapes match the Anthropic Messages API, but the traffic flows through your Azure resource instead of `api.anthropic.com`.
29+
30+
Use this generator when your organization standardizes on Azure for model hosting (billing, networking, compliance) but still wants to work against Claude. If you don't need Azure, prefer `AnthropicChatGenerator`.
31+
32+
The default model is `claude-sonnet-4-5`. Other models known to work include `claude-opus-4-6`, `claude-sonnet-4-6`, `claude-opus-4-5`, `claude-opus-4-1`, and `claude-haiku-4-5`. This list is not exhaustive — the actual catalog depends on what is deployed in your Foundry resource. See the [Anthropic model overview](https://docs.anthropic.com/en/docs/about-claude/models) for guidance on picking a model.
33+
34+
### Parameters
35+
36+
`AnthropicFoundryChatGenerator` needs two things to talk to Azure: credentials and an endpoint.
37+
38+
**Credentials.** Pick one of:
39+
40+
- The `ANTHROPIC_FOUNDRY_API_KEY` environment variable (recommended).
41+
- The `api_key` init parameter using the Haystack [Secret](../../concepts/secret-management.mdx) API: `Secret.from_token("your-api-key-here")`.
42+
- A callable passed as `azure_ad_token_provider` that returns a fresh Azure AD token on demand. Use this for Entra ID / managed-identity setups where a static key isn't appropriate.
43+
44+
**Endpoint.** Pick one of:
45+
46+
- The `resource` init parameter (or the `ANTHROPIC_FOUNDRY_RESOURCE` environment variable) — the short Foundry resource name, used to derive the URL.
47+
- The `endpoint` init parameter — a full URL, useful for custom domains or non-standard routes.
48+
49+
Once configured, pass any text-generation parameter supported by the Anthropic [Messages API](https://docs.anthropic.com/en/api/messages) through `generation_kwargs`, either at init or per call. Common keys include `system`, `max_tokens`, `temperature`, `top_p`, `top_k`, `stop_sequences`, `metadata`, and `extra_headers`. You can also tune `timeout` and `max_retries` to control client-side resilience.
50+
51+
The component takes a list of `ChatMessage` objects. `ChatMessage` is a data class that holds a message, a role (`user`, `assistant`, `system`, or `function`), and optional metadata. Only text input is supported.
52+
53+
### Tool Support
54+
55+
`AnthropicFoundryChatGenerator` supports function calling through the `tools` parameter, which accepts:
56+
57+
- **A list of Tool objects**: Pass individual tools as a list.
58+
- **A single Toolset**: Pass an entire Toolset directly.
59+
- **Mixed Tools and Toolsets**: Combine multiple Toolsets with standalone tools in a single list.
60+
61+
```python
62+
from haystack.tools import Tool, Toolset
63+
from haystack_integrations.components.generators.anthropic import AnthropicFoundryChatGenerator
64+
65+
weather_tool = Tool(name="weather", description="Get weather info", ...)
66+
math_toolset = Toolset([add_tool, subtract_tool, multiply_tool])
67+
68+
generator = AnthropicFoundryChatGenerator(
69+
resource="my-resource",
70+
tools=[math_toolset, weather_tool],
71+
)
72+
```
73+
74+
Tools passed to `run()` override any tools set at init time. For more details, see the [Tool](../../tools/tool.mdx) and [Toolset](../../tools/toolset.mdx) documentation.
75+
76+
### Streaming
77+
78+
You can stream output as it's generated. Pass a callback to `streaming_callback`. The built-in `print_streaming_chunk` prints text tokens and tool events to stdout.
79+
80+
```python
81+
from haystack.components.generators.utils import print_streaming_chunk
82+
from haystack.dataclasses import ChatMessage
83+
from haystack_integrations.components.generators.anthropic import AnthropicFoundryChatGenerator
84+
85+
generator = AnthropicFoundryChatGenerator(
86+
resource="my-resource",
87+
streaming_callback=print_streaming_chunk,
88+
)
89+
generator.run([ChatMessage.from_user("Your question here")])
90+
```
91+
92+
:::info
93+
Streaming works only with a single response. If a provider supports multiple candidates, set `n=1`.
94+
:::
95+
96+
See [Streaming Support](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) for how `StreamingChunk` works and how to write a custom callback. Prefer `print_streaming_chunk` unless you need a specific transport (such as SSE or WebSocket) or custom UI formatting.
97+
98+
### Async
99+
100+
`run_async` mirrors `run` and is wired up automatically — useful inside an async pipeline or web handler.
101+
102+
```python
103+
import asyncio
104+
from haystack.dataclasses import ChatMessage
105+
from haystack_integrations.components.generators.anthropic import AnthropicFoundryChatGenerator
106+
107+
async def main():
108+
generator = AnthropicFoundryChatGenerator(resource="my-resource")
109+
result = await generator.run_async([ChatMessage.from_user("Hello!")])
110+
print(result["replies"][0].text)
111+
112+
asyncio.run(main())
113+
```
114+
115+
## Usage
116+
117+
Install the `anthropic-haystack` package to use the `AnthropicFoundryChatGenerator`:
118+
119+
```shell
120+
pip install anthropic-haystack
121+
```
122+
123+
### On its own
124+
125+
```python
126+
from haystack.dataclasses import ChatMessage
127+
from haystack.utils import Secret
128+
from haystack_integrations.components.generators.anthropic import AnthropicFoundryChatGenerator
129+
130+
generator = AnthropicFoundryChatGenerator(
131+
model="claude-sonnet-4-5",
132+
api_key=Secret.from_env_var("ANTHROPIC_FOUNDRY_API_KEY"),
133+
resource="my-resource",
134+
)
135+
136+
response = generator.run([ChatMessage.from_user("What's Natural Language Processing?")])
137+
print(response)
138+
```
139+
140+
### In a pipeline
141+
142+
```python
143+
from haystack import Pipeline
144+
from haystack.components.builders import ChatPromptBuilder
145+
from haystack.dataclasses import ChatMessage
146+
from haystack_integrations.components.generators.anthropic import AnthropicFoundryChatGenerator
147+
148+
pipe = Pipeline()
149+
pipe.add_component("prompt_builder", ChatPromptBuilder())
150+
pipe.add_component(
151+
"llm",
152+
AnthropicFoundryChatGenerator(resource="my-resource"),
153+
)
154+
pipe.connect("prompt_builder", "llm")
155+
156+
country = "Germany"
157+
messages = [
158+
ChatMessage.from_system(
159+
"You are an assistant giving out valuable information to language learners.",
160+
),
161+
ChatMessage.from_user("What's the official language of {{ country }}?"),
162+
]
163+
164+
res = pipe.run(
165+
data={
166+
"prompt_builder": {
167+
"template_variables": {"country": country},
168+
"template": messages,
169+
},
170+
},
171+
)
172+
print(res)
173+
```

‎docs-website/sidebars.js‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -414,6 +414,7 @@ export default {
414414
'pipeline-components/generators/amazonbedrockgenerator',
415415
'pipeline-components/generators/aimllapichatgenerator',
416416
'pipeline-components/generators/anthropicchatgenerator',
417+
'pipeline-components/generators/anthropicfoundrychatgenerator',
417418
'pipeline-components/generators/anthropicgenerator',
418419
'pipeline-components/generators/anthropicvertexchatgenerator',
419420
'pipeline-components/generators/azureopenaichatgenerator',

‎docs-website/versioned_docs/version-2.30/pipeline-components/generators.mdx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ Generators are responsible for generating text after you give them a prompt. The
1515
| [AmazonBedrockGenerator](generators/amazonbedrockgenerator.mdx) | Enables text generation using models through Amazon Bedrock service. | ✅ |
1616
| [AIMLAPIChatGenerator](generators/aimllapichatgenerator.mdx) | Enables chat completion using AI models through the AIMLAPI. | ✅ |
1717
| [AnthropicChatGenerator](generators/anthropicchatgenerator.mdx) | This component enables chat completions using Anthropic large language models (LLMs). | ✅ |
18+
| [AnthropicFoundryChatGenerator](generators/anthropicfoundrychatgenerator.mdx) | This component enables chat completions using Anthropic models served through Azure Foundry. | ✅ |
1819
| [AnthropicVertexChatGenerator](generators/anthropicvertexchatgenerator.mdx) | This component enables chat completions using AnthropicVertex API. | ✅ |
1920
| [AnthropicGenerator](generators/anthropicgenerator.mdx) | This component enables text completions using Anthropic large language models (LLMs). | ✅ |
2021
| [AzureOpenAIChatGenerator](generators/azureopenaichatgenerator.mdx) | Enables chat completion using OpenAI's LLMs through Azure services. | ✅ |

0 commit comments

Comments
 (0)