Skip to content

Commit feaa186

Browse files
julian-rischjulian-risch
authored
Add Agent tutorial
1 parent 3ee35f0 commit feaa186

1 file changed

Lines changed: 368 additions & 0 deletions

File tree

‎tutorials/43_Building_a_Tool_Calling_Agent.ipynb‎

Lines changed: 368 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,368 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {
6+
"id": "2OvkPji9O-qX"
7+
},
8+
"source": [
9+
"# Tutorial: Building a Tool-Calling Agent\n",
10+
"\n",
11+
"- **Level**: Beginner\n",
12+
"- **Time to complete**: 15 minutes\n",
13+
"- **Components Used**: [`Agent`](https://docs.haystack.deepset.ai/docs/agent), [`OpenAIChatGenerator`](https://docs.haystack.deepset.ai/docs/openaichatgenerator), [`SerperDevWebSearch`](https://docs.haystack.deepset.ai/docs/serperdevwebsearch), [`ComponentTool`](https://docs.haystack.deepset.ai/docs/componenttool)\n",
14+
"- **Prerequisites**: You must have an [OpenAI API Key](https://platform.openai.com/api-keys) and a [SerperDev API Key](https://serper.dev/api-key)\n",
15+
"- **Goal**: After completing this tutorial, you'll have learned how to create an Agent that can use tools to answer questions and perform tasks, both as a standalone component and integrated with a pipeline."
16+
]
17+
},
18+
{
19+
"cell_type": "markdown",
20+
"metadata": {
21+
"id": "LFqHcXYPO-qZ"
22+
},
23+
"source": [
24+
"## Overview\n",
25+
"\n",
26+
"In this tutorial, you'll learn how to create an agent that can use tools to answer questions and perform tasks. We'll explore two approaches:\n",
27+
"\n",
28+
"1. Using the Agent as a standalone component with a simple web search tool\n",
29+
"2. Integrating the Agent with a more complex pipeline with multiple components\n",
30+
"\n",
31+
"The Agent component allows you to create AI assistants that can use tools to gather information, perform calculations, or interact with external systems. It uses a large language model (LLM) to understand user queries and decide which tools to use to answer them."
32+
]
33+
},
34+
{
35+
"cell_type": "markdown",
36+
"metadata": {
37+
"id": "QXjVlbPiO-qZ"
38+
},
39+
"source": [
40+
"## Preparing the Environment\n",
41+
"\n",
42+
"First, let's install Haystack and set up the environment:"
43+
]
44+
},
45+
{
46+
"cell_type": "code",
47+
"execution_count": null,
48+
"metadata": {
49+
"id": "UQbU8GUfO-qZ"
50+
},
51+
"outputs": [],
52+
"source": [
53+
"%%bash\n",
54+
"\n",
55+
"pip install haystack-ai"
56+
]
57+
},
58+
{
59+
"cell_type": "markdown",
60+
"metadata": {
61+
"id": "_lvfew16O-qa"
62+
},
63+
"source": [
64+
"### Enter API Keys\n",
65+
"\n",
66+
"Enter your API keys for OpenAI and SerperDev:"
67+
]
68+
},
69+
{
70+
"cell_type": "code",
71+
"execution_count": null,
72+
"metadata": {
73+
"id": "CbVN-s5LO-qa"
74+
},
75+
"outputs": [],
76+
"source": [
77+
"from getpass import getpass\n",
78+
"import os\n",
79+
"\n",
80+
"if \"OPENAI_API_KEY\" not in os.environ:\n",
81+
" os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter OpenAI API key:\")\n",
82+
"if \"SERPERDEV_API_KEY\" not in os.environ:\n",
83+
" os.environ[\"SERPERDEV_API_KEY\"] = getpass(\"Enter SerperDev API key: \")"
84+
]
85+
},
86+
{
87+
"cell_type": "markdown",
88+
"metadata": {
89+
"id": "yL8nuJdWO-qa"
90+
},
91+
"source": [
92+
"## Using Agent as a Standalone Component\n",
93+
"\n",
94+
"We start with a simple example of using the Agent as a standalone component with a web search tool:"
95+
]
96+
},
97+
{
98+
"cell_type": "code",
99+
"execution_count": null,
100+
"metadata": {
101+
"id": "XvLVaFHTO-qb"
102+
},
103+
"outputs": [],
104+
"source": [
105+
"from haystack.components.agents import Agent\n",
106+
"from haystack.components.generators.chat import OpenAIChatGenerator\n",
107+
"from haystack.components.websearch import SerperDevWebSearch\n",
108+
"from haystack.dataclasses import ChatMessage\n",
109+
"from haystack.tools.component_tool import ComponentTool\n",
110+
"\n",
111+
"# Create a web search tool using SerperDevWebSearch\n",
112+
"web_tool = ComponentTool(\n",
113+
" component=SerperDevWebSearch(),\n",
114+
")\n",
115+
"\n",
116+
"# Create the agent with the web search tool\n",
117+
"agent = Agent(\n",
118+
" chat_generator=OpenAIChatGenerator(),\n",
119+
" tools=[web_tool],\n",
120+
")\n",
121+
"\n",
122+
"# Run the agent with a query\n",
123+
"result = agent.run(\n",
124+
" messages=[ChatMessage.from_user(\"Find information about Haystack\")]\n",
125+
")\n",
126+
"\n",
127+
"# Print the final response\n",
128+
"print(result[\"messages\"][-1].text)"
129+
]
130+
},
131+
{
132+
"cell_type": "markdown",
133+
"metadata": {},
134+
"source": [
135+
"\n",
136+
"The Agent has a couple of optional parameters that let you customize it's behavior:\n",
137+
"- `system_prompt` for defining a system prompt with instructions for the Agent's LLM\n",
138+
"- `exit_conditions` that will cause the agent to return. It's a list of strings and the items can be `\"text\"`, which means that the Agent will exit as soon as the LLM replies only with a text response,\n",
139+
"or specific tool names.\n",
140+
"- `state_schema` for the State that is shared across one agent invocation run. It defines extra information – such as documents or context – that tools can read from or write to during execution. You can use this schema to pass parameters that tools can both produce and consume.\n",
141+
"- `streaming_callback` to stream the tokens from the LLM directly to output.\n",
142+
"- `raise_on_tool_invocation_failure` to decide if the agent should raise an exception when a tool invocation fails. If set to False, the exception will be turned into a chat message and passed to the LLM. It can then try to improve with the next tool invocation.\n",
143+
"- `max_agent_steps` to limit how many times the Agent can call tools and prevent endless loops.\n",
144+
"\n",
145+
"In the previous example, the Agent was allowed to call the tool multiple times until the default exit condition was met: a text response. Now, let's return right after calling `web_tool` or after the LLM generated text, whatever happens first. Let's also use streaming so that we see the tokens of the response while they are being generated."
146+
]
147+
},
148+
{
149+
"cell_type": "code",
150+
"execution_count": null,
151+
"metadata": {},
152+
"outputs": [],
153+
"source": [
154+
"agent = Agent(\n",
155+
" chat_generator=OpenAIChatGenerator(),\n",
156+
" tools=[web_tool],\n",
157+
" exit_conditions=[\"text\", \"web_tool\"],\n",
158+
" streaming_callback=lambda chunk: print(chunk.content, end=\"\", flush=True),\n",
159+
")\n",
160+
"\n",
161+
"result = agent.run(\n",
162+
" messages=[ChatMessage.from_user(\"Find information about Haystack\")]\n",
163+
")"
164+
]
165+
},
166+
{
167+
"cell_type": "markdown",
168+
"metadata": {},
169+
"source": [
170+
"You can easily switch out the ChatGenerator used in the Agent. Currently all of the following ChatGenerators support tools and thus can be used with Agent:\n",
171+
"\n",
172+
"- AmazonBedrockChatGenerator\n",
173+
"- AnthropicChatGenerator\n",
174+
"- AzureOpenAIChatGenerator\n",
175+
"- CohereChatGenerator\n",
176+
"- GoogleAIGeminiChatGenerator\n",
177+
"- HuggingFaceAPIChatGenerator\n",
178+
"- HuggingFaceLocalChatGenerator\n",
179+
"- MistralChatGenerator\n",
180+
"- OllamaChatGenerator\n",
181+
"- OpenAIChatGenerator\n",
182+
"- VertexAIGeminiChatGenerator\n",
183+
"\n",
184+
"For example, if you have a `HF_API_TOKEN` and `huggingface_hub[inference]>=0.27.0` installed, all you need to do is replace OpenAIChatGenerator by HuggingFaceAPIChatGenerator and run `from haystack.components.generators.chat import HuggingFaceAPIChatGenerator`"
185+
]
186+
},
187+
{
188+
"cell_type": "markdown",
189+
"metadata": {
190+
"id": "HryYZP9ZO-qb"
191+
},
192+
"source": [
193+
"## Using Agent with a Pipeline as Tool\n",
194+
"\n",
195+
"Now, for a more sophisticated example, let's build a research assistant that can search the web, fetch content from links, and generate comprehensive answers. We'll start with a Haystack Pipeline that the Agent can use as a tool:"
196+
]
197+
},
198+
{
199+
"cell_type": "code",
200+
"execution_count": null,
201+
"metadata": {
202+
"id": "INdC3WvLO-qb"
203+
},
204+
"outputs": [],
205+
"source": [
206+
"from haystack.components.agents import Agent\n",
207+
"from haystack.components.generators.chat import OpenAIChatGenerator\n",
208+
"from haystack.components.builders.answer_builder import AnswerBuilder\n",
209+
"from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder\n",
210+
"from haystack.components.converters.html import HTMLToDocument\n",
211+
"from haystack.components.fetchers.link_content import LinkContentFetcher\n",
212+
"from haystack.components.websearch.serper_dev import SerperDevWebSearch\n",
213+
"from haystack.dataclasses import ChatMessage\n",
214+
"from haystack.core.pipeline import Pipeline\n",
215+
"from haystack.core.super_component import SuperComponent\n",
216+
"from haystack.tools import ComponentTool\n",
217+
"\n",
218+
"search_pipeline = Pipeline()\n",
219+
"\n",
220+
"search_pipeline.add_component(\"search\", SerperDevWebSearch(top_k=10))\n",
221+
"search_pipeline.add_component(\"fetcher\", LinkContentFetcher(timeout=3, raise_on_failure=False, retry_attempts=2))\n",
222+
"search_pipeline.add_component(\"converter\", HTMLToDocument())\n",
223+
"search_pipeline.add_component(\"builder\", ChatPromptBuilder(\n",
224+
" template=[ChatMessage.from_user(\"\"\"\n",
225+
"{% for doc in docs %}\n",
226+
"<search-result url=\"{{ doc.meta.url }}\">\n",
227+
"{{ doc.content|default|truncate(25000) }}\n",
228+
"</search-result>\n",
229+
"{% endfor %}\n",
230+
"\"\"\")],\n",
231+
" variables=[\"docs\"],\n",
232+
" required_variables=[\"docs\"]\n",
233+
"))\n",
234+
"\n",
235+
"search_pipeline.connect(\"search.links\", \"fetcher.urls\")\n",
236+
"search_pipeline.connect(\"fetcher.streams\", \"converter.sources\")\n",
237+
"search_pipeline.connect(\"converter.documents\", \"builder.docs\")\n",
238+
"\n",
239+
"# Wrap in SuperComponent + ComponentTool\n",
240+
"search_component = SuperComponent(pipeline=search_pipeline)\n",
241+
"search_tool = ComponentTool(\n",
242+
" name=\"search\",\n",
243+
" description=\"Use this tool to search for information on the internet.\",\n",
244+
" component=search_component\n",
245+
")\n",
246+
"\n",
247+
"agent = Agent(\n",
248+
" chat_generator=OpenAIChatGenerator(),\n",
249+
" tools=[search_tool],\n",
250+
" system_prompt=\"\"\"\n",
251+
"You are a deep research assistant.\n",
252+
"You create comprehensive research reports to answer the user's questions.\n",
253+
"You use the 'search'-tool to answer any questions.\n",
254+
"You perform multiple searches until you have the information you need to answer the question.\n",
255+
"Make sure you research different aspects of the question.\n",
256+
"Use markdown to format your response.\n",
257+
"When you use information from the websearch results, cite your sources using markdown links.\n",
258+
"It is important that you cite accurately.\n",
259+
"\"\"\",\n",
260+
" exit_conditions=[\"text\"],\n",
261+
" max_agent_steps=20,\n",
262+
")"
263+
]
264+
},
265+
{
266+
"cell_type": "markdown",
267+
"metadata": {},
268+
"source": [
269+
"\n",
270+
"Our Agent is ready to use! It is good practice to call `agent.warm_up()` before running an Agent, which makes sure models are loaded in case that's required. "
271+
]
272+
},
273+
{
274+
"cell_type": "code",
275+
"execution_count": null,
276+
"metadata": {},
277+
"outputs": [],
278+
"source": [
279+
"query = \"What are the latest updates on the Artemis moon mission?\"\n",
280+
"messages = [ChatMessage.from_user(query)]\n",
281+
"\n",
282+
"agent.warm_up()\n",
283+
"agent_output = agent.run(messages=messages)\n",
284+
"\n",
285+
"# Filter replies with valid 'text' only\n",
286+
"valid_replies = [msg for msg in agent_output[\"messages\"] if getattr(msg, \"text\", None)]\n",
287+
"\n",
288+
"# Answer builder\n",
289+
"answer_builder = AnswerBuilder()\n",
290+
"answers = answer_builder.run(query=query, replies=valid_replies)\n",
291+
"\n",
292+
"# Print the result\n",
293+
"for answer in answers[\"answers\"]:\n",
294+
" print(answer)"
295+
]
296+
},
297+
{
298+
"cell_type": "markdown",
299+
"metadata": {
300+
"id": "czMjWwnxPA-3"
301+
},
302+
"source": [
303+
"Let's break down this last example in the tutorial.\n",
304+
"The **Agent** is the main component that orchestrates the interaction between the LLM and tools.\n",
305+
"We use **ComponentTool** as a wrapper that allows Haystack components to be used as tools by the agent.\n",
306+
"The **SuperComponent** wraps entire pipelines so that they can be used as components and thus also as tools.\n",
307+
"\n",
308+
"We created a sophisticated search pipeline that:\n",
309+
"1. Searches the web using SerperDevWebSearch\n",
310+
"2. Fetches content from the found links\n",
311+
"3. Converts HTML content to Documents\n",
312+
"4. Formats the results for the Agent\n",
313+
"\n",
314+
"The Agent then uses this pipeline as a tool to gather information and generate comprehensive answers.\n",
315+
"\n",
316+
"By the way, did you know that the Agent is a Haystack component itself? That means you can use and combine an Agent in your pipelines just like any other component!"
317+
]
318+
},
319+
{
320+
"cell_type": "markdown",
321+
"metadata": {
322+
"id": "9y4iJE_SrS4K"
323+
},
324+
"source": [
325+
"## What's next\n",
326+
"\n",
327+
"🎉 Congratulations! You've learned how to create a tool-calling Agent with Haystack. You can now:\n",
328+
"- Create simple agents with basic tools\n",
329+
"- Build complex pipelines with multiple components\n",
330+
"- Use the Agent component to create sophisticated AI assistants\n",
331+
"- Combine web search, content fetching, and document processing in your applications\n",
332+
"\n",
333+
"If you liked this tutorial, you may also enjoy reusing pipelines from the following tutorials and make them tools of a powerful Agent:\n",
334+
"- [Creating Your First QA Pipeline with Retrieval-Augmentation](https://haystack.deepset.ai/tutorials/27_first_rag_pipeline)\n",
335+
"- [Building Fallbacks with Conditional Routing](https://haystack.deepset.ai/tutorials/36_building_fallbacks_with_conditional_routing)\n",
336+
"\n",
337+
"To stay up to date on the latest Haystack developments, you can [subscribe to our newsletter](https://landing.deepset.ai/haystack-community-updates) and [join Haystack discord community](https://discord.gg/haystack).\n",
338+
"\n",
339+
"Thanks for reading!"
340+
]
341+
}
342+
],
343+
"metadata": {
344+
"accelerator": "GPU",
345+
"colab": {
346+
"gpuType": "T4",
347+
"provenance": []
348+
},
349+
"kernelspec": {
350+
"display_name": "Python 3",
351+
"name": "python3"
352+
},
353+
"language_info": {
354+
"codemirror_mode": {
355+
"name": "ipython",
356+
"version": 3
357+
},
358+
"file_extension": ".py",
359+
"mimetype": "text/x-python",
360+
"name": "python",
361+
"nbconvert_exporter": "python",
362+
"pygments_lexer": "ipython3",
363+
"version": "3.9.6"
364+
}
365+
},
366+
"nbformat": 4,
367+
"nbformat_minor": 0
368+
}

0 commit comments

Comments
 (0)