diff --git a/docs-website/docs/concepts/pipelines/pipeline-breakpoints.mdx b/docs-website/docs/concepts/pipelines/pipeline-breakpoints.mdx index 0660f68bd33..80fa18a2bcc 100644 --- a/docs-website/docs/concepts/pipelines/pipeline-breakpoints.mdx +++ b/docs-website/docs/concepts/pipelines/pipeline-breakpoints.mdx @@ -2,12 +2,12 @@ title: "Pipeline Breakpoints" id: pipeline-breakpoints slug: "/pipeline-breakpoints" -description: "Learn how to pause and resume Haystack pipeline or Agent execution using breakpoints to debug, inspect, and continue workflows from saved snapshots." +description: "Learn how to pause and resume Haystack pipeline execution using breakpoints to debug, inspect, and continue workflows from saved snapshots." --- # Pipeline Breakpoints -Learn how to pause and resume Haystack pipeline or Agent execution using breakpoints to debug, inspect, and continue workflows from saved snapshots. +Learn how to pause and resume Haystack pipeline execution using breakpoints to debug, inspect, and continue workflows from saved snapshots. ## Introduction @@ -15,8 +15,6 @@ Haystack pipelines support breakpoints for debugging complex execution flows. A You can set a `Breakpoint` on any component in a pipeline with a specific visit count. When triggered, the system stops the execution of the `Pipeline` and captures a snapshot of the current pipeline state. The state can be saved to a JSON file when snapshot file saving is enabled, see [Snapshot file saving](#snapshot-file-saving) below. You can inspect and modify the snapshot and use it to resume execution from the exact point where it stopped. -You can also set breakpoints on an Agent, specifically on the `ChatGenerator` component or on any of the `Tool` specified in the `ToolInvoker` component . - ## Setting a `Breakpoint` on a Regular Component Create a `Breakpoint` by specifying the component name and the visit count at which to trigger it. This is useful for pipelines with loops. The default `visit_count` value is 0. @@ -49,7 +47,7 @@ To access the pipeline state during the breakpoint we can both catch the excepti ## Using a custom snapshot callback -You can pass a `snapshot_callback` to `Pipeline.run()` or `Agent.run()` to handle snapshots yourself instead of saving to a file. When a breakpoint is triggered or a snapshot is created on error, the callback is invoked with the `PipelineSnapshot` object. This is useful for saving snapshots to a database, sending them to a remote service, or custom logging. +You can pass a `snapshot_callback` to `Pipeline.run()` to handle snapshots yourself instead of saving to a file. When a breakpoint is triggered or a snapshot is created on error, the callback is invoked with the `PipelineSnapshot` object. This is useful for saving snapshots to a database, sending them to a remote service, or custom logging. ```python from haystack.core.errors import BreakpointException @@ -72,7 +70,7 @@ except BreakpointException as e: print(f"Breakpoint triggered: {e.component}") ``` -When `snapshot_callback` is provided, file-saving is skipped and the callback is responsible for handling the snapshot. The same parameter is available on `Agent.run()` for agent breakpoints. +When `snapshot_callback` is provided, file-saving is skipped and the callback is responsible for handling the snapshot. ## Snapshot file saving @@ -111,58 +109,6 @@ result = pipeline.run(data={}, pipeline_snapshot=snapshot) print(result["llm"]["replies"]) ``` -## Setting a Breakpoint on an Agent - -You can also set breakpoints in an Agent component. An Agent supports two types of breakpoints: - -1. **Chat Generator Breakpoint**: Pauses before LLM calls. -2. **Tool Invoker Breakpoint**: Pauses before any tool execution. - -A `ChatGenerator` breakpoint is defined as shown below. You need to define a `Breakpoint` as for a pipeline breakpoint and then an `AgentBreakpoint` where you pass the breakpoint defined before and the name of Agent component. - -```python -from haystack.dataclasses.breakpoints import AgentBreakpoint, Breakpoint, ToolBreakpoint - -# Break at chat generator (LLM calls) -chat_bp = Breakpoint(component_name="chat_generator", visit_count=0) -agent_breakpoint = AgentBreakpoint(break_point=chat_bp, agent_name="my_agent") -``` - -To set a breakpoint on a Tool in an Agent, do the following: - -First, define a `ToolBreakpoint` specifying the `ToolInvoker` component whose name is `tool_invoker` and then the tool associated with the breakpoint, in this case – a `weather_tool` . - -Then, define an `AgentBreakpoint` passing the `ToolBreakpoint` defined before as the breakpoint. - -```python -from haystack.dataclasses.breakpoints import AgentBreakpoint, Breakpoint, ToolBreakpoint - -# Break at tool invoker (tool calls) -tool_bp = ToolBreakpoint( - component_name="tool_invoker", - visit_count=0, - tool_name="weather_tool", # Specific tool, or None for any tool -) -agent_breakpoint = AgentBreakpoint(break_point=tool_bp, agent_name="my_agent") -``` - -### Resuming Agent Execution - -When an Agent breakpoint is triggered, you can resume execution using the saved snapshot. Similar to the regular component in a pipeline, pass the JSON file with the snapshot to the `run()` method of the pipeline. - -```python -from haystack.core.pipeline.breakpoint import load_pipeline_snapshot - -# Load the snapshot -snapshot_file = "./agent_debug/agent_chat_generator_2025_07_11_23_23.json" -snapshot = load_pipeline_snapshot(snapshot_file) - -# Resume pipeline execution -result = pipeline.run(data={}, pipeline_snapshot=snapshot) -print("Pipeline resumed successfully") -print(f"Final result: {result}") -``` - ## Error Recovery with Snapshots Pipelines automatically create a snapshot of the last valid state if a run fails. The snapshot contains inputs, visit counts, and intermediate outputs up to the failure. You can inspect it, fix the issue, and resume execution from that checkpoint instead of restarting the whole run. diff --git a/docs-website/docs/development/tracing.mdx b/docs-website/docs/development/tracing.mdx index 3224ca5c122..a0c82519f7a 100644 --- a/docs-website/docs/development/tracing.mdx +++ b/docs-website/docs/development/tracing.mdx @@ -23,26 +23,17 @@ Instrumented applications typically send traces to a trace collector or a tracin | [LoggingTracer](tracing/logging-tracer.mdx) | Inspect the data flowing through your pipeline in real time through logs, with no backend setup. | | [Custom Tracer](tracing/custom-tracer.mdx) | Connect any tracing backend by implementing the `Tracer` interface. | -## Disabling Auto Tracing +## Enabling and Disabling Tracing -Haystack automatically detects and enables tracing under the following circumstances: +Haystack never enables tracing automatically. To enable it, either call `haystack.tracing.enable_tracing(...)` with the tracer of your choice, or add a tracing connector component such as the [`OpenTelemetryConnector`](tracing/opentelemetry.mdx) or the [`DatadogConnector`](tracing/datadog.mdx) to your pipeline. -- If `opentelemetry-sdk` is installed and configured for OpenTelemetry. Note that this auto-enabling is deprecated and will be removed in Haystack 3.0 – use the [`OpenTelemetryConnector`](tracing/opentelemetry.mdx) to enable OpenTelemetry tracing instead. -- If `ddtrace` is installed for Datadog. Note that this auto-enabling is deprecated and will be removed in Haystack 3.0 – use the [`DatadogConnector`](tracing/datadog.mdx) to enable Datadog tracing instead. +To disable an enabled tracer: -To disable this behavior, there are two options: +```python +from haystack.tracing import disable_tracing -- Set the environment variable `HAYSTACK_AUTO_TRACE_ENABLED` to `false` when running your Haystack application - -— or — - -- Disable tracing in Python: - - ```python - from haystack.tracing import disable_tracing - - disable_tracing() - ``` +disable_tracing() +``` ## Content Tracing diff --git a/docs-website/docs/overview/migrating-from-langgraphlangchain-to-haystack.mdx b/docs-website/docs/overview/migrating-from-langgraphlangchain-to-haystack.mdx index 233201d1019..52d9315c1ff 100644 --- a/docs-website/docs/overview/migrating-from-langgraphlangchain-to-haystack.mdx +++ b/docs-website/docs/overview/migrating-from-langgraphlangchain-to-haystack.mdx @@ -43,7 +43,7 @@ Here's a table of key concepts and their approximate equivalents between the two | Multi-Agent Collaboration (LangChain) | Multi-Agent System | Using [`ComponentTool`](../tools/componenttool.mdx), agents can use other agents as tools, enabling [multi-agent architectures](https://haystack.deepset.ai/tutorials/45_creating_a_multi_agent_system) within one framework. | | Model Context Protocol `load_mcp_tools` `MultiServerMCPClient` | Model Context Protocol - `MCPTool`, `MCPToolset`, `StdioServerInfo`, `StreamableHttpServerInfo` | Haystack provides [various MCP primitives](https://haystack.deepset.ai/integrations/mcp) for connecting multiple MCP servers and organizing MCP toolsets. | | Memory (State, short-term, long-term) | Memory (Agent State, short-term, long-term) | Agent [State](../pipeline-components/agents-1/state.mdx) provides a structured way to share data between tools and store intermediate results during agent execution. For short-term memory, Haystack offers a [ChatMessage Store](/reference/experimental-chatmessage-store-api) to persist chat history. More memory options are coming soon. | -| Time travel (Checkpoints) | Breakpoints (Breakpoint, AgentBreakpoint, ToolBreakpoint, Snapshot) | [Breakpoints](../concepts/pipelines/pipeline-breakpoints.mdx) let you pause, inspect, modify, and resume a pipeline, agent, or tool for debugging or iterative development. | +| Time travel (Checkpoints) | Breakpoints (Breakpoint, PipelineSnapshot) | [Breakpoints](../concepts/pipelines/pipeline-breakpoints.mdx) let you pause, inspect, modify, and resume a pipeline for debugging or iterative development. | | Human-in-the-Loop (Interrupts / Commands) | Human-in-the-loop (`ConfirmationHook` with confirmation strategies) | Haystack applies [confirmation strategies](https://haystack.deepset.ai/tutorials/47_human_in_the_loop_agent) through a `ConfirmationHook` registered under the Agent's `before_tool` [hook point](../pipeline-components/agents-1/hooks.mdx) to pause or block the execution to gather user feedback | ## Ecosystem and Tooling Mapping: LangChain → Haystack diff --git a/docs-website/docs/pipeline-components/generators/googleaigeminichatgenerator.mdx b/docs-website/docs/pipeline-components/generators/googleaigeminichatgenerator.mdx index 40e266934f7..2d781290e4a 100644 --- a/docs-website/docs/pipeline-components/generators/googleaigeminichatgenerator.mdx +++ b/docs-website/docs/pipeline-components/generators/googleaigeminichatgenerator.mdx @@ -98,7 +98,7 @@ def get_current_weather( tool = create_tool_from_function(get_current_weather) ``` -Create a new instance of `GoogleAIGeminiChatGenerator` to set the tools and a [ToolInvoker](../tools/toolinvoker.mdx) to invoke the tools. +Create a new instance of `GoogleAIGeminiChatGenerator` to set the tools and a `ToolInvoker` to invoke the tools. ```python import os diff --git a/docs-website/docs/pipeline-components/generators/guides-to-generators/function-calling.mdx b/docs-website/docs/pipeline-components/generators/guides-to-generators/function-calling.mdx index 3467e57037f..c1efa59cd61 100644 --- a/docs-website/docs/pipeline-components/generators/guides-to-generators/function-calling.mdx +++ b/docs-website/docs/pipeline-components/generators/guides-to-generators/function-calling.mdx @@ -74,7 +74,7 @@ At this point the model has only *requested* a call. Nothing has been executed y ### 3. Actually Invoke the Tool -To execute the requested tool calls, use the [`ToolInvoker`](../../tools/toolinvoker.mdx) component. It takes a list of [`ChatMessage`](../../../concepts/data-classes/chatmessage.mdx) objects containing tool calls, runs the corresponding tools, and returns new `ChatMessage` objects (with role `tool`) carrying the results as `ToolCallResult`s. To get a final natural-language answer, feed those results back to the ChatGenerator. +To execute the requested tool calls, use the `ToolInvoker` component. It takes a list of [`ChatMessage`](../../../concepts/data-classes/chatmessage.mdx) objects containing tool calls, runs the corresponding tools, and returns new `ChatMessage` objects (with role `tool`) carrying the results as `ToolCallResult`s. To get a final natural-language answer, feed those results back to the ChatGenerator. ```python from haystack.components.tools import ToolInvoker diff --git a/docs-website/docs/pipeline-components/generators/vertexaigeminichatgenerator.mdx b/docs-website/docs/pipeline-components/generators/vertexaigeminichatgenerator.mdx index 2d601cef711..4bf43908642 100644 --- a/docs-website/docs/pipeline-components/generators/vertexaigeminichatgenerator.mdx +++ b/docs-website/docs/pipeline-components/generators/vertexaigeminichatgenerator.mdx @@ -101,7 +101,7 @@ def get_current_weather( tool = create_tool_from_function(get_current_weather) ``` -Create a new instance of `VertexAIGeminiChatGenerator` to set the tools and a [ToolInvoker](../tools/toolinvoker.mdx) to invoke the tools.: +Create a new instance of `VertexAIGeminiChatGenerator` to set the tools and a `ToolInvoker` to invoke the tools.: ```python from haystack_integrations.components.generators.google_vertex import ( diff --git a/docs-website/docs/pipeline-components/tools/toolinvoker.mdx b/docs-website/docs/pipeline-components/tools/toolinvoker.mdx deleted file mode 100644 index 634bfd72179..00000000000 --- a/docs-website/docs/pipeline-components/tools/toolinvoker.mdx +++ /dev/null @@ -1,214 +0,0 @@ ---- -title: "ToolInvoker" -id: toolinvoker -slug: "/toolinvoker" -description: "This component is designed to execute tool calls prepared by language models. It acts as a bridge between the language model's output and the actual execution of functions or tools that perform specific tasks." ---- - -# ToolInvoker - -This component is designed to execute tool calls prepared by language models. It acts as a bridge between the language model's output and the actual execution of functions or tools that perform specific tasks. - -
- -| | | -| --- | --- | -| **Most common position in a pipeline** | After a Chat Generator | -| **Mandatory init variables** | `tools`: A list of [`Tools`](../../tools/tool.mdx) that can be invoked | -| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects from a Chat Generator containing tool calls | -| **Output variables** | `tool_messages`: A list of `ChatMessage` objects with tool role. Each `ChatMessage` objects wraps the result of a tool invocation. | -| **API reference** | [Tools](/reference/tools-api) | -| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/tools/tool_invoker.py | -| **Package name** | `haystack-ai` | - -
- -## Overview - -A `ToolInvoker` is a component that processes `ChatMessage` objects containing tool calls. It invokes the corresponding tools and returns the results as a list of `ChatMessage` objects. Each tool is defined with a name, description, parameters, and a function that performs the task. The `ToolInvoker` manages these tools and handles the invocation process. - -You can pass multiple tools to the `ToolInvoker` component, and it will automatically choose the right tool to call based on tool calls produced by a Language Model. - -The `ToolInvoker` has two additionally helpful parameters: - -- `convert_result_to_json_string`: Use `json.dumps` (when True) or `str` (when False) to convert the result into a string. -- `raise_on_failure`: If True, it will raise an exception in case of errors. If False, it will return a `ChatMessage` object with `error=True` and a description of the error in `result`. Use this, for example, when you want to keep the Language Model running in a loop and fixing its errors. - -:::info[ChatMessage and Tool Data Classes] - -Follow the links to learn more about [ChatMessage](../../concepts/data-classes/chatmessage.mdx) and [Tool](../../tools/tool.mdx) data classes. -::: - -## Usage - -### On its own - -```python -from haystack.dataclasses import ChatMessage, ToolCall -from haystack.components.tools import ToolInvoker -from haystack.tools import Tool - - -# Tool definition -def dummy_weather_function(city: str): - return f"The weather in {city} is 20 degrees." - - -parameters = { - "type": "object", - "properties": {"city": {"type": "string"}}, - "required": ["city"], -} -tool = Tool( - name="weather_tool", - description="A tool to get the weather", - function=dummy_weather_function, - parameters=parameters, -) - -# Usually, the ChatMessage with tool_calls is generated by a Language Model -# Here, we create it manually for demonstration purposes -tool_call = ToolCall(tool_name="weather_tool", arguments={"city": "Berlin"}) -message = ChatMessage.from_assistant(tool_calls=[tool_call]) - -# ToolInvoker initialization and run -invoker = ToolInvoker(tools=[tool]) -result = invoker.run(messages=[message]) - -print(result) -``` - -``` ->> { ->> 'tool_messages': [ ->> ChatMessage( ->> _role=, ->> _content=[ ->> ToolCallResult( ->> result='"The weather in Berlin is 20 degrees."', ->> origin=ToolCall( ->> tool_name='weather_tool', ->> arguments={'city': 'Berlin'}, ->> id=None ->> ) ->> ) ->> ], ->> _meta={} ->> ) ->> ] ->> } -``` - -### In a pipeline - -The following code snippet shows how to process a user query about the weather. First, we define a `Tool` for fetching weather data, then we initialize a `ToolInvoker` to execute this tool, while using an `OpenAIChatGenerator` to generate responses. A `ConditionalRouter` is used in this pipeline to route messages based on whether they contain tool calls. The pipeline connects these components, processes a user message asking for the weather in Berlin, and outputs the result. - -```python -from haystack.dataclasses import ChatMessage -from haystack.components.tools import ToolInvoker -from haystack.components.generators.chat import OpenAIChatGenerator -from haystack.components.routers import ConditionalRouter -from haystack.tools import Tool -from haystack import Pipeline -from typing import List # Ensure List is imported - -# Define a dummy weather tool -import random - - -def dummy_weather(location: str): - return { - "temp": f"{random.randint(-10, 40)} °C", - "humidity": f"{random.randint(0, 100)}%", - } - - -weather_tool = Tool( - name="weather", - description="A tool to get the weather", - function=dummy_weather, - parameters={ - "type": "object", - "properties": {"location": {"type": "string"}}, - "required": ["location"], - }, -) - -# Initialize the ToolInvoker with the weather tool -tool_invoker = ToolInvoker(tools=[weather_tool]) - -# Initialize the ChatGenerator -chat_generator = OpenAIChatGenerator(model="gpt-4o-mini", tools=[weather_tool]) - -# Define routing conditions -routes = [ - { - "condition": "{{replies[0].tool_calls | length > 0}}", - "output": "{{replies}}", - "output_name": "there_are_tool_calls", - "output_type": List[ChatMessage], # Use direct type - }, - { - "condition": "{{replies[0].tool_calls | length == 0}}", - "output": "{{replies}}", - "output_name": "final_replies", - "output_type": List[ChatMessage], # Use direct type - }, -] - -# Initialize the ConditionalRouter -router = ConditionalRouter(routes, unsafe=True) - -# Create the pipeline -pipeline = Pipeline() -pipeline.add_component("generator", chat_generator) -pipeline.add_component("router", router) -pipeline.add_component("tool_invoker", tool_invoker) - -# Connect components -pipeline.connect("generator.replies", "router") -pipeline.connect( - "router.there_are_tool_calls", - "tool_invoker.messages", -) # Correct connection - -# Example user message -user_message = ChatMessage.from_user("What is the weather in Berlin?") - -# Run the pipeline -result = pipeline.run({"messages": [user_message]}) - -# Print the result -print(result) -``` - -``` -{ - "tool_invoker":{ - "tool_messages":[ - "ChatMessage(_role=", - "_content="[ - "ToolCallResult(result=""{'temp': '33 °C', 'humidity': '79%'}", - "origin=ToolCall(tool_name=""weather", - "arguments="{ - "location":"Berlin" - }, - "id=""call_pUVl8Cycssk1dtgMWNT1T9eT"")", - "error=False)" - ], - "_name=None", - "_meta="{ - - }")" - ] - } -} -``` - -## Additional References - -🧑‍🍳 Cookbooks: - -- [Define & Run Tools](https://haystack.deepset.ai/cookbook/tools_support) -- [Newsletter Sending Agent with Haystack Tools](https://haystack.deepset.ai/cookbook/newsletter-agent) -- [Create a Swarm of Agents](https://haystack.deepset.ai/cookbook/swarm) diff --git a/docs-website/docs/tools/ready-made-tools.mdx b/docs-website/docs/tools/ready-made-tools.mdx index 1aaa4284e51..521d2ee37b1 100644 --- a/docs-website/docs/tools/ready-made-tools.mdx +++ b/docs-website/docs/tools/ready-made-tools.mdx @@ -7,7 +7,7 @@ description: "Ready-made Tools and Toolsets in Haystack provide prebuilt capabil # Ready-Made Tools -Ready-made Tools and Toolsets provide prebuilt capabilities for common Agent workflows. You can pass them directly to an [Agent](../pipeline-components/agents-1/agent.mdx), use them with a [ToolInvoker](../pipeline-components/tools/toolinvoker.mdx), or inspect their function and parameter schema when you need to customize their behavior. +Ready-made Tools and Toolsets provide prebuilt capabilities for common Agent workflows. You can pass them directly to an [Agent](../pipeline-components/agents-1/agent.mdx), which executes them for you, or inspect their function and parameter schema when you need to customize their behavior. These are the ready-made Tools and Toolsets available in Haystack: diff --git a/docs-website/docs/tools/tool.mdx b/docs-website/docs/tools/tool.mdx index 7530c080bb8..cc9bce697b7 100644 --- a/docs-website/docs/tools/tool.mdx +++ b/docs-website/docs/tools/tool.mdx @@ -21,7 +21,7 @@ If you are looking for the details of this data class's methods and parameters, A tool is a function for which Language Models can prepare a call. -The `Tool` class is used in Chat Generators and provides a consistent experience across models. `Tool` is also used in the [`ToolInvoker`](../pipeline-components/tools/toolinvoker.mdx) component that executes calls prepared by Language Models. +The `Tool` class is used in Chat Generators and provides a consistent experience across models. `Tool` is also used in the `ToolInvoker` component that executes calls prepared by Language Models. ```python @dataclass @@ -442,7 +442,7 @@ print(res_w_tool_call) #### Executing Tool Calls -To execute prepared tool calls, you can use the [`ToolInvoker`](../pipeline-components/tools/toolinvoker.mdx) component. +To execute prepared tool calls, you can use the `ToolInvoker` component. This component acts as the execution engine for tools, processing the calls prepared by the Language Model. Here’s an example: diff --git a/docs-website/docs/tools/toolset.mdx b/docs-website/docs/tools/toolset.mdx index 27147ac9226..108d7e31913 100644 --- a/docs-website/docs/tools/toolset.mdx +++ b/docs-website/docs/tools/toolset.mdx @@ -22,7 +22,7 @@ Group multiple Tools into a single unit. ## Overview -A `Toolset` groups multiple Tool instances into a single manageable unit. It simplifies passing tools to components like Chat Generators, [`ToolInvoker`](../pipeline-components/tools/toolinvoker.mdx), or [`Agent`](../pipeline-components/agents-1/agent.mdx), and supports filtering, serialization, and reuse. +A `Toolset` groups multiple Tool instances into a single manageable unit. It simplifies passing tools to components like Chat Generators, `ToolInvoker`, or [`Agent`](../pipeline-components/agents-1/agent.mdx), and supports filtering, serialization, and reuse. Additionally, by subclassing `Toolset`, you can create implementations that dynamically load tools from external sources like OpenAPI URLs, MCP servers, or other resources. diff --git a/docs-website/sidebars.js b/docs-website/sidebars.js index a4e0655f993..2c5dfff479d 100644 --- a/docs-website/sidebars.js +++ b/docs-website/sidebars.js @@ -643,13 +643,6 @@ export default { 'pipeline-components/samplers/toppsampler', ], }, - { - type: 'category', - label: 'Tools', - items: [ - 'pipeline-components/tools/toolinvoker', - ], - }, { type: 'category', label: 'Translators',