diff --git a/docs-website/docs/concepts/agents/multi-agent-systems.mdx b/docs-website/docs/concepts/agents/multi-agent-systems.mdx index bcfbccdee85..0af576578a9 100644 --- a/docs-website/docs/concepts/agents/multi-agent-systems.mdx +++ b/docs-website/docs/concepts/agents/multi-agent-systems.mdx @@ -147,9 +147,9 @@ components: tools: null tools_strict: false type: haystack.components.generators.chat.openai.OpenAIChatGenerator - confirmation_strategies: null exit_conditions: - text + hooks: null max_agent_steps: 100 raise_on_tool_invocation_failure: false required_variables: null @@ -157,7 +157,8 @@ components: streaming_callback: null system_prompt: You are a coordinator. Delegate research tasks to the research specialist. Keep your final answer concise. - tool_invoker_kwargs: null + tool_concurrency_limit: 4 + tool_streaming_callback_passthrough: false tools: - data: component: @@ -180,9 +181,9 @@ components: tools: null tools_strict: false type: haystack.components.generators.chat.openai.OpenAIChatGenerator - confirmation_strategies: null exit_conditions: - text + hooks: null max_agent_steps: 100 raise_on_tool_invocation_failure: false required_variables: null @@ -190,7 +191,8 @@ components: streaming_callback: null system_prompt: You are a research specialist. Search the web to find information. Return a concise summary of your findings in 3-5 sentences. - tool_invoker_kwargs: null + tool_concurrency_limit: 4 + tool_streaming_callback_passthrough: false tools: - data: component: @@ -242,7 +244,7 @@ Returning just `result["last_message"].text` (with `@tool`) or using `outputs_to When covering multiple topics, the coordinator can call the same specialist tool several times in a single response. All tool calls from one LLM response are executed concurrently using a thread pool. -Control the level of parallelism with `max_workers` in `tool_invoker_kwargs` (default: `4`). +Control the level of parallelism with the `tool_concurrency_limit` init parameter (default: `4`). The example below asks the coordinator about two topics: it calls `research` twice and both specialists run in parallel. @@ -319,7 +321,7 @@ coordinator = Agent( "Keep your final answer concise." ), streaming_callback=print_streaming_chunk, - tool_invoker_kwargs={"max_workers": 4}, # run up to 4 specialist calls in parallel + tool_concurrency_limit=4, # run up to 4 specialist calls in parallel ) result = coordinator.run( diff --git a/docs-website/docs/overview/migrating-from-langgraphlangchain-to-haystack.mdx b/docs-website/docs/overview/migrating-from-langgraphlangchain-to-haystack.mdx index bfde61efe58..233201d1019 100644 --- a/docs-website/docs/overview/migrating-from-langgraphlangchain-to-haystack.mdx +++ b/docs-website/docs/overview/migrating-from-langgraphlangchain-to-haystack.mdx @@ -44,7 +44,7 @@ Here's a table of key concepts and their approximate equivalents between the two | Model Context Protocol `load_mcp_tools` `MultiServerMCPClient` | Model Context Protocol - `MCPTool`, `MCPToolset`, `StdioServerInfo`, `StreamableHttpServerInfo` | Haystack provides [various MCP primitives](https://haystack.deepset.ai/integrations/mcp) for connecting multiple MCP servers and organizing MCP toolsets. | | Memory (State, short-term, long-term) | Memory (Agent State, short-term, long-term) | Agent [State](../pipeline-components/agents-1/state.mdx) provides a structured way to share data between tools and store intermediate results during agent execution. For short-term memory, Haystack offers a [ChatMessage Store](/reference/experimental-chatmessage-store-api) to persist chat history. More memory options are coming soon. | | Time travel (Checkpoints) | Breakpoints (Breakpoint, AgentBreakpoint, ToolBreakpoint, Snapshot) | [Breakpoints](../concepts/pipelines/pipeline-breakpoints.mdx) let you pause, inspect, modify, and resume a pipeline, agent, or tool for debugging or iterative development. | -| Human-in-the-Loop (Interrupts / Commands) | Human-in-the-loop ( ConfirmationStrategy / ConfirmationPolicy) | Haystack uses [confirmation strategies](https://haystack.deepset.ai/tutorials/47_human_in_the_loop_agent) to pause or block the execution to gather user feedback | +| Human-in-the-Loop (Interrupts / Commands) | Human-in-the-loop (`ConfirmationHook` with confirmation strategies) | Haystack applies [confirmation strategies](https://haystack.deepset.ai/tutorials/47_human_in_the_loop_agent) through a `ConfirmationHook` registered under the Agent's `before_tool` [hook point](../pipeline-components/agents-1/hooks.mdx) to pause or block the execution to gather user feedback | ## Ecosystem and Tooling Mapping: LangChain → Haystack diff --git a/docs-website/docs/pipeline-components/agents-1/agent.mdx b/docs-website/docs/pipeline-components/agents-1/agent.mdx index 8024bf2d12b..53935adc754 100644 --- a/docs-website/docs/pipeline-components/agents-1/agent.mdx +++ b/docs-website/docs/pipeline-components/agents-1/agent.mdx @@ -69,8 +69,9 @@ print(response["tool_call_counts"]) # {"calculator": 1} - `streaming_callback`: A callback invoked for each streamed token. Use the built-in `print_streaming_chunk` for console output. - `max_agent_steps`: Maximum number of LLM + tool call iterations before the agent stops. Defaults to `100`. - `raise_on_tool_invocation_failure`: If `True`, raises an exception when a tool call fails. If `False` (default), the error is passed back to the LLM as a message so it can recover. -- `confirmation_strategies`: A dict mapping tool names (or tuples of tool names) to a `ConfirmationStrategy`, enabling human review of tool calls before execution. See [Human in the Loop](./human-in-the-loop.mdx). -- `tool_invoker_kwargs`: Additional keyword arguments forwarded to the internal `ToolInvoker`. +- `hooks`: A dict mapping a hook point (`"before_llm"`, `"before_tool"`, `"after_tool"`, `"on_exit"`) to a list of hooks the agent runs at that point. Hooks receive the live `State` and influence the run by mutating it — for example, to build run-time context or require human confirmation of tool calls. See [Hooks](./hooks.mdx) and [Human in the Loop](./human-in-the-loop.mdx). +- `tool_concurrency_limit`: Maximum number of tool calls to execute at the same time. Defaults to `4`; set to `1` to disable parallel tool execution. +- `tool_streaming_callback_passthrough`: If `True`, passes the streaming callback to tools that accept it. ### Runtime overrides @@ -78,6 +79,7 @@ print(response["tool_call_counts"]) # {"calculator": 1} - `tools`: Pass a list of `Tool`/`Toolset` objects, or a list of tool name strings to select a subset of the agent's configured tools for this run. - `generation_kwargs`: Additional keyword arguments forwarded to the LLM, overriding any set at init time (e.g. `{“temperature”: 0.2}`). +- `hook_context`: A dict of request-scoped resources made available to [hooks](./hooks.mdx) via `state.data["hook_context"]` — for example, a user ID or a WebSocket connection. :::info For the full parameter reference, see the [Agents API Documentation](/reference/agents-api). @@ -243,9 +245,9 @@ components: tools: null tools_strict: false type: haystack.components.generators.chat.openai.OpenAIChatGenerator - confirmation_strategies: null exit_conditions: - text + hooks: null max_agent_steps: 5 raise_on_tool_invocation_failure: false required_variables: null @@ -253,7 +255,8 @@ components: streaming_callback: null system_prompt: You are a helpful assistant. Use the web search tool to find information when needed. - tool_invoker_kwargs: null + tool_concurrency_limit: 4 + tool_streaming_callback_passthrough: false tools: - data: component: @@ -472,6 +475,7 @@ Agents work with MCP in two directions: 📖 Related docs: - [State](./state.mdx) — managing shared data between tools +- [Hooks](./hooks.mdx) — running custom logic at defined points of the run loop - [Human in the Loop](./human-in-the-loop.mdx) — intercepting tool calls for human review 📚 Tutorials: diff --git a/docs-website/docs/pipeline-components/agents-1/hooks.mdx b/docs-website/docs/pipeline-components/agents-1/hooks.mdx new file mode 100644 index 00000000000..50def82e72b --- /dev/null +++ b/docs-website/docs/pipeline-components/agents-1/hooks.mdx @@ -0,0 +1,202 @@ +--- +title: "Hooks" +id: hooks +slug: "/hooks" +description: "Hooks let you run custom logic at defined points of an Agent's run loop — before each LLM call, before and after tool execution, and on exit." +--- + +# Hooks + +Hooks let you run custom logic at defined points of an [`Agent`](./agent.mdx)'s run loop — before each LLM call, before and after tool execution, and on exit. + +
+ +| | | +| --- | --- | +| **Configured on** | The [`Agent`](./agent.mdx) component via the `hooks` parameter | +| **Key classes** | `hook` (decorator), `FunctionHook`, `Hook` (protocol) | +| **Import path** | `haystack.hooks` | +| **API reference** | [Hooks](/reference/hooks-api) | +| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/hooks/ | +| **Package name** | `haystack-ai` | + +
+ +## Overview + +Pass `hooks` to the `Agent` as a dictionary mapping a *hook point* to a list of hooks the Agent runs at that point. Each hook receives the live [`State`](./state.mdx) and influences the run by mutating it in place. Hooks for a hook point run in list order, and the same hook can be registered under multiple hook points. + +This enables patterns such as building run-time system context, retrieving memories before the first LLM call, auditing or intercepting tool calls, and requiring a condition to hold before the Agent is allowed to finish. + +### Hook points + +- `before_llm`: Runs before each chat-generator call. +- `before_tool`: Runs after the model requests tool calls, before any tools run. After these hooks run, the Agent re-reads the current last message from `state.data["messages"]`. If that message contains tool calls, those calls are executed. If it does not, no tools run for that step, no tool-based exit condition is triggered, and the Agent loops back to the next LLM call unless `max_agent_steps` has been reached. +- `after_tool`: Runs after tools execute, once their result messages are in `state.data["messages"]`, before the exit-condition check and the next LLM call. Use it to rewrite the freshly produced tool-result messages — for example, to offload, redact, truncate, or summarize results. It does not run on the plain-text exit step. It does still run when a `before_tool` hook removed the pending tool calls: no tools executed on that step, so don't assume the last message is a fresh tool result. +- `on_exit`: Runs when the Agent is about to stop on an exit condition. An `on_exit` hook can keep the Agent running by setting the `continue_run` control flag (`state.set("continue_run", True)`), usually alongside a message telling the model what to do next. `on_exit` hooks run when the Agent stops on an exit condition, but not when it stops because `max_agent_steps` is reached. + +Registering a hook under an unknown hook point raises a `ValueError` at construction. A hook class can declare an `allowed_hook_points` attribute listing the hook points it supports; the Agent validates it and fails fast if the hook is registered somewhere it doesn't belong. + +### State keys for hooks + +The Agent manages a few state keys that hooks interact with. Like the run-metadata keys (`step_count`, `token_usage`, `tool_call_counts`), they are reserved — using any of them in your own `state_schema` raises a `ValueError`. See [State](./state.mdx#schema-definition) for the full list: + +- `continue_run`: Set by an `on_exit` hook to keep the Agent running. +- `tools`: The tools available in the current step, for hooks to inspect. +- `hook_context`: Request-scoped resources passed to `Agent.run(hook_context={...})` / `run_async(hook_context={...})`. Hooks read it with `state.data["hook_context"]` or `state.data.get("hook_context")` — use it for per-request resources such as a user ID, a WebSocket, or a database client. Avoid the plain `state.get("hook_context")` here: `State.get` returns a deep copy of the value, which often fails for the kinds of resources stored in this dict (such as a WebSocket or a database client). + +Hooks can also read the automatically tracked run metadata: `step_count`, `token_usage`, and `tool_call_counts`. + +## Creating hooks + +### With the `@hook` decorator + +The `@hook` decorator wraps a function taking a single `State` argument into a hook. A regular function becomes the hook's sync path, a coroutine function its async path. To give a single hook both paths, construct a `FunctionHook` directly with both `function` and `async_function`. + +The example below registers a hook at each of `before_llm`, `before_tool`, and `on_exit` to show what hooks can do: + +```python +from datetime import datetime, timezone +from typing import Annotated + +from haystack.components.agents import Agent +from haystack.components.agents.state import State, replace_values +from haystack.components.generators.chat import OpenAIChatGenerator +from haystack.dataclasses import ChatMessage +from haystack.hooks import hook +from haystack.tools import tool + + +@tool +def search(query: Annotated[str, "The search query"]) -> str: + """Search the web.""" + # Placeholder: would call a real search API + return "Fusion startups reported net-energy-gain milestones this year." + + +@hook +def build_context(state: State) -> None: + # before_llm: build run-time system context once, before the first model call. + if state.get("step_count") == 0: + now = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC") + system = ChatMessage.from_system( + f"You are a research assistant. The current time is {now}.", + ) + state.set( + "messages", + [system, *state.data["messages"]], + handler_override=replace_values, + ) + + +@hook +def audit_tool_calls(state: State) -> None: + # before_tool: see which tools the model is about to run. + pending = state.data["messages"][-1].tool_calls + print(f"about to run: {[tc.tool_name for tc in pending]}") + + +@hook +def require_search(state: State) -> None: + # on_exit: keep going until the agent has actually searched. + if state.get("tool_call_counts", {}).get("search", 0) == 0: + state.set("messages", [ChatMessage.from_system("Search before answering.")]) + state.set("continue_run", True) + + +agent = Agent( + chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"), + tools=[search], + hooks={ + "before_llm": [build_context], + "before_tool": [audit_tool_calls], + "on_exit": [require_search], + }, +) + +result = agent.run( + messages=[ + ChatMessage.from_user("What are the latest developments in fusion energy?"), + ], +) +print(result["last_message"].text) +``` + +### Class-based hooks + +A hook is any object with a `run(state)` method; it may additionally define `run_async(state)` for true async behavior. Class-based hooks may also implement the optional lifecycle methods `warm_up` / `warm_up_async` and `close` / `close_async`. The Agent calls them from its own `warm_up` / `close`, so a hook can defer opening clients or reading credentials until warm-up and release them on close. + +When a class-based hook should be serializable (so an Agent using it can be serialized), implement `to_dict` / `from_dict`: store serializable constructor arguments on the hook and rebuild runtime clients from those values. + +The example below is an `on_exit` hook that grades the Agent's answer with its own LLM and asks the Agent to improve a weak answer before finishing: + +```python +from typing import Any + +from haystack.components.agents import Agent +from haystack.components.agents.state import State +from haystack.components.generators.chat import OpenAIChatGenerator +from haystack.core.serialization import default_from_dict, default_to_dict +from haystack.dataclasses import ChatMessage + + +class GradeFinalAnswer: + """Grade the Agent's answer with an LLM and ask it to improve a weak answer before finishing.""" + + def __init__(self, model: str = "gpt-5.4-nano"): + self.model = model + self._judge = OpenAIChatGenerator(model=self.model) + + def warm_up(self) -> None: + # Warm up the judge's own client during the Agent's warm-up. + self._judge.warm_up() + + def close(self) -> None: + # Release the judge's client during the Agent's close. + self._judge.close() + + def run(self, state: State) -> None: + answer = state.data["messages"][-1].text or "" + verdict = ( + self._judge.run( + messages=[ + ChatMessage.from_user( + f"Reply with only PASS or FAIL. Is this answer complete?\n\n{answer}", + ), + ], + )["replies"][0].text + or "" + ) + if "FAIL" in verdict.upper(): + state.set( + "messages", + [ + ChatMessage.from_user( + "Your answer was incomplete. Please improve it.", + ), + ], + ) + state.set("continue_run", True) + + def to_dict(self) -> dict[str, Any]: + return default_to_dict(self, model=self.model) + + @classmethod + def from_dict(cls, data: dict[str, Any]) -> "GradeFinalAnswer": + return default_from_dict(cls, data) + + +agent = Agent( + chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"), + hooks={"on_exit": [GradeFinalAnswer()]}, +) +result = agent.run(messages=[ChatMessage.from_user("Explain how vaccines work.")]) +print(result["last_message"].text) +``` + +## Ready-made hooks + +Haystack ships two ready-made hooks: + +- `ConfirmationHook`: A `before_tool` hook that applies Human-in-the-Loop confirmation strategies to pending tool calls — a human can confirm, modify, or reject the tool calls the model requested before they run. See [Human in the Loop](./human-in-the-loop.mdx). +- `ToolResultOffloadHook`: An `after_tool` hook that offloads tool results to a `ToolResultStore` (such as `FileSystemToolResultStore`) and replaces them in the conversation with a compact pointer, so the next LLM call sees a reference instead of the full result. Per-tool policies (`AlwaysOffload`, `NeverOffload`, `OffloadOverChars`) control which results are offloaded. For the model to retrieve an offloaded result when it needs the full content, give the Agent a tool that can read from the store — for example, a tool that reads files from disk when using `FileSystemToolResultStore`. Import it from `haystack.hooks.tool_result_offloading`, and see the [Hooks API reference](/reference/hooks-api) for a complete configuration example. diff --git a/docs-website/docs/pipeline-components/agents-1/human-in-the-loop.mdx b/docs-website/docs/pipeline-components/agents-1/human-in-the-loop.mdx index 3899fc6106f..cc8c02b36cd 100644 --- a/docs-website/docs/pipeline-components/agents-1/human-in-the-loop.mdx +++ b/docs-website/docs/pipeline-components/agents-1/human-in-the-loop.mdx @@ -15,8 +15,8 @@ This is useful for high-stakes operations - such as sending emails, modifying da | | | | --- | --- | -| **Configured on** | The [`Agent`](./agent.mdx) component via `confirmation_strategies` | -| **Key classes** | `BlockingConfirmationStrategy`, `AlwaysAskPolicy`, `AskOncePolicy`, `NeverAskPolicy`, `RichConsoleUI`, `SimpleConsoleUI` | +| **Configured on** | The [`Agent`](./agent.mdx) component, as a `ConfirmationHook` registered under the `before_tool` [hook point](./hooks.mdx) | +| **Key classes** | `ConfirmationHook`, `BlockingConfirmationStrategy`, `AlwaysAskPolicy`, `AskOncePolicy`, `NeverAskPolicy`, `RichConsoleUI`, `SimpleConsoleUI` | | **Import path** | `haystack.human_in_the_loop` | | **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/human_in_the_loop/ | | **Package name** | `haystack-ai` | @@ -25,8 +25,11 @@ This is useful for high-stakes operations - such as sending emails, modifying da ## Overview -The HITL system is composed of three layers: +HITL is one application of the Agent's general [hooks](./hooks.mdx) mechanism: a `ConfirmationHook` registered under the `before_tool` hook point intercepts the tool calls the model requested before they run, and confirms, modifies, or rejects them by rewriting the conversation in the Agent's `State`. +The HITL system is composed of these layers: + +- **`ConfirmationHook`** - the `before_tool` hook that applies your confirmation strategies to pending tool calls. Its `confirmation_strategies` mapping accepts a single tool name, a tuple of tool names, or the wildcard `"*"` that applies to any tool without a more specific entry. - **Strategy** - decides what to do when a tool is about to be called. The built-in `BlockingConfirmationStrategy` pauses execution and asks a human. - **Policy** - decides *when* to ask. Built-in policies: `AlwaysAskPolicy`, `NeverAskPolicy`, `AskOncePolicy`. - **UI** - the interface used to ask the human. Built-in UIs: `RichConsoleUI` (requires `rich`) and `SimpleConsoleUI` (stdlib only). @@ -40,6 +43,10 @@ If the policy says to ask, the UI prompts the human with the tool name, descript The agent then continues with the human's decision. +:::info +Strategies see only the arguments the model produced for a tool call. Values injected from [`State`](./state.mdx) via a tool's `inputs_from_state` mapping are not included in what is presented for confirmation — that injection happens at tool execution time. +::: + ## Usage ### Basic setup @@ -52,6 +59,7 @@ from haystack.dataclasses import ChatMessage from haystack.human_in_the_loop import ( AlwaysAskPolicy, BlockingConfirmationStrategy, + ConfirmationHook, SimpleConsoleUI, ) from haystack.tools import tool @@ -75,7 +83,11 @@ strategy = BlockingConfirmationStrategy( agent = Agent( chat_generator=OpenAIChatGenerator(model="gpt-5.4-mini"), tools=[send_email], - confirmation_strategies={"send_email": strategy}, + hooks={ + "before_tool": [ + ConfirmationHook(confirmation_strategies={"send_email": strategy}), + ], + }, ) result = agent.run( @@ -116,7 +128,7 @@ strategy = BlockingConfirmationStrategy( ### Applying strategies to multiple tools -You can configure different strategies per tool, or share one strategy across a group of tools using a tuple key: +You can configure different strategies per tool, share one strategy across a group of tools using a tuple key, or set a default for all tools with the wildcard `"*"` (applied to any tool without a more specific entry): ```python @tool @@ -145,15 +157,19 @@ ask_strategy = BlockingConfirmationStrategy( confirmation_ui=SimpleConsoleUI(), ) -agent = Agent( - chat_generator=OpenAIChatGenerator(model="gpt-5.4-mini"), - tools=[send_email, delete_record, update_record, search], +confirmation_hook = ConfirmationHook( confirmation_strategies={ # Share one strategy across multiple sensitive tools using a tuple key ("send_email", "delete_record", "update_record"): ask_strategy, # search has no strategy - always executes without asking }, ) + +agent = Agent( + chat_generator=OpenAIChatGenerator(model="gpt-5.4-mini"), + tools=[send_email, delete_record, update_record, search], + hooks={"before_tool": [confirmation_hook]}, +) ``` ### Customizing feedback messages @@ -269,7 +285,7 @@ Returned by the strategy to the agent. The [hitl-hayhooks-redis-openwebui](https://github.com/deepset-ai/hitl-hayhooks-redis-openwebui) repository shows a full production-style HITL setup using a Haystack Agent served via [Hayhooks](https://github.com/deepset-ai/hayhooks) with approval dialogs rendered in [Open WebUI](https://github.com/open-webui/open-webui). -The key pattern it demonstrates is a custom `RedisConfirmationStrategy` that uses `confirmation_strategy_context` to pass per-request resources - a Redis client and an async event queue - into the strategy at runtime: +The key pattern it demonstrates is a custom `RedisConfirmationStrategy` that receives per-request resources - a Redis client and an async event queue - at runtime. Pass such resources via the generic `hook_context` run argument (`agent.run(messages=[...], hook_context={"redis": client})`). `ConfirmationHook` reads this dict from state with `state.data["hook_context"]` (not `state.get`, which returns a deep copy that fails for live resources like clients and queues - see [Hooks](./hooks.mdx)) and passes it to each strategy's `run()` as the `confirmation_strategy_context` keyword argument, which is how a custom strategy receives the Redis client and event queue: - When a tool call is about to execute, the strategy emits a `tool_call_start` SSE event and blocks on `Redis BLPOP` waiting for an approval decision. - The Open WebUI Pipe function receives the SSE event, shows the user a confirmation dialog, then writes `approved` or `rejected` to Redis via `LPUSH`. diff --git a/docs-website/sidebars.js b/docs-website/sidebars.js index 230448083fe..43d6fffd63f 100644 --- a/docs-website/sidebars.js +++ b/docs-website/sidebars.js @@ -156,6 +156,7 @@ export default { label: 'Agents', items: [ 'pipeline-components/agents-1/agent', + 'pipeline-components/agents-1/hooks', 'pipeline-components/agents-1/human-in-the-loop', 'pipeline-components/agents-1/state', ],