Skip to content

Commit c13cc91

Browse files
julian-rischclaude
andauthored
docs: add Agent Hooks page and recast Human-in-the-Loop as a before_tool hook (#11878)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent e69572a commit c13cc91

6 files changed

Lines changed: 245 additions & 20 deletions

File tree

docs-website/docs/concepts/agents/multi-agent-systems.mdx

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -147,17 +147,18 @@ components:
147147
tools: null
148148
tools_strict: false
149149
type: haystack.components.generators.chat.openai.OpenAIChatGenerator
150-
confirmation_strategies: null
151150
exit_conditions:
152151
- text
152+
hooks: null
153153
max_agent_steps: 100
154154
raise_on_tool_invocation_failure: false
155155
required_variables: null
156156
state_schema: {}
157157
streaming_callback: null
158158
system_prompt: You are a coordinator. Delegate research tasks to the research
159159
specialist. Keep your final answer concise.
160-
tool_invoker_kwargs: null
160+
tool_concurrency_limit: 4
161+
tool_streaming_callback_passthrough: false
161162
tools:
162163
- data:
163164
component:
@@ -180,17 +181,18 @@ components:
180181
tools: null
181182
tools_strict: false
182183
type: haystack.components.generators.chat.openai.OpenAIChatGenerator
183-
confirmation_strategies: null
184184
exit_conditions:
185185
- text
186+
hooks: null
186187
max_agent_steps: 100
187188
raise_on_tool_invocation_failure: false
188189
required_variables: null
189190
state_schema: {}
190191
streaming_callback: null
191192
system_prompt: You are a research specialist. Search the web to find
192193
information. Return a concise summary of your findings in 3-5 sentences.
193-
tool_invoker_kwargs: null
194+
tool_concurrency_limit: 4
195+
tool_streaming_callback_passthrough: false
194196
tools:
195197
- data:
196198
component:
@@ -242,7 +244,7 @@ Returning just `result["last_message"].text` (with `@tool`) or using `outputs_to
242244

243245
When covering multiple topics, the coordinator can call the same specialist tool several times in a single response.
244246
All tool calls from one LLM response are executed concurrently using a thread pool.
245-
Control the level of parallelism with `max_workers` in `tool_invoker_kwargs` (default: `4`).
247+
Control the level of parallelism with the `tool_concurrency_limit` init parameter (default: `4`).
246248

247249
The example below asks the coordinator about two topics: it calls `research` twice and both specialists run in parallel.
248250

@@ -319,7 +321,7 @@ coordinator = Agent(
319321
"Keep your final answer concise."
320322
),
321323
streaming_callback=print_streaming_chunk,
322-
tool_invoker_kwargs={"max_workers": 4}, # run up to 4 specialist calls in parallel
324+
tool_concurrency_limit=4, # run up to 4 specialist calls in parallel
323325
)
324326
325327
result = coordinator.run(

docs-website/docs/overview/migrating-from-langgraphlangchain-to-haystack.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ Here's a table of key concepts and their approximate equivalents between the two
4444
| Model Context Protocol `load_mcp_tools` `MultiServerMCPClient` | Model Context Protocol - `MCPTool`, `MCPToolset`, `StdioServerInfo`, `StreamableHttpServerInfo` | Haystack provides [various MCP primitives](https://haystack.deepset.ai/integrations/mcp) for connecting multiple MCP servers and organizing MCP toolsets. |
4545
| Memory (State, short-term, long-term) | Memory (Agent State, short-term, long-term) | Agent [State](../pipeline-components/agents-1/state.mdx) provides a structured way to share data between tools and store intermediate results during agent execution. For short-term memory, Haystack offers a [ChatMessage Store](/reference/experimental-chatmessage-store-api) to persist chat history. More memory options are coming soon. |
4646
| Time travel (Checkpoints) | Breakpoints (Breakpoint, AgentBreakpoint, ToolBreakpoint, Snapshot) | [Breakpoints](../concepts/pipelines/pipeline-breakpoints.mdx) let you pause, inspect, modify, and resume a pipeline, agent, or tool for debugging or iterative development. |
47-
| Human-in-the-Loop (Interrupts / Commands) | Human-in-the-loop ( ConfirmationStrategy / ConfirmationPolicy) | Haystack uses [confirmation strategies](https://haystack.deepset.ai/tutorials/47_human_in_the_loop_agent) to pause or block the execution to gather user feedback |
47+
| Human-in-the-Loop (Interrupts / Commands) | Human-in-the-loop (`ConfirmationHook` with confirmation strategies) | Haystack applies [confirmation strategies](https://haystack.deepset.ai/tutorials/47_human_in_the_loop_agent) through a `ConfirmationHook` registered under the Agent's `before_tool` [hook point](../pipeline-components/agents-1/hooks.mdx) to pause or block the execution to gather user feedback |
4848

4949
## Ecosystem and Tooling Mapping: LangChain → Haystack
5050

docs-website/docs/pipeline-components/agents-1/agent.mdx

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -69,15 +69,17 @@ print(response["tool_call_counts"]) # {"calculator": 1}
6969
- `streaming_callback`: A callback invoked for each streamed token. Use the built-in `print_streaming_chunk` for console output.
7070
- `max_agent_steps`: Maximum number of LLM + tool call iterations before the agent stops. Defaults to `100`.
7171
- `raise_on_tool_invocation_failure`: If `True`, raises an exception when a tool call fails. If `False` (default), the error is passed back to the LLM as a message so it can recover.
72-
- `confirmation_strategies`: A dict mapping tool names (or tuples of tool names) to a `ConfirmationStrategy`, enabling human review of tool calls before execution. See [Human in the Loop](./human-in-the-loop.mdx).
73-
- `tool_invoker_kwargs`: Additional keyword arguments forwarded to the internal `ToolInvoker`.
72+
- `hooks`: A dict mapping a hook point (`"before_llm"`, `"before_tool"`, `"after_tool"`, `"on_exit"`) to a list of hooks the agent runs at that point. Hooks receive the live `State` and influence the run by mutating it — for example, to build run-time context or require human confirmation of tool calls. See [Hooks](./hooks.mdx) and [Human in the Loop](./human-in-the-loop.mdx).
73+
- `tool_concurrency_limit`: Maximum number of tool calls to execute at the same time. Defaults to `4`; set to `1` to disable parallel tool execution.
74+
- `tool_streaming_callback_passthrough`: If `True`, passes the streaming callback to tools that accept it.
7475

7576
### Runtime overrides
7677

7778
`run()` also accepts parameters that override the init-time configuration for a single call:
7879

7980
- `tools`: Pass a list of `Tool`/`Toolset` objects, or a list of tool name strings to select a subset of the agent's configured tools for this run.
8081
- `generation_kwargs`: Additional keyword arguments forwarded to the LLM, overriding any set at init time (e.g. `{“temperature”: 0.2}`).
82+
- `hook_context`: A dict of request-scoped resources made available to [hooks](./hooks.mdx) via `state.data["hook_context"]` — for example, a user ID or a WebSocket connection.
8183

8284
:::info
8385
For the full parameter reference, see the [Agents API Documentation](/reference/agents-api).
@@ -243,17 +245,18 @@ components:
243245
tools: null
244246
tools_strict: false
245247
type: haystack.components.generators.chat.openai.OpenAIChatGenerator
246-
confirmation_strategies: null
247248
exit_conditions:
248249
- text
250+
hooks: null
249251
max_agent_steps: 5
250252
raise_on_tool_invocation_failure: false
251253
required_variables: null
252254
state_schema: {}
253255
streaming_callback: null
254256
system_prompt: You are a helpful assistant. Use the web search tool to find
255257
information when needed.
256-
tool_invoker_kwargs: null
258+
tool_concurrency_limit: 4
259+
tool_streaming_callback_passthrough: false
257260
tools:
258261
- data:
259262
component:
@@ -472,6 +475,7 @@ Agents work with MCP in two directions:
472475
📖 Related docs:
473476

474477
- [State](./state.mdx) — managing shared data between tools
478+
- [Hooks](./hooks.mdx) — running custom logic at defined points of the run loop
475479
- [Human in the Loop](./human-in-the-loop.mdx) — intercepting tool calls for human review
476480

477481
📚 Tutorials:
Lines changed: 202 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,202 @@
1+
---
2+
title: "Hooks"
3+
id: hooks
4+
slug: "/hooks"
5+
description: "Hooks let you run custom logic at defined points of an Agent's run loop — before each LLM call, before and after tool execution, and on exit."
6+
---
7+
8+
# Hooks
9+
10+
Hooks let you run custom logic at defined points of an [`Agent`](./agent.mdx)'s run loop — before each LLM call, before and after tool execution, and on exit.
11+
12+
<div className="key-value-table">
13+
14+
| | |
15+
| --- | --- |
16+
| **Configured on** | The [`Agent`](./agent.mdx) component via the `hooks` parameter |
17+
| **Key classes** | `hook` (decorator), `FunctionHook`, `Hook` (protocol) |
18+
| **Import path** | `haystack.hooks` |
19+
| **API reference** | [Hooks](/reference/hooks-api) |
20+
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/hooks/ |
21+
| **Package name** | `haystack-ai` |
22+
23+
</div>
24+
25+
## Overview
26+
27+
Pass `hooks` to the `Agent` as a dictionary mapping a *hook point* to a list of hooks the Agent runs at that point. Each hook receives the live [`State`](./state.mdx) and influences the run by mutating it in place. Hooks for a hook point run in list order, and the same hook can be registered under multiple hook points.
28+
29+
This enables patterns such as building run-time system context, retrieving memories before the first LLM call, auditing or intercepting tool calls, and requiring a condition to hold before the Agent is allowed to finish.
30+
31+
### Hook points
32+
33+
- `before_llm`: Runs before each chat-generator call.
34+
- `before_tool`: Runs after the model requests tool calls, before any tools run. After these hooks run, the Agent re-reads the current last message from `state.data["messages"]`. If that message contains tool calls, those calls are executed. If it does not, no tools run for that step, no tool-based exit condition is triggered, and the Agent loops back to the next LLM call unless `max_agent_steps` has been reached.
35+
- `after_tool`: Runs after tools execute, once their result messages are in `state.data["messages"]`, before the exit-condition check and the next LLM call. Use it to rewrite the freshly produced tool-result messages — for example, to offload, redact, truncate, or summarize results. It does not run on the plain-text exit step. It does still run when a `before_tool` hook removed the pending tool calls: no tools executed on that step, so don't assume the last message is a fresh tool result.
36+
- `on_exit`: Runs when the Agent is about to stop on an exit condition. An `on_exit` hook can keep the Agent running by setting the `continue_run` control flag (`state.set("continue_run", True)`), usually alongside a message telling the model what to do next. `on_exit` hooks run when the Agent stops on an exit condition, but not when it stops because `max_agent_steps` is reached.
37+
38+
Registering a hook under an unknown hook point raises a `ValueError` at construction. A hook class can declare an `allowed_hook_points` attribute listing the hook points it supports; the Agent validates it and fails fast if the hook is registered somewhere it doesn't belong.
39+
40+
### State keys for hooks
41+
42+
The Agent manages a few state keys that hooks interact with. Like the run-metadata keys (`step_count`, `token_usage`, `tool_call_counts`), they are reserved — using any of them in your own `state_schema` raises a `ValueError`. See [State](./state.mdx#schema-definition) for the full list:
43+
44+
- `continue_run`: Set by an `on_exit` hook to keep the Agent running.
45+
- `tools`: The tools available in the current step, for hooks to inspect.
46+
- `hook_context`: Request-scoped resources passed to `Agent.run(hook_context={...})` / `run_async(hook_context={...})`. Hooks read it with `state.data["hook_context"]` or `state.data.get("hook_context")` — use it for per-request resources such as a user ID, a WebSocket, or a database client. Avoid the plain `state.get("hook_context")` here: `State.get` returns a deep copy of the value, which often fails for the kinds of resources stored in this dict (such as a WebSocket or a database client).
47+
48+
Hooks can also read the automatically tracked run metadata: `step_count`, `token_usage`, and `tool_call_counts`.
49+
50+
## Creating hooks
51+
52+
### With the `@hook` decorator
53+
54+
The `@hook` decorator wraps a function taking a single `State` argument into a hook. A regular function becomes the hook's sync path, a coroutine function its async path. To give a single hook both paths, construct a `FunctionHook` directly with both `function` and `async_function`.
55+
56+
The example below registers a hook at each of `before_llm`, `before_tool`, and `on_exit` to show what hooks can do:
57+
58+
```python
59+
from datetime import datetime, timezone
60+
from typing import Annotated
61+
62+
from haystack.components.agents import Agent
63+
from haystack.components.agents.state import State, replace_values
64+
from haystack.components.generators.chat import OpenAIChatGenerator
65+
from haystack.dataclasses import ChatMessage
66+
from haystack.hooks import hook
67+
from haystack.tools import tool
68+
69+
70+
@tool
71+
def search(query: Annotated[str, "The search query"]) -> str:
72+
"""Search the web."""
73+
# Placeholder: would call a real search API
74+
return "Fusion startups reported net-energy-gain milestones this year."
75+
76+
77+
@hook
78+
def build_context(state: State) -> None:
79+
# before_llm: build run-time system context once, before the first model call.
80+
if state.get("step_count") == 0:
81+
now = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
82+
system = ChatMessage.from_system(
83+
f"You are a research assistant. The current time is {now}.",
84+
)
85+
state.set(
86+
"messages",
87+
[system, *state.data["messages"]],
88+
handler_override=replace_values,
89+
)
90+
91+
92+
@hook
93+
def audit_tool_calls(state: State) -> None:
94+
# before_tool: see which tools the model is about to run.
95+
pending = state.data["messages"][-1].tool_calls
96+
print(f"about to run: {[tc.tool_name for tc in pending]}")
97+
98+
99+
@hook
100+
def require_search(state: State) -> None:
101+
# on_exit: keep going until the agent has actually searched.
102+
if state.get("tool_call_counts", {}).get("search", 0) == 0:
103+
state.set("messages", [ChatMessage.from_system("Search before answering.")])
104+
state.set("continue_run", True)
105+
106+
107+
agent = Agent(
108+
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
109+
tools=[search],
110+
hooks={
111+
"before_llm": [build_context],
112+
"before_tool": [audit_tool_calls],
113+
"on_exit": [require_search],
114+
},
115+
)
116+
117+
result = agent.run(
118+
messages=[
119+
ChatMessage.from_user("What are the latest developments in fusion energy?"),
120+
],
121+
)
122+
print(result["last_message"].text)
123+
```
124+
125+
### Class-based hooks
126+
127+
A hook is any object with a `run(state)` method; it may additionally define `run_async(state)` for true async behavior. Class-based hooks may also implement the optional lifecycle methods `warm_up` / `warm_up_async` and `close` / `close_async`. The Agent calls them from its own `warm_up` / `close`, so a hook can defer opening clients or reading credentials until warm-up and release them on close.
128+
129+
When a class-based hook should be serializable (so an Agent using it can be serialized), implement `to_dict` / `from_dict`: store serializable constructor arguments on the hook and rebuild runtime clients from those values.
130+
131+
The example below is an `on_exit` hook that grades the Agent's answer with its own LLM and asks the Agent to improve a weak answer before finishing:
132+
133+
```python
134+
from typing import Any
135+
136+
from haystack.components.agents import Agent
137+
from haystack.components.agents.state import State
138+
from haystack.components.generators.chat import OpenAIChatGenerator
139+
from haystack.core.serialization import default_from_dict, default_to_dict
140+
from haystack.dataclasses import ChatMessage
141+
142+
143+
class GradeFinalAnswer:
144+
"""Grade the Agent's answer with an LLM and ask it to improve a weak answer before finishing."""
145+
146+
def __init__(self, model: str = "gpt-5.4-nano"):
147+
self.model = model
148+
self._judge = OpenAIChatGenerator(model=self.model)
149+
150+
def warm_up(self) -> None:
151+
# Warm up the judge's own client during the Agent's warm-up.
152+
self._judge.warm_up()
153+
154+
def close(self) -> None:
155+
# Release the judge's client during the Agent's close.
156+
self._judge.close()
157+
158+
def run(self, state: State) -> None:
159+
answer = state.data["messages"][-1].text or ""
160+
verdict = (
161+
self._judge.run(
162+
messages=[
163+
ChatMessage.from_user(
164+
f"Reply with only PASS or FAIL. Is this answer complete?\n\n{answer}",
165+
),
166+
],
167+
)["replies"][0].text
168+
or ""
169+
)
170+
if "FAIL" in verdict.upper():
171+
state.set(
172+
"messages",
173+
[
174+
ChatMessage.from_user(
175+
"Your answer was incomplete. Please improve it.",
176+
),
177+
],
178+
)
179+
state.set("continue_run", True)
180+
181+
def to_dict(self) -> dict[str, Any]:
182+
return default_to_dict(self, model=self.model)
183+
184+
@classmethod
185+
def from_dict(cls, data: dict[str, Any]) -> "GradeFinalAnswer":
186+
return default_from_dict(cls, data)
187+
188+
189+
agent = Agent(
190+
chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
191+
hooks={"on_exit": [GradeFinalAnswer()]},
192+
)
193+
result = agent.run(messages=[ChatMessage.from_user("Explain how vaccines work.")])
194+
print(result["last_message"].text)
195+
```
196+
197+
## Ready-made hooks
198+
199+
Haystack ships two ready-made hooks:
200+
201+
- `ConfirmationHook`: A `before_tool` hook that applies Human-in-the-Loop confirmation strategies to pending tool calls — a human can confirm, modify, or reject the tool calls the model requested before they run. See [Human in the Loop](./human-in-the-loop.mdx).
202+
- `ToolResultOffloadHook`: An `after_tool` hook that offloads tool results to a `ToolResultStore` (such as `FileSystemToolResultStore`) and replaces them in the conversation with a compact pointer, so the next LLM call sees a reference instead of the full result. Per-tool policies (`AlwaysOffload`, `NeverOffload`, `OffloadOverChars`) control which results are offloaded. For the model to retrieve an offloaded result when it needs the full content, give the Agent a tool that can read from the store — for example, a tool that reads files from disk when using `FileSystemToolResultStore`. Import it from `haystack.hooks.tool_result_offloading`, and see the [Hooks API reference](/reference/hooks-api) for a complete configuration example.

0 commit comments

Comments
 (0)