diff --git a/docs-website/docs/pipeline-components/agents-1/agent.mdx b/docs-website/docs/pipeline-components/agents-1/agent.mdx index 8c6e3c1aa61..8024bf2d12b 100644 --- a/docs-website/docs/pipeline-components/agents-1/agent.mdx +++ b/docs-website/docs/pipeline-components/agents-1/agent.mdx @@ -38,16 +38,33 @@ The `Agent` returns a dictionary containing: - `messages`: the full conversation history, - `last_message`: the final `ChatMessage` from the agent, +- `step_count`: the number of steps the agent ran, +- `token_usage`: aggregated token usage summed across every LLM call in the run, +- `tool_call_counts`: how many times each tool was invoked, keyed by tool name, - Additional dynamic keys based on `state_schema`. +### Run Metadata + +The `step_count`, `token_usage`, and `tool_call_counts` outputs are populated automatically during a run. They are added to the agent's `state_schema` behind the scenes, so tools registered with `inputs_from_state` can read them mid-run. They are outputs only — they cannot be passed as inputs to `run()` or `run_async()`, and using them as keys in your own `state_schema` raises a `ValueError`. See [State](./state.mdx#schema-definition) for details. + +```python +response = agent.run(messages=[ChatMessage.from_user("What is 7 * (4 + 2)?")]) + +print(response["step_count"]) # 2 +print( + response["token_usage"], +) # {"prompt_tokens": 512, "completion_tokens": 86, ...} +print(response["tool_call_counts"]) # {"calculator": 1} +``` + ## Parameters `chat_generator` is the only mandatory parameter — an instance of a Chat Generator that supports tools. All other parameters are optional. -- `tools`: A list of tool or toolset instances the agent can call. Supported types: [`Tool`](../../tools/tool.mdx), [`ComponentTool`](../../tools/componenttool.mdx), [`PipelineTool`](../../tools/pipelinetool.mdx), [`MCPTool`](../../tools/mcptool.mdx), [`Toolset`](../../tools/toolset.mdx), [`MCPToolset`](../../tools/mcptoolset.mdx), [`SearchableToolset`](../../tools/searchabletoolset.mdx). +- `tools`: A list of tool or toolset instances the agent can call. Supported types: [`Tool`](../../tools/tool.mdx), [`ComponentTool`](../../tools/componenttool.mdx), [`PipelineTool`](../../tools/pipelinetool.mdx), [`MCPTool`](../../tools/mcptool.mdx), [`Toolset`](../../tools/toolset.mdx), [`MCPToolset`](../../tools/mcptoolset.mdx), [`SearchableToolset`](../../tools/searchabletoolset.mdx). Tool names must be unique; duplicate names are detected at the start of each agent step, before the chat generator is called. - `system_prompt`: A plain string or Jinja2 template used as the system message for every run. If the template contains Jinja2 variables, those variables become additional inputs to `run()`. - `user_prompt`: A Jinja2 template appended to the user-provided messages on each run. Template variables become additional inputs to `run()`. Use `required_variables` to enforce which variables must be provided. -- `exit_conditions`: List of conditions that cause the agent to stop. Use `”text”` to stop when the LLM replies without a tool call, or a tool name to stop once that tool has been executed. Defaults to `[“text”]`. +- `exit_conditions`: List of conditions that cause the agent to stop. Use `”text”` to stop when the LLM replies without a tool call, or a tool name to stop once that tool has been executed. Defaults to `[“text”]`. Exit conditions are evaluated at runtime rather than validated at initialization, so a condition can name a tool that is only loaded later — for example, a tool passed at runtime via `run(tools=...)` or one discovered by a [`SearchableToolset`](../../tools/searchabletoolset.mdx). - `state_schema`: Defines the agent's runtime state — a dict mapping key names to type configs (e.g. `{“docs”: {“type”: list[Document]}}`). Tools can read from and write to state keys via `inputs_from_state` and `outputs_to_state`. See [State](./state.mdx) for full details. - `streaming_callback`: A callback invoked for each streamed token. Use the built-in `print_streaming_chunk` for console output. - `max_agent_steps`: Maximum number of LLM + tool call iterations before the agent stops. Defaults to `100`. diff --git a/docs-website/docs/pipeline-components/agents-1/state.mdx b/docs-website/docs/pipeline-components/agents-1/state.mdx index 1a124636dcc..9c089f8d7a6 100644 --- a/docs-website/docs/pipeline-components/agents-1/state.mdx +++ b/docs-website/docs/pipeline-components/agents-1/state.mdx @@ -67,6 +67,15 @@ The schema defines what data can be stored and how values are updated. Each sche If you don't specify a handler, State automatically assigns a default based on the type. +:::info Reserved keys +The `Agent` manages some state keys itself and rejects them in a user-provided `state_schema` with a `ValueError`: + +- The run-metadata keys `step_count`, `token_usage`, and `tool_call_counts`, which the Agent populates automatically during a run: tools can read them mid-run via `inputs_from_state`, and they are returned in the result dictionary. +- The hook-facing keys `continue_run` (set by an `on_exit` hook to keep the Agent running), `tools` (the tools available in the current step, for hooks to inspect), and `hook_context` (the request-scoped resources passed to `Agent.run(hook_context={...})`). + +If one of your state keys clashes, rename it (for example, `my_token_usage`). +::: + ### Default Handlers State provides two built-in merge behaviors (importable from `haystack.components.agents.state`): diff --git a/docs-website/docs/pipeline-components/generators.mdx b/docs-website/docs/pipeline-components/generators.mdx index f18a6503a00..a5cfc6b3f44 100644 --- a/docs-website/docs/pipeline-components/generators.mdx +++ b/docs-website/docs/pipeline-components/generators.mdx @@ -35,6 +35,7 @@ Generators are responsible for generating text after you give them a prompt. The | [LlamaStackChatGenerator](generators/llamastackchatgenerator.mdx) | Enables chat completions using an LLM model made available via Llama Stack server | ✅ | | [MetaLlamaChatGenerator](generators/metallamachatgenerator.mdx) | Enables chat completion with any model hosted available with Meta Llama API. | ✅ | | [MistralChatGenerator](generators/mistralchatgenerator.mdx) | Enables chat completion using Mistral's text generation models. | ✅ | +| [MockChatGenerator](generators/mockchatgenerator.mdx) | Returns predefined responses without calling any API — a deterministic, zero-cost stand-in for real Chat Generators in tests and prototypes. | ✅ | | [NvidiaChatGenerator](generators/nvidiachatgenerator.mdx) | Enables chat completion using Nvidia-hosted models. | ✅ | | [NvidiaGenerator](generators/nvidiagenerator.mdx) | Provides an interface for generating text using LLMs self-hosted with NVIDIA NIM or models hosted on the NVIDIA API catalog. | ❌ | | [OllamaChatGenerator](generators/ollamachatgenerator.mdx) | Enables chat completion using an LLM running on Ollama. | ✅ | diff --git a/docs-website/docs/pipeline-components/generators/mockchatgenerator.mdx b/docs-website/docs/pipeline-components/generators/mockchatgenerator.mdx new file mode 100644 index 00000000000..123744a90bc --- /dev/null +++ b/docs-website/docs/pipeline-components/generators/mockchatgenerator.mdx @@ -0,0 +1,109 @@ +--- +title: "MockChatGenerator" +id: mockchatgenerator +slug: "/mockchatgenerator" +description: "A Chat Generator that returns predefined responses without calling any API, for tests and quick prototypes." +--- + +# MockChatGenerator + +A Chat Generator that returns predefined responses without calling any API, for tests and quick prototypes. + +
+ +| | | +| --- | --- | +| **Most common position in a pipeline** | In place of a real Chat Generator, in tests and prototypes | +| **Mandatory init variables** | None | +| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects | +| **Output variables** | `replies`: A list of generated `ChatMessage` objects | +| **API reference** | [Generators](/reference/generators-api) | +| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/components/generators/chat/mock.py | +| **Package name** | `haystack-ai` | + +
+ +## Overview + +`MockChatGenerator` is a deterministic, zero-cost drop-in replacement for real Chat Generators such as `OpenAIChatGenerator`. It implements `run`, `run_async`, streaming callbacks, and serialization but never contacts an external service, which makes it ideal for unit tests, smoke tests, and quick prototypes. + +The response is selected based on how the component is configured: + +- **Fixed response**: Pass a single string or `ChatMessage` via `responses`. The same reply is returned on every call. A `ChatMessage` passed as a response must have the `assistant` role. +- **Cycling responses**: Pass a list of strings and/or `ChatMessage` objects via `responses`. Each call returns the next item, wrapping around to the start once the list is exhausted. This is useful to drive multi-step flows such as Agents, where the first call returns a tool call and a later call returns the final answer. +- **Dynamic response**: Pass a `response_fn` callable that receives the input messages and returns the reply as a string or an assistant `ChatMessage`. Use this when the reply should depend on the input. To support serialization, pass a named function. +- **Echo (default)**: With no configuration, the component echoes back the text of the last message that has text content, so it is usable out of the box. + +`responses` and `response_fn` are mutually exclusive. + +Further optional parameters: + +- `model`: The model name reported in the response metadata. Defaults to `"mock-model"`. +- `meta`: Additional metadata merged into the `meta` of every returned `ChatMessage`. A per-response `ChatMessage`'s own metadata takes precedence. +- `streaming_callback`: An optional callback invoked with `StreamingChunk` objects reconstructed from the predefined response. It lets the mock exercise streaming code paths without a real model. + +## Usage + +### On its own + +```python +from haystack.components.generators.chat import MockChatGenerator +from haystack.dataclasses import ChatMessage + +# Fixed response +generator = MockChatGenerator(responses="Hello, this is a mock response.") +result = generator.run([ChatMessage.from_user("Hi!")]) +print(result["replies"][0].text) # "Hello, this is a mock response." + +# Echo mode (default): returns the last message with text content +generator = MockChatGenerator() +result = generator.run([ChatMessage.from_user("Repeat after me")]) +print(result["replies"][0].text) # "Repeat after me" +``` + +### Driving an Agent + +Pass `ChatMessage` objects (rather than plain strings) to return tool calls or reasoning content. With cycling responses, you can script a full agent loop without a real model: + +```python +from haystack.components.agents import Agent +from haystack.components.generators.chat import MockChatGenerator +from haystack.dataclasses import ChatMessage, ToolCall +from haystack.tools import tool + + +@tool +def search(query: str) -> str: + """Search for information.""" + return f"Results for: {query}" + + +generator = MockChatGenerator( + responses=[ + ChatMessage.from_assistant( + tool_calls=[ToolCall(tool_name="search", arguments={"query": "Haystack"})], + ), + "Here is the final answer.", + ], +) + +agent = Agent(chat_generator=generator, tools=[search]) +result = agent.run(messages=[ChatMessage.from_user("Tell me about Haystack")]) +print(result["last_message"].text) # "Here is the final answer." +``` + +### Input-dependent responses + +```python +from haystack.components.generators.chat import MockChatGenerator +from haystack.dataclasses import ChatMessage + + +def shout_back(messages: list[ChatMessage]) -> str: + return messages[-1].text.upper() + + +generator = MockChatGenerator(response_fn=shout_back) +result = generator.run([ChatMessage.from_user("hello")]) +print(result["replies"][0].text) # "HELLO" +``` diff --git a/docs-website/docs/tools/searchabletoolset.mdx b/docs-website/docs/tools/searchabletoolset.mdx index 3cfa6a32ae2..b4e1ef3465a 100644 --- a/docs-website/docs/tools/searchabletoolset.mdx +++ b/docs-website/docs/tools/searchabletoolset.mdx @@ -46,6 +46,14 @@ subsequent iterations. `SearchableToolset` does not support adding new tools after initialization or merging with other toolsets. Use `catalog` to provide all tools upfront. ::: +### Warm-up + +`SearchableToolset` builds its search index during `warm_up()`. When used with an [`Agent`](../pipeline-components/agents-1/agent.mdx), constructing the Agent does not trigger this — warm-up happens when you call `Agent.warm_up()` or automatically at run time. + +All tool names in the catalog must be unique: `warm_up()` raises a `ValueError` if the catalog contains tools with duplicate names, since a search hit could otherwise resolve to the wrong tool. + +The Agent evaluates its `exit_conditions` at runtime, so an exit condition can name any tool in the catalog, even one the agent has not discovered yet. + ## Usage ### Basic usage with an Agent diff --git a/docs-website/sidebars.js b/docs-website/sidebars.js index c137c1d80f5..e4d7c0abe9f 100644 --- a/docs-website/sidebars.js +++ b/docs-website/sidebars.js @@ -433,6 +433,7 @@ export default { 'pipeline-components/generators/llamastackchatgenerator', 'pipeline-components/generators/metallamachatgenerator', 'pipeline-components/generators/mistralchatgenerator', + 'pipeline-components/generators/mockchatgenerator', 'pipeline-components/generators/nvidiachatgenerator', 'pipeline-components/generators/nvidiagenerator', 'pipeline-components/generators/ollamachatgenerator',