Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 44 additions & 1 deletion docs-website/docs/concepts/pipelines/serialization.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ with open("/content/test.yml", "w") as file:

You can convert a YAML pipeline back into Python. Use the `loads()` method to convert a string representation of a pipeline (`str`, `bytes` or `bytearray`) or the `load()` method to convert a pipeline represented in a file-like object into a corresponding Python object.

Both loading methods support callbacks that let you modify components during the deserialization process. Therefore, loading a serialized pipeline or component assumes that the serialized definition originates from a trusted source and has been reviewed by the user.
Both loading methods support callbacks that let you modify components during the deserialization process. Deserialization is gated by a trusted-module allowlist, so pipelines referencing classes outside of it fail to load until you extend the allowlist — see [Deserialization Security](#deserialization-security) below.

Here is an example script:

Expand Down Expand Up @@ -96,6 +96,49 @@ pipe = Pipeline.loads(
)
```

## Deserialization Security

Loading a pipeline instantiates the classes referenced in the serialized data. To prevent a crafted YAML file from importing and instantiating arbitrary classes, `Pipeline.load`, `Pipeline.loads`, and `Pipeline.from_dict` refuse to import classes from modules outside a trusted-module allowlist and raise a `DeserializationError` instead.

By default, the allowlist contains `haystack`, `haystack_integrations`, `haystack_experimental`, `builtins`, `typing`, and `collections`. Dangerous builtins such as `eval`, `exec`, `compile`, `__import__`, `open`, and `getattr` are blocked even though `builtins` is allowlisted.

### Allowing Custom Modules

Pipelines that reference custom components or callables in other packages fail to load until you add the modules to the allowlist. You can extend it in three ways:

```python
from haystack import Pipeline

# 1. Per call: pass additional module patterns for this deserialization only
pipe = Pipeline.load(open("pipeline.yaml"), allowed_modules=["mypkg.*"])

# 2. Process-wide: extend the allowlist programmatically
from haystack.core.serialization import allow_deserialization_module

allow_deserialization_module("mypkg")
```

```shell
# 3. Environment variable with comma-separated patterns, read on every deserialization call
export HAYSTACK_DESERIALIZATION_ALLOWLIST="mypkg.*,otherpkg.*"
```

Patterns are matched as prefixes by default (`"mypkg"` matches `mypkg` and any of its submodules), or as `fnmatch` globs if they contain `*`, `?`, or `[` somewhere other than a trailing `.*`. A trailing `.*` is treated as a prefix match, so `"mypkg"` and `"mypkg.*"` behave identically.

If the source of the serialized data is fully trusted, you can bypass the allowlist entirely with `unsafe=True`:

```python
pipe = Pipeline.load(open("pipeline.yaml"), unsafe=True)
```

Only use `unsafe=True` when you fully trust where the serialized pipeline comes from — it also lifts the block on dangerous builtins.

### Nested Init Parameter Validation

As an additional safeguard, deserialization validates the keys of `init_parameters` against the class's `__init__` signature before recursing into any nested `{"type": "...", "init_parameters": {...}}` dictionary. A nested dictionary whose key is not an accepted parameter name is rejected with a `DeserializationError` *before* the nested type is imported, which blocks attempts to smuggle untrusted classes into unused parameter slots. Classes whose constructor takes `**kwargs` are exempt, since their accepted parameter set cannot be statically determined.

This validation may surface pre-existing bugs in YAML files — for example typos, leftovers from renamed or removed parameters, or stale snapshots from older Haystack versions. The fix is to update the YAML so each nested-component key matches a real `__init__` parameter of the parent class.

## Default Serialization Behavior

The serialization system uses `default_to_dict` and `default_from_dict` to handle many object types automatically. You typically do **not** need to implement custom `to_dict`/`from_dict` for:
Expand Down
2 changes: 2 additions & 0 deletions docs-website/docs/tools/componenttool.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ It does input type conversion and offers support for components with run methods
- Lists of dataclasses (such as List[Document])
- Parameters with mixed types (such as List[Document], str...)

If the wrapped component defines a `run_async` method, `ComponentTool` automatically wires an async invoker as well, so the tool supports async invocation (for example, from `Agent.run_async`) without extra configuration. See [Async Tools](tool.mdx#async-tools) for details.

### Parameters

- `component` is mandatory and must be a Haystack component instance, either an existing one or a custom component.
Expand Down
2 changes: 2 additions & 0 deletions docs-website/docs/tools/pipelinetool.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ It replaces the older workflow of first wrapping a pipeline in a `SuperComponent
`PipelineTool` builds the tool parameter schema from the pipeline’s input sockets and uses the underlying components’ docstrings for input descriptions. You can choose which pipeline inputs and outputs to expose with
`input_mapping` and `output_mapping`. It can be used in a pipeline with `ToolInvoker` or directly with the `Agent` component.

`PipelineTool` also supports async invocation: since every `Pipeline` exposes a native `run_async`, the tool can be awaited (for example, from `Agent.run_async`) without extra configuration. See [Async Tools](tool.mdx#async-tools) for details.

### Parameters

- `pipeline` is mandatory and must be a `Pipeline` instance.
Expand Down
119 changes: 119 additions & 0 deletions docs-website/docs/tools/skilltoolset.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
---
title: "SkillToolset"
id: skilltoolset
slug: "/skilltoolset"
description: "Let agents discover and read skills — reusable instruction sets with bundled files — through progressive disclosure."
---

# SkillToolset

Let agents discover and read skills — reusable instruction sets with bundled files — through progressive disclosure.

<div className="key-value-table">

| | |
| --- | --- |
| **Mandatory init variables** | `store`: A `SkillStore` instance that provides the skills |
| **API reference** | [SkillToolset](/reference/tools-api#skilltoolset) |
| **GitHub link** | https://github.com/deepset-ai/haystack/blob/main/haystack/tools/skills/skill_toolset.py |
| **Package name** | `haystack-ai` |

</div>

## Overview

A *skill* is a directory (or equivalent storage unit) containing a `SKILL.md` file with YAML frontmatter (`description` is required; `name` is optional and defaults to the directory name) and a markdown body of instructions. Skills may bundle additional files, such as reference docs, examples, or templates.

`SkillToolset` lets an [`Agent`](../pipeline-components/agents-1/agent.mdx) use skills through *progressive disclosure*, similar to how coding assistants like Claude Code expose skills: the model first sees only each skill's name and description, loads the full instructions when a task calls for them, and fetches bundled files only when the instructions reference them. This keeps the context small even with many detailed skills.

The toolset exposes two tools:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be worth adding a note how users can enable executing scripts bundled with tools. E.g. Include their own custom bash tool or script execution tool alongside the SkillsToolset.

Otherwise by default SkillsToolset only supports reading skill files and not executing anything.


- `load_skill`: Returns a skill's full instructions on demand, plus a manifest of its bundled files. The names and descriptions of all discovered skills are baked into this tool's description at warm-up, so the model can see which skills exist without any system prompt injection.
- `read_skill_file`: Reads a file bundled with a skill (with path-traversal protection).

Skills are discovered when the toolset is warmed up — the `Agent` does this automatically before a run. Constructing the toolset does not read any skills.

`SkillToolset` is backed by a `SkillStore`. Use the built-in `FileSystemSkillStore` to load skills from a local directory, or implement the `SkillStore` protocol (`list_skills`, `load_skill`, `read_skill_file`, plus serialization methods) to back the toolset with any storage system — a database, a remote API, and so on.

:::info
The tool names `load_skill` and `read_skill_file` are fixed, so an `Agent` can use at most one `SkillToolset`. It also does not support adding tools or concatenation with other toolsets — to combine it with other tools, pass it to the `Agent` alongside them, for example `tools=[skills_toolset, other_tool]`. To serve skills from multiple sources, back a single toolset with a custom store that merges them.
:::

### Skill format

`FileSystemSkillStore` expects one sub-directory per skill under a root directory:

```
skills/
pdf-forms/
SKILL.md # frontmatter (description required, name optional) + markdown instructions
reference/forms.md # optional bundled file
```

A minimal `SKILL.md` looks like this:

```markdown
---
name: pdf-forms
description: Fill in PDF forms programmatically. Use when the user asks to complete or fill a PDF form.
---

# Filling PDF forms

1. Inspect the form fields first...
2. For the full field reference, read `reference/forms.md`.
```

Only the frontmatter of each `SKILL.md` is read at warm-up to build the catalog; instruction bodies and bundled files are read lazily when the agent calls the corresponding tool.

### Multimodal skill assets

`read_skill_file` returns text files as strings, images as [`ImageContent`](../concepts/data-classes/imagecontent.mdx), and PDFs as [`FileContent`](../concepts/data-classes/filecontent.mdx). Image and file results are passed to the model as content parts of the tool result instead of being converted to a string, so an `Agent` backed by a multimodal chat generator that supports these inputs (for example, `OpenAIResponsesChatGenerator`) can read a skill's visual assets — such as a reference screenshot or a showcase PDF — directly. Binary files that are neither images nor PDFs are rejected with an error.

### Executing bundled scripts

`SkillToolset` only *reads* skills — `load_skill` and `read_skill_file` never execute anything. If your skills bundle executable scripts (for example, a Python helper that the instructions tell the model to run), pass a script-execution tool of your own to the `Agent` alongside the toolset:

```python
agent = Agent(
chat_generator=OpenAIChatGenerator(),
tools=[skills_toolset, run_shell_command_tool], # your own execution tool
)
```

The agent can then read a bundled script with `read_skill_file` and run it through your execution tool. Since such a tool runs model-chosen commands, scope it carefully — restrict what it can execute, sandbox it, or guard it with a [Human in the Loop](../pipeline-components/agents-1/human-in-the-loop.mdx) confirmation strategy.

## Usage

### With an Agent

```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.skill_stores.file_system import FileSystemSkillStore
from haystack.tools import SkillToolset

store = FileSystemSkillStore("skills/")
skills_toolset = SkillToolset(store)

agent = Agent(chat_generator=OpenAIChatGenerator(), tools=skills_toolset)

# The agent sees the available skills in the `load_skill` tool description,
# loads the matching skill, and follows its instructions.
result = agent.run(messages=[ChatMessage.from_user("Fill in this PDF form for me.")])
print(result["last_message"].text)
```

### Inspecting discovered skills

The `skills` property returns the metadata of all discovered skills as a mapping of skill name to `SkillInfo` (warming up the toolset first if needed):

```python
from haystack.skill_stores.file_system import FileSystemSkillStore
from haystack.tools import SkillToolset

skills_toolset = SkillToolset(FileSystemSkillStore("skills/"))
for name, info in skills_toolset.skills.items():
print(f"{name}: {info.description}")
```
30 changes: 28 additions & 2 deletions docs-website/docs/tools/tool.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -29,16 +29,18 @@ class Tool:
name: str
description: str
parameters: Dict[str, Any]
function: Callable
function: Callable | None = None
outputs_to_string: dict[str, Any] | None = None
inputs_from_state: dict[str, str] | None = None
outputs_to_state: dict[str, dict[str, Any]] | None = None
async_function: Callable | None = None
```

- `name` is the name of the Tool.
- `description` is a string describing what the Tool does.
- `parameters` is a JSON schema describing the expected parameters.
- `function` is invoked when the Tool is called.
- `function` is invoked when the Tool is called. It must be a regular (sync) function.
- `async_function` (optional) is a coroutine function awaited when the Tool is invoked in an async context. See [Async Tools](#async-tools) below.
- `outputs_to_string` (optional) controls how parts of the tool’s output are converted into one or more strings (e.g. for LLM consumption).
- `inputs_from_state` (optional) maps values from the agent state to the tool’s input parameters (e.g. to share info between tools)
- `outputs_to_state` (optional) specifies how tool outputs are written back into the agent state, with optional handlers.
Expand Down Expand Up @@ -297,6 +299,30 @@ print(result["last_message"].text)
# Red apple with stem resting on straw.
```

## Async Tools

Tools support native async invocation. A `Tool` can carry an `async_function` (a coroutine function) alongside or instead of the sync `function`. When an [`Agent`](../pipeline-components/agents-1/agent.mdx) runs via `run_async`, it awaits the tool's `async_function` if one is available; `Tool.invoke_async` does the same when calling a tool directly.

The `@tool` decorator and `create_tool_from_function` route `async def` callables to `async_function` automatically, so decorating an `async def` is all it takes to produce an async tool:

```python
from typing import Annotated
from haystack.tools import tool


@tool
async def weather(city: Annotated[str, "The name of the city"]) -> str:
"""Get the weather for a city."""
...
```

How the two fields interact:

- If only `function` is set, `invoke_async` falls back to running the sync function in a worker thread, so sync tools keep working in async contexts.
- If only `async_function` is set, the tool can only be invoked asynchronously — calling the sync `invoke` raises a `ToolInvocationError`.

[`ComponentTool`](componenttool.mdx) automatically wires an async invoker for components that define `run_async`, and [`PipelineTool`](pipelinetool.mdx) inherits this behavior — in Haystack 3.0 every `Pipeline` exposes a native `run_async`, so pipeline tools support the async path out of the box.

## Toolset

A Toolset groups multiple Tool instances into a single manageable unit.
Expand Down
1 change: 1 addition & 0 deletions docs-website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -701,6 +701,7 @@ export default {
'tools/mcptool',
'tools/mcptoolset',
'tools/searchabletoolset',
'tools/skilltoolset',
{
type: 'category',
label: 'Ready-made Tools',
Expand Down