feat: Add compaction hook

[sjrl]

Related Issues

• part 1 of Add a Compaction Tool and Hook #10866

Proposed Changes:

Added context compaction for the Agent: a ContextCompactionHook that shortens the conversation once it
grows past a threshold, and a Compactor protocol defining how the shortening is done. The first built-in
compactor is SlidingWindowCompactor, which keeps the leading system messages and the most recent messages and drops what is in between, leaving a short note in their place. Register the hook under the before_llm hook point:

from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.hooks.compaction import ContextCompactionHook, SlidingWindowCompactor

hook = ContextCompactionHook(
    compactor=SlidingWindowCompactor(keep_last_n_messages=20), threshold_tokens=100_000
)
agent = Agent(
    chat_generator=OpenAIChatGenerator(model="gpt-5.4-nano"),
    tools=[web_search],
    hooks={"before_llm": [hook]},
)

The trigger is the context_tokens state key, which the Agent refreshes after each chat-generator call from the
reply's reported token usage. The Agent's Chat Generator must report usage in meta["usage"], otherwise context_tokens stays at 0, the threshold is never reached, and the hook never compacts. The hook logs a
warning once per run when it detects this. Leave headroom between the threshold and the model's context window: context_tokens was measured at the previous call, so the tool results appended since and the upcoming reply must still fit within the window.

Compaction never leaves a tool result whose originating tool call was removed, which chat-completion APIs reject: the boundary of the retained window is moved backwards onto the assistant message holding the call, so slightly more may be kept than keep_last_n_messages asks for. Implement the Compactor protocol to define your own strategy. Return None unless the conversation actually gets smaller, and never drop the final message, because the Agent may be mid-step with a pending tool call whose result is appended right after compaction.

Note that compaction is lossy. After it runs the Agent works from a shorter record of the run, and with SlidingWindowCompactor anything outside the retained window is gone rather than summarized.

How did you test it?

Added new tests.

Also tried it out locally with
compaction_live_check.py

Notes for the reviewer

This one a few PRs.

1. I'll add proper docs pages in a separate PR.
2. I plan on adding more Compactor types (e.g. summarization, tool result pruning) in a separate PR.
3. The Compaction Tool (which will reuse the introduced Compactor type) will be a separate PR.

Checklist

• I have read the contributors guidelines and the code of conduct.
• I have updated the related issue with new insights and changes.
• I have added unit tests and updated the docstrings.
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
• I have documented my code.
• I have added a release note file, following the contributors guidelines.
• I have run pre-commit hooks and fixed any issue.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
haystack-docs	Ignored	Preview	Jul 29, 2026 9:25am

[github-actions]

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
haystack/hooks/compaction
__init__.py
hooks.py					159, 171-172
sliding_window.py
utils.py
haystack/hooks/compaction/types
__init__.py
protocol.py					55
Project Total

This report was generated by python-coverage-comment-action

[anakin87]

I left a few initial comments

[anakin87]

In this design

• the hook knows when to run the compactor (based on tokens)
• the compactor knows how many messages to keep

Have you also thought of other design options?
Like a single context_budget parameter?

[sjrl]

@anakin87 could you explain a bit more what the context_budget parameter would be exactly? Some constraints we should be aware of:

• We don't have an exact token count at compaction time since we haven't counted the tokens yet from the tool results picked up from the last turn. So we are definitely under counting when compaction is triggered. --> We could try an approximate this using a provided tokenizer like tiktoken.
• We don't know what the context window size is of the model that is currently driving the Agent. --> This could be solved by asking for this info at init time in the Compactor.

[anakin87]

Yes, I understand the limitations.

My main concern is that we trigger on a certain token threshold and then we cut based on messages; the Compactor also does not know if it reached a number below the threshold. For these reasons, I'm not sure that the current protocol will be correct in the future (maybe the Compactor needs to know about the threshold).
Maybe we should flag this feature as experimental?

LC SummarizationMiddleware: interesting to take a look, not perfect

[sjrl]

Yeah that's a fair point. I'm looking into potential solutions now. It seems like we will need to introduce some sort of approximate token counter for this to be more robust. I'll see if I can rework this PR to do something like this and then I'd be happy to split it up into smaller ones to help keep the review size manageable.

[anakin87]

2. I plan on adding more Compactor types (e.g. summarization, tool result pruning) in a separate PR.

Could you explain tool result pruning?

3. The Compaction Tool (which will reuse the introduced Compactor type) will be a separate PR.

What's the use case for having a separate Compaction Tool?

[sjrl]

Could you explain tool result pruning?

Sure! Tool result pruning is the idea that to shorten context we could simply prune older tool results especially long ones. The idea being in a long running Agent session think 100s of tool calls the earlier ones in the history may no longer be relevant and just pollute the context window. So this compactor would go in and replace old tool results with a placeholder message like "[Tool result removed to free up context. Call {tool_name} again if you need it.]" where we could insert the actual tool name.

There are some blog posts that mention this concept either under the name pruning or trimming.

What's the use case for having a separate Compaction Tool?

The point of having a compaction tool is to let the Agent decide when to compact. Mostly motivated by this blog post https://www.langchain.com/blog/autonomous-context-compression allowing users to follow the philosophy of giving more power to the Agent in making decisions. E.g. a quote from the blog post

We are generally bullish on the idea that harnesses should, where possible, “get out of the way” and take advantage of improvements in the underlying reasoning models. This is an instance of the bitter lesson: can we give agents more control over their own context to avoid tuning their harness by hand?