feat: add token counters

[sjrl]

Related Issues

• part of Add a Compaction Tool and Hook #10866
• Determined in the thread feat: Add compaction hook #12176 (comment) that for a better UX we should implement a way to count the tokens used up by a list of chat messages

Proposed Changes:

Added haystack.token_counters: a TokenCounter protocol for estimating how many tokens a list of
ChatMessage objects occupies, with two implementations.

Providers report token usage only after a call, and only for the call as a whole, so anything that needs a size beforehand - deciding whether a conversation still fits a model's context window, or how much of it to drop - has to
estimate one.

ApproximateTokenCounter needs no deps and estimates from text length at a configurable chars_per_token. TiktokenCounter counts with OpenAI's byte-pair encoder using tiktoken.

Neither can measure an image or a file, since these are text only tokenizers. Both charge a flat tokens_per_image and tokens_per_file instead, counting images a tool returned inside its result as well as those a message carries directly. Raise those values if you send large images or long documents. Similar approach as LC.

from haystack.dataclasses import ChatMessage
from haystack.token_counters import ApproximateTokenCounter, TiktokenCounter

messages = [ChatMessage.from_user("Hello, how are you?")]

# No dependencies: estimates from text length.
ApproximateTokenCounter(chars_per_token=4.0).count(messages)

# Closer for OpenAI models; needs: pip install tiktoken
TiktokenCounter(encoding="o200k_base").count(messages)

Follow-up: provider-specific token counters

Every major provider exposes an endpoint that counts exactly, including images and tool schemas, at the cost of a network round trip. The TokenCounter protocol exists so these can be added as separate implementations:

Provider	API	Counts	Docs
OpenAI	POST /v1/responses/input_tokens (client.responses.input_tokens.count)	messages, images, files, tool schemas	Counting tokens
Anthropic	client.messages.count_tokens()	messages, images, PDFs, tools. Free, rate-limited	Token counting
Google Gemini	models.countTokens	text, images, audio, video, PDFs; returns a per-modality breakdown via ModalityTokenCount	Understand and count tokens · API reference
Amazon Bedrock	CountTokens	messages for Bedrock-hosted models	Count tokens

How did you test it?

New tests

Notes for the reviewer

If we think this route makes sense I can open up issues for adding token counters for each of the different providers.

Checklist

• I have read the contributors guidelines and the code of conduct.
• I have updated the related issue with new insights and changes.
• I have added unit tests and updated the docstrings.
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
• I have documented my code.
• I have added a release note file, following the contributors guidelines.
• I have run pre-commit hooks and fixed any issue.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
haystack-docs	Ignored	Preview	Jul 30, 2026 12:45pm

[github-actions]

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
haystack/token_counters
__init__.py
approximate_counter.py
tiktoken_counter.py
utils.py
haystack/token_counters/types
__init__.py
protocol.py
Project Total

This report was generated by python-coverage-comment-action

[anakin87]

I'd like to see a short example of how you'd use this in the compaction Hook

[sjrl]

I'd like to see a short example of how you'd use this in the compaction Hook

It's still a work in progress but you can see how these token counters would update the compaction hook here #12196 It's a new PR that builds the compaction hook off this one.

[anakin87]

Looks good.

• I think we can further iterate on them while working on Hooks
• Let's create an issue to track documentation for these classes