feat: add Presidio integration for PII detection and anonymization by SyedShahmeerAli12 · Pull Request #3075 · deepset-ai/haystack-core-integrations

SyedShahmeerAli12 · 2026-04-01T07:53:47Z

What this adds

Three new Haystack components using Microsoft Presidio:

PresidioDocumentCleaner — anonymizes PII in list[Document], returns new Documents without mutating inputs
PresidioTextCleaner — anonymizes PII in list[str], useful for sanitizing user queries before LLM calls
PresidioEntityExtractor — detects PII entities and stores them in Document metadata under the "entities" key

Usage example

from haystack_integrations.components.preprocessors.presidio import PresidioDocumentCleaner
from haystack import Document

cleaner = PresidioDocumentCleaner()

result = cleaner.run(
    documents=[Document(content="My name is shahhmeer, email: ashahmeer73@gmail.com")]
)

# → "My name is <PERSON>, email: <EMAIL_ADDRESS>"

sjrl · 2026-04-01T07:59:59Z

@SyedShahmeerAli12 thanks for contributing! Please read the contribution guidelines here https://github.com/deepset-ai/haystack-core-integrations/blob/main/CONTRIBUTING.md#create-a-new-integration and run the scaffolding script to help fill in some missing aspects of your contribution.

Also make sure to run the linter and the tests

SyedShahmeerAli12 · 2026-04-01T08:14:16Z

▎ Hello @sjrl I've implemented all three components proposed in this issue:

PresidioDocumentCleaner : anonymizes PII in list[Document]
PresidioTextCleaner : anonymizes PII in list[str] for query sanitization
PresidioEntityExtractor : detects PII entities and stores them in Document metadata

All CI checks are passing. Happy to make any changes based on your feedback

integrations/presidio/README.md

.github/workflows/presidio.yml

integrations/presidio/pyproject.toml

sjrl · 2026-04-02T07:58:38Z

@SyedShahmeerAli12 a few more high-level comments before I do an indepth review:

Please add a new entry in .github/labeler.yml
We recently add code coverage support so please add a new entry in .github/workflows/CI_coverage_comment.yml
Add the new integration to the table in the root README.md https://github.com/deepset-ai/haystack-core-integrations/blob/main/README.md

Implements three Haystack components using Microsoft Presidio: - PresidioDocumentCleaner: anonymizes PII in list[Document] - PresidioTextCleaner: anonymizes PII in list[str] (for query sanitization) - PresidioEntityExtractor: detects PII entities and stores them in Document metadata

…e type mismatch

…d coverage entries

.github/workflows/CI_coverage_comment.yml

README.md

…tical ordering

SyedShahmeerAli12 · 2026-04-09T12:18:20Z

Addressed all comments removed the ## Contributing header from README to match the pgvector format, fixed alphabetical ordering in both the root README table and CI_coverage_comment.yml. Python 3.14 was already in place.

...dio/src/haystack_integrations/components/preprocessors/presidio/presidio_document_cleaner.py

...dio/src/haystack_integrations/components/preprocessors/presidio/presidio_entity_extractor.py

...residio/src/haystack_integrations/components/preprocessors/presidio/presidio_text_cleaner.py

...dio/src/haystack_integrations/components/preprocessors/presidio/presidio_entity_extractor.py

...residio/src/haystack_integrations/components/preprocessors/presidio/presidio_text_cleaner.py

…, type hints, dataclasses.replace, doc links - Add keyword-only arguments (*, ) to all three component __init__ methods - Move AnalyzerEngine/AnonymizerEngine initialization to warm_up() since they load spaCy ML models - Fix run() return types from dict[str, Any] to proper typed dicts - Use dataclasses.replace() in PresidioEntityExtractor instead of Document() - Add Presidio documentation links for language, entities, and score_threshold params - Update integration tests to call warm_up() before run() - Add missing _anonymizer mock in test_run_skips_on_error tests

SyedShahmeerAli12 · 2026-04-09T14:00:09Z

warm_up() — AnalyzerEngine/AnonymizerEngine do load spaCy ML models, so moved engine init into warm_up()
across all three components. Updated integration tests accordingly.
Keyword-only args — Added *, to all three init methods.
Return types — Replaced dict[str, Any] with proper typed returns, removed unused Any import.
dataclasses.replace() — Used in PresidioEntityExtractor instead of constructing a new Document.
Doc links — Added https://microsoft.github.io/presidio/supported_languages/,
https://microsoft.github.io/presidio/supported_entities/, and https://microsoft.github.io/presidio/analyzer/
links to docstrings in all three components.

sjrl · 2026-04-10T07:59:45Z

...dio/src/haystack_integrations/components/preprocessors/presidio/presidio_document_cleaner.py

+
+        :param language:
+            Language code for PII detection. Defaults to `"en"`.
+            See [Presidio supported languages](https://microsoft.github.io/presidio/supported_languages/).


This link leads to a 404

let me check

sjrl · 2026-04-10T08:04:49Z

...dio/src/haystack_integrations/components/preprocessors/presidio/presidio_document_cleaner.py

+        if self._analyzer is None:
+            self._analyzer = AnalyzerEngine()
+        if self._anonymizer is None:
+            self._anonymizer = AnonymizerEngine()


We prefer to use the pattern

Suggested change

if self._analyzer is None:

self._analyzer = AnalyzerEngine()

if self._anonymizer is None:

self._anonymizer = AnonymizerEngine()

if self._is_warmed_up:

return

self._analyzer = AnalyzerEngine()

self._anonymizer = AnonymizerEngine()

self._is_warmed_up = True

and then add self._is_warmed_up = False to the init method

sjrl · 2026-04-10T08:05:23Z

...dio/src/haystack_integrations/components/preprocessors/presidio/presidio_document_cleaner.py

+        """
+        cleaned: list[Document] = []


We auto-warm up our components at first run time.

Suggested change

"""

cleaned: list[Document] = []

"""

if not self._is_warmed_up:

self.warm_up()

cleaned: list[Document] = []

sjrl · 2026-04-10T08:06:12Z

...dio/src/haystack_integrations/components/preprocessors/presidio/presidio_document_cleaner.py

+
+    Documents without text content are passed through unchanged.
+
+    Call `warm_up()` before running this component to load the Presidio analyzer and anonymizer engines.


Once this comment is addressed https://github.com/deepset-ai/haystack-core-integrations/pull/3075/changes#r3062974319 this should be dropped. Or updated to explain that the engines are loaded on first run.

SyedShahmeerAli12 requested a review from a team as a code owner April 1, 2026 07:53

SyedShahmeerAli12 requested review from sjrl and removed request for a team April 1, 2026 07:53

github-actions bot added topic:CI type:documentation Improvements or additions to documentation labels Apr 1, 2026

sjrl self-assigned this Apr 1, 2026

sjrl removed their assignment Apr 2, 2026

sjrl reviewed Apr 2, 2026

View reviewed changes

integrations/presidio/README.md Show resolved Hide resolved

sjrl reviewed Apr 2, 2026

View reviewed changes

.github/workflows/presidio.yml Outdated Show resolved Hide resolved

sjrl reviewed Apr 2, 2026

View reviewed changes

integrations/presidio/pyproject.toml Show resolved Hide resolved

SyedShahmeerAli12 force-pushed the feat/presidio-integration branch from 081b9d3 to 82de46b Compare April 2, 2026 14:40

SyedShahmeerAli12 added 7 commits April 2, 2026 19:44

fix(presidio): add missing README.md required by hatchling build

b7f0359

fix(presidio): fix lint errors and add missing pydoc config

e162949

fix(presidio): apply ruff format to test files

cc518d1

fix(presidio): add py.typed marker for mypy type checking

a363f89

fix(presidio): suppress mypy arg-type error for presidio cross-packag…

a8b2004

…e type mismatch

Address PR review: update README, add Python 3.14 support, labeler an…

80c8c1d

…d coverage entries

SyedShahmeerAli12 force-pushed the feat/presidio-integration branch from 82de46b to 80c8c1d Compare April 2, 2026 14:45