feat: Add better language support presidio

[sjrl]

Related Issues

• fixes #issue-number

Proposed Changes:

Update the presidio integration to properly pass language to the supported_languages param of AnalyzerEngine and add a new param called models which is required if using languages other than English to tell Presidio what model to use to power the NlpEngine.

For example, to run the document cleaner in german this now works and the german model is automatically downloaded at warm up time.

from haystack import Document
from haystack_integrations.components.preprocessors.presidio import PresidioDocumentCleaner

cleaner = PresidioDocumentCleaner(
    language="de",
    models=[{"lang_code": "de", "model_name": "de_core_news_lg"}],
)
cleaner.warm_up()
docs = [Document(content="Mein Name ist Hans Müller und meine E-Mail ist hans@example.com")]
result = cleaner.run(documents=docs)

How did you test it?

Added unit and integration tests.

Notes for the reviewer

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.

[github-actions]

Coverage report (presidio)

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
integrations/presidio/src/haystack_integrations/components/common/presidio
utils.py
integrations/presidio/src/haystack_integrations/components/extractors/presidio
presidio_entity_extractor.py					101
integrations/presidio/src/haystack_integrations/components/preprocessors/presidio
presidio_document_cleaner.py					99
presidio_text_cleaner.py					96
Project Total

This report was generated by python-coverage-comment-action

[bogdankostic]

Looks good in principle, I'm just wondering if we should remove the language parameter, given that if a user wants to use a non-english model, they have to set the models parameter. In that case self.language is not used to initialize the AnalyzerEngine.

[sjrl]

Looks good in principle, I'm just wondering if we should remove the language parameter, given that if a user wants to use a non-english model, they have to set the models parameter. In that case self.language is not used to initialize the AnalyzerEngine.

Maybe instead we could add a set of pre-defined mappings between language and model so users could just use the language param?

[bogdankostic]

Looks good in principle, I'm just wondering if we should remove the language parameter, given that if a user wants to use a non-english model, they have to set the models parameter. In that case self.language is not used to initialize the AnalyzerEngine.

Maybe instead we could add a set of pre-defined mappings between language and model so users could just use the language param?

SOunds good, that way users wouldn't need to figure out which model to use for their language.

[bogdankostic]

Looking good to me now! Just added a minor suggestion to improve the doc string of the class var.