Skip to content

feat: add support for elasticsearch sparse inference embedding retriever#3166

Merged
davidsbatista merged 6 commits intodeepset-ai:mainfrom
srini047:es-inference-sparse-retriever
Apr 20, 2026
Merged

feat: add support for elasticsearch sparse inference embedding retriever#3166
davidsbatista merged 6 commits intodeepset-ai:mainfrom
srini047:es-inference-sparse-retriever

Conversation

@srini047
Copy link
Copy Markdown
Contributor

Related Issues

Proposed Changes:

Add support for ElasticsearchInferenceSparseRetriever which uses inference_id and query for retrieval.

How did you test it?

Add tests supporting the new ElasticsearchInferenceSparseRetriever.

Notes for the reviewer

  • Should integration tests all be added?
  • Use from haystack.core.serialization import component_from_dict, component_to_dict for serialization instead of to_dict and from_dict methods?

Checklist

@srini047 srini047 requested a review from a team as a code owner April 15, 2026 08:13
@srini047 srini047 requested review from anakin87 and removed request for a team April 15, 2026 08:13
@srini047 srini047 changed the title feat: add support for elastic sparse inference embedding retriever feat: add support for elasticsearch sparse inference embedding retriever Apr 15, 2026
@github-actions github-actions bot added the type:documentation Improvements or additions to documentation label Apr 15, 2026
@anakin87 anakin87 requested review from davidsbatista and removed request for anakin87 April 15, 2026 08:14
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 15, 2026

Coverage report (elasticsearch)

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  integrations/elasticsearch/src/haystack_integrations/components/retrievers/elasticsearch
  bm25_retriever.py
  embedding_retriever.py
  inference_sparse_retriever.py
  sparse_embedding_retriever.py
  integrations/elasticsearch/src/haystack_integrations/document_stores/elasticsearch
  document_store.py 562-563, 566-567
  filters.py
Project Total  

This report was generated by python-coverage-comment-action

@davidsbatista davidsbatista removed the type:documentation Improvements or additions to documentation label Apr 17, 2026
@github-actions github-actions bot added the type:documentation Improvements or additions to documentation label Apr 17, 2026
@davidsbatista
Copy link
Copy Markdown
Contributor

@srini047 thanks for another contribution, I did a few more adjustments, mainly testing against an Elastic Search cloud account.

inference_sparse_retriever.py

  • Removed unused logger and logging import
  • Removed dead SparseEmbedding import from docstring example
  • Fixed :raises ValueError: docstring to cover both document_store and inference_id cases

document_store.py

  • Fixed _deserialize_document to handle ELSER string token keys gracefully (try/except around int() conversion)
  • Removed redundant inference_id validation from _create_sparse_retrieval_inference_body

test_inference_sparse_retriever.py

  • Fixed test_run_async: assert_called_once_with → assert_awaited_once_with
  • Added full integration test suite (TestElasticsearchInferenceSparseRetrieverIntegration) covering basic retrieval, top_k, filters, filter policies, async, and document shape
  • Fixed _index_documents_with_inference to flatten meta keys to top level (matching Haystack's Document.to_dict() layout)
  • Batched inference API calls: one round-trip per test instead of one per document
  • Fixed test_retrieval_respects_top_k to assert results were actually found (0 < len <= 2)

test_document_store.py

  • Added test_sparse_vector_retrieval_inference_builds_query_with_filters
  • Added test_sparse_vector_retrieval_inference_async_builds_query_with_filters

conftest.py

  • Added inference_sparse_document_store fixture (auto-skips when cloud credentials are absent)

Copy link
Copy Markdown
Contributor

@davidsbatista davidsbatista left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! As I mentioned earlier, I did some adjustments.

@davidsbatista davidsbatista merged commit 055f7c7 into deepset-ai:main Apr 20, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

integration:elasticsearch type:documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ElasticsearchInferenceSparseRetriever (ES inference for sparse)

2 participants