Skip to content

feat(opensearch): Add support for nested fields in OpenSearchDocumentStore#3117

Open
bogdankostic wants to merge 4 commits intomainfrom
opensearch-support-nested
Open

feat(opensearch): Add support for nested fields in OpenSearchDocumentStore#3117
bogdankostic wants to merge 4 commits intomainfrom
opensearch-support-nested

Conversation

@bogdankostic
Copy link
Copy Markdown
Contributor

@bogdankostic bogdankostic commented Apr 7, 2026

Related Issues

Proposed Changes:

Add support for OpenSearch nested field type in OpenSearchDocumentStore, enabling multi-condition filtering on array-of-objects metadata fields (e.g., filtering where refs.law == "bgb" AND refs.section == "1" matches within the same array element rather than across different elements).

Key changes:

  • New nested_fields parameter on OpenSearchDocumentStore.__init__: accepts either explicit field names (e.g., ["refs", "tags"]) or ["*"] for auto-detection from list[dict] metadata values
  • Filter translation: normalize_filters now wraps conditions targeting nested sub-fields in OpenSearch nested queries, grouping conditions on the same nested path for correct within-element matching
  • Auto-detection mode (["*"]): detects list[dict] metadata fields during write_documents and dynamically updates the index mapping via put_mapping, supporting incremental detection across batches
  • Existing index support: when the index already exists, nested fields are discovered from the live mapping at initialization
  • Both sync and async code paths are updated

How did you test it?

  • Unit tests for filter translation, nested path detection helpers, document store methods (serialization, default mappings, field detection, wildcard detection, incremental detection)
  • Integration tests covering explicit nested fields (single filter, combined AND, mixed nested+flat, OR, different paths) and wildcard auto-detection (auto-detection, filtering, incremental detection)

Notes for the reviewer

Checklist

@github-actions github-actions bot added integration:opensearch type:documentation Improvements or additions to documentation labels Apr 7, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 7, 2026

Coverage report (opensearch)

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  integrations/opensearch/src/haystack_integrations/document_stores/opensearch
  document_store.py 324-325, 338, 365-366, 375-384, 393-406, 421-423, 457-459, 643-652, 858-864, 888-894, 918-924, 958-964, 996-998, 1031, 1433-1437, 1538-1542, 1605-1615, 1716-1721, 1731-1736, 1752-1753, 1759-1761
  filters.py 19-20, 28, 55
Project Total  

This report was generated by python-coverage-comment-action

@bogdankostic bogdankostic marked this pull request as ready for review April 7, 2026 20:02
@bogdankostic bogdankostic requested a review from a team as a code owner April 7, 2026 20:02
@bogdankostic bogdankostic requested review from anakin87 and removed request for a team April 7, 2026 20:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

integration:opensearch type:documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OpenSearch: Support nested Field Type for Correct Multi-Condition Array Filtering

1 participant