feat: adding count with filtering operations to OpenSearchDocumentStore

[davidsbatista]

Related Issues

• fixes add the following operations to OpenSearchDocumentStore #2635

Proposed Changes:

• count_documents_by_filter() - count documents matching filter criteria
• count_distinct_values_by_filter()- get distinct value counts for metadata fields with optional filtering
• get_fields_info() - retrieve field type information from index mapping
• get_field_min_max() - get min/max values for numeric metadata fields
• get_field_unique_values() - get unique values for a field with pagination and content-based filtering
• query_sql() - execute SQL queries against OpenSearch with support for multiple response formats (JSON, CSV, JDBC, RAW)

How did you test it?

• added integrations tests covering the new methods both or sync and async versions

Notes for the reviewer

• added httpx>=0.28.1 dependency
• the query_sql() method performs a raw http request (based on httpx) if the specified response format is not JSON

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.

[sjrl]

Hey @tstadel I'd also appreciate your review on this since we want to make sure it will in platform as well.

[tstadel]

Note that this only works till 10k. Overall the pagination is not ideal here. I'd suggest to switch over to composite aggregations which support proper pagination. However you'd need to change the signature to support the after param instead of from.

[tstadel]

I'd suggest to remove the from param altogether as it gives incorrect impressions here. That would be the quickest measure.

For full pagination support I'd recommend:

1. support from and after on protocol level (as some document stores might support from-based and some after-based pagination)
2. raise NotSupportedError for OpenSearch when from is set
3. implement after-based pagination for OpenSearch using composite aggregations

[davidsbatista]

I went with option 3.

def get_metadata_field_unique_values(
        self,
        metadata_field: str,
        search_term: str | None = None,
        size: int | None = 10000,
        after: dict[str, Any] | None = None,
    )

Regarding the Protocol - I support your suggestion of adding both from and after - but let's see how this applies to other DocumentStores, and then we can see if it's easier to add to Protocol.

I think there are common operations that can already be added to the Protocol. I need to review it in detail and make use of it to avoid having so much duplicated code for tests. There's an issue with it, and I will probably take it soon.

[tstadel]

@davidsbatista I left some comments. Biggest issue I see is concerning the pagination of unique_values. I'd probably just not support it here. We can always come back and implement a proper solution.

[tstadel]

LGTM now!

[sjrl]

One minor comment, otherwise looks good!