Skip to content

fix: >= and < filters give wrong results for equal datetimes in different string formats #11583

Description

@Ayushhgit

Describe the bug
The >= and < metadata filter operators in haystack/utils/filters.py return wrong results when comparing datetime strings that represent the same moment in time but are formatted differently.

The cause: _greater_than_equal is implemented as _equal(...) or _greater_than(...). _greater_than parses string operands as datetimes before comparing, but _equal compares the raw strings. So for two equivalent representations of the same datetime (e.g. 2025-01-01 vs 2025-01-01T00:00:00, or the same instant expressed with different timezone offsets), the equality check fails and the strict comparison is also False, making >= return False. _less_than is defined as not _greater_than_equal(...), so it returns True for those equal datetimes.

Net effect: date range filters silently drop boundary documents in InMemoryDocumentStore (and any other component using document_matches_filter).

Error message
No error. Documents are silently excluded from (or wrongly included in) filter results.

Expected behavior
>= should return True and < should return False when the document value and the filter value parse to the same datetime, regardless of string formatting. > and <= already behave correctly because they compare parsed datetimes directly.

Additional context
The asymmetry: <= (not _greater_than) and > are correct, while >= and < (which both go through the string-based _equal) are wrong. So the bug only bites on the boundary value of range queries, which makes it easy to miss in production.

To Reproduce

from haystack.dataclasses import Document
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.utils.filters import document_matches_filter

doc = Document(content=x, meta={date: 2025-01-01T00:00:00})

f_gte = {field: meta.date, operator: >=, value: 2025-01-01}
f_lt = {field: meta.date, operator: <, value: 2025-01-01}

print(document_matches_filter(f_gte, doc))  # False, expected True
print(document_matches_filter(f_lt, doc))   # True, expected False

# Same instant, different timezone offsets
doc2 = Document(content=x, meta={date: 2025-01-01T12:00:00+02:00})
f2 = {field: meta.date, operator: >=, value: 2025-01-01T10:00:00+00:00}
print(document_matches_filter(f2, doc2))    # False, expected True

# End to end: boundary document dropped from a date range query
store = InMemoryDocumentStore()
store.write_documents([doc])
res = store.filter_documents({field: meta.date, operator: >=, value: 2025-01-01})
print(len(res))  # 0, expected 1

FAQ Check

System:

  • OS: Windows 11
  • Haystack version: main (2.x)

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions