Skip to content

fix: reconcile tz-aware and naive datetimes in ordering filters - #11943

Merged
anakin87 merged 1 commit into
deepset-ai:mainfrom
otiscuilei:fix/filters-datetime-tz-aware-comparison
Jul 10, 2026
Merged

fix: reconcile tz-aware and naive datetimes in ordering filters#11943
anakin87 merged 1 commit into
deepset-ai:mainfrom
otiscuilei:fix/filters-datetime-tz-aware-comparison

Conversation

@otiscuilei

Copy link
Copy Markdown
Contributor

Anyone who filters documents with an ordering operator (>, >=, <, <=) where both the filter value and the document meta value are datetime objects, but the two differ in timezone-awareness (one naive, one tz-aware), hits an uncaught TypeError: can't compare offset-naive and offset-aware datetimes. Because the exception is not a FilterError, it propagates out of document_matches_filter / DocumentStore.filter_documents and aborts the entire query.

The root cause is in _prepare_ordering_comparison in haystack/utils/filters.py. The naive/aware reconciliation helper _ensure_both_dates_naive_or_aware was only invoked inside the isinstance(value, str) or isinstance(filter_value, str) branch. When both operands arrive as datetime objects (no string involved), that branch is skipped and the code falls through directly to the raw > / < comparison, raising. The string-date path already handles this awareness mismatch gracefully, so the behavior was inconsistent depending on whether a date came in as a string or a datetime.

The fix moves the reconciliation out of the string-only branch so it runs whenever both operands are datetime instances:

if isinstance(value, str) or isinstance(filter_value, str):
    if not isinstance(value, datetime):
        value = _parse_date(value)
    if not isinstance(filter_value, datetime):
        filter_value = _parse_date(filter_value)

if isinstance(value, datetime) and isinstance(filter_value, datetime):
    value, filter_value = _ensure_both_dates_naive_or_aware(value, filter_value)

Before:

from datetime import datetime, timezone
from haystack import Document
from haystack.utils.filters import document_matches_filter

document_matches_filter(
    {"field": "meta.date", "operator": ">", "value": datetime(2023, 1, 1, tzinfo=timezone.utc)},
    Document(meta={"date": datetime(2024, 1, 1)}),  # naive
)
# TypeError: can't compare offset-naive and offset-aware datetimes

After:

document_matches_filter(
    {"field": "meta.date", "operator": ">", "value": datetime(2023, 1, 1, tzinfo=timezone.utc)},
    Document(meta={"date": datetime(2024, 1, 1)}),
)
# True

A focused regression test is added to test/utils/test_filters.py covering both the aware-filter/naive-doc and naive-filter/aware-doc directions; it fails on the pre-fix code with the TypeError and passes with the fix. A release note is included under releasenotes/notes/.

@otiscuilei
otiscuilei requested a review from a team as a code owner July 10, 2026 05:56
@otiscuilei
otiscuilei requested review from anakin87 and removed request for a team July 10, 2026 05:56
@vercel

vercel Bot commented Jul 10, 2026

Copy link
Copy Markdown

@Otis0408 is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions github-actions Bot added topic:tests type:documentation Improvements on the docs labels Jul 10, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  haystack/utils
  filters.py
Project Total  

This report was generated by python-coverage-comment-action

@anakin87 anakin87 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me.

Thank you!

@anakin87
anakin87 merged commit 3cfbaa2 into deepset-ai:main Jul 10, 2026
23 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic:tests type:documentation Improvements on the docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants