Skip to content

🐛 Fixed incorrect date filtering on SQLite#28952

Closed
frenck wants to merge 2 commits into
TryGhost:mainfrom
frenck:frenck/fix-sqlite-date-filter
Closed

🐛 Fixed incorrect date filtering on SQLite#28952
frenck wants to merge 2 commits into
TryGhost:mainfrom
frenck:frenck/fix-sqlite-date-filter

Conversation

@frenck

@frenck frenck commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

Got some code for us? Awesome 🎊!

Please take a minute to explain the change you're making:

  • Why are you making it?

    Filtering by an absolute date returns the wrong rows on SQLite. Using filter="published_at:>'...'" in a theme's {{#get}}, or the same filter via the Content/Admin API, includes posts from the wrong side of the boundary. Fixes Cannot get posts from the same date (SQLite3-specific?) #23441.

  • What does it do?

    Date columns are stored as YYYY-MM-DD HH:MM:SS (UTC). NQL already normalizes relative dates (now-30d) to that format, but absolute values from a filter (for example published_at:>'2025-02-27T19:03:00.000-05:00') are passed through untouched. On SQLite, datetimes are stored as text and compared lexically, so the T sorts after the space separator (T > ) and the comparison is wrong. Relative dates work, absolute ISO dates don't.

    This adds a model filter plugin that normalizes absolute date values to the stored format before the query runs. It wraps applyDefaultAndCustomFilters with a schema-aware transformer that, for dateTime columns only, reparses the value (handling ISO 8601 with timezone offsets) and reformats it as UTC YYYY-MM-DD HH:mm:ss. Non-date columns and values it can't parse as a date are left untouched, so a post titled 2025-02-27 is never rewritten. Offsets are converted to UTC, which also corrects the same comparison on MySQL.

  • Why is this something Ghost users or developers need?

    Date-based filtering is a documented, widely used feature (date pagination, "newer/older than" lists, API queries). Today it quietly returns incorrect results on SQLite, the default database for self-hosted installs.

Please check your PR against these items:

  • I've read and followed the Contributor Guide
  • I've explained my change
  • I've written an automated test to prove my change works

We appreciate your contribution! 🙏

@coderabbitai

coderabbitai Bot commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: d3a99d60-ae5f-41c7-a3db-89704acfc4ff

📥 Commits

Reviewing files that changed from the base of the PR and between 6c9f433 and cf26321.

📒 Files selected for processing (2)
  • ghost/core/core/server/models/base/bookshelf.js
  • ghost/core/core/server/models/base/plugins/date-filter.js
🚧 Files skipped from review as they are similar to previous changes (2)
  • ghost/core/core/server/models/base/bookshelf.js
  • ghost/core/core/server/models/base/plugins/date-filter.js

Walkthrough

A new Bookshelf date-filter plugin is added and registered after the existing filter plugin. It detects dateTime columns from schemaTables, normalizes ISO date strings in filter trees to YYYY-MM-DD HH:mm:ss, and chains this normalization after any mongoTransformer in applyDefaultAndCustomFilters. Unit tests cover the normalizer, and integration tests verify published_at boundary filtering in SQLite.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title directly summarizes the main fix: incorrect SQLite date filtering.
Description check ✅ Passed The description clearly explains the SQLite date-filter bug and the normalization fix.
Linked Issues check ✅ Passed The change normalizes absolute date filters for dateTime columns, matching the issue's SQLite comparison fix.
Out of Scope Changes check ✅ Passed The added plugin and regression tests stay within the date-filtering bug fix.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@ghost/core/core/server/models/base/plugins/date-filter.js`:
- Around line 60-72: The normalizeFieldValue logic in date-filter.js is treating
any object as an operator map, which incorrectly converts Date instances into
empty objects. Update normalizeFieldValue to only enter the
operator-normalization branch for plain objects containing $-prefixed operators,
and leave Date values unchanged; keep the array handling and normalizeValue
usage intact.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: bc801781-9315-4c24-9fb0-c35e4eac0a55

📥 Commits

Reviewing files that changed from the base of the PR and between d9e9681 and 3babb68.

📒 Files selected for processing (4)
  • ghost/core/core/server/models/base/bookshelf.js
  • ghost/core/core/server/models/base/plugins/date-filter.js
  • ghost/core/test/integration/model/post-date-filter.test.js
  • ghost/core/test/unit/server/models/base/date-filter.test.js

Comment thread ghost/core/core/server/models/base/plugins/date-filter.js
@frenck frenck force-pushed the frenck/fix-sqlite-date-filter branch from 3babb68 to 6c9f433 Compare June 27, 2026 22:53
frenck and others added 2 commits June 30, 2026 16:25
fixes TryGhost#23441

- date columns are stored as "YYYY-MM-DD HH:MM:SS", but absolute date values in filters were passed through untouched while only relative dates (now-30d) were normalized
- on SQLite, datetimes are stored as text and compared lexically, so the "T" in an ISO value sorts after the stored space separator and greater/less-than filters returned the wrong rows
- normalizes date-column filter values to the stored UTC format before the query is built, leaving non-date columns and unparseable values untouched
The date-filter normalization rewrites bare date filter values to the
stored `YYYY-MM-DD HH:MM:SS` format, so the existing query-binding
assertions for `published_at:>'2015-07-20'` now expect
`2015-07-20 00:00:00`.
@9larsons 9larsons force-pushed the frenck/fix-sqlite-date-filter branch from 6c9f433 to cf26321 Compare June 30, 2026 21:31
@9larsons

9larsons commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Thanks @frenck, great writeup, and the dx is spot on, but I'd like to fix this a layer up in NQL rather than in Ghost core. Relative dates (now-30d) are already normalized to the stored YYYY-MM-DD HH:mm:ss UTC format inside @tryghost/nql-lang and this would be its sibling.

Going to close this in favour of that approach. I'll raise the nql-lang PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot get posts from the same date (SQLite3-specific?)

2 participants