|
| 1 | +# ADR-0053: `date` is a timezone-naive calendar day; `datetime` is an instant rendered in a reference timezone |
| 2 | + |
| 3 | +**Status**: Proposed (2026-06-16) |
| 4 | +**Deciders**: ObjectStack Protocol Architects |
| 5 | +**Builds on**: [ADR-0032](./0032-unified-expression-layer.md) (unified expression layer — CEL dialect, `today()`/`daysFromNow()`), [ADR-0014](./0014-record-form-field-type.md) (field types) |
| 6 | +**Consumers**: `@objectstack/spec` (`Field.date`/`Field.datetime`), `@objectstack/driver-sql` (`coerceFilterValue`, `formatInput`/`formatOutput`, `dateFields`/`datetimeFields`), `@objectstack/formula` (`stdlib` time functions, `cel-engine` hydration), `@objectstack/objectql` (`applyFormulaPlan`), schedule/cron executors, report/analytics date bucketing, `sys-user-preference.timezone`. |
| 7 | +**Surfaced by**: the formula/flow guardrail series (#1928) and the templates time-relative bug family (#1874). Browser testing of `example-crm` and `templates` found that a `Field.date` compared for equality against a time function (`end_date == daysFromNow(60)`, `expires_on: { $in: [daysFromNow(30)] }`) **silently matches nothing** — and that the value is stored as a full timestamp, not a calendar day. |
| 8 | + |
| 9 | +--- |
| 10 | + |
| 11 | +## TL;DR |
| 12 | + |
| 13 | +A `Field.date` is meant to be a **calendar day** ("close date", "due date", |
| 14 | +"birthday"). Today ObjectStack stores and compares it as a **JS `Date` / full |
| 15 | +timestamp** — an *instant*. That is the textbook "date-as-instant" mistake, and |
| 16 | +it produces two failures: |
| 17 | + |
| 18 | +1. **Silent equality miss.** The write path stores the full timestamp |
| 19 | + (`formatInput` does not normalize — `sql-driver.ts:1967`), but the filter path |
| 20 | + normalizes the query value to `YYYY-MM-DD` (`coerceFilterValue` — |
| 21 | + `sql-driver.ts:1543`). So `date == <date-only>` compares |
| 22 | + `"2026-08-15T17:24Z"` against `"2026-08-15"` → never equal. Range filters |
| 23 | + (`$gte`/`$lt`) only work by accident of lexicographic ISO ordering. |
| 24 | +2. **Off-by-one across timezones.** A date stored as UTC-midnight |
| 25 | + (`new Date("2026-06-16")` = `2026-06-16T00:00Z`) renders as **June 15** for a |
| 26 | + user at UTC-8. Every mature platform avoids this by treating a date as a |
| 27 | + timezone-naive string and never converting it to an instant. |
| 28 | + |
| 29 | +A third, related defect: `daysFromNow(n)`/`daysAgo(n)` keep the current |
| 30 | +**wall-clock time** (`addDaysUtc` — `stdlib.ts:36`), unlike `today()` which |
| 31 | +truncates to UTC midnight (`startOfDayUtc` — `stdlib.ts:19`). And `today()` |
| 32 | +is computed in **UTC**, not the user/org timezone, even though a |
| 33 | +`sys-user-preference.timezone` exists but is never read by the engine. |
| 34 | + |
| 35 | +**Decision.** Adopt the industry-standard split, staged by risk: |
| 36 | + |
| 37 | +- **`date` = a timezone-naive calendar day**, represented as a `YYYY-MM-DD` |
| 38 | + string end-to-end (storage, query, CEL). It is **never** converted to an |
| 39 | + instant and **never** timezone-shifted. |
| 40 | +- **`datetime` = an instant**, stored as UTC, **rendered in a reference |
| 41 | + timezone** (org default, optionally overridden per user). |
| 42 | +- **Phase 1 (low risk, do first):** make `Field.date` a `YYYY-MM-DD` string at |
| 43 | + the driver write/read boundary, aligning storage with the filter layer's |
| 44 | + *already-existing* date-only contract. This fixes the equality miss with no |
| 45 | + new semantics. |
| 46 | +- **Phase 2 (needs review):** introduce a **reference-timezone** model so |
| 47 | + `today()`/`daysFromNow()` and `datetime` rendering are timezone-aware. |
| 48 | + |
| 49 | +--- |
| 50 | + |
| 51 | +## Context |
| 52 | + |
| 53 | +### The current three-layer asymmetry (verified) |
| 54 | + |
| 55 | +| Layer | Treatment of `Field.date` | Evidence | |
| 56 | +|-------|---------------------------|----------| |
| 57 | +| Column | `date` → `table.date()` (SQLite has no real DATE type — TEXT affinity) | `sql-driver.ts:1816` | |
| 58 | +| **Write** | **no normalization** — the JS `Date` is stored verbatim, keeping its time | `formatInput`, `sql-driver.ts:1967` | |
| 59 | +| Read | no normalization — returns the stored string with its time | empirical: `dev.db` holds `"2026-07-15T17:24:56.533Z"` | |
| 60 | +| **Filter** | **normalizes the query value to `YYYY-MM-DD`** (date-only string compare) | `coerceFilterValue`, `sql-driver.ts:1543-1554` | |
| 61 | +| Formula | the stored string is hydrated to a `Date` (date-only → UTC midnight) and compared against the time-function `Date` | `applyFormulaPlan` (`engine.ts`), `hydrateOverloadStrings` (`cel-engine.ts`) | |
| 62 | + |
| 63 | +The write/filter mismatch is the proximate cause: the filter layer already |
| 64 | +*assumes* date-only, but the write layer does not deliver it. |
| 65 | + |
| 66 | +### The time functions disagree with each other |
| 67 | + |
| 68 | +- `today()` → start-of-day **UTC** (`startOfDayUtc`, `stdlib.ts:19,57`). |
| 69 | +- `daysFromNow(n)`/`daysAgo(n)` → `now() ± n*24h`, **keeping wall-clock time** |
| 70 | + (`addDaysUtc`, `stdlib.ts:36`). Two calls a minute apart differ. |
| 71 | +- CEL's only temporal type is `google.protobuf.Timestamp` (a UTC instant) — |
| 72 | + there is no `PlainDate`. So a date field flowing into CEL is forced into an |
| 73 | + instant, which is exactly what we want to avoid. |
| 74 | + |
| 75 | +### How mature platforms model this (all converge) |
| 76 | + |
| 77 | +| Platform | date type (tz-naive) | datetime type (instant) | |
| 78 | +|----------|----------------------|--------------------------| |
| 79 | +| PostgreSQL | `DATE` | `timestamptz` (UTC stored, session-TZ rendered) | |
| 80 | +| **Salesforce** (closest analog) | **Date** — tz-independent, same for all users | **Date/Time** — UTC stored, rendered in running user's TZ; `TODAY()` is user-TZ | |
| 81 | +| java.time | `LocalDate` | `Instant` / `ZonedDateTime` | |
| 82 | +| JS Temporal | `Temporal.PlainDate` | `Temporal.Instant` / `ZonedDateTime` | |
| 83 | +| Rails / Django | `Date` (naive) | UTC stored + active-zone render | |
| 84 | +| Airtable | Date field, "include time" off | "include time" on + timezone setting | |
| 85 | + |
| 86 | +The universal rule: **a calendar date is timezone-naive and is never stored as |
| 87 | +an instant; an instant is stored in UTC and rendered in a timezone.** The choice |
| 88 | +between them is precisely "does this concept depend on a timezone?" |
| 89 | + |
| 90 | +--- |
| 91 | + |
| 92 | +## Decision — staged by risk |
| 93 | + |
| 94 | +### Phase 1 — `Field.date` is a `YYYY-MM-DD` string end-to-end (low risk) |
| 95 | + |
| 96 | +1. **Driver write** (`formatInput`): for every field in `dateFields[table]`, |
| 97 | + normalize the value to `YYYY-MM-DD` before insert/update (reuse the exact |
| 98 | + truncation already in `coerceFilterValue`). A `Date`, a full ISO string, or a |
| 99 | + `YYYY-MM-DD` string all collapse to the calendar day (UTC calendar day for a |
| 100 | + `Date`, matching the existing filter coercion). |
| 101 | +2. **Driver read** (`formatOutput`): coerce stored `date` values to |
| 102 | + `YYYY-MM-DD` (slice any time component). This transparently repairs legacy |
| 103 | + rows that already hold a timestamp, so equality works without a data |
| 104 | + migration. |
| 105 | +3. **CEL/formula**: a `date` field is a `YYYY-MM-DD` string. Date-only |
| 106 | + comparisons (`==`, `<`, `>=`) operate on the string; ISO-8601 sorts |
| 107 | + lexicographically = chronologically, so range comparisons stay correct. |
| 108 | + `today()`/`daysFromNow()` used against a date field are compared date-only. |
| 109 | +4. **No change to `Field.datetime`** — it keeps full-instant semantics |
| 110 | + (`datetimeFields`, stored as UTC ms — `sql-driver.ts:1500`). |
| 111 | + |
| 112 | +After Phase 1, `date == daysFromNow(n)` works (both sides are the same calendar |
| 113 | +day), `$in` of dates works, and the day-window range pattern keeps working. The |
| 114 | +#1950 lint becomes a belt-and-suspenders hint rather than the only mitigation. |
| 115 | + |
| 116 | +### Phase 2 — reference-timezone model (needs review; behavior change) |
| 117 | + |
| 118 | +5. **Reference timezone**: an org-level default timezone setting, optionally |
| 119 | + overridden by `sys-user-preference.timezone` (which exists but is currently |
| 120 | + unread). A single resolver computes "the active timezone" for an execution |
| 121 | + context (interactive user vs. scheduled job → org default). |
| 122 | +6. **`today()`/`daysFromNow()`/`daysAgo()` become timezone-aware**: "what |
| 123 | + calendar day is it" is computed in the reference timezone, not UTC. (At |
| 124 | + 23:00 UTC-8 on the 15th, `today()` must be the 15th, not UTC's 16th.) For a |
| 125 | + genuine sub-day time offset, authors use `now() + duration("Nh")` — the |
| 126 | + documented escape hatch. |
| 127 | +7. **`datetime` rendering** uses the reference timezone at the presentation |
| 128 | + boundary (console, templates, reports). Storage stays UTC. |
| 129 | + |
| 130 | +Phase 2 is gated on an explicit review because it changes the meaning of |
| 131 | +`today()`/`daysFromNow()` and touches scheduling, reporting/date-bucketing, and |
| 132 | +rendering. |
| 133 | + |
| 134 | +--- |
| 135 | + |
| 136 | +## Consequences |
| 137 | + |
| 138 | +- **Phase 1 fixes the silent equality bug at the root** and is shippable on its |
| 139 | + own; it aligns write with the filter contract rather than inventing a |
| 140 | + semantic. Risk is confined to anything that *relied* on a time component |
| 141 | + living inside a `Field.date` — which is already semantically illegitimate and |
| 142 | + should be a `Field.datetime`. |
| 143 | +- **Legacy data**: read-side normalization (step 2) repairs old timestamped |
| 144 | + `date` rows on the fly. An optional one-time migration can rewrite them at |
| 145 | + rest. `Field.datetime` data is untouched. |
| 146 | +- **Migration audit**: grep for code that reads a `Field.date` expecting a time |
| 147 | + component; convert those fields to `Field.datetime`. |
| 148 | +- **Phase 2 blast radius**: cron/schedule day-boundaries, analytics date |
| 149 | + buckets (`dateGranularity`), and any UI that renders datetimes shift from |
| 150 | + implicit-UTC to reference-TZ. Each needs a test pass. |
| 151 | +- **Rollback**: Phase 1 is a localized driver change (revertable); Phase 2 is |
| 152 | + feature-flaggable behind "reference timezone unset → UTC" (today's behavior). |
| 153 | + |
| 154 | +--- |
| 155 | + |
| 156 | +## Non-goals |
| 157 | + |
| 158 | +- Recurring events / RRULE, business-calendar/working-day arithmetic, and |
| 159 | + holiday calendars — out of scope. |
| 160 | +- Per-field timezone overrides (à la Airtable's per-field setting) — the model |
| 161 | + here is org-default + user-override only. |
| 162 | +- Changing the CEL substrate to add a native `PlainDate` type — we represent |
| 163 | + dates as strings rather than extending cel-js. |
| 164 | + |
| 165 | +--- |
| 166 | + |
| 167 | +## Alternatives considered |
| 168 | + |
| 169 | +1. **Status quo + the #1950 lint only.** Rejected: the lint warns but the root |
| 170 | + cause (date stored as instant, write/filter asymmetry, UTC `today()`) |
| 171 | + remains, and the tz off-by-one is untouched. |
| 172 | +2. **Make everything `datetime`.** Rejected: erases the legitimate, useful |
| 173 | + distinction every analog platform keeps; "close date" genuinely is a |
| 174 | + tz-naive calendar day. |
| 175 | +3. **Store `date` as a UTC-midnight instant and compare date-only everywhere.** |
| 176 | + Rejected: still tz-shifts on render (the off-by-one), and re-introduces the |
| 177 | + instant we are trying to remove. |
| 178 | +4. **Truncate `daysFromNow()`/`daysAgo()` to midnight but leave storage as-is.** |
| 179 | + Rejected as a complete fix: it helps the formula path but not the |
| 180 | + write/filter asymmetry, and it bakes in UTC midnight (Phase 2's tz-aware |
| 181 | + `today()` supersedes it). |
0 commit comments