fix(driver-sql): Field.time canonical storage — HH:MM:SS[.fff] wall-clock text on every dialect (#3994) (#4020)

* feat(spec,ci): temporal hooks onto the IDataDriver contract; conformance job with live non-UTC servers

Closes the two debts the datetime storage work (#3912/#3942/#3954) left
open in ADR-0053.

D-A2 — contract promotion. temporalFilterValue and temporalFilterColumnSql
were duck-typed: analytics probed `typeof driver.x === 'function'` against
a locally-invented interface, and nothing at the type level said a driver
must implement both or neither. #3912's lesson is precisely that coercing
the comparand without normalising the column reintroduces half the bug, so
a driver implementing one hook alone would silently regress. Both are now
optional members of IDataDriver, documented as a PAIR with
"absent = identity" semantics for drivers whose storage form is the wire
form. SqlDriver `implements IDataDriver`, so its signatures are
compile-checked from here on; analytics Picks the contract instead of
inventing a local shape, keeping the runtime typeof guards as the correct
way to consume an optional member. coerceFilterValueForSql already sits
behind the hook as the last-resort boolean/number recovery — the demotion
D-A2 asked for — and stays in exactly that role. ADR-0053 records the
resolution.

D-A3 — the conformance backstop. The live-server suites from #3912/#3942
are opt-in (OS_TEST_POSTGRES_URL / OS_TEST_MYSQL_URL) and skip without a
server, so nothing ran them in CI and the seam could regress silently.
New `temporal-conformance` job: postgres:16 and mysql:8.0 service
containers, servers pointed at +08:00 post-start (services cannot
override the image command), the Node process at America/New_York, and
assertions in UTC — the three-way skew that caught every bug in this
family, with both suites asserting a non-UTC server so a mis-provisioned
service fails loudly rather than passing vacuously. The WHOLE driver-sql
suite runs under the skewed zone, so a TZ-sensitive assumption anywhere
in the driver's tests fails here before it ships. mysql:8.0 also covers
the other half of the #3942 compatibility claim — the hands-on
verification ran on MariaDB 10.11, the stricter dialect.

Verified locally with the exact CI command against live PG 16
(Asia/Shanghai) and MariaDB (+08:00): 47 files, 480/480, zero skips.
Full pnpm build + test green (132 tasks); spec doc gates green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TPxNwPnjcn599ujXpU3ibJ

* fix(driver-sql): Field.time gets a canonical storage form — HH:MM:SS[.fff] wall-clock text on every dialect (#3994)

Field.time repeated the pre-#3912 Field.datetime pattern: writes were never
normalised, only reads were repaired, so one SQLite column accumulated bare
time-of-day TEXT, full-timestamp TEXT and INTEGER epoch ms side by side.
find() looked right; everything comparing the STORED form was wrong —
measured: a business-hours window filter silently dropped 4 of 7 rows, ORDER
BY sorted 14:30 before 08:00, a full-ISO write failed the statement on both
Postgres and MySQL, a bound Date stored a process-timezone wall clock on pg,
MySQL's bare TIME rounded fractional seconds, and NOW() defaults read three
different clocks across the three dialects.

The #3912→#3942→#3954 construction, transplanted (ADR-0053 D-C1..D-C3):

- One canonicalTimeOfDay — HH:MM:SS, .fff only when non-zero; Date/epoch/
  full-timestamp fold to the UTC time-of-day — applied on write (formatInput),
  to filter comparands (coerceFilterValue → temporalFilterValue) and on read
  (toTimeOnly).
- SQLite: backfillCanonicalTimes converges legacy columns at schema sync
  (IS NOT-guarded UPDATE, log-and-swallow); until then filters wrap the
  column in sqliteCanonicalTimeSql — correct, just unindexed. os migrate
  plan lists the work as normalize_time_storage with a row count.
- MySQL: new time columns are TIME(3); legacy TIME(0) widens at schema sync
  (migrateMysqlTimeColumns, plan kind widen_time_columns) since
  zero-precision TIME rounds fractional writes.
- NOW() defaults read the UTC clock on every dialect; MySQL 8.0 rejects a
  plain CURRENT_TIMESTAMP default on TIME entirely, so the expression
  default is also a compatibility fix.
- distinct()/aggregate() present time columns exactly as find() does
  (ReadPresentationKind gains 'time').

HH:MM:SS writes round-trip byte-identically (field-zoo f_time, #2022); a
minutes-only HH:MM completes to HH:MM:00; uninterpretable values pass
through untouched.

Verified on live servers: SQLite, PG 16 at Asia/Shanghai, MariaDB 10.11 at
+08:00, Node at America/New_York — 505/505 driver tests with zero skips.

Closes #3994

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TPxNwPnjcn599ujXpU3ibJ

---------

Co-authored-by: Claude <noreply@anthropic.com>

Author: os-zhuang

.changeset/field-time-canonical-storage.md

@@ -0,0 +1,40 @@
+---
+"@objectstack/driver-sql": minor
+"@objectstack/cli": patch
+"@objectstack/spec": patch
+---
+
+fix(driver-sql): `Field.time` gets a canonical storage form — `HH:MM:SS[.fff]` wall-clock text on every dialect (#3994)
+
+`Field.time` repeated the pre-#3912 `Field.datetime` pattern: writes were never
+normalised and only reads were repaired, so one SQLite column accumulated bare
+time-of-day TEXT, full-timestamp TEXT and INTEGER epoch ms side by side.
+`find()` looked right; everything that compared the STORED form was wrong —
+measured: a business-hours window filter silently dropped 4 of 7 rows, ORDER BY
+sorted 14:30 before 08:00, a full-ISO write failed the statement outright on
+both Postgres and MySQL, a bound `Date` stored a process-timezone wall clock on
+pg, MySQL's bare `TIME` rounded `…00.500` up to `…01`, and a `NOW()` default
+resolved against three different clocks on the three dialects.
+
+The #3912→#3942→#3954 construction, transplanted (ADR-0053 D-C1..D-C3):
+
+- One `canonicalTimeOfDay` — `HH:MM:SS`, `.fff` only when non-zero; `Date`/
+  epoch/full-timestamp fold to the UTC time-of-day — applied on write
+  (`formatInput`), to filter comparands (`coerceFilterValue`, and thereby the
+  `temporalFilterValue` contract hook) and on read (`toTimeOnly`).
+- SQLite: legacy columns converge at schema sync (`backfillCanonicalTimes`,
+  same `IS NOT`-guarded UPDATE, same log-and-swallow policy); until then the
+  filter paths wrap the column in the repair expression — correct, just
+  unindexed. `os migrate plan` lists the work as `normalize_time_storage` with
+  a row count.
+- MySQL: new time columns are `TIME(3)`; legacy `TIME(0)` columns widen at
+  schema sync (`migrateMysqlTimeColumns`, plan kind `widen_time_columns`),
+  since zero-precision TIME *rounds* fractional writes.
+- `NOW()` defaults read the UTC clock on every dialect (Postgres previously
+  used the server zone, MySQL the inserting session's zone — and MySQL 8.0
+  rejects a plain `CURRENT_TIMESTAMP` default on TIME entirely).
+- `distinct()`/`aggregate()` present time columns exactly as `find()` does.
+
+`HH:MM:SS` writes round-trip byte-identically (the field-zoo `f_time`
+contract); a minutes-only `HH:MM` now completes to `HH:MM:00`, and uninterpretable
+values still pass through untouched.

docs/adr/0053-date-and-datetime-semantics.md

@@ -1,6 +1,6 @@
 # ADR-0053: `date` is a timezone-naive calendar day; `datetime` is an instant rendered in a reference timezone
 
-**Status**: Accepted (2026-06-16) — Phase 1 + addendum D-A1 implemented (`sql-driver.ts` `toDateOnly` write/read/filter normalization; analytics `coerceTemporalFilterValue`), Phase 2 landing incrementally; D-A2 resolved 2026-07-30: `temporalFilterValue` + `temporalFilterColumnSql` are optional `IDataDriver` contract members with identity semantics, and analytics types its driver seam from the contract. **Partly superseded (2026-07-29, addendum D-B1..D-B4):** Phase 1's "`Field.datetime` stays stored as UTC epoch ms" is replaced by one canonical UTC instant per dialect — `YYYY-MM-DDTHH:MM:SS.sssZ` text on SQLite, `timestamptz` on Postgres, `DATETIME(3)` on MySQL — applied on write and to filter comparands alike (#3912, #3942).
+**Status**: Accepted (2026-06-16) — Phase 1 + addendum D-A1 implemented (`sql-driver.ts` `toDateOnly` write/read/filter normalization; analytics `coerceTemporalFilterValue`), Phase 2 landing incrementally; D-A2 resolved 2026-07-30: `temporalFilterValue` + `temporalFilterColumnSql` are optional `IDataDriver` contract members with identity semantics, and analytics types its driver seam from the contract. **Partly superseded (2026-07-29, addendum D-B1..D-B4):** Phase 1's "`Field.datetime` stays stored as UTC epoch ms" is replaced by one canonical UTC instant per dialect — `YYYY-MM-DDTHH:MM:SS.sssZ` text on SQLite, `timestamptz` on Postgres, `DATETIME(3)` on MySQL — applied on write and to filter comparands alike (#3912, #3942). **Extended (2026-07-30, addendum D-C1..D-C3):** `Field.time` takes the same construction — canonical `HH:MM:SS[.fff]` wall-clock text, one function on write/filter/read, `TIME(3)` on MySQL, UTC `NOW()` defaults on every dialect (#3994).
 **Deciders**: ObjectStack Protocol Architects
 **Builds on**: [ADR-0032](./0032-unified-expression-layer.md) (unified expression layer — CEL dialect, `today()`/`daysFromNow()`), [ADR-0014](./0014-record-form-field-type.md) (field types)
 **Consumers**: `@objectstack/spec` (`Field.date`/`Field.datetime`), `@objectstack/driver-sql` (`coerceFilterValue`, `formatInput`/`formatOutput`, `dateFields`/`datetimeFields`), `@objectstack/formula` (`stdlib` time functions, `cel-engine` hydration), `@objectstack/objectql` (`applyFormulaPlan`), schedule/cron executors, report/analytics date bucketing, `sys-user-preference.timezone`.
@@ -574,3 +574,78 @@ The same caveat as Postgres applies and is worth stating plainly: the migration
 Regression cover is `sql-driver-datetime-mysql-storage.test.ts`, opt-in via
 `OS_TEST_MYSQL_URL` (CI provisions no server), asserting a non-UTC server so it
 cannot pass vacuously. 10 of its 13 cases fail without this change.
+
+---
+
+## Addendum (2026-07-30) — `Field.time` gets the same storage convention (D-C1..D-C3, #3994)
+
+`Field.time` was the last temporal field type with **no storage convention at
+all** — the exact meta-problem #3912 exposed for `Field.datetime`. `formatInput`
+never touched it, `coerceFilterValue` returned its comparands unchanged, and the
+read paths papered over the drift so well (`toTimeOnly`) that `find()` always
+looked right. Measured on real servers (SQLite; PG 16 at `Asia/Shanghai`;
+MariaDB 10.11 / MySQL 8.0 at `+08:00`; Node at `America/New_York`), the same
+failure family followed: a business-hours window filter silently dropped 4 of 7
+rows on SQLite (INTEGER epoch rows fail `>= '09:00'` because `INTEGER < TEXT`;
+full-timestamp text fails `<= '18:00'` lexicographically), ORDER BY put 14:30
+before 08:00, a full-ISO write failed the statement outright on PG **and**
+MySQL, a bound `Date` stored a process-timezone wall clock on pg, MySQL's bare
+`TIME` rounded `…00.500` up to `…01`, and a `NOW()` default resolved against
+three different clocks on the three dialects.
+
+### D-C1 — The canonical form is `HH:MM:SS`, `.fff` only when non-zero
+
+A `Field.time` is a **timezone-naive wall-clock time-of-day** (#2004), so the
+canonical text carries no zone: `HH:MM:SS`, extended to `HH:MM:SS.fff` exactly
+when the milliseconds are non-zero. One function (`canonicalTimeOfDay`) produces
+it and is applied on write (`formatInput`), to filter comparands
+(`coerceFilterValue`/`temporalFilterValue`) and on read (`toTimeOnly`), the
+D-B1 construction transplanted.
+
+Why variable-width rather than datetime's fixed `.sss`: `.` sorts below every
+digit, so lexicographic order is still chronological order across mixed widths
+(`'14:30:00.100' < '14:30:01'`), and the zero-millisecond spelling `HH:MM:SS`
+is what every dialect's native TIME emits — the field-zoo `f_time` round-trip
+(#2022) keeps holding byte-for-byte. Determinism is what matters for equality
+and `distinct()`, and each wall clock has exactly one spelling. A minutes-only
+`'14:30'` completes to `'14:30:00'`.
+
+A `Date` / epoch-ms / full-timestamp value folds to its **UTC** time-of-day —
+matching the platform's instant semantics, the SQLite read repair's historical
+behaviour, and `nowColumnDefault`; never the process or server timezone.
+Uninterpretable values (including out-of-range wall clocks like `'25:00'`) pass
+through untouched.
+
+### D-C2 — Physical form per dialect
+
+- **SQLite**: canonical TEXT (no native type). Legacy columns converge at
+  schema sync via `backfillCanonicalTimes` — one `UPDATE` per column whose SET
+  expression IS the read-repair SQL (`sqliteCanonicalTimeSql`), `IS NOT` guard,
+  log-and-swallow failure policy, `os migrate plan` row-counted entry
+  (`normalize_time_storage`) — D-B3 verbatim. Until it runs, filters wrap the
+  column in the repair expression: correct, just unindexed.
+- **Postgres**: native `time` (µs precision). The canonical literal binds
+  unambiguously; the pre-fix failures were the *input* shapes, not the column.
+- **MySQL**: `TIME(3)` — the `DATETIME(3)` precedent applied: bare `TIME` is
+  zero-precision and **rounds** fractional literals, changing the stored wall
+  clock. Legacy `TIME(0)` columns widen at schema sync
+  (`migrateMysqlTimeColumns`, plan kind `widen_time_columns`), restating a
+  `NOW()` expression default where declared.
+
+### D-C3 — `NOW()` defaults are the UTC wall clock on every dialect
+
+`knex.fn.now()` compiles to `CURRENT_TIMESTAMP`, which a TIME column resolves
+in the **server's** zone on Postgres and the **inserting session's** zone on
+MySQL (so the same column's default depended on who inserted). MySQL 8.0
+additionally rejects a plain `CURRENT_TIMESTAMP` default on TIME. The defaults
+are now expression-spelled per dialect — `strftime('%H:%M:%f','now')` with a
+canonical `.000` trim on SQLite, `timezone('utc', now())::time(3)` on Postgres,
+`(cast(utc_timestamp(3) as time(3)))` on MySQL (8.0.13+/MariaDB 10.2+ for the
+expression-default syntax) — all reading the UTC clock.
+
+The D-B3 caveat applies unchanged: the backfill converges *shapes*; a wall
+clock the old path never recorded correctly (a pg `Date` bind serialised in the
+host's zone) is not recoverable. Regression cover:
+`sql-driver-time-canonical-storage.test.ts`, `sql-driver-time-of-day.test.ts`
+(SQLite), `sql-driver-time-live-dialects.test.ts` (live PG + MySQL, in the CI
+temporal-conformance job's non-UTC matrix).

packages/cli/src/utils/schema-migrate.pending-render.test.ts

@@ -41,6 +41,8 @@ const ADDITIVE: PendingSchemaWork[] = [
 const IN_PLACE: PendingSchemaWork[] = [
   { table: 'evt', kind: 'normalize_datetime_storage', columns: ['at'], rows: 1234567 },
   { table: 'legacy', kind: 'widen_datetime_columns', columns: ['at', 'created_at'], rows: 42 },
+  { table: 'shift', kind: 'normalize_time_storage', columns: ['starts_at'], rows: 7 },
+  { table: 'shift_my', kind: 'widen_time_columns', columns: ['starts_at'], rows: 9 },
 ];
 
 describe('renderPendingSchemaWork (#3954)', () => {
@@ -74,6 +76,11 @@ describe('renderPendingSchemaWork (#3954)', () => {
     expect(out()).toContain('widen_datetime_columns: at, created_at');
     // A MySQL widen is ALTER … MODIFY — a rebuild, said outright.
     expect(out()).toContain('42 row table rebuild');
+    // The Field.time twins (#3994) take the same rendering rules.
+    expect(out()).toContain('normalize_time_storage: starts_at');
+    expect(out()).toContain('7 row update(s)');
+    expect(out()).toContain('widen_time_columns: starts_at');
+    expect(out()).toContain('9 row table rebuild');
   });
 
   it('shows both sections when both kinds are pending', () => {
@@ -102,7 +109,7 @@ describe('summarizePendingSchemaWork (#3954)', () => {
     const summary = summarizePendingSchemaWork([...ADDITIVE, ...IN_PLACE]);
     expect(summary).toContain('1 table(s) to create');
     expect(summary).toContain('1 column(s) to add');
-    expect(summary).toContain('3 datetime column(s) to converge in place');
-    expect(summary).toContain('~1,234,609 rows');
+    expect(summary).toContain('5 temporal column(s) to converge in place');
+    expect(summary).toContain('~1,234,625 rows');
   });
 });

packages/cli/src/utils/schema-migrate.ts

@@ -285,17 +285,14 @@ export function renderPendingSchemaWork(pending: PendingSchemaWork[]): void {
   if (inPlace.length > 0) {
     console.log(`  ${chalk.bold('In place (existing rows converged when you apply)')}`);
     for (const p of inPlace) {
-      const label = p.kind === 'normalize_datetime_storage'
-        ? 'normalize_datetime_storage'
-        : 'widen_datetime_columns';
       // A MySQL widen is `ALTER … MODIFY`, i.e. a full table rebuild holding a
       // metadata lock — worth saying outright, not just implying via the count.
-      const cost = p.kind === 'widen_datetime_columns'
+      const cost = p.kind === 'widen_datetime_columns' || p.kind === 'widen_time_columns'
         ? `${formatRows(p.rows)} row table rebuild`
         : `${formatRows(p.rows)} row update(s)`;
       console.log(
         `    ${chalk.yellow('~')} ${chalk.yellow(p.table)} ` +
-        `${chalk.dim(`[${label}: ${p.columns.join(', ')} — ${cost}]`)}`,
+        `${chalk.dim(`[${p.kind}: ${p.columns.join(', ')} — ${cost}]`)}`,
       );
     }
     console.log('');
@@ -320,7 +317,7 @@ export function summarizePendingSchemaWork(pending: PendingSchemaWork[]): string
   if (inPlace.length > 0) {
     const cols = inPlace.reduce((n, p) => n + p.columns.length, 0);
     const rows = inPlace.reduce((n, p) => n + (p.rows ?? 0), 0);
-    parts.push(`${cols} datetime column(s) to converge in place (~${formatRows(rows)} rows)`);
+    parts.push(`${cols} temporal column(s) to converge in place (~${formatRows(rows)} rows)`);
   }
   return parts.join(', ');
 }

packages/plugins/driver-sql/src/legacy-datetime-storage.testkit.ts

@@ -39,6 +39,24 @@ export class LegacyStorageDriver extends SqlDriver {
     this.canonicalDatetimeFields[table]?.delete(field);
   }
 
+  /**
+   * The `Field.time` twin (#3994): insert raw pre-canonical time values and
+   * clear the column's canonical marker.
+   */
+  async seedLegacyTimeRows(
+    table: string,
+    field: string,
+    rows: Array<Record<string, unknown>>,
+  ): Promise<void> {
+    await this.knex(table).insert(rows);
+    this.forgetCanonicalTime(table, field);
+  }
+
+  /** Drop the "already backfilled" marker for one `Field.time` column. */
+  forgetCanonicalTime(table: string, field: string): void {
+    this.canonicalTimeFields[table]?.delete(field);
+  }
+
   /** Raw stored form of a column, for asserting on the fixture's premise. */
   async storedForms(table: string, field: string): Promise<Array<{ id: string; type: string; value: unknown }>> {
     const res: any = await this.knex.raw(

packages/plugins/driver-sql/src/schema-drift.ts

@@ -133,23 +133,26 @@ export interface PendingSchemaWork {
 /**
  * What kind of physical work a {@link PendingSchemaWork} entry represents.
  *
- * The first two are purely additive and never touch existing rows. The datetime
- * pair is NOT: `normalize_datetime_storage` rewrites rows in place (the SQLite
- * canonical-UTC backfill) and `widen_datetime_columns` rebuilds a column (the
- * MySQL `TIMESTAMP` → `DATETIME(3)` widening) — both from #3912/#3942. They are
- * rendered under their own heading for that reason: the additive section tells
- * the operator the work is never data-losing, and that claim must not silently
- * come to cover a row rewrite.
+ * The first two are purely additive and never touch existing rows. The rest are
+ * NOT: `normalize_datetime_storage` / `normalize_time_storage` rewrite rows in
+ * place (the SQLite canonical-text backfills, #3912/#3994) and
+ * `widen_datetime_columns` / `widen_time_columns` rebuild a column (the MySQL
+ * `TIMESTAMP` → `DATETIME(3)` and `TIME` → `TIME(3)` widenings, #3942/#3994).
+ * They are rendered under their own heading for that reason: the additive
+ * section tells the operator the work is never data-losing, and that claim must
+ * not silently come to cover a row rewrite.
  */
 export type PendingSchemaWorkKind =
   | 'create_table'
   | 'add_columns'
   | 'normalize_datetime_storage'
-  | 'widen_datetime_columns';
+  | 'normalize_time_storage'
+  | 'widen_datetime_columns'
+  | 'widen_time_columns';
 
 /** True for the kinds that rewrite or rebuild existing data rather than adding to it. */
 export function isInPlaceSchemaWork(kind: PendingSchemaWorkKind): boolean {
-  return kind === 'normalize_datetime_storage' || kind === 'widen_datetime_columns';
+  return kind !== 'create_table' && kind !== 'add_columns';
 }
 
 /** Ops that act on an index rather than a column — reconciled without a table rebuild. */