fix(cli,driver-sql): os migrate plan lists the datetime storage convergence (#3954) (#3964)

The datetime canonicalisation (#3912/#3942) added two steps to initObjects'
physical path: a row-rewriting backfill on SQLite and a TIMESTAMP →
DATETIME(3) column rebuild on MySQL. Both already respected the DDL
deferral (#3917), so plan performed neither and apply performed both — the
behaviour was never wrong. The reporting was.

PendingSchemaWork.kind could only express create_table / add_columns, so
neither step could appear in the plan. An operator saw two added columns,
confirmed, and apply additionally rewrote every row of a datetime column —
or took a metadata lock to rebuild one on a large table. The plan promises
to show what apply will do, so anything added to initObjects' physical
path has to be representable there; the type now says so.

- Two new kinds, normalize_datetime_storage and widen_datetime_columns,
  plus an optional `rows` carrying how much data the step touches:
  row-writes for the backfill, the table's size for the rebuild, because
  ALTER … MODIFY is a full rebuild holding a metadata lock and that is the
  number deciding "now" versus "in a maintenance window".
- previewDeferredSchemaWork measures both without performing either. Each
  probe reuses the exact predicate its migration uses — the backfill's
  whole WHERE, the widening's own information_schema filter, extracted to
  legacyMysqlTimestampColumns and shared — so the plan and the apply
  cannot name different sets. A probe that cannot run is swallowed to an
  unlisted step, never to a failed plan.
- The CLI renders them under their own heading rather than folding them
  into the additive section, whose "created when you apply" framing
  carries an implicit promise that the work is never data-losing.
  summarizePendingSchemaWork never omits in-place work, and is unchanged
  when there is none. The documented sample output in deployment/cli.mdx
  gains the section too.

Verified on SQLite and against a real MariaDB 10.11: plan lists the
columns and row count without touching the schema, apply performs exactly
that, re-plan is empty.

Closes #3954

Author: os-zhuang

.changeset/migrate-plan-lists-datetime-convergence.md

@@ -0,0 +1,31 @@
+---
+"@objectstack/driver-sql": minor
+"@objectstack/cli": patch
+---
+
+fix(cli,driver-sql): `os migrate plan` lists the datetime storage convergence (#3954)
+
+The datetime canonicalisation (#3912/#3942) added two steps to `initObjects`'
+physical path: a row-rewriting backfill on SQLite and a `TIMESTAMP` →
+`DATETIME(3)` column rebuild on MySQL. Both already respected the DDL deferral,
+so `plan` performed neither and `apply` performed both — the behaviour was never
+wrong. The reporting was.
+
+`PendingSchemaWork` could only express `create_table` / `add_columns`, so an
+operator saw a plan listing two added columns, confirmed it, and `apply`
+additionally rewrote every row of a datetime column — or took a metadata lock to
+rebuild one on a large table. The plan promises to show what apply will do.
+
+- `PendingSchemaWork.kind` gains `normalize_datetime_storage` and
+  `widen_datetime_columns`, plus an optional `rows` carrying how much data the
+  step touches: row-writes for the backfill, the table's size for the rebuild —
+  the number that decides "now" versus "in a maintenance window".
+- `previewDeferredSchemaWork()` measures both without performing either, reusing
+  the exact predicate each migration uses (the backfill's whole `WHERE`, the
+  widening's own `information_schema` filter) so the plan and the apply cannot
+  name different sets. A probe that cannot run is swallowed to "unlisted", never
+  to a failed plan.
+- The CLI renders them under their own heading rather than folding them into the
+  additive section, whose "created when you apply" framing carries an implicit
+  promise that the work is never data-losing. `summarizePendingSchemaWork` — the
+  line read just before typing `y` — never omits in-place work.

content/docs/deployment/cli.mdx

@@ -511,12 +511,27 @@ performed. So `plan` really is a dry run, and everything `apply` is about to do
     + crm_quote [create_table, 9 column(s)]
     + crm_contact [add_columns: nickname, region]
 
+  In place (existing rows converged when you apply)
+    ~ crm_contact [normalize_datetime_storage: signed_at — 1,240 row update(s)]
+
   Safe (loosening — applied without --allow-destructive)
     ✓ crm_contact.email [relax_not_null]
 ```
 
 Answering `n` leaves the database exactly as it was.
 
+The two upper sections differ in a way worth reading carefully. **New** is
+purely additive — it creates tables and columns and never touches a row. **In
+place** rewrites existing data: the storage-form convergence a `Field.datetime`
+column needs when the database predates the canonical UTC storage (ADR-0053
+addendum D-B1..D-B4). It carries a row count because that is the number
+deciding whether to run it now; on MySQL it reads `widen_datetime_columns` and
+is an `ALTER … MODIFY` table rebuild that holds a metadata lock for its
+duration.
+
+Both are safe to apply — the convergence preserves every stored instant and is
+idempotent — but only the second takes time proportional to your data.
+
 #### Occupancy check (SQLite)
 
 A running `os dev` / `os serve` holding the same SQLite file open is the usual

packages/cli/src/utils/schema-migrate.pending-render.test.ts

@@ -0,0 +1,108 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * #3954 — how `os migrate plan` renders the pending work.
+ *
+ * The additive section carries an implicit promise: what it lists is created,
+ * never data-losing, and needs no `--allow-destructive` thought. The datetime
+ * convergence (#3912/#3942) rewrites rows and rebuilds columns, so folding it
+ * into that section would quietly extend the promise to cover a table rewrite.
+ * These tests pin the split, and pin that the summary line — the one an operator
+ * reads before typing "yes" — never omits in-place work.
+ */
+
+import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest';
+import {
+  renderPendingSchemaWork,
+  summarizePendingSchemaWork,
+  type PendingSchemaWork,
+} from './schema-migrate.js';
+
+let lines: string[];
+
+beforeEach(() => {
+  lines = [];
+  vi.spyOn(console, 'log').mockImplementation((...args: unknown[]) => {
+    lines.push(args.map(String).join(' '));
+  });
+});
+
+afterEach(() => {
+  vi.restoreAllMocks();
+});
+
+const out = () => lines.join('\n');
+
+const ADDITIVE: PendingSchemaWork[] = [
+  { table: 'widgets', kind: 'create_table', columns: ['sku', 'qty'] },
+  { table: 'orders', kind: 'add_columns', columns: ['note'] },
+];
+
+const IN_PLACE: PendingSchemaWork[] = [
+  { table: 'evt', kind: 'normalize_datetime_storage', columns: ['at'], rows: 1234567 },
+  { table: 'legacy', kind: 'widen_datetime_columns', columns: ['at', 'created_at'], rows: 42 },
+];
+
+describe('renderPendingSchemaWork (#3954)', () => {
+  it('renders nothing at all when there is nothing pending', () => {
+    renderPendingSchemaWork([]);
+    expect(out()).toBe('');
+  });
+
+  it('keeps the additive section exactly as it was when only additive work is pending', () => {
+    renderPendingSchemaWork(ADDITIVE);
+    expect(out()).toContain('New (additive — created when you apply)');
+    expect(out()).toContain('widgets');
+    expect(out()).toContain('[create_table, 2 column(s)]');
+    expect(out()).toContain('[add_columns: note]');
+    // No second heading appears when there is no in-place work.
+    expect(out()).not.toContain('In place');
+  });
+
+  it('puts the datetime convergence under its OWN heading, not the additive one', () => {
+    renderPendingSchemaWork(IN_PLACE);
+    expect(out()).toContain('In place (existing rows converged when you apply)');
+    // The additive heading claims the work is never data-losing; a row rewrite
+    // must never be listed beneath it.
+    expect(out()).not.toContain('New (additive');
+  });
+
+  it('names the columns and the size of each in-place step', () => {
+    renderPendingSchemaWork(IN_PLACE);
+    expect(out()).toContain('normalize_datetime_storage: at');
+    expect(out()).toContain('1,234,567 row update(s)');
+    expect(out()).toContain('widen_datetime_columns: at, created_at');
+    // A MySQL widen is ALTER … MODIFY — a rebuild, said outright.
+    expect(out()).toContain('42 row table rebuild');
+  });
+
+  it('shows both sections when both kinds are pending', () => {
+    renderPendingSchemaWork([...ADDITIVE, ...IN_PLACE]);
+    expect(out()).toContain('New (additive — created when you apply)');
+    expect(out()).toContain('In place (existing rows converged when you apply)');
+  });
+
+  it('reads an unmeasured count as unknown rather than zero', () => {
+    renderPendingSchemaWork([{ table: 'evt', kind: 'normalize_datetime_storage', columns: ['at'] }]);
+    expect(out()).toContain('? row update(s)');
+    expect(out()).not.toContain('0 row update(s)');
+  });
+});
+
+describe('summarizePendingSchemaWork (#3954)', () => {
+  it('is unchanged for purely additive work', () => {
+    expect(summarizePendingSchemaWork(ADDITIVE)).toBe('1 table(s) to create, 1 column(s) to add');
+  });
+
+  it('is unchanged when nothing is pending', () => {
+    expect(summarizePendingSchemaWork([])).toBe('0 table(s) to create, 0 column(s) to add');
+  });
+
+  it('never omits in-place work — this is the line read before confirming', () => {
+    const summary = summarizePendingSchemaWork([...ADDITIVE, ...IN_PLACE]);
+    expect(summary).toContain('1 table(s) to create');
+    expect(summary).toContain('1 column(s) to add');
+    expect(summary).toContain('3 datetime column(s) to converge in place');
+    expect(summary).toContain('~1,234,609 rows');
+  });
+});

packages/cli/src/utils/schema-migrate.ts

@@ -14,6 +14,7 @@
  */
 import chalk from 'chalk';
 import type { ManagedDriftEntry, DriftCategory, PendingSchemaWork } from '@objectstack/driver-sql';
+import { isInPlaceSchemaWork } from '@objectstack/driver-sql';
 import { describeDriverConnection } from './connection-display.js';
 
 export type { PendingSchemaWork };
@@ -251,29 +252,75 @@ export function summarize(drift: ManagedDriftEntry[]): string {
 }
 
 /**
- * Render the additive work the boot sync was held back from doing (#3917).
+ * Render the work the boot sync was held back from doing (#3917), in two
+ * sections split by whether it touches existing data (#3954).
  *
- * Deliberately its own section rather than a `DriftCategory`: this is not
- * divergence between metadata and an existing column — it is the create/add
- * that used to happen silently at boot, now shown before it runs. Purely
- * additive and never data-losing, so it carries no `--allow-destructive` gate.
+ * Deliberately its own block rather than a `DriftCategory`: this is not
+ * divergence between metadata and an existing column — it is what used to
+ * happen silently at boot, now shown before it runs.
+ *
+ * The split matters. The additive section tells the operator the work is never
+ * data-losing, and that promise must not quietly come to cover the datetime
+ * convergence, which rewrites rows (SQLite) or rebuilds a column (MySQL). Those
+ * get their own heading, and their row counts, because "how long will this hold
+ * the table" is the question they raise and the additive kinds do not.
  */
 export function renderPendingSchemaWork(pending: PendingSchemaWork[]): void {
   if (pending.length === 0) return;
-  console.log(`  ${chalk.bold('New (additive — created when you apply)')}`);
-  for (const p of pending) {
-    const detail = p.kind === 'create_table'
-      ? `[create_table, ${p.columns.length} column(s)]`
-      : `[add_columns: ${p.columns.join(', ')}]`;
-    console.log(`    ${chalk.cyan('+')} ${chalk.cyan(p.table)} ${chalk.dim(detail)}`);
+
+  const additive = pending.filter((p) => !isInPlaceSchemaWork(p.kind));
+  const inPlace = pending.filter((p) => isInPlaceSchemaWork(p.kind));
+
+  if (additive.length > 0) {
+    console.log(`  ${chalk.bold('New (additive — created when you apply)')}`);
+    for (const p of additive) {
+      const detail = p.kind === 'create_table'
+        ? `[create_table, ${p.columns.length} column(s)]`
+        : `[add_columns: ${p.columns.join(', ')}]`;
+      console.log(`    ${chalk.cyan('+')} ${chalk.cyan(p.table)} ${chalk.dim(detail)}`);
+    }
+    console.log('');
+  }
+
+  if (inPlace.length > 0) {
+    console.log(`  ${chalk.bold('In place (existing rows converged when you apply)')}`);
+    for (const p of inPlace) {
+      const label = p.kind === 'normalize_datetime_storage'
+        ? 'normalize_datetime_storage'
+        : 'widen_datetime_columns';
+      // A MySQL widen is `ALTER … MODIFY`, i.e. a full table rebuild holding a
+      // metadata lock — worth saying outright, not just implying via the count.
+      const cost = p.kind === 'widen_datetime_columns'
+        ? `${formatRows(p.rows)} row table rebuild`
+        : `${formatRows(p.rows)} row update(s)`;
+      console.log(
+        `    ${chalk.yellow('~')} ${chalk.yellow(p.table)} ` +
+        `${chalk.dim(`[${label}: ${p.columns.join(', ')} — ${cost}]`)}`,
+      );
+    }
+    console.log('');
   }
-  console.log('');
+}
+
+/** `rows` is optional on the type; an unmeasured count reads as unknown, not zero. */
+function formatRows(rows: number | undefined): string {
+  return rows === undefined ? '?' : rows.toLocaleString('en-US');
 }
 
 export function summarizePendingSchemaWork(pending: PendingSchemaWork[]): string {
   const creates = pending.filter((p) => p.kind === 'create_table').length;
   const columns = pending
     .filter((p) => p.kind === 'add_columns')
     .reduce((n, p) => n + p.columns.length, 0);
-  return `${creates} table(s) to create, ${columns} column(s) to add`;
+  const parts = [`${creates} table(s) to create`, `${columns} column(s) to add`];
+
+  // Only mentioned when there is some, so the common in-sync summary is
+  // unchanged — but never omitted when there is, which is the #3954 point.
+  const inPlace = pending.filter((p) => isInPlaceSchemaWork(p.kind));
+  if (inPlace.length > 0) {
+    const cols = inPlace.reduce((n, p) => n + p.columns.length, 0);
+    const rows = inPlace.reduce((n, p) => n + (p.rows ?? 0), 0);
+    parts.push(`${cols} datetime column(s) to converge in place (~${formatRows(rows)} rows)`);
+  }
+  return parts.join(', ');
 }

packages/plugins/driver-sql/src/index.ts

@@ -21,6 +21,7 @@ export {
   diffManagedIndexes,
   expectedIndexes,
   isIndexDriftOp,
+  isInPlaceSchemaWork,
   isManagedIndexName,
   legacyUniqueIndexNames,
   legacyUniqueReplacements,
@@ -38,6 +39,7 @@ export type {
   ExpectedIndex,
   LegacyUniqueReplacement,
   PendingSchemaWork,
+  PendingSchemaWorkKind,
   FieldDef as DriftFieldDef,
 } from './schema-drift.js';
 

packages/plugins/driver-sql/src/schema-drift.ts

@@ -91,20 +91,60 @@ export type DriftOp =
     };
 
 /**
- * Physical schema work the *additive* boot sync is holding back (#3917).
+ * Physical work the boot sync is holding back (#3917).
  *
  * Distinct from {@link DriftOp}: drift is divergence between metadata and an
  * EXISTING column/index that only a deliberate reconcile may resolve, whereas
- * this is the create-table / add-column work `initObjects` performs on its own
- * — captured rather than executed while the driver runs with DDL deferred, so
- * `os migrate plan` can show it and `os migrate apply` can gate it behind the
- * confirmation prompt.
+ * this is the work `initObjects` performs on its own — captured rather than
+ * executed while the driver runs with DDL deferred, so `os migrate plan` can
+ * show it and `os migrate apply` can gate it behind the confirmation prompt.
+ *
+ * The plan's promise is that it shows what `apply` will do, so **anything added
+ * to `initObjects`' physical path has to be representable here** — otherwise an
+ * operator confirms a two-column plan and `apply` additionally rewrites a table.
+ * That is the gap #3954 closed for the datetime convergence; keep it closed.
  */
 export interface PendingSchemaWork {
   table: string;
-  kind: 'create_table' | 'add_columns';
-  /** Declared columns for a create; the missing ones for an add. */
+  kind: PendingSchemaWorkKind;
+  /**
+   * Declared columns for a create; the missing ones for an add; the columns
+   * being converged for the two datetime steps.
+   */
   columns: string[];
+  /**
+   * How much data the step touches, when that is knowable up front and worth
+   * knowing — absent for the additive kinds, which touch none.
+   *
+   * For `normalize_datetime_storage` it is the number of row-writes (summed
+   * across `columns`, since each is its own `UPDATE`). For
+   * `widen_datetime_columns` it is the table's row count, because MySQL's
+   * `ALTER … MODIFY` is a full rebuild holding a metadata lock — which is the
+   * number that decides "now" versus "in a maintenance window".
+   */
+  rows?: number;
+}
+
+/**
+ * What kind of physical work a {@link PendingSchemaWork} entry represents.
+ *
+ * The first two are purely additive and never touch existing rows. The datetime
+ * pair is NOT: `normalize_datetime_storage` rewrites rows in place (the SQLite
+ * canonical-UTC backfill) and `widen_datetime_columns` rebuilds a column (the
+ * MySQL `TIMESTAMP` → `DATETIME(3)` widening) — both from #3912/#3942. They are
+ * rendered under their own heading for that reason: the additive section tells
+ * the operator the work is never data-losing, and that claim must not silently
+ * come to cover a row rewrite.
+ */
+export type PendingSchemaWorkKind =
+  | 'create_table'
+  | 'add_columns'
+  | 'normalize_datetime_storage'
+  | 'widen_datetime_columns';
+
+/** True for the kinds that rewrite or rebuild existing data rather than adding to it. */
+export function isInPlaceSchemaWork(kind: PendingSchemaWorkKind): boolean {
+  return kind === 'normalize_datetime_storage' || kind === 'widen_datetime_columns';
 }
 
 /** Ops that act on an index rather than a column — reconciled without a table rebuild. */