feat(core,platform-objects,spec): ADR-0119 D2 — the migration-journal runner (#4617)

A migration killed mid-run is now either resumable to completion or
compensable to clean, with journal rows proving which.

ADR-0119 D1 made `engine.transaction()` contract-reachable, which answers
multi-write atomicity that fits in one transaction. Migration-class work does
not fit: a million-row backfill cannot hold one write-lock, driver-memory's
`beginTransaction` deep-clones the whole database, `transaction()` binds the
default driver only, and a KILLED process defeats in-process rollback
entirely. So the unit of atomicity is the chunk, and durability across chunks
is a journal.

- `runMigrationJournal` (@objectstack/core): preflight dry-run across every
  step before any step writes; chunked writes each inside `engine.transaction()`;
  LIFO compensation newest-first on failure; re-entrant forward recovery under
  a per-plan `onCrash` policy; at-least-once with an `attempt` counter, reusing
  bulk-write.ts's delivery contract rather than re-deriving it.
- `sys_migration_journal` (@objectstack/platform-objects): rows keyed
  (run_id, seq) under a unique index, registered unconditionally beside
  sys_migration so recovery is discoverable with zero host wiring (ADR-0078).
  Distinct in grain from sys_migration, which holds one verdict per named
  migration; this holds many rows per run.
- Row contract + object-name constant in @objectstack/spec/system, so core's
  runner writes the journal without depending on platform-objects.

The invariant carrying the design: `chunk_done(i)` is written INSIDE the
chunk's transaction so `done ⇔ committed` holds by construction, while
`chunk_started(i)` is written autonomously before it. That asymmetry gives
`started ∧ ¬done` exactly one meaning — outcome unknown — which is the only
state a crash leaves and the only state recovery reasons about.

The runner refuses rather than degrades: no rollback capability, a failed
preflight, an uncompensable plan declaring onCrash:'compensate', or a resume
whose plan hash disagrees with the journal. A compensation failure halts and
is journalled, and the run ends `failed` rather than `compensated` — a
database in a state no clean story covers must not be reported as a tidy
rollback.

`engineCanRollBack` is now shared: the two-level probe was the same condition
in this runner and in batchData's atomic gate, and two copies drift by one
clause and leave one caller believing it has atomicity it does not have. It
moves to @objectstack/core as a type predicate; metadata-protocol imports it.

Boot reconciliation and `os migrate resume` land separately; the discovery
primitive they consume, `findInterruptedRuns`, is exported here.

Docs: ADR-0118 (plugin-reachable transactions) is renumbered ADR-0119. It
merged a day after an unrelated ADR-0118 (非用户 actor 的平台契约), and the
earlier merge holds the number. Its Status line now cites the implementing
PR and its tests instead of a dangling "this PR", and records that D2/D3
remain unimplemented.

Refs: ADR-0119 D2, #4617, #4612, ADR-0034, ADR-0060, ADR-0078, ADR-0117 D8

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NKcGqCYCCpMkB5UW8jNPXx

Author: claude

.changeset/adr-0119-d2-migration-journal.md

@@ -0,0 +1,78 @@
+---
+"@objectstack/spec": minor
+"@objectstack/platform-objects": minor
+"@objectstack/core": minor
+"@objectstack/metadata-protocol": patch
+---
+
+feat(core,platform-objects,spec): the ADR-0119 D2 migration-journal runner — a migration killed mid-run is resumable to completion or compensable to clean, with journal rows proving which (#4617)
+
+**The gap D1 left open.** ADR-0119 D1 made `engine.transaction()` reachable
+through the contract, which is the right answer for multi-write atomicity that
+fits in one transaction. Migration-class work does not fit: a million-row
+backfill cannot hold one write-lock for its duration, `driver-memory`'s
+`beginTransaction` deep-clones the entire database (O(db) per begin),
+`ObjectQL.transaction()` binds the **default driver only** so a multi-datasource
+migration silently commits part of its work outside it, and a process **killed**
+— as distinct from a thrown error — defeats in-process rollback entirely. So the
+unit of atomicity is the *chunk*, and durability across chunks is a journal.
+
+Four consumers had each converged on the same four moves — dry-run preflight,
+undo journal, LIFO compensation, re-entrant forward recovery (ADR-0105 D13
+promotion, ADR-0117 D8's ownership backfill, the org lifecycle transitions, and
+D10 master-data distribution #4585). One copy is engineering; four is platform
+debt, and the fourth author would have had to rediscover the invariant below
+from scratch.
+
+**New: `runMigrationJournal` (`@objectstack/core`).** Preflight runs every
+step's read-only validator before any step writes, so a plan that would fail at
+step 3 has not written step 1. Rows are chunked per the `bulk-write.ts`
+discipline; each chunk's writes run inside `engine.transaction()`. On failure,
+committed chunks are compensated newest-first, each in its own transaction. On
+restart, a rediscovered run resumes forward from the first chunk lacking
+`chunk_done`, or unwinds, per the plan's `onCrash` policy. Forward and
+compensate callbacks receive an `attempt` counter; `attempt > 1` means the prior
+outcome is UNKNOWN and the callback must recheck by natural key before
+re-writing — the same at-least-once contract `bulk-write.ts` already documents,
+reused rather than re-derived.
+
+**The invariant that carries the design:** `chunk_done(i)` is written **inside**
+the chunk's own transaction, so `done ⇔ committed` holds by construction;
+`chunk_started(i)` is written autonomously **before** it. That asymmetry is what
+gives `started ∧ ¬done` exactly one meaning — *the outcome is unknown* — which
+is the only state a crash can leave and the only state recovery reasons about.
+Making both writes symmetric would look tidier and would destroy recovery.
+
+**New: `sys_migration_journal` (`@objectstack/platform-objects`).** Rows keyed
+`(run_id, seq)` under a unique index, so a resumed run that miscomputes its next
+sequence fails loudly rather than double-recording an event. Registered
+unconditionally alongside `sys_migration` because recovery must be discoverable
+with **zero host wiring** — a journal some kernels compose and others do not is
+a journal a boot scanner cannot rely on (ADR-0078). Distinct in grain from
+`sys_migration`, which holds one durable verdict per named migration; this holds
+many rows per *run*. Read-only over the API; writes go through the runner in
+system context.
+
+**The runner refuses rather than degrades**, in four places: the runtime cannot
+roll back; any preflight fails; the plan declares `onCrash: 'compensate'` but a
+step cannot compensate; or a resume's plan hash disagrees with the journal
+(resuming a changed plan would apply chunk boundaries the journal never
+described). A compensation failure halts and is journalled — never swallowed —
+and the run ends `failed`, not `compensated`, because a database in a state no
+clean story covers must not be reported as a tidy rollback.
+
+**`engineCanRollBack` is now shared.** The two-level probe (engine method AND
+default-driver `beginTransaction`) was the same condition written twice — here
+and in `batchData`'s atomic gate. It now lives in `@objectstack/core` and
+`@objectstack/metadata-protocol` imports it, as a type predicate so callers do
+not each re-narrow the optional member by hand. Two copies of "can this runtime
+actually roll back?" drift by one clause and leave one caller believing it has
+atomicity it does not have.
+
+Boot reconciliation and `os migrate resume` land separately; `findInterruptedRuns`
+is the discovery primitive they will consume, and is exported here.
+
+**Docs:** ADR-0118 (plugin-reachable transactions) is renumbered **ADR-0119**.
+It merged one day after an unrelated ADR-0118 (非用户 actor 的平台契约) and the
+earlier merge holds the number; citations of "ADR-0118 D1/D2/D3/D4" written
+before 2026-08-03 mean the renumbered record.

…dr-0118-plugin-reachable-transactions.md …dr-0119-plugin-reachable-transactions.md.changeset/adr-0118-plugin-reachable-transactions.md renamed to .changeset/adr-0119-plugin-reachable-transactions.md .changeset/adr-0118-plugin-reachable-transactions.md renamed to .changeset/adr-0119-plugin-reachable-transactions.md

@@ -3,7 +3,7 @@
 "@objectstack/metadata-protocol": minor
 ---
 
-feat(spec,metadata-protocol): `IObjectQLEngine.transaction` joins the slot contract, and `batchData`'s `atomic` flag becomes real — rollback or refusal, never silent best-effort (ADR-0118 D1/D4, #4612)
+feat(spec,metadata-protocol): `IObjectQLEngine.transaction` joins the slot contract, and `batchData`'s `atomic` flag becomes real — rollback or refusal, never silent best-effort (ADR-0119 D1/D4, #4612)
 
 **D1 — the contract fix.** `ObjectQL.transaction()` — ADR-0034's ambient
 transaction, shipped since v8.0.0 — was reachable from plugin space only
@@ -53,7 +53,7 @@ If you were passing `atomic: true` and relying on partial results surviving a
 failure, that was the bug — switch to `atomic: false` (or omit it) for
 best-effort semantics.
 
-ADR-0118 also rules on two items landing separately: D2 specifies a
+ADR-0119 also rules on two items landing separately: D2 specifies a
 framework-owned migration-journal runner for multi-step migrations too large
 for one transaction, and D3 retires the declared-but-unimplemented
 `IDataEngine.batch?`.

content/docs/references/system/migration.mdx

@@ -34,8 +34,8 @@ irreversibly on migrated data gate on the flag instead of the version.
 ## TypeScript Usage
 
 ```typescript
-import { AddFieldOperation, ChangeSetSchema, CreateObjectOperation, DataMigrationFlagSchema, DeleteObjectOperation, ExecuteSqlOperation, MigrationDependencySchema, MigrationOperationSchema, ModifyFieldOperation, RemoveFieldOperation, RenameObjectOperation } from '@objectstack/spec/system';
-import type { ChangeSet, DataMigrationFlag, MigrationOperation } from '@objectstack/spec/system';
+import { AddFieldOperation, ChangeSetSchema, CreateObjectOperation, DataMigrationFlagSchema, DeleteObjectOperation, ExecuteSqlOperation, MigrationDependencySchema, MigrationJournalEventSchema, MigrationOperationSchema, ModifyFieldOperation, RemoveFieldOperation, RenameObjectOperation } from '@objectstack/spec/system';
+import type { ChangeSet, DataMigrationFlag, MigrationJournalEvent, MigrationOperation } from '@objectstack/spec/system';
 
 // Validate data
 const result = AddFieldOperation.parse(data);
@@ -153,6 +153,27 @@ Dependency reference to another migration that must run first
 | **package** | `string` | optional | Package that owns the dependency migration |
 
 
+---
+
+## MigrationJournalEvent
+
+One event in a migration run journal — the durable trace that lets a killed run be resumed forward or compensated back, with rows proving which
+
+### Properties
+
+| Property | Type | Required | Description |
+| :--- | :--- | :--- | :--- |
+| **run_id** | `string` | ✅ | Identifies one run. Rows are keyed (run_id, seq) |
+| **seq** | `integer` | ✅ | Monotonic per-run sequence. Ordering authority — wall-clock timestamps can tie or skew |
+| **kind** | `Enum<'run_started' \| 'chunk_started' \| 'chunk_done' \| 'compensated' \| 'run_done' \| 'run_failed'>` | ✅ | Event kind |
+| **migration_id** | `string` | optional | The named migration this run belongs to, when it has one — joins to sys_migration.id |
+| **plan_hash** | `string` | optional | On run_started: hash of the plan shape. A resume whose plan hash differs REFUSES rather than resuming a changed plan against an old journal |
+| **chunk_index** | `integer` | optional | On chunk_started / chunk_done / compensated: the run-global chunk index |
+| **attempt** | `integer` | optional | Which attempt produced this event. attempt > 1 means a prior outcome was unknown and the callback was asked to recheck by natural key |
+| **detail** | `string` | optional | JSON-encoded payload — the chunk plan on run_started, the error on run_failed / a failed compensation |
+| **created_at** | `string` | optional | Wall-clock stamp, for humans. Never the ordering authority — that is seq |
+
+
 ---
 
 ## MigrationOperation

…-transactions-and-honest-atomic-batch.md …-transactions-and-honest-atomic-batch.mddocs/adr/0118-plugin-reachable-transactions-and-honest-atomic-batch.md renamed to docs/adr/0119-plugin-reachable-transactions-and-honest-atomic-batch.md docs/adr/0118-plugin-reachable-transactions-and-honest-atomic-batch.md renamed to docs/adr/0119-plugin-reachable-transactions-and-honest-atomic-batch.md

@@ -1,6 +1,7 @@
-# ADR-0118: Multi-write atomicity is reachable through the contract, `atomic` means atomic or refuses, and migrations too big for one transaction get a journal runner
+# ADR-0119: Multi-write atomicity is reachable through the contract, `atomic` means atomic or refuses, and migrations too big for one transaction get a journal runner
 
-**Status**: Accepted (2026-08-02) — D1/D4 implemented in this PR; D2 tracked in [#4617](https://github.com/objectstack-ai/objectstack/issues/4617); D3 tracked in [#4618](https://github.com/objectstack-ai/objectstack/issues/4618)
+**Status**: Accepted (2026-08-02) — D1/D4 implemented in [#4623](https://github.com/objectstack-ai/objectstack/pull/4623): D1 in `packages/spec/src/contracts/objectql-engine.ts` (test `packages/objectql/src/protocol-batch-atomic.test.ts`), D4 in `packages/metadata-protocol/src/protocol.ts` (test `packages/metadata-protocol/src/protocol.batch-atomic.test.ts`). D2 tracked in [#4617](https://github.com/objectstack-ai/objectstack/issues/4617); D3 tracked in [#4618](https://github.com/objectstack-ai/objectstack/issues/4618) — neither is implemented, so this record is *not* wholly "implemented".
+**Renumbered**: published for one day as ADR-0118. Renumbered to 0119 because [ADR-0118 (非用户 actor 的平台契约)](./0118-non-user-actor-contract.md) merged first (10:37 vs 12:11 on 2026-08-02) and holds the number. Citations of "ADR-0118 D1/D2/D3/D4" written before 2026-08-03 mean this record.
 **Deciders**: ObjectStack Protocol Architects
 **Builds on**: [ADR-0034](./0034-transactional-writes-and-ambient-transaction.md) (the ambient `AsyncLocalStorage` transaction D1 declares — this ADR adds no mechanism to it), [ADR-0067](./0067-commit-history-and-rollback-for-ai-authoring.md) (D2 — the join-don't-nest rule that makes an outer transaction the sole owner of commit/rollback), [ADR-0049](./0049-no-unenforced-security-properties.md) (enforce-or-remove — the disposition method applied to `batch?` in D3 and to the `atomic` flag in D4), [ADR-0087](./0087-metadata-protocol-upgrade-contract.md) (D3's replayable migration chain — the metadata-side analogue of the data-side runner D2 specifies), [ADR-0008](./0008-metadata-repository-and-change-log.md) (the JSONL change log — the journal shape D2 deliberately does *not* reuse), [ADR-0060](./0060-conformance-ledger-platform-pattern.md) (framework-owned ledger pattern — the precedent for `sys_migration_journal`), [ADR-0117](./0117-owning-business-unit-record-stamp.md) (D8 — backfill plus a fail-closed enable gate, the migration posture D2 and D4 both inherit), [ADR-0078](./0078-no-silently-inert-metadata.md) (no silently inert declarations — why D2 rejects a pluggable journal store)
 **Consumers**: `@objectstack/spec` (`contracts/objectql-engine.ts`, `api/batch.zod.ts`), `@objectstack/metadata-protocol` (`protocol.ts`, `host-engine.ts`, `sys-metadata-repository.ts`), `@objectstack/objectql` (the implementation — unchanged by D1), `@objectstack/rest` (the `/batch` routes — unchanged), and for D2: `@objectstack/core`, `@objectstack/platform-objects`

packages/core/src/index.ts

@@ -30,6 +30,11 @@ export * from './utils/datetime.js';
 // Export the shared batched-write helper (framework#2678)
 export * from './utils/bulk-write.js';
 
+// Export the migration-journal runner (ADR-0119 D2, #4617) — chunk-atomic
+// migrations with durable recovery, plus the shared `engineCanRollBack` gate
+// that `@objectstack/metadata-protocol`'s atomic `batchData` also uses.
+export * from './utils/migration-journal.js';
+
 // Export the runtime filter-placeholder resolver (framework#3582)
 export * from './utils/filter-tokens.js';