ref(migrations): Align journals with SQL cutover

Drop the retired core and plugin data migration capsules after the bridge-release cutover. Keep mixed journal support for future entries while preserving the current core SQL ledger and upgrade guard.

Author: dcramer

packages/docs/src/content/docs/cli/upgrade.md

@@ -1,8 +1,8 @@
 ---
 title: "junior upgrade"
-description: "Apply Junior schema and data migrations."
+description: "Apply Junior and plugin schema or data migrations."
 type: reference
-summary: Move configured SQL schemas and persisted state forward after upgrades.
+summary: Bring the configured Junior SQL database up to the installed schema.
 prerequisites:
   - /start-here/quickstart/
 related:
@@ -11,100 +11,65 @@ related:
   - /cli/snapshot-create/
 ---
 
-Use `junior upgrade` after installing a Junior release that includes schema or
-data migrations. The command mutates the configured SQL database and state
-stores, so run it from the same app environment that has the production state
-and SQL environment variables configured for the deployment you are upgrading.
+Use `junior upgrade` after installing a new Junior release. The command applies
+pending SQL schema and TypeScript data entries from the ordered Drizzle journals
+owned by Junior and its enabled plugins.
 
 ## Usage
 
-Run it from a project that already has `@sentry/junior` installed:
+Run the command from your Junior app:
 
 ```bash
 pnpm exec junior upgrade
 ```
 
-The command takes no extra arguments.
+The command takes no arguments. It migrates Junior first, then enabled plugins.
+Already-applied entries are recognized as up to date and are not rerun.
 
-## What it does
+## Upgrade bridge for older databases
 
-`junior upgrade` runs migrations sequentially. Core and plugin migration
-directories use Drizzle Kit's ordered journal and may contain either generated
-SQL schema migrations or TypeScript data migrations targeting a versioned
-host capability API. Current
-upgrade work includes:
+An existing Junior database without core Drizzle migration history must
+complete the `0.107.1` upgrade before upgrading to a later release. Before
+starting this bridge, block new ingress, drain active and resumable work, stop
+all old workers and queue consumers, and keep them stopped until the later
+release is ready. If Junior reports this unsupported database state:
 
-- Apply core and enabled-plugin schema and data journal entries.
-- Rewrite retained turn-session records from legacy storage shapes before the
-  new runtime reads them.
-- Move legacy `junior:conversation-work:*` Redis state into the newer conversation record and index state used by the durable worker and dashboard feed.
-- Backfill retained conversation records into the shared Junior SQL database. The upgrade requires `DATABASE_URL`.
-- Apply the SQL schema cutover and rewrite legacy Pi-message rows into canonical conversation events.
-- Repair legacy token and estimated-cost rollups from durable SQL conversation events in bounded batches. Conversations that are active during the repair are left unchanged and can be repaired by rerunning the command after they become idle.
+1. Install `@sentry/junior@0.107.1`.
+2. Run `pnpm exec junior upgrade` and confirm it completes successfully.
+3. Restore the intended Junior version.
+4. Run `pnpm exec junior upgrade` again.
+5. Deploy the intended version, then restart workers and reopen ingress.
 
-Completed journal entries are tracked individually, and TypeScript migrations
-can checkpoint progress for safe retries. Legacy backfills remain idempotent:
-rerunning them skips records that were already moved, removes stale legacy
-index entries that no longer have a record, and upserts SQL conversation rows.
-The conversation-history import paginates through every conversation in the
-retained activity index; orphaned or expired Redis keys outside that index are
-not treated as retained history. After cutover, SQL owns durable conversation
-metadata and event history.
-
-## Hard-cutover upgrade sequence
-
-The canonical conversation-event cutover is not rolling-compatible. Do not run it inside a Vercel build while the previous deployment can still accept work. Use this operator sequence:
-
-1. Block new ingress and enqueueing while leaving the previous release's workers and continuation consumers running.
-2. Let existing work drain, then verify that no turns remain running or awaiting resume.
-3. Stop every old worker, queue consumer, and heartbeat. Keep the old deployment stopped for the rest of the procedure.
-4. Run the upgrade from an operator environment with the production `REDIS_URL`, `JUNIOR_STATE_KEY_PREFIX`, and `DATABASE_URL`.
-5. Confirm the history import and message-event seal complete with no missing rows.
-6. Run `junior check`, deploy the new release, and only then reopen ingress and start the new workers.
-
-Run the upgrade as a separate operator command:
-
-```bash
-pnpm exec junior upgrade
-pnpm exec junior check
-```
-
-The checkpoint and message-event rewrites fail closed if resumable work remains. After the drain succeeds, each rewrite invalidates stale resume state before changing physical event positions. Checkpoint normalization closes deletion gaps, and the message migration resequences the streams it changes while preserving reporting summaries.
-
-If the command exits nonzero, leave the deployment stopped, correct the reported state, and rerun it. Do not restart workers after only part of the sequence completes.
+Fresh databases without Junior tables do not require the bridge release.
 
 ## Example output
 
-Typical logs look like this:
+An already-current database reports its migrations as existing:
 
 ```text
-Running Junior upgrade migrations...
-Running migration core-migrations...
-Finished migration core-migrations: scanned=10 migrated=6 existing=4 missing=0 skipped=0
-Running migration plugin-migrations...
-Finished migration plugin-migrations: scanned=8 migrated=8 existing=0 missing=0 skipped=0
-Junior upgrade complete.
+Checking database migrations...
+  junior: up to date (7 migrations)
+  junior-github: up to date (4 migrations)
+  junior-memory: up to date (5 migrations)
+  junior-scheduler: up to date (2 migrations)
+Database is up to date (18 migrations).
 ```
 
 ## Failure behavior
 
-If the configured state store is unavailable or a legacy record is malformed, the CLI exits non-zero and prints the underlying error:
-
-```text
-junior command failed: Legacy conversation work state is invalid for slack:C123:1712345.0001
-```
-
-Treat that as a deploy blocker for the affected environment. Check `REDIS_URL`, `JUNIOR_STATE_KEY_PREFIX`, `DATABASE_URL`, and the reported legacy record before retrying.
+The command exits nonzero when it cannot connect to SQL, encounters an
+unsupported pre-Drizzle database, or a migration fails. Treat that as a deploy
+blocker: correct the reported database or migration error, then rerun the
+command.
 
 ## Verification
 
-After running the command:
-
-1. Confirm the final log line includes `Junior upgrade complete`.
-2. Confirm `backfill-conversation-events-sql` scanned the complete retained activity index and did not stop at one page.
-3. Confirm `move-conversation-messages-to-events` reports `missing=0`. The runtime now uses the copied events. The legacy message table remains available so a later upgrade can recover messages written by old workers during deployment.
-4. Run `pnpm exec junior check` before building or deploying the app.
+Confirm that the command exits successfully and lists `junior` plus each
+enabled plugin that owns migrations. The final line reports whether the
+database was already current or how many migrations were applied.
 
 ## Next step
 
-Run [junior check](/cli/check/) after the upgrade, then continue with [junior snapshot create](/cli/snapshot-create/) if your plugins need sandbox dependencies.
+Run [junior check](/cli/check/) before deploying, then continue with
+[junior snapshot create](/cli/snapshot-create/) if your plugins need sandbox
+dependencies.

packages/junior-migrations/README.md

@@ -32,7 +32,8 @@ schema snapshot while preserving the mixed journal order.
 Drizzle Kit remains the supported authoring tool. Drizzle ORM's stock
 `migrate()` function is not a supported executor for mixed journals because it
 requires every entry to have a SQL file. Call `runMigrationJournal` instead and
-provide the host database adapter, state adapter, and TypeScript loader. The
+provide the host database adapter and, for TypeScript entries, a context and
+loader. The
 same database adapter drives the journal ledger and is exposed as
 `context.database`, so migration files never own connection or driver setup.
 
@@ -44,8 +45,8 @@ so it must not be used to upgrade an existing installation.
 
 The runner rejects runtime imports of application source, relative modules,
 and unversioned `@sentry/junior` modules. Migrations may import the append-only
-`@sentry/junior/migration-helpers/v1` surface for parsing primitives, adapters,
-stores, and key resolution. One-off migration decisions and data transforms
+`@sentry/junior/migration-helpers/v1` surface for stable parsing primitives and
+other reusable infrastructure. One-off migration decisions and data transforms
 must still remain in the journal entry. Add a new helper or capability version
 rather than changing an existing contract.
 

packages/junior/migrations/README.md

@@ -1,4 +1,4 @@
-# SQL migrations
+# Migrations
 
 `src/db/schema.ts` is the Drizzle schema entrypoint. This directory is the
 append-only history used to bring an existing database to that schema. It
@@ -33,21 +33,20 @@ TypeScript migrations target the versioned migration capability API and must
 not import Junior runtime internals or current feature schemas. Their complete
 implementation remains in the migration file and uses only the stable
 database, state, Redis, and progress capabilities supplied by the runner. The
-host database adapter owns connections, drivers, transactions, and locks so
-those infrastructure modules are not frozen into every migration.
+host database adapter owns connections, drivers, transactions, and locks.
 
 Reusable infrastructure belongs in the append-only
-`@sentry/junior/migration-helpers/v1` export. It may provide stable parsers,
-stores, adapter projections, and key resolution, but must not implement a
-specific data migration. The journal entry remains the only owner of one-off
-record transformations.
+`@sentry/junior/migration-helpers/v1` export. It may provide stable parsing and
+other reusable infrastructure, but must not implement a specific data
+migration. The journal entry remains the only owner of one-off record
+transformations.
 
 The `0000_initial.sql` baseline represents the schema already deployed by the
-pre-Drizzle Junior migration runner. During upgrade, existing installations
-adopt that baseline once; new installations execute it normally. All later
-migrations are applied by Junior in journal order. Drizzle ORM's stock
-`migrate()` function is not compatible with TypeScript entries in this folder.
-
-Migration loading, locking, and legacy baseline adoption live in
-`src/chat/conversations/sql/migrations.ts`. Their integration coverage lives in
-`tests/integration/conversation-sql.test.ts`.
+pre-Drizzle Junior migration runner. Existing databases without core Drizzle
+history must first run the documented bridge release; current upgrades do not
+infer or adopt legacy state. Fresh databases execute the complete journal
+normally. Drizzle ORM's stock `migrate()` function is not compatible with
+TypeScript entries in this folder.
+
+Migration loading, locking, and the bridge-release guard live in
+`src/chat/conversations/sql/migrations.ts`.