Skip to content

Commit 5a06ffa

Browse files
authored
[db_metadata] Parse schema on migration, await backfill (#9878)
Non-atomic schema changes ARE real and CAN hurt you - Adds CRDB-specific preprocessing and `sqlparser` parsing to each schema change within Nexus - Verifies that each change contains "at most one" schema-modifying DDL statement - Adds a secondary transaction to each "apply schema change" step, which verifies that the underlying backfill operation has completed. The objective here is to transform "delayed, async DDL statements" into synchronous operations, so we can know "it succeeded fully" or "it failed" before moving on Part of #9866 Fixes #9888
1 parent ccede3c commit 5a06ffa

254 files changed

Lines changed: 3097 additions & 4 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.claude/skills/crdb-change/SKILL.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,11 @@ In `nexus/db-model/src/schema_versions.rs`, bump `SCHEMA_VERSION`.
7777

7878
In `nexus/db-model/src/schema_versions.rs`, add the new version to the `KNOWN_VERSIONS` list.
7979

80-
### Step 8: Test the migration
80+
### Step 8: Generate verification files
81+
82+
Run `EXPECTORATE=overwrite cargo nextest run -p nexus-db-model 'test_migration_verification_files'` to auto-generate `.verify.sql` files for any migration steps that contain backfill-prone DDL (`CREATE INDEX`, `ADD CONSTRAINT`, `ALTER COLUMN SET NOT NULL`, or `ADD COLUMN` with `NOT NULL` / non-null `DEFAULT` / `STORED` computed columns). Check in any generated files alongside the migration. If your migration doesn't contain backfill-prone DDL, no files are generated and this step is a no-op.
83+
84+
### Step 9: Test the migration
8185

8286
Run `cargo nextest run -p omicron-nexus schema` to verify that the migration is correct.
8387

Cargo.lock

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

nexus/db-model/Cargo.toml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,7 @@ tufaceous-artifact.workspace = true
6464
[dev-dependencies]
6565
camino-tempfile.workspace = true
6666
expectorate.workspace = true
67-
regex.workspace = true
6867
proptest.workspace = true
68+
regex.workspace = true
69+
sqlparser.workspace = true
6970
test-strategy.workspace = true

0 commit comments

Comments
 (0)