Skip to content

Commit b41e03b

Browse files
committed
fix(tests): install eql_v3 via sqlx Migrator, not a hand-rolled installer (CIP-3141)
c012fdc made the property suites self-install eql_v3 with an embedded copy of the release SQL guarded by a hand-rolled advisory lock. That raced: nextest runs each test in its OWN process, several hit the empty shard DB at once, and concurrent `CREATE SCHEMA eql_v3` failed with `duplicate key … pg_namespace`. The lock attempt was reinventing — incorrectly — what sqlx already does. Use the real thing: `ensure_eql_installed` now runs `sqlx::migrate!("./migrations")` against the base pool — the SAME embedded migration set `#[sqlx::test]` applies to every other test in the suite. `Migrator::run` records applied versions in `_sqlx_migrations` and skips them (idempotent, so a developer's pre-migrated local DB is a no-op) and holds a database-level advisory lock for the run, so the per-test processes serialise: exactly one applies each migration, the rest observe it already applied. No bespoke installer, no bespoke lock. The migrator is built in the test target (property/mod.rs `migrator()`) so the lib never embeds the gitignored generated `001_install_eql.sql`. Keeps the `{e:#}` change from c012fdc that surfaces anyhow's cause chain (it is what revealed the duplicate-key error in the first place). Compiles clean; CI's sharded nextest run (process-per-test) is the verification that the migrator serialises the concurrent install.
1 parent c012fdc commit b41e03b

4 files changed

Lines changed: 38 additions & 55 deletions

File tree

tests/sqlx/src/property.rs

Lines changed: 20 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -22,49 +22,30 @@ use tokio::sync::Mutex;
2222
/// `fixtures.eql_v2_<T>` table exactly once.
2323
static FIXTURE_LOADED: OnceLock<Mutex<HashSet<&'static str>>> = OnceLock::new();
2424

25-
/// Per-process guard ensuring the EQL surface (the `eql_v3` schema the oracle
26-
/// queries cast to) is installed into the connected DB exactly once.
27-
static EQL_INSTALLED: OnceLock<Mutex<bool>> = OnceLock::new();
28-
29-
/// Ensure the EQL surface (the `eql_v3` schema + scalar domains/operators the
30-
/// oracle queries cast to) is present in the DB behind `pool`.
25+
/// Apply the SQLx migrations (the EQL install in `001_install_eql.sql`, plus the
26+
/// regression-data migrations) to the DB behind `pool`.
3127
///
3228
/// The property suites connect via `connect_pool()` to the base test database
33-
/// (`DATABASE_URL`), NOT through `#[sqlx::test]`'s migrated per-test scratch
34-
/// DBs. In a CI shard that base DB is a stock Postgres with no EQL installed —
35-
/// only the `build-archive` job ran `sqlx migrate run`, and against a different
36-
/// Postgres — so every `::eql_v3.<T>_eq` cast would raise
37-
/// `schema "eql_v3" does not exist`. This installs the surface on demand so the
38-
/// suites are self-sufficient regardless of where they run (CI shard, local,
39-
/// fork), instead of silently depending on a pre-installed base DB.
29+
/// (`DATABASE_URL`) rather than through `#[sqlx::test]`'s migrated per-test
30+
/// scratch DBs, because their proptest case loop is synchronous and cannot take
31+
/// `#[sqlx::test]`'s injected pool. In a CI shard that base DB is a stock
32+
/// Postgres with no EQL installed, so every `::eql_v3.<T>_eq` cast would raise
33+
/// `schema "eql_v3" does not exist`. This brings the base DB up to the same
34+
/// migrated state the rest of the suite gets for free.
4035
///
41-
/// `install_sql` is the EQL installer (`migrations/001_install_eql.sql`),
42-
/// `include_str!`-embedded into the test binary at compile time (see
43-
/// `property/mod.rs`) so it travels inside the prebuilt nextest archive — the
44-
/// same mechanism the fixture corpus uses. A process-wide async mutex
45-
/// guarantees exactly-once execution across the parallel proptest threads, and
46-
/// a presence check (`eql_v3.int4_eq`) skips the install when the DB already
47-
/// has the surface (a developer's pre-installed local DB), where re-running the
48-
/// non-idempotent installer would error on duplicate objects.
49-
pub async fn ensure_eql_installed(pool: &PgPool, install_sql: &str) -> Result<()> {
50-
let guard = EQL_INSTALLED.get_or_init(|| Mutex::new(false));
51-
let mut installed = guard.lock().await;
52-
if *installed {
53-
return Ok(());
54-
}
55-
// Presence check: skip the installer if the surface is already there. int4
56-
// is the reference scalar type and is always part of the surface.
57-
let present: bool = sqlx::query_scalar("SELECT to_regtype('eql_v3.int4_eq') IS NOT NULL")
58-
.fetch_one(pool)
36+
/// `migrator` is `sqlx::migrate!("./migrations")` — the SAME embedded migration
37+
/// set `#[sqlx::test]` runs, passed in from the test binary so the lib does not
38+
/// embed the (gitignored, generated) migration files. `Migrator::run` is
39+
/// idempotent (it records applied versions in `_sqlx_migrations` and skips
40+
/// them) and process-safe (it takes a database-level advisory lock for the
41+
/// duration), so the separate OS processes nextest runs each test in serialise
42+
/// correctly — exactly one applies each migration, the rest observe it already
43+
/// applied. A developer's already-migrated local DB is a no-op.
44+
pub async fn ensure_eql_installed(pool: &PgPool, migrator: &sqlx::migrate::Migrator) -> Result<()> {
45+
migrator
46+
.run(pool)
5947
.await
60-
.context("probing for an existing eql_v3 install")?;
61-
if !present {
62-
sqlx::raw_sql(install_sql)
63-
.execute(pool)
64-
.await
65-
.context("installing the EQL surface into the property-test DB")?;
66-
}
67-
*installed = true;
48+
.context("applying EQL migrations to the property-test DB")?;
6849
Ok(())
6950
}
7051

tests/sqlx/tests/encrypted_domain/property/e2e_oracle.rs

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -67,8 +67,9 @@ where
6767
.build()?;
6868
let pool: PgPool = rt.block_on(connect_pool())?;
6969
// The base DB this pool connects to is not migrated by `#[sqlx::test]`; in a
70-
// CI shard it has no `eql_v3` surface, so install it before any cast/query.
71-
rt.block_on(ensure_eql_installed(&pool, super::EQL_INSTALL_SQL))?;
70+
// CI shard it has no `eql_v3` surface, so apply the migrations (idempotent +
71+
// process-safe via the migrator's advisory lock) before any cast/query.
72+
rt.block_on(ensure_eql_installed(&pool, &super::migrator()))?;
7273

7374
// Shrinking is disabled for the e2e suite: every failed shrink attempt would
7475
// trigger another ZeroKMS batch, and ciphertext cannot be meaningfully

tests/sqlx/tests/encrypted_domain/property/fixture_oracle.rs

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -65,8 +65,9 @@ fn embedded_fixture_sql<T: ScalarType>() -> &'static str {
6565
/// `#[sqlx::test]`'s ephemeral DBs by default, not the pool we connect to).
6666
async fn load_fixture_rows<T: ScalarType>(pool: &PgPool) -> Result<Vec<Row<T>>> {
6767
// The base DB this pool connects to is not migrated by `#[sqlx::test]`; in a
68-
// CI shard it has no `eql_v3` surface, so install it before any cast/query.
69-
ensure_eql_installed(pool, super::EQL_INSTALL_SQL).await?;
68+
// CI shard it has no `eql_v3` surface, so apply the migrations (idempotent +
69+
// process-safe via the migrator's advisory lock) before any cast/query.
70+
ensure_eql_installed(pool, &super::migrator()).await?;
7071
ensure_fixture_loaded::<T>(pool, embedded_fixture_sql::<T>()).await?;
7172
let sql = format!(
7273
"SELECT plaintext, payload::text FROM {} ORDER BY id",

tests/sqlx/tests/encrypted_domain/property/mod.rs

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -4,18 +4,18 @@
44
//! `scalars::<X>::` test-name prefix, so a `scalars::property::…` test would be
55
//! mis-read as a scalar type and break the catalog cross-check.
66
7-
/// The EQL installer (`migrations/001_install_eql.sql`, the full release),
8-
/// embedded at compile time so the property suites can install the `eql_v3`
9-
/// surface into their base DB on demand (see `property::ensure_eql_installed`).
10-
/// Same embed-into-the-archive rationale as the per-type fixture corpus in
11-
/// `fixture_oracle.rs`: the file is produced by `test:sqlx:prep` before the
12-
/// nextest archive is built, and the CI shards run from that archive without a
13-
/// checkout of the gitignored generated SQL. The path resolves against the
14-
/// `eql_tests` crate root (`tests/sqlx`).
15-
pub(crate) const EQL_INSTALL_SQL: &str = include_str!(concat!(
16-
env!("CARGO_MANIFEST_DIR"),
17-
"/migrations/001_install_eql.sql"
18-
));
7+
/// The embedded SQLx migration set (`tests/sqlx/migrations`) — the SAME one
8+
/// `#[sqlx::test]` applies to its scratch DBs. The property suites connect to
9+
/// the base DB directly (their proptest case loop is sync and can't take
10+
/// `#[sqlx::test]`'s injected pool), so they apply this themselves to reach the
11+
/// migrated state the rest of the suite gets for free (see
12+
/// `property::ensure_eql_installed`). The macro embeds the files at compile
13+
/// time and resolves `./migrations` against the `eql_tests` crate root
14+
/// (`tests/sqlx`); kept in the test target, not the lib, so the lib never
15+
/// embeds the gitignored generated `001_install_eql.sql`.
16+
pub(crate) fn migrator() -> sqlx::migrate::Migrator {
17+
sqlx::migrate!("./migrations")
18+
}
1919

2020
// NULL / blocker / CHECK-constraint unit tests.
2121
mod edge_cases;

0 commit comments

Comments
 (0)