|
| 1 | +# Property-based tests for the `eql_v3` encrypted scalar domains |
| 2 | + |
| 3 | +These tests assert that SQL operator results on the `eql_v3` encrypted-domain |
| 4 | +types agree with a **plaintext oracle** across a *generated* input space — the |
| 5 | +fixed-pivot scalar matrix only checks hand-picked values, so property tests are |
| 6 | +what catch operator/oracle disagreements across the whole value space. Origin: |
| 7 | +CIP-3141. |
| 8 | + |
| 9 | +## The three suites |
| 10 | + |
| 11 | +The harness is **one unit-level suite plus two integration suites**. They are |
| 12 | +named for what they operate on, not by an abstract tier letter: |
| 13 | + |
| 14 | +| Suite | Location | Kind | Inputs | DB / creds | |
| 15 | +|-------|----------|------|--------|------------| |
| 16 | +| **catalog** | [`crates/eql-scalars/src/proptest_invariants.rs`](../../../../../crates/eql-scalars/src/proptest_invariants.rs) | unit (pure Rust) | generated terms / kinds | none — runs in fork CI | |
| 17 | +| **fixture** | [`fixture_oracle.rs`](./fixture_oracle.rs) | integration | committed fixture corpus (real ciphertext) | shared test DB | |
| 18 | +| **e2e** | [`e2e_oracle.rs`](./e2e_oracle.rs) | integration | freshly generated plaintexts, encrypted each run | shared test DB **+ ZeroKMS creds** | |
| 19 | + |
| 20 | +Plus [`edge_cases.rs`](./edge_cases.rs): example-based unit tests for |
| 21 | +NULL propagation, blockers raising on unsupported operators (including the |
| 22 | +native-`jsonb` `->`/`@>` domain-fallback paths), `timestamptz` ordering |
| 23 | +deferral, and CHECK rejection of malformed payloads. |
| 24 | + |
| 25 | +### catalog — catalog invariants, no database |
| 26 | + |
| 27 | +Pure-Rust `proptest` over the `eql-scalars` catalog: term/operator/extractor |
| 28 | +consistency, "every blocker is non-`STRICT` + `plpgsql`", payload-key set == |
| 29 | +declared terms, integer-range ordering. No DB, no encryption, no creds, so it |
| 30 | +runs in the lean `cargo test -p eql-scalars` path (and on fork PRs). This is the |
| 31 | +only suite where `proptest` shrinking is meaningful and enabled. |
| 32 | + |
| 33 | +### fixture — oracle over committed ciphertext |
| 34 | + |
| 35 | +Runs the shared all-pairs oracle engine over the real, committed fixture corpus |
| 36 | +(`fixtures.eql_v2_<T>.sql`, generated by `cipherstash-client` during |
| 37 | +`mise run test:sqlx:prep`). `proptest` selects a sub-multiset of fixture rows |
| 38 | +(with repeats), and the engine checks **every ordered pair**. No new encryption, |
| 39 | +so it runs whenever the fixtures are present, and it generalises across the whole |
| 40 | +catalog for free (every fixtured type gets an `eq_oracle`, ordered types also get |
| 41 | +an `ord_oracle`). |
| 42 | + |
| 43 | +### e2e — oracle over fresh end-to-end encryption |
| 44 | + |
| 45 | +Same oracle engine, but each case **generates fresh random plaintexts and |
| 46 | +encrypts them end-to-end through ZeroKMS** (one batched call per case) before |
| 47 | +querying. Gated behind the `proptest-e2e` cargo feature — `mise run test:sqlx` |
| 48 | +enables it (CI has the secrets); a bare `cargo test` compiles it out. It is the |
| 49 | +**only** suite that can exercise "same plaintext, *different* ciphertext" |
| 50 | +(equality across independently-encrypted values), because the committed fixture |
| 51 | +corpus has no duplicate plaintexts. Integer scalars only for now (random `T` |
| 52 | +generation is trivial for integers). |
| 53 | + |
| 54 | +## The shared oracle engine |
| 55 | + |
| 56 | +`assert_eq_oracle` / `assert_ord_oracle` in |
| 57 | +[`../../../src/property.rs`](../../../src/property.rs) take a corpus of |
| 58 | +`(plaintext, payload_json)` rows and check, over every ordered pair, that: |
| 59 | + |
| 60 | +- `=` / `<>` on the `_eq` domain agree with plaintext `==` / `!=`, and |
| 61 | +- (ordered domains) `<` `<=` `>` `>=` and `ord_term` sort order agree with the |
| 62 | + plaintext comparison. |
| 63 | + |
| 64 | +The fixture and e2e suites differ only in **where the rows come from**; the |
| 65 | +engine is identical. |
| 66 | + |
| 67 | +## Why ciphertext can't be `Arbitrary`-derived |
| 68 | + |
| 69 | +A valid payload's `hm`/`ob` terms are real ciphertext from `cipherstash-client` |
| 70 | +(`encrypt_eql`), which needs a live ZeroKMS handshake — there is no offline/mock |
| 71 | +cipher. `proptest` can only generate **plaintexts**; turning them into payloads |
| 72 | +means encrypting them. That is exactly why there are two integration suites: the |
| 73 | +fixture suite reuses already-encrypted values, and the e2e suite pays for fresh |
| 74 | +encryption to reach inputs the fixtures can't. |
| 75 | + |
| 76 | +## Conventions and footguns |
| 77 | + |
| 78 | +- **Never put these tests under a `scalars::` module.** `mise run test:matrix:inventory` |
| 79 | + discovers scalar types from every `scalars::<X>::` test-name prefix, so a |
| 80 | + `scalars::property::…` test would be mis-read as a scalar type and break the |
| 81 | + catalog cross-check. They live under `property::` for this reason. |
| 82 | +- **The e2e suite is gated behind `proptest-e2e`.** Keep it that way so a |
| 83 | + credential-less `cargo test` (and the lean crate path) still compiles. |
| 84 | +- **The e2e suite disables shrinking and failure persistence.** Every shrink |
| 85 | + attempt would burn another ZeroKMS batch, ciphertext can't be meaningfully |
| 86 | + shrunk, and fresh-each-run ciphertext can't be replayed. |
| 87 | +- **proptest + async bridge.** The `proptest!` body is sync; the DB-backed suites |
| 88 | + run their async oracle on a per-case current-thread `tokio` runtime and connect |
| 89 | + via `connect_pool()` (they cannot use `#[sqlx::test]`'s injected pool from a |
| 90 | + sync body). Operator evaluation is read-only `SELECT`, so no per-test schema |
| 91 | + isolation is needed. |
| 92 | +- **Equality-true must actually fire.** Random integer pairs almost never |
| 93 | + collide, so the e2e corpus injects deliberate duplicate plaintexts (plus signed |
| 94 | + extremes and zero) to exercise the `a == b ⇒ eq` branch across distinct |
| 95 | + ciphertexts. |
| 96 | + |
| 97 | +## Running |
| 98 | + |
| 99 | +```bash |
| 100 | +# catalog suite only (no DB, no creds) |
| 101 | +cargo test -p eql-scalars proptest_invariants |
| 102 | + |
| 103 | +# fixture + edge-case suites (needs a prepared DB) |
| 104 | +mise run test:sqlx:prep |
| 105 | +cd tests/sqlx && cargo test --test encrypted_domain property::fixture_oracle property::edge_cases |
| 106 | + |
| 107 | +# all suites incl. e2e (needs DB + CS_* creds) |
| 108 | +mise run test:sqlx # enables --features proptest-e2e |
| 109 | +``` |
0 commit comments