Skip to content

Commit 12a7705

Browse files
committed
docs(walkthroughs): fix review findings (counts, ste_vec framing, line refs)
1 parent be5ef03 commit 12a7705

2 files changed

Lines changed: 73 additions & 59 deletions

File tree

docs/walkthroughs/fixture-generator.md

Lines changed: 44 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -32,10 +32,14 @@ Consequences that shape the whole pipeline:
3232
- `CS_CLIENT_ID` + `CS_CLIENT_KEY` — client key material, via `EnvKeyProvider`.
3333
(`tasks/fixtures.toml:12-17`; built in `tests/sqlx/src/fixtures/cipherstash.rs:44-53`.)
3434
- Generated fixture SQL is **gitignored** and regenerated on every test run
35-
(`.gitignore:225-230`), so a stale fixture cannot rot in the tree.
35+
(`.gitignore:227-230`), so a stale fixture cannot rot in the tree.
3636
- Static/committed fixtures are forbidden as a way to dodge the creds dependency.
37-
There is exactly **one** committed exception, `tests/sqlx/fixtures/v3_ste_vec.sql`,
38-
pending a SteVec-document generator — a gap, not a pattern to copy.
37+
Every fixture is generated — including the SteVec document
38+
`tests/sqlx/fixtures/v3_ste_vec.sql`, which is gitignored (`.gitignore:228`) and
39+
rebuilt each run by its own generator (`fixtures::v3_ste_vec::generate()`).
40+
(`CLAUDE.md` still calls `v3_ste_vec.sql` "the one committed exception … pending a
41+
SteVec-document generator", but that note is stale: the generator exists and the file
42+
is gitignored like the rest.)
3943

4044
The plaintext **values** are single-sourced in the Rust catalog
4145
`crates/eql-scalars` so the test oracle (the expected-result computation) and the
@@ -49,16 +53,16 @@ row.
4953
```mermaid
5054
flowchart TD
5155
subgraph catalog["crates/eql-scalars (zero-dep catalog)"]
52-
SPEC["ScalarSpec rows in CATALOG<br/>token + kind + domains + fixtures<br/>lib.rs:301-427"]
53-
FIX["Fixture value lists<br/>INT4_FIXTURES, NUMERIC_FIXTURES, …<br/>lib.rs:242-299"]
54-
MAC["int_values! / text_values! macros<br/>materialise typed const slices<br/>lib.rs:439-510"]
55-
VALS["INT4_VALUES: &[i32]<br/>INT2_VALUES, INT8_VALUES, TEXT_VALUES<br/>lib.rs:476-510"]
56+
SPEC["ScalarSpec rows in CATALOG<br/>token + kind + domains + fixtures<br/>lib.rs:312-438"]
57+
FIX["Fixture value lists<br/>INT4_FIXTURES, NUMERIC_FIXTURES, …<br/>lib.rs:253-310"]
58+
MAC["int_values! / text_values! macros<br/>materialise typed const slices<br/>lib.rs:450-521"]
59+
VALS["INT4_VALUES: &[i32]<br/>INT2_VALUES, INT8_VALUES, TEXT_VALUES<br/>lib.rs:487-521"]
5660
SPEC --> FIX --> MAC --> VALS
5761
end
5862
5963
subgraph harness["tests/sqlx (harness)"]
6064
ST["ScalarType::fixture_values()<br/>scalar_domains.rs:53"]
61-
TEMP["temporal_values! / hand impls<br/>date/timestamptz/numeric → LazyLock<Vec<T>><br/>scalar_domains.rs:192-468"]
65+
TEMP["temporal_values! / hand impls<br/>date/timestamptz/numeric → LazyLock<Vec<T>><br/>scalar_domains.rs:192-495"]
6266
MOD["scalar_fixture! → spec()<br/>FixtureSpec builder per type<br/>scalar_fixture.rs:174-194"]
6367
VALS --> ST
6468
SPEC --> TEMP --> ST
@@ -110,9 +114,9 @@ Step by step:
110114

111115
```mermaid
112116
flowchart LR
113-
ROW["INT4 ScalarSpec<br/>token=int4, kind=I32<br/>fixtures=INT4_FIXTURES<br/>lib.rs:301-306"]
114-
LIST["INT4_FIXTURES: &[Fixture]<br/>Min, N(-100), …, Max<br/>lib.rs:244-246"]
115-
IV["int_values!(INT4_VALUES, i32, INT4)<br/>const-eval resolve + range check<br/>lib.rs:476"]
117+
ROW["INT4 ScalarSpec<br/>token=int4, kind=I32<br/>fixtures=INT4_FIXTURES<br/>lib.rs:312-317"]
118+
LIST["INT4_FIXTURES: &[Fixture]<br/>Min, N(-100), …, Max<br/>lib.rs:255-257"]
119+
IV["int_values!(INT4_VALUES, i32, INT4)<br/>const-eval resolve + range check<br/>lib.rs:487"]
116120
CONST["INT4_VALUES: &[i32]<br/>(committed source of truth,<br/>NOT a generated .rs)"]
117121
118122
ROW --> LIST
@@ -129,7 +133,7 @@ flowchart LR
129133

130134
The key invariant: the **oracle** (`expected_forward` filtering `fixture_values()`)
131135
and the **encryption input** (`spec().values()` fed to `encrypt_store`) are the same
132-
`&[i32]` const. The catalog comment makes this explicit (`lib.rs:429-435`):
136+
`&[i32]` const. The catalog comment makes this explicit (`lib.rs:440-446`):
133137

134138
> This is the **single-sourced** plaintext list the SQLx test matrix reads via
135139
> `ScalarType::fixture_values()` and the fixture generator encrypts — derived from
@@ -140,8 +144,8 @@ and the **encryption input** (`spec().values()` fed to `encrypt_store`) are the
140144

141145
| Kind | Catalog literal | Materialized via | Result type |
142146
|------|-----------------|------------------|-------------|
143-
| integers (`int4`/`int2`/`int8`) | `Fixture::Int` / `Min`/`Max`/`Zero` | `int_values!` macro, **`const`** (`lib.rs:439-478`) | `&'static [iN]` |
144-
| `text` | `Fixture::Text(&str)` | `text_values!` macro, **`const`** (`lib.rs:486-510`) | `&'static [&str]` |
147+
| integers (`int4`/`int2`/`int8`) | `Fixture::Int` / `Min`/`Max`/`Zero` | `int_values!` macro, **`const`** (`lib.rs:450-485`) | `&'static [iN]` |
148+
| `text` | `Fixture::Text(&str)` | `text_values!` macro, **`const`** (`lib.rs:497-519`) | `&'static [&str]` |
145149
| `date` / `timestamptz` | `Fixture::Date`/`Timestamptz(&str)` | `temporal_values!``LazyLock<Vec<T>>` (`scalar_domains.rs:192-320`) | `&'static [chrono::…]` |
146150
| `numeric` | `Fixture::Numeric(&str)` | hand-written `LazyLock<Vec<Decimal>>` (`scalar_domains.rs:449-468`) | `&'static [Decimal]` |
147151

@@ -150,23 +154,23 @@ are not `const`, so those parse the catalog strings once into a `LazyLock<Vec<T>
150154
The catalog itself stays **zero-dep** (no chrono, no rust_decimal) — parsing happens
151155
in the harness, not in `eql-scalars`. The `int_values!` macro does a compile-time
152156
range check so an out-of-range literal is a build error, not a silent `as` truncation
153-
(`lib.rs:456-462`).
157+
(`lib.rs:467-473`).
154158

155159
---
156160

157161
## 4. Worked example: `int4` from catalog row to SQL INSERT
158162

159163
### 4a. The catalog row and value list
160164

161-
`crates/eql-scalars/src/lib.rs:244-246`:
165+
`crates/eql-scalars/src/lib.rs:255-257`:
162166

163167
```rust
164168
const INT4_FIXTURES: &[Fixture] = fixtures!(int i32;
165169
Min, N(-100), N(-1), Zero, N(1), N(2), N(5), N(10), N(17), N(25),
166170
N(42), N(50), N(100), N(250), N(1000), N(9999), Max);
167171
```
168172

169-
`lib.rs:301-306`:
173+
`lib.rs:312-317`:
170174

171175
```rust
172176
const INT4: ScalarSpec = ScalarSpec {
@@ -178,18 +182,18 @@ const INT4: ScalarSpec = ScalarSpec {
178182
```
179183

180184
The `fixtures!` macro range-checks every `N(..)` literal against `i32` at its
181-
definition site (`lib.rs:191-193`), and `Min`/`Max`/`Zero` are resolved to the
185+
definition site (`lib.rs:202-204`), and `Min`/`Max`/`Zero` are resolved to the
182186
kind's bounds.
183187

184188
### 4b. Materialized into a typed const
185189

186-
`lib.rs:476`:
190+
`lib.rs:487`:
187191

188192
```rust
189193
int_values!(INT4_VALUES, i32, INT4);
190194
```
191195

192-
The macro (`lib.rs:439-474`) walks `INT4.fixtures`, calls `numeric_value(kind)` to
196+
The macro (`lib.rs:450-485`) walks `INT4.fixtures`, calls `numeric_value(kind)` to
193197
resolve each `Fixture` (`Min``i32::MIN`, `N(-100)``-100`, `Zero``0`, …), and
194198
const-evaluates the whole thing into `pub const INT4_VALUES: &[i32]`. So
195199
`INT4_VALUES == [-2147483648, -100, -1, 0, 1, 2, 5, 10, 17, 25, 42, 50, 100, 250, 1000, 9999, 2147483647]`.
@@ -237,7 +241,7 @@ let outputs = encrypt_eql(cipher, prepared, &opts).await?; // ONE ZeroKMS roun
237241
```
238242

239243
`value.to_plaintext()` is the `EqlPlaintext` lift — for `i32` it is
240-
`Plaintext::Int(Some(*self))` (`eql_plaintext.rs:152-158`). The `Cast::INT` maps to
244+
`Plaintext::Int(Some(*self))` (`eql_plaintext.rs:153-159`). The `Cast::INT` maps to
241245
`ColumnType::Int` (`cipherstash.rs:92-110`), and the `Unique`+`Ore` indexes map to
242246
the unique + ORE `IndexType`s (`cipherstash.rs:118-135`). `EqlOperation::Store`
243247
yields a full storage payload `{"k":"ct","v":2,"i":…,"c":…,"hm":…,"ob":…}`.
@@ -267,8 +271,8 @@ INSERT INTO fixtures.eql_v2_int4 (id, plaintext, payload) VALUES ('1', '-2147483
267271
```
268272

269273
The `plaintext` column (`-2147483648`) is the committed oracle; the `payload` is the
270-
real encrypted document. At test time `fetch_fixture_payload` (`scalar_domains.rs:726-743`)
271-
looks up a payload by plaintext, and `assert_scalar_plaintexts` (`scalar_domains.rs:763-778`)
274+
real encrypted document. At test time `fetch_fixture_payload` (`scalar_domains.rs:729-746`)
275+
looks up a payload by plaintext, and `assert_scalar_plaintexts` (`scalar_domains.rs:766-781`)
272276
compares the DB query result against `expected_forward` over `INT4_VALUES`.
273277

274278
---
@@ -278,18 +282,19 @@ compares the DB query result against `expected_forward` over `INT4_VALUES`.
278282
`generate_all` also runs three hand-written fixtures that aren't `CATALOG` scalars
279283
(`generate_all_fixtures.rs:37-61`):
280284

281-
- **`v3_ste_vec`** — a SteVec JSONB document fixture. **The one committed
282-
exception** (`.gitignore` re-lists it but `CLAUDE.md` documents it as a gap pending
283-
a SteVec-document generator). A hand-written `FixtureSpec<serde_json::Value>` riding
284-
the same `run()` pipeline.
285+
- **`v3_ste_vec`** — a SteVec JSONB document fixture. A hand-written
286+
`FixtureSpec<serde_json::Value>` riding the same `run()` pipeline
287+
(`fixtures::v3_ste_vec::generate()`), gitignored and regenerated like every other
288+
fixture. (`CLAUDE.md` still calls it "the one committed exception"; that wording is
289+
stale — see §6.)
285290
- **`v3_doc_int4`** — a scalar-shaped SteVec document, one `{"field": <int4>}` per
286291
`INT4_VALUES`. A **split** fixture: the encryption input is the jsonb document but
287292
the plaintext oracle column is the bare `int4`, so it uses the
288293
`run_with_payloads` seam (`driver.rs:242-303`) instead of `run()`.
289294
- **`v3_numeric_collision`** — the `[1, 1.0, 2]` scale-equivalence collision
290295
(`tests/sqlx/src/fixtures/v3_numeric_collision.rs`). It cannot live in the
291296
catalog `eql_v2_numeric` fixture because the catalog distinctness guard
292-
(`numeric_value_guards::fixtures_are_distinct_by_value`, `scalar_domains.rs:508-523`)
297+
(`numeric_value_guards::fixtures_are_distinct_by_value`, `scalar_domains.rs:509-523`)
293298
forbids the value-equal pair `1`/`1.0`. Encrypted at numeric ORE width via the
294299
standard `.run()` driver, addressed by `id` rather than `plaintext` (since
295300
`WHERE plaintext = 1` would match both).
@@ -316,14 +321,16 @@ through the prep flow, which requires:
316321
Why not just commit the SQL and skip the creds? Because the ciphertexts must be the
317322
*actual* output of the crypto. `CLAUDE.md`:
318323

319-
> Do NOT add static/committed fixtures to dodge the creds dependency. The one
320-
> committed exception, `tests/sqlx/fixtures/v3_ste_vec.sql`, is a gap pending a
321-
> SteVec-document generator … not a pattern to copy.
324+
> Do NOT add static/committed fixtures to dodge the creds dependency.
322325
323-
The gitignore enforces it mechanically (`.gitignore:225-230`): every
324-
`eql_v2*` fixture plus `v3_doc_int4.sql` and `v3_numeric_collision.sql` are ignored
325-
and regenerated on every `mise run test:sqlx`. A stale or hand-edited fixture can't
326-
survive a run. The live round-trip is additionally smoke-tested by the
326+
(`CLAUDE.md` goes on to name `tests/sqlx/fixtures/v3_ste_vec.sql` as "the one committed
327+
exception … pending a SteVec-document generator", but that clause is stale: the SteVec
328+
generator now exists and the file is gitignored and regenerated like the rest.)
329+
330+
The gitignore enforces it mechanically (`.gitignore:227-230`): every
331+
`eql_v2*` fixture plus `v3_ste_vec.sql`, `v3_doc_int4.sql`, and `v3_numeric_collision.sql`
332+
are ignored and regenerated on every `mise run test:sqlx`. A stale or hand-edited fixture
333+
can't survive a run. The live round-trip is additionally smoke-tested by the
327334
`#[ignore]` `live_tests` in `cipherstash.rs:309-412`, which assert the real Store
328335
payload shape (`v=2`, non-null `hm`/`ob`/`c`/`i`) and that distinct plaintexts yield
329336
distinct `hm` terms — so an SDK API drift surfaces there before the whole pipeline
@@ -335,12 +342,12 @@ breaks.
335342

336343
| Concern | Path |
337344
|---------|------|
338-
| Catalog rows, fixture lists, `int_values!`/`text_values!` | `crates/eql-scalars/src/lib.rs:184-510` |
345+
| Catalog rows, fixture lists, `int_values!`/`text_values!` | `crates/eql-scalars/src/lib.rs:195-521` |
339346
| `generate_all` entry point | `tests/sqlx/tests/generate_all_fixtures.rs:25-62` |
340347
| `fixture:generate:all` mise task | `tasks/fixtures.toml:1-29` |
341348
| `test:sqlx:prep` flow | `mise.toml:49-80` |
342349
| `ScalarType` trait + `fixture_values()` + oracle | `tests/sqlx/src/scalar_domains.rs:17-128` |
343-
| `temporal_values!` (date/timestamptz), numeric/text impls | `tests/sqlx/src/scalar_domains.rs:192-468` |
350+
| `temporal_values!` (date/timestamptz), numeric/text impls | `tests/sqlx/src/scalar_domains.rs:192-495` |
344351
| `scalar_types!` harness token list | `tests/sqlx/src/scalar_types.rs:39-63` |
345352
| `scalar_fixture!` (spec builder + generator test) | `tests/sqlx/src/fixtures/scalar_fixture.rs` |
346353
| `FixtureSpec` builder + SQL renderers | `tests/sqlx/src/fixtures/spec.rs` |

0 commit comments

Comments
 (0)