Skip to content

Commit 2d3fa18

Browse files
committed
docs(v3): changelog + reference for N-block ORE, ordered numeric/timestamptz
CHANGELOG: rewrite the timestamptz entry (ordering now ships), add the numeric ordered-domain entry, and add a Fixed entry for the N-block ORE comparator generalization + the eql_v3 ore_block_u64_8_256 -> ore_block_256 rename. Reference guide: document the fourth (numeric/Decimal) fixture discriminator — proc-macro routing, scalar_fixture arm, numeric_values accessor + distinctness guard, rust_decimal dep.
1 parent 8db938a commit 2d3fa18

2 files changed

Lines changed: 42 additions & 1 deletion

File tree

CHANGELOG.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,8 @@ Each entry that ships in a published release links to the PR that introduced it.
2626
- **`eql_v3.int2` encrypted-domain type family.** Four jsonb-backed domains for encrypted `int2` columns — `eql_v3.int2` (storage-only), `eql_v3.int2_eq` (`=` / `<>` via HMAC), and `eql_v3.int2_ord` / `eql_v3.int2_ord_ore` (also `<` `<=` `>` `>=` via ORE block terms, with `MIN` / `MAX` aggregates) — generated from the `int2` row in `eql-scalars::CATALOG` by the same materializer as the `eql_v3.int4` reference. Index via a functional index on the `eql_v3.eq_term` / `eql_v3.ord_term` extractors, not an operator class on the domain. Why: a type-safe, per-capability encrypted `smallint` column, proving the scalar generator generalizes beyond the `int4` reference. ([#243](https://github.com/cipherstash/encrypt-query-language/pull/243))
2727
- **`eql_v3.int8` encrypted-domain type family.** Four jsonb-backed domains for encrypted `int8` columns — `eql_v3.int8` (storage-only), `eql_v3.int8_eq` (`=` / `<>` via HMAC), and `eql_v3.int8_ord` / `eql_v3.int8_ord_ore` (also `<` `<=` `>` `>=` via ORE block terms, with `MIN` / `MAX` aggregates) — generated from the `int8` row in `eql-scalars::CATALOG` by the same materializer as the `eql_v3.int4` reference. Index via a functional index on the `eql_v3.eq_term` / `eql_v3.ord_term` extractors, not an operator class on the domain. Why: a type-safe, per-capability encrypted `bigint` column, extending the scalar generator across the full 64-bit integer width. ([#253](https://github.com/cipherstash/encrypt-query-language/pull/253))
2828
- **`eql_v3.date` encrypted-domain type family.** Four jsonb-backed domains for encrypted `date` columns — `eql_v3.date` (storage-only), `eql_v3.date_eq` (`=` / `<>` via HMAC), and `eql_v3.date_ord` / `eql_v3.date_ord_ore` (also `<` `<=` `>` `>=` via ORE block terms, with `MIN` / `MAX` aggregates) — generated from the `date` row in `eql-scalars::CATALOG` by the same materializer as the `eql_v3.int4` reference. Plaintexts encrypt under the `date` cast and compare via the same ORE block terms as the integer scalars (ORE is plaintext-agnostic — dates order like integers). Index via a functional index on the `eql_v3.eq_term` / `eql_v3.ord_term` extractors, not an operator class on the domain. Why: the first **non-integer ordered** scalar encrypted-domain type — a type-safe, per-capability encrypted `date` column — proving the generator and SQLx test matrix generalize beyond fixed-width integers. ([#256](https://github.com/cipherstash/encrypt-query-language/pull/256))
29-
- **`eql_v3.timestamptz` encrypted-domain type family (equality-only).** Two jsonb-backed domains for encrypted `timestamptz` columns — `eql_v3.timestamptz` (storage-only) and `eql_v3.timestamptz_eq` (`=` / `<>` via HMAC) — generated from the `timestamptz` row in `eql-scalars::CATALOG` by the same materializer as the `eql_v3.date` family. Values are **UTC-normalized** (cipherstash has no timezone-preserving type): plaintexts encrypt under the `timestamp` cast. Index via a functional index on the `eql_v3.eq_term` extractor, not an operator class on the domain. **Ordering (`<` `<=` `>` `>=`, `MIN` / `MAX`) is deferred:** cipherstash encrypts `Plaintext::Timestamp` at native 12-block ORE width, but EQL's only ORE comparator (`eql_v2.compare_ore_block_u64_8_256_term`) is hardcoded to 8 blocks, so ordered timestamptz domains would silently mis-order. There are no `eql_v3.timestamptz_ord` / `_ord_ore` domains and no timestamptz `MIN` / `MAX` aggregates until a wide-ORE (12-block) term lands — tracked in [#241](https://github.com/cipherstash/encrypt-query-language/issues/241). Why: a type-safe, equality-searchable encrypted UTC-timestamp column, stacking on the `date` temporal-scalar foundation; ordering follows once the comparator supports the native ciphertext width. ([#257](https://github.com/cipherstash/encrypt-query-language/pull/257))
29+
- **`eql_v3.timestamptz` encrypted-domain type family (ordered).** Four jsonb-backed domains for encrypted `timestamptz` columns — `eql_v3.timestamptz` (storage-only), `eql_v3.timestamptz_eq` (`=` / `<>` via HMAC), and `eql_v3.timestamptz_ord` / `eql_v3.timestamptz_ord_ore` (also `<` `<=` `>` `>=`, `MIN` / `MAX` via 12-block ORE) — generated from the `timestamptz` row in `eql-scalars::CATALOG` by the same materializer as the `eql_v3.date` family. Values are **UTC-normalized** (cipherstash has no timezone-preserving type): plaintexts encrypt under the `timestamp` cast. Ordering works because the `eql_v3` ORE block comparator now derives its block count from the ciphertext width (see the comparator entry below) instead of assuming 8. Index via a functional index on the `eql_v3.eq_term` / `eql_v3.ord_term` extractors, not an operator class on the domain. ([#257](https://github.com/cipherstash/encrypt-query-language/pull/257))
30+
- **`eql_v3.numeric` encrypted-domain type family (ordered).** Four jsonb-backed domains for encrypted `numeric` / `decimal` columns — `eql_v3.numeric` (storage-only), `eql_v3.numeric_eq` (`=` / `<>` via HMAC), and `eql_v3.numeric_ord` / `eql_v3.numeric_ord_ore` (also `<` `<=` `>` `>=`, `MIN` / `MAX` via 14-block ORE) — generated from the `numeric` row in `eql-scalars::CATALOG`. cipherstash encrypts `Plaintext::Decimal` at native 14-block ORE width; ordering matches `rust_decimal::Decimal` ordering exactly (equivalent scales such as `1` and `1.0` collide, like Postgres `numeric`). Index via a functional index on the `eql_v3.eq_term` / `eql_v3.ord_term` extractors. Why: a type-safe, ordered encrypted decimal column, the first scalar to exercise an ORE term wider than 8 blocks. ([#241](https://github.com/cipherstash/encrypt-query-language/issues/241))
3031
- **Per-domain `MIN` / `MAX` aggregates for the encrypted-domain family.** `eql_v3.min(eql_v3.<T>_ord)` / `eql_v3.max(eql_v3.<T>_ord)` (and the `_ord_ore` twin) are generated for every ord-capable scalar variant, giving type-safe extrema on domain-typed columns — comparison routes through the variant's `<` / `>` operator (ORE block term, no decryption). The aggregates are declared `PARALLEL = SAFE` with a combine function (the state function itself — min/max are associative), so PostgreSQL can use partial/parallel aggregation on large `GROUP BY` workloads. Why: the new domain types previously had no equivalent of the composite-type aggregates. The existing `eql_v2.min(eql_v2_encrypted)` / `eql_v2.max(eql_v2_encrypted)` aggregates are **retained** and continue to work on `eql_v2_encrypted` columns; the per-domain aggregates are additive and coexist with them. ([#239](https://github.com/cipherstash/encrypt-query-language/pull/239))
3132
- **`eql_v3.text` encrypted-domain family (`text`, `text_eq`, `text_match`, `text_ord`, `text_ord_ore`).** Adds equality (`=` / `<>` via HMAC), match (`@>` / `<@` via a new self-contained `eql_v3.bloom_filter` SEM index term), and ORE ordering (`<` `<=` `>` `>=`, `min` / `max`) for encrypted text, at parity with EQL v2 text — generated from the `text` row in `eql-scalars::CATALOG` by the same materializer as the `eql_v3.int4` reference. `text` is the first scalar to add a new index `Term` (`Bloom`) and the first non-integer, unbounded ordered kind (lexicographic pivots, hand-written `impl ScalarType`). Index via a functional index on the `eql_v3.eq_term` / `eql_v3.ord_term` / `eql_v3.match_term` extractors, not an operator class on the domain. Why: brings searchable encrypted text to the namespaced, `eql_v2`-free `eql_v3` surface. Match is exposed as bloom-filter containment on the `text_match` domain — deliberately *not* SQL `LIKE` (no wildcard/anchoring; probabilistic ngram containment) — and never backs equality (which always routes through `Hm`). ([#260](https://github.com/cipherstash/encrypt-query-language/pull/260))
3233
- **Self-contained `eql_v3` schema + standalone `release/cipherstash-encrypt-v3.sql` installer.** The `eql_v3` encrypted-domain surface no longer depends on `eql_v2` at runtime: it now owns its own copies of the searchable-encrypted-metadata (SEM) index-term types — `eql_v3.hmac_256` and `eql_v3.ore_block_256` (with its btree operator class) — so the `eql_v3.eq_term` / `eql_v3.ord_term` extractors return `eql_v3` types and no `eql_v2.<symbol>` appears anywhere in the v3 SQL. The whole v3 surface relocated under a single `src/v3/` tree (`src/v3/sem/` for the hand-written SEM types, `src/v3/scalars/` for the generated domain families). A new build variant ships the `eql_v3` schema on its own as `release/cipherstash-encrypt-v3.sql`, installable into a database with no `eql_v2` present; a CI gate greps that artifact and its dependency closure to keep it `eql_v2`-free. Why: a clean foundation for the per-scalar encrypted-domain model to stand alone, ahead of it replacing the `eql_v2_encrypted` composite column type. This is additive — a new schema and a new artifact — and leaves `eql_v2` byte-for-byte unchanged. ([#255](https://github.com/cipherstash/encrypt-query-language/pull/255))
@@ -35,6 +36,10 @@ Each entry that ships in a published release links to the PR that introduced it.
3536

3637
- **Scalar encrypted-domain types are now defined in a Rust catalog, not TOML manifests; the Python codegen toolchain is removed.** Adding a scalar encrypted-domain type (`int4`, `int8`, …) is now one row in `eql-scalars::CATALOG` (`crates/eql-scalars/src/lib.rs`) instead of authoring `tasks/codegen/types/<T>.toml`. `mise run build` regenerates the gitignored SQL surface via `cargo run -p eql-codegen` (Rust, std-only) rather than the Python generator. The catalog row's `Fixture` list is the single source of truth for that type's plaintext fixtures: the SQLx test matrix reads it directly as a compile-time-materialised const (`eql_scalars::INT4_VALUES` / `INT2_VALUES`, `ScalarType::FIXTURE_VALUES`), so there is no longer a generated, committed `tests/sqlx/src/fixtures/<T>_values.rs` — a Rust source of truth no longer round-trips through generated Rust. The shipped SQL is unchanged — `release/*.sql` is byte-identical across the cutover — so there is no change for callers installing EQL; this only affects contributors who extend the scalar domain families. The `python` mise tool, the `pytest`-based `test:codegen` (now `cargo test -p eql-scalars -p eql-codegen`), the per-type `mise run codegen:domain` tasks, and the per-type `tests/sqlx/snapshots/<T>_matrix_tests.txt` baselines (collapsed into one catalog-reconciled `tests/sqlx/snapshots/matrix_tests.txt`) are gone. Why: a single compiler-validated source of truth shared by the generator and the SQLx test harness, and one fewer toolchain in the build/test path — building and testing EQL no longer needs Python (Python remains only for the separate docs-markdown tooling). ([#252](https://github.com/cipherstash/encrypt-query-language/pull/252))
3738

39+
### Fixed
40+
41+
- **The `eql_v3` ORE block comparator now orders ciphertexts of any block count, not just 8.** `eql_v3.compare_ore_block_256_term` derives the block count `N` from the term length (`octet_length = 49·N + 16`) instead of hardcoding 8, so encrypted types whose native ORE width exceeds 8 blocks — `numeric` (14) and `timestamptz` (12) — order, range-query, `ORDER BY`, and `MIN`/`MAX` correctly instead of silently mis-ordering. Malformed terms (length not `49·N + 16` for `N ≥ 1`) now raise instead of returning a bogus comparison. The self-contained `eql_v3` SEM type was renamed `eql_v3.ore_block_u64_8_256 → eql_v3.ore_block_256` to reflect that it is width-agnostic (the `eql_v2` type is unchanged). No effect on existing 8-block types (a no-op for `N = 8`). ([#241](https://github.com/cipherstash/encrypt-query-language/issues/241))
42+
3843
## [2.3.1] — 2026-05-21
3944

4045
### Fixed

docs/reference/adding-a-scalar-encrypted-domain-type.md

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -326,6 +326,42 @@ first added) also needs, in `tests/sqlx/src/fixtures/eql_plaintext.rs`:
326326
`Plaintext::*` variant (`Plaintext::NaiveDate` / `Plaintext::Timestamp` /
327327
`Plaintext::Text`), plus the three mirrored `#[test]`s.
328328

329+
#### A fourth fixture shape: non-integer, non-chrono, non-text (`numeric` / `Decimal`)
330+
331+
`numeric` (backed by `rust_decimal::Decimal`, 14-block ORE — the first scalar
332+
whose ORE term is wider than 8 blocks) is ordered but is **neither** the integer
333+
materialiser, **nor** chrono (`temporal`), **nor** `text` (it owns a `Decimal`,
334+
not a `String`, and has no `Match` index). It therefore introduces a **fourth
335+
fixture discriminator**, which means touching the proc-macro routing, not just
336+
the type list. Beyond the §3.1 `eql_plaintext.rs` wiring above (`Cast::DECIMAL`,
337+
`PlaintextSqlType::NUMERIC`, the `cast_for_kind` / `plaintext_sql_type_for_kind`
338+
arms, `Sealed for Decimal`, `EqlPlaintext for Decimal``Plaintext::Decimal`),
339+
it also needs:
340+
341+
- an **`is_numeric_token`** arm in `crates/eql-tests-macros/src/lib.rs`'s
342+
fixture-module router — without it `scalar_types!(fixture_modules)` panics at
343+
compile time on the unrecognised kind (the router handled only `temporal` /
344+
`text` before);
345+
- a **`numeric` arm** in the `scalar_fixture!` macro
346+
(`tests/sqlx/src/fixtures/scalar_fixture.rs`) — the temporal arm's twin
347+
(`[Unique, Ore]`, pivot-presence asserts via `OrderedScalar`), but no `Match`
348+
and no chrono;
349+
- a hand-written **`numeric_values()`** accessor plus `impl ScalarType` /
350+
`OrderedScalar for Decimal` in `tests/sqlx/src/scalar_domains.rs` — parsing the
351+
catalog's `Fixture::Numeric` strings into a `LazyLock<Vec<Decimal>>` (the
352+
catalog stays zero-dep; the parse lives in the harness). `Decimal: Ord` supplies
353+
the expected sort order — `ore-rs` guarantees the ciphertext order agrees, and
354+
equivalent scales (`1``1.0`) collide like `Decimal`'s own `Ord`. Add a
355+
**`fixtures_are_distinct_by_value`** guard (parse → `HashSet`): the zero-dep
356+
catalog only dedupes by literal string, so `"1"` / `"1.0"` would slip past it
357+
but collide in both the ORE ciphertext and the fixture table;
358+
- the **`rust_decimal` dependency** + the sqlx **`rust_decimal` feature** in
359+
`tests/sqlx/Cargo.toml` (in `[dependencies]`, not `[dev-dependencies]` — the
360+
`Decimal` impls live in the crate's library code).
361+
362+
See `docs/plans/2026-06-11-ore-block-comparator-n-blocks-design.md` for the full
363+
worked example (and the N-block ORE comparator change the wide term relies on).
364+
329365
### New-capability domains (e.g. `_match` / `Bloom`)
330366

331367
A domain carrying a capability the matrix does not model — `text`'s `_match`

0 commit comments

Comments
 (0)