Skip to content

Commit f281167

Browse files
committed
feat(stack): strongly-typed v3 Drizzle adapter (envelope typing, docs, perf)
- A3: type the Drizzle customType around the stored Encrypted envelope, not plaintext — inserts take pre-encrypted rows, select returns envelopes; the codec is bigint-safe. Removes the as-never/Record casts on the insert path. - M1: type createEncryptionOperatorsV3 structurally ({ encrypt }) so the EncryptionV3() return value is accepted without a cast; add operators.test-d guarding it (the typecheck surface that catches the regression). - Add bigint drizzle coverage (unit + live round-trip). - B6: memoize per-column context so inArray builds once, not per element. - C10: extract a shared drizzle:Name helper. C8: document the intentional v2/v3 EncryptionOperatorError fork. m3/m4: correct stale public.* comment and error text. TSDoc for the factory and ~20 operators. - README: document @cipherstash/stack/eql/v3/drizzle.
1 parent 8123839 commit f281167

12 files changed

Lines changed: 516 additions & 84 deletions

File tree

packages/stack/README.md

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -366,6 +366,87 @@ For columns with `searchableJson: true`, three JSONB operators are available:
366366

367367
These operators encrypt the JSON path selector using the `steVecSelector` query type and cast it to `eql_v2_encrypted` for use with the EQL PostgreSQL functions.
368368

369+
### EQL v3 Concrete-Type Columns (Drizzle)
370+
371+
The `@cipherstash/stack/eql/v3/drizzle` module targets EQL v3, where each encrypted column is a **concrete Postgres domain type** (`eql_v3.text_eq`, `eql_v3.integer_ord`, …). Instead of toggling capability flags, you pick a concrete type from the `types` namespace and its query capabilities are fixed by that choice. The v2 `@cipherstash/stack/drizzle` module above is unchanged — this is an additive export.
372+
373+
Declare a Drizzle table using the `types` factories. The suffix encodes the capability: `*Eq` (equality), `*Ord` (order + range, which also covers equality), `*Match` / `TextSearch` (free-text search), and the bare name (e.g. `Text`, `Bigint`) is storage-only.
374+
375+
```ts
376+
import { pgTable, integer } from "drizzle-orm/pg-core"
377+
import { drizzle } from "drizzle-orm/postgres-js"
378+
import {
379+
types,
380+
createEncryptionOperatorsV3,
381+
extractEncryptionSchemaV3,
382+
} from "@cipherstash/stack/eql/v3/drizzle"
383+
import { EncryptionV3 } from "@cipherstash/stack/v3"
384+
385+
// Capabilities come from the concrete type — no flags to configure.
386+
const users = pgTable("users", {
387+
id: integer("id").primaryKey().generatedAlwaysAsIdentity(),
388+
email: types.TextEq("email"), // equality: eq / ne / inArray
389+
age: types.IntegerOrd("age"), // order + range: gt/gte/lt/lte, between, asc/desc
390+
bio: types.TextMatch("bio"), // free-text search: contains
391+
balance: types.Bigint("balance"), // storage only (no query capability)
392+
})
393+
```
394+
395+
Derive the v3 schema from the table, build the typed client, and create the capability-checked operators:
396+
397+
```ts
398+
const usersSchema = extractEncryptionSchemaV3(users)
399+
const client = await EncryptionV3({ schemas: [usersSchema] })
400+
const ops = createEncryptionOperatorsV3(client)
401+
402+
const db = drizzle({ client: sqlClient })
403+
```
404+
405+
The operators auto-encrypt their operands and validate them against the column's concrete type. Applying an operator the type doesn't support throws `EncryptionOperatorError`:
406+
407+
```ts
408+
// Equality — email is TextEq
409+
const exact = await db.select().from(users)
410+
.where(await ops.eq(users.email, "alice@example.com"))
411+
412+
// Range + ordering — age is IntegerOrd
413+
const adults = await db.select().from(users)
414+
.where(await ops.gte(users.age, 18))
415+
.orderBy(ops.asc(users.age))
416+
417+
const midBand = await db.select().from(users)
418+
.where(await ops.between(users.age, 25, 40))
419+
420+
// Set membership — built on equality
421+
const listed = await db.select().from(users)
422+
.where(await ops.inArray(users.email, ["alice@example.com", "bob@example.com"]))
423+
424+
// Free-text token containment — bio is TextMatch
425+
const coffee = await db.select().from(users)
426+
.where(await ops.contains(users.bio, "coffee"))
427+
```
428+
429+
Rows are **pre-encrypted** with `client.bulkEncryptModels(...)` before they reach `db.insert(...).values(...)` — Drizzle never sees plaintext. `Bigint` columns take a native JS `bigint`:
430+
431+
```ts
432+
const rows = await client.bulkEncryptModels(
433+
[
434+
{ email: "alice@example.com", age: 30, bio: "climbing and coffee", balance: 100_000n },
435+
{ email: "bob@example.com", age: 41, bio: "cycling and coffee", balance: 250_000n },
436+
],
437+
usersSchema,
438+
)
439+
if (rows.failure) throw new Error(rows.failure.message)
440+
441+
await db.insert(users).values(rows.data)
442+
```
443+
444+
Notes:
445+
446+
- **Free-text search uses `ops.contains`** (token containment on a `match` index), not SQL `like` / `ilike`. It matches whole indexed tokens, so `contains(users.bio, "coffee")` finds rows whose `bio` contains the token `coffee`.
447+
- **The concrete type defines the legal operators.** `TextEq` supports `eq` / `ne` / `inArray` / `notInArray`; `*Ord` types add `gt` / `gte` / `lt` / `lte` / `between` / `notBetween` and `asc` / `desc`; `*Match` and `TextSearch` add `contains`; a bare `Text` / `Integer` / `Bigint` column is storage-only. Using an unsupported operator throws `EncryptionOperatorError`.
448+
- Combine conditions with `ops.and` / `ops.or`, and do NULL checks with `ops.isNull` / `ops.isNotNull` (the where-clause operators are `async` and must be `await`ed; `ops.asc` / `ops.desc` are synchronous).
449+
369450
## Identity-Aware Encryption
370451

371452
Lock encryption to a specific user by requiring a valid JWT for decryption.
@@ -664,6 +745,8 @@ csValue(valueName) // returns ProtectValue (for nested values)
664745
| `@cipherstash/stack/secrets` | `Secrets` class and secrets types |
665746
| `@cipherstash/stack/client` | Client-safe exports (schema builders and types only - no native FFI) |
666747
| `@cipherstash/stack/types` | All TypeScript types (`Encrypted`, `Decrypted`, `ClientConfig`, `EncryptionClientConfig`, query types, etc.) |
748+
| `@cipherstash/stack/v3` | `EncryptionV3` typed client plus the EQL v3 authoring DSL (`encryptedTable`, `types`, v3 type helpers) |
749+
| `@cipherstash/stack/eql/v3/drizzle` | EQL v3 Drizzle integration: `types` column factories, `createEncryptionOperatorsV3`, `extractEncryptionSchemaV3`, `makeEqlV3Column`, `EncryptionOperatorError` |
667750

668751
## Migration from @cipherstash/protect
669752

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
import { type SQL, sql } from 'drizzle-orm'
2+
import { integer, PgDialect, pgTable } from 'drizzle-orm/pg-core'
3+
import { describe, expect, it, vi } from 'vitest'
4+
import { getEqlV3Column, isEqlV3Column } from '@/eql/v3/drizzle/column'
5+
import {
6+
createEncryptionOperatorsV3,
7+
EncryptionOperatorError,
8+
} from '@/eql/v3/drizzle/operators'
9+
import { types } from '@/eql/v3/drizzle/types'
10+
11+
// A representative encrypted envelope — what `client.encrypt` actually returns
12+
// and what a bigint column stores. Deliberately NOT a plaintext bigint.
13+
const TERM = { c: 'ct', v: 1 }
14+
const TERM_JSON = JSON.stringify(TERM)
15+
16+
function chainable(result: unknown) {
17+
const op = result as {
18+
withLockContext: ReturnType<typeof vi.fn>
19+
audit: ReturnType<typeof vi.fn>
20+
}
21+
op.withLockContext = vi.fn(() => op)
22+
op.audit = vi.fn(() => op)
23+
return op
24+
}
25+
26+
function setup() {
27+
const encrypt = vi.fn(() => chainable(Promise.resolve({ data: TERM })))
28+
const ops = createEncryptionOperatorsV3({ encrypt })
29+
const dialect = new PgDialect()
30+
const render = (s: SQL) => dialect.sqlToQuery(s)
31+
return { ops, encrypt, render }
32+
}
33+
34+
// A statically-typed table via the drizzle `types` namespace (no dynamic
35+
// matrix, no casts) — the column set is known at compile time.
36+
const accounts = pgTable('accounts', {
37+
id: integer().primaryKey(),
38+
balance: types.BigintOrd('balance'),
39+
ledgerId: types.BigintEq('ledger_id'),
40+
archived: types.Bigint('archived'),
41+
})
42+
43+
describe('v3 drizzle bigint columns', () => {
44+
it('detects a bigint column and reports its concrete public.bigint domain', () => {
45+
const storage = types.Bigint('n')
46+
const ord = types.BigintOrd('n_ord')
47+
48+
expect(isEqlV3Column(storage)).toBe(true)
49+
expect(getEqlV3Column('n', storage)?.getEqlType()).toBe('public.bigint')
50+
expect(getEqlV3Column('n_ord', ord)?.getEqlType()).toBe('public.bigint_ord')
51+
})
52+
53+
it('emits the concrete public.bigint* SQL type through pgTable', () => {
54+
expect(accounts.balance.getSQLType()).toBe('public.bigint_ord')
55+
expect(accounts.ledgerId.getSQLType()).toBe('public.bigint_eq')
56+
expect(accounts.archived.getSQLType()).toBe('public.bigint')
57+
})
58+
59+
it('encrypts a native bigint operand for eq without JSON-stringifying it', async () => {
60+
const { ops, encrypt, render } = setup()
61+
// A value beyond Number.MAX_SAFE_INTEGER to prove the bigint is passed
62+
// through untouched (a JSON.stringify of a bigint would have thrown).
63+
const value = 9223372036854775807n
64+
const q = render(await ops.eq(accounts.ledgerId, value))
65+
66+
expect(q.sql).toContain('eql_v3.eq("accounts"."ledger_id", $1::jsonb)')
67+
expect(q.params).toEqual([TERM_JSON])
68+
expect(encrypt.mock.calls[0]?.[0]).toBe(value)
69+
})
70+
71+
it('emits ordering and range operators for a bigint_ord column', async () => {
72+
const { ops, render } = setup()
73+
74+
const gt = render(await ops.gt(accounts.balance, 10n))
75+
expect(gt.sql).toContain('eql_v3.gt("accounts"."balance", $1::jsonb)')
76+
77+
const between = render(await ops.between(accounts.balance, -5n, 5n))
78+
expect(between.sql).toContain('eql_v3.gte("accounts"."balance", $1::jsonb)')
79+
expect(between.sql).toContain('eql_v3.lte("accounts"."balance", $2::jsonb)')
80+
81+
const asc = render(ops.asc(accounts.balance))
82+
expect(asc.sql).toContain('eql_v3.ord_term("accounts"."balance")')
83+
})
84+
85+
it('rejects equality/ordering on a storage-only bigint column', async () => {
86+
const { ops } = setup()
87+
await expect(ops.eq(accounts.archived, 1n)).rejects.toBeInstanceOf(
88+
EncryptionOperatorError,
89+
)
90+
expect(() => ops.asc(accounts.archived)).toThrow(EncryptionOperatorError)
91+
})
92+
93+
it('rejects a bare SQL expression that is not an encrypted column', async () => {
94+
const { ops } = setup()
95+
await expect(ops.eq(sql`1`, 1n)).rejects.toBeInstanceOf(
96+
EncryptionOperatorError,
97+
)
98+
})
99+
})

packages/stack/__tests__/drizzle-v3/codec.test.ts

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,12 @@ describe('v3 codec', () => {
2020
expect(v3FromDriver(obj)).toBe(obj)
2121
})
2222

23-
it('passes null/undefined through on read', () => {
23+
it('normalises null/undefined to SQL NULL (JS null) on read', () => {
2424
expect(v3FromDriver(null)).toBeNull()
25-
expect(v3FromDriver(undefined)).toBeUndefined()
25+
expect(v3FromDriver(undefined)).toBeNull()
26+
})
27+
28+
it('does not throw on a stray bigint in the envelope (defensive)', () => {
29+
expect(v3ToDriver({ v: 1n } as never)).toBe('{"v":"1"}')
2630
})
2731
})

packages/stack/__tests__/drizzle-v3/operators-live-pg.test.ts

Lines changed: 86 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ import {
1515
createEncryptionOperatorsV3,
1616
EncryptionOperatorError,
1717
extractEncryptionSchemaV3,
18+
types as v3drizzle,
1819
} from '@/eql/v3/drizzle'
1920
import { makeEqlV3Column } from '@/eql/v3/drizzle/column'
2021
import { installEqlV3IfNeeded } from '../helpers/eql-v3'
@@ -54,7 +55,7 @@ const matrixTable = pgTable(TABLE_NAME, {
5455
V3_MATRIX['public.text_eq'].builder('nullable_text_eq'),
5556
),
5657
...matrixColumns,
57-
} as never)
58+
})
5859

5960
const accountsTable = pgTable(ACCOUNT_TABLE_NAME, {
6061
id: integer('id').primaryKey().generatedAlwaysAsIdentity(),
@@ -63,12 +64,31 @@ const accountsTable = pgTable(ACCOUNT_TABLE_NAME, {
6364
testRunId: text('test_run_id').notNull(),
6465
})
6566

67+
// A statically-typed encrypted table (via the drizzle `types` namespace) with
68+
// concrete bigint columns. Unlike the dynamic matrix table, its column set is
69+
// known at compile time, so it exercises A3 end-to-end with ZERO casts: the
70+
// insert takes envelope rows and the select yields `Encrypted` values ready for
71+
// decrypt.
72+
const BIGINT_TABLE_NAME = 'protect_ci_v3_drizzle_bigint'
73+
const bigintTable = pgTable(BIGINT_TABLE_NAME, {
74+
id: integer('id').primaryKey().generatedAlwaysAsIdentity(),
75+
rowKey: text('row_key').notNull(),
76+
testRunId: text('test_run_id').notNull(),
77+
balance: v3drizzle.BigintOrd('balance'),
78+
ledger: v3drizzle.BigintEq('ledger'),
79+
})
80+
81+
// Full i64 bounds — proves protect-ffi 0.28 round-trips a JS bigint beyond
82+
// Number.MAX_SAFE_INTEGER losslessly through the encrypted column.
83+
const BIGINT_BALANCE = 9223372036854775807n
84+
const BIGINT_LEDGER = -9223372036854775808n
85+
6686
const schema = extractEncryptionSchemaV3(matrixTable)
87+
const bigintSchema = extractEncryptionSchemaV3(bigintTable)
6788

6889
type PlainValue = string | number | boolean | Date
6990
type RowKey = typeof ROW_A | typeof ROW_B
7091
type MatrixPlainRow = Record<string, PlainValue | null | string>
71-
type MatrixDbRow = Record<string, unknown>
7292
type SelectRow = { rowKey: string }
7393
type Db = ReturnType<typeof drizzle>
7494
type Client = Awaited<ReturnType<typeof EncryptionV3>>
@@ -180,7 +200,7 @@ function encryptedInsertRows(): MatrixPlainRow[] {
180200
beforeAll(async () => {
181201
if (!LIVE_EQL_V3_PG_ENABLED) return
182202
await installEqlV3IfNeeded(sqlClient)
183-
client = await EncryptionV3({ schemas: [schema] })
203+
client = await EncryptionV3({ schemas: [schema, bigintSchema] })
184204
ops = createEncryptionOperatorsV3(client)
185205
db = drizzle({ client: sqlClient })
186206

@@ -205,21 +225,42 @@ beforeAll(async () => {
205225
test_run_id TEXT NOT NULL
206226
)
207227
`)
228+
await sqlClient.unsafe(`
229+
CREATE TABLE IF NOT EXISTS ${BIGINT_TABLE_NAME} (
230+
id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY,
231+
row_key TEXT NOT NULL,
232+
test_run_id TEXT NOT NULL,
233+
balance public.bigint_ord NOT NULL,
234+
ledger public.bigint_eq NOT NULL
235+
)
236+
`)
208237

209-
const encryptedRows = unwrap<MatrixDbRow[]>(
238+
const encryptedRows = unwrap(
210239
await client.bulkEncryptModels(encryptedInsertRows(), schema),
211240
)
212241
await db.insert(matrixTable).values(encryptedRows)
213242
await db.insert(accountsTable).values([
214243
{ rowKey: ROW_A, label: 'primary', testRunId: RUN },
215244
{ rowKey: ROW_B, label: 'secondary', testRunId: RUN },
216245
])
246+
247+
// A3 end-to-end, cast-free: encrypt a native bigint model, insert the
248+
// resulting envelope rows (typed against the column's `Encrypted` data slot),
249+
// no `as never` anywhere.
250+
const bigintRows = unwrap(
251+
await client.bulkEncryptModels(
252+
[{ rowKey: ROW_A, testRunId: RUN, balance: BIGINT_BALANCE, ledger: BIGINT_LEDGER }],
253+
bigintSchema,
254+
),
255+
)
256+
await db.insert(bigintTable).values(bigintRows)
217257
}, 120000)
218258

219259
afterAll(async () => {
220260
if (!LIVE_EQL_V3_PG_ENABLED) return
221261
await sqlClient`DELETE FROM ${sqlClient(TABLE_NAME)} WHERE test_run_id = ${RUN}`
222262
await sqlClient`DELETE FROM ${sqlClient(ACCOUNT_TABLE_NAME)} WHERE test_run_id = ${RUN}`
263+
await sqlClient`DELETE FROM ${sqlClient(BIGINT_TABLE_NAME)} WHERE test_run_id = ${RUN}`
223264
await sqlClient.end()
224265
}, 30000)
225266

@@ -556,4 +597,45 @@ describeLivePg('v3 drizzle operators (live pg matrix)', () => {
556597
// Only '' is excluded (ROW_A); ROW_B ('ada@example.com') survives.
557598
expect(rows).toEqual([ROW_B])
558599
}, 30000)
600+
601+
// A3 + bigint lock: a statically-typed bigint table round-trips a real i64
602+
// value through encrypt → insert → select → decrypt with no casts. The select
603+
// yields `Encrypted`-typed columns (the envelope), fed straight to decrypt.
604+
it('round-trips a native bigint through a statically-typed encrypted column', async () => {
605+
const encrypted = await db
606+
.select({ balance: bigintTable.balance, ledger: bigintTable.ledger })
607+
.from(bigintTable)
608+
.where(drizzleEq(bigintTable.testRunId, RUN))
609+
expect(encrypted).toHaveLength(1)
610+
611+
const decrypted = unwrap(
612+
await client.decryptModel(encrypted[0], bigintSchema),
613+
)
614+
expect(decrypted.balance).toBe(BIGINT_BALANCE)
615+
expect(decrypted.ledger).toBe(BIGINT_LEDGER)
616+
}, 30000)
617+
618+
it('filters a bigint column by encrypted equality and ordering', async () => {
619+
const byLedger = (await db
620+
.select({ rowKey: bigintTable.rowKey })
621+
.from(bigintTable)
622+
.where(
623+
and(
624+
drizzleEq(bigintTable.testRunId, RUN),
625+
await ops.eq(bigintTable.ledger, BIGINT_LEDGER),
626+
),
627+
)) as SelectRow[]
628+
expect(byLedger.map((row) => row.rowKey)).toEqual([ROW_A])
629+
630+
const byBalance = (await db
631+
.select({ rowKey: bigintTable.rowKey })
632+
.from(bigintTable)
633+
.where(
634+
and(
635+
drizzleEq(bigintTable.testRunId, RUN),
636+
await ops.gt(bigintTable.balance, 0n),
637+
),
638+
)) as SelectRow[]
639+
expect(byBalance.map((row) => row.rowKey)).toEqual([ROW_A])
640+
}, 30000)
559641
})
Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
import { describe, expectTypeOf, it } from 'vitest'
2+
import type { EncryptionClient } from '@/encryption'
3+
import type { EncryptionV3 } from '@/encryption/v3'
4+
import { createEncryptionOperatorsV3 } from '@/eql/v3/drizzle'
5+
6+
/**
7+
* Static regression guard for M1: `createEncryptionOperatorsV3` must accept the
8+
* `TypedEncryptionClient` that `EncryptionV3` resolves to — the documented
9+
* `createEncryptionOperatorsV3(await EncryptionV3({ schemas }))` usage — as well
10+
* as the nominal `EncryptionClient` and a hand-rolled `{ encrypt }` double,
11+
* none requiring a cast. Typing the parameter to `EncryptionClient` (the
12+
* original bug) makes the first call below a compile error, which this suite
13+
* would then catch. Lives in a `*.test-d.ts` so it is inside the existing
14+
* typecheck scope without dragging the loose-typed runtime suites in.
15+
*/
16+
describe('createEncryptionOperatorsV3 - client parameter (M1)', () => {
17+
type V3Client = Awaited<ReturnType<typeof EncryptionV3>>
18+
19+
it('accepts the client EncryptionV3 returns with no cast', () => {
20+
expectTypeOf(createEncryptionOperatorsV3).toBeCallableWith(
21+
{} as V3Client,
22+
)
23+
})
24+
25+
it('still accepts the nominal EncryptionClient', () => {
26+
expectTypeOf(createEncryptionOperatorsV3).toBeCallableWith(
27+
{} as EncryptionClient,
28+
)
29+
})
30+
31+
it('accepts a minimal structural { encrypt } double', () => {
32+
const double = {
33+
encrypt: (_plaintext: never, _opts: never) => ({}) as unknown,
34+
}
35+
expectTypeOf(createEncryptionOperatorsV3).toBeCallableWith(double)
36+
})
37+
})

0 commit comments

Comments
 (0)