Skip to content

Commit ffadd95

Browse files
committed
feat(stack): bulkEncrypt/bulkDecrypt on the wasm-inline entry (CIP-3584)
The WASM entry exposed no bulk operations at all — only encrypt/decrypt — so rendering an N-row list on Deno, Cloudflare Workers, or Supabase Edge Functions cost N sequential ZeroKMS round trips. Combined with the ~6.6s cold start that made list endpoints impractical on the edge, and it left the entry with no access to the bulk path AGENTS.md names as the way to exercise ZeroKMS bulk speed. Surfaced as M1 in the rc.3 skilltester run. The FFI already had the primitives — `encryptBulk`, `decryptBulkFallible` are exported from protect-ffi's WASM build (identical in 0.29 and 0.30). This wires them up; no new capability was needed underneath. Signature deliberately differs from the native entry's bulk methods. It follows `encryptQueryBulk`, the bulk primitive already on this client: an index-aligned array with per-item table/column routing, throwing rather than a `{ data } | { failure }` envelope. No `{ id, plaintext }` payload envelopes either — protect-ffi's `EncryptPayload` has no `id` field, so the native one is dropped at the boundary and buys nothing once positions are stable. Consistency within this surface beats parity with a different entry point, since a WASM caller is reading this module's API. Per-item routing is what makes the saving real: one call covers several columns across many rows, where a single-column batch would still cost one round trip per column. Two failure modes are handled rather than assumed away: - `bulkDecrypt` builds on the FALLIBLE primitive, so when items fail it throws once naming every failing index and reason, instead of surfacing the first and discarding work the caller already paid for. - Both methods assert the FFI returned as many results as were sent. Matching is positional (no correlation id), so a short response would silently leave trailing slots null — indistinguishable from "this row had no value". That is silent wrong data, so it throws. Model helpers stay Node-only: this entry has no single-model operation to build them on, so adding bulk-model alone would be incoherent. Noted in the class docblock as a separate port. 14 new unit tests: one-FFI-call-per-batch, payload shape, mixed tables/columns with declared-vs-property column names, position stability across null/undefined, all-null and empty short-circuits, per-index failure reporting, and both length-mismatch guards. 823 tests pass; typecheck and build clean; 0 biome errors. Skill updated — `stash-encryption` documented only the native bulk shape, which would not compile on the edge.
1 parent 0e96d36 commit ffadd95

5 files changed

Lines changed: 543 additions & 4 deletions

File tree

.changeset/wasm-inline-bulk-ops.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
---
2+
'@cipherstash/stack': minor
3+
'stash': patch
4+
---
5+
6+
Add `bulkEncrypt` / `bulkDecrypt` to `@cipherstash/stack/wasm-inline`, so a list of encrypted rows costs **one** ZeroKMS round trip instead of one per row.
7+
8+
The WASM entry previously exposed no bulk operations at all — only `encrypt` / `decrypt` — so rendering an N-row list on Deno, Cloudflare Workers, or Supabase Edge Functions meant N sequential ZeroKMS calls. Combined with the WASM cold start, that made list endpoints impractical on the edge, and it left the entry with no access to the bulk path the docs recommend for throughput.
9+
10+
```typescript
11+
// Write: several columns across many rows, one round trip
12+
const encrypted = await client.bulkEncrypt([
13+
{ plaintext: "alice@example.com", table: users, column: users.email },
14+
{ plaintext: "hello", table: users, column: users.bio },
15+
])
16+
17+
// Read: a whole page in one call
18+
const emails = await client.bulkDecrypt(rows.map((r) => r.email))
19+
```
20+
21+
Both are index-aligned with their input, and `null` / `undefined` entries yield `null` at the same index without reaching ZeroKMS (an all-null batch makes no call at all). Because each entry names its own table and column, a single `bulkEncrypt` can cover several columns across many rows.
22+
23+
**The signature deliberately differs from the native entry's** `bulkEncrypt` / `bulkDecrypt`. It follows `encryptQueryBulk`, the bulk primitive already on this client: a plain index-aligned array with per-item routing, and errors that throw rather than a `{ data } | { failure }` envelope. There are no `{ id, plaintext }` payload envelopes — protect-ffi's `EncryptPayload` has no `id` field, so the native one is dropped at the FFI boundary and buys nothing when positions are already stable.
24+
25+
`bulkDecrypt` builds on the fallible FFI primitive, so when items fail it throws once and names **every** failing index with its reason, rather than surfacing the first and discarding the rest.
26+
27+
The model helpers (`encryptModel` / `decryptModel` and their bulk forms) remain Node-only: the WASM entry has no single-model operation to build them on, so those need their own port.

packages/stack/__tests__/helpers/stub-protect-ffi-wasm-inline.ts

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,12 +14,24 @@ export const decrypt = (): never => {
1414
)
1515
}
1616

17+
export const decryptBulkFallible = (): never => {
18+
throw new Error(
19+
'[test stub]: protect-ffi/wasm-inline decryptBulkFallible not implemented',
20+
)
21+
}
22+
1723
export const encrypt = (): never => {
1824
throw new Error(
1925
'[test stub]: protect-ffi/wasm-inline encrypt not implemented',
2026
)
2127
}
2228

29+
export const encryptBulk = (): never => {
30+
throw new Error(
31+
'[test stub]: protect-ffi/wasm-inline encryptBulk not implemented',
32+
)
33+
}
34+
2335
export const encryptQuery = (): never => {
2436
throw new Error(
2537
'[test stub]: protect-ffi/wasm-inline encryptQuery not implemented',
Lines changed: 265 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,265 @@
1+
import { beforeEach, describe, expect, it, vi } from 'vitest'
2+
3+
// #737: the WASM entry exposed no bulk operations, so an N-row list on the
4+
// edge cost N ZeroKMS round trips. These tests pin the new surface: the FFI
5+
// payload shape (`{ plaintext, table, column }` / `{ ciphertext }` — the
6+
// protect-ffi `EncryptPayload` has no `id`, unlike the native entry's), that
7+
// exactly ONE FFI call is made per batch (the whole point), position
8+
// stability across nulls, and the per-index failure reporting that
9+
// `decryptBulkFallible` makes possible. protect-ffi is mocked; live coverage
10+
// runs in the Deno e2e.
11+
12+
const ffi = vi.hoisted(() => ({
13+
newClient: vi.fn(async () => ({ handle: 'wasm-client' })),
14+
encrypt: vi.fn(async () => ({ v: 3, i: {}, c: 'ct' })),
15+
decrypt: vi.fn(async () => 'plain'),
16+
isEncrypted: vi.fn(() => true),
17+
encryptQuery: vi.fn(async () => ({ v: 3, i: {} })),
18+
encryptQueryBulk: vi.fn(async () => []),
19+
encryptBulk: vi.fn(
20+
async (_client: unknown, { plaintexts }: { plaintexts: unknown[] }) =>
21+
plaintexts.map((_, n) => ({ v: 3, i: {}, c: `ct-${n}` })),
22+
),
23+
decryptBulkFallible: vi.fn(
24+
async (_client: unknown, { ciphertexts }: { ciphertexts: unknown[] }) =>
25+
ciphertexts.map((_, n) => ({ data: `plain-${n}` })),
26+
),
27+
}))
28+
vi.mock('@cipherstash/protect-ffi/wasm-inline', () => ffi)
29+
vi.mock('@cipherstash/auth/wasm-inline', () => ({
30+
AccessKeyStrategy: {
31+
create: vi.fn(() => ({
32+
data: { getToken: async () => ({ token: 'test' }) },
33+
})),
34+
},
35+
OidcFederationStrategy: {},
36+
}))
37+
38+
import { encryptedTable, types } from '../src/eql/v3'
39+
import { Encryption } from '../src/wasm-inline'
40+
41+
const users = encryptedTable('users', {
42+
email: types.TextEq('email'),
43+
bio: types.TextSearch('bio'),
44+
})
45+
46+
// A column whose declared DB name differs from the property, so the payload
47+
// assertion proves `getColumnName` is applied rather than the property key.
48+
const accounts = encryptedTable('accounts', {
49+
emailAddress: types.TextEq('email_address'),
50+
})
51+
52+
async function client() {
53+
return Encryption({
54+
schemas: [users, accounts],
55+
config: {
56+
workspaceCrn: 'crn:test:ws',
57+
accessKey: 'test-key',
58+
clientId: 'id',
59+
clientKey: 'key',
60+
},
61+
})
62+
}
63+
64+
const ct = (c: string) => ({ v: 3, i: { t: 'users', c: 'email' }, c }) as never
65+
66+
describe('WasmEncryptionClient.bulkEncrypt', () => {
67+
beforeEach(() => {
68+
vi.clearAllMocks()
69+
})
70+
71+
it('sends one FFI call for the whole batch', async () => {
72+
const c = await client()
73+
await c.bulkEncrypt([
74+
{ plaintext: 'a@b.com', table: users, column: users.email },
75+
{ plaintext: 'hello', table: users, column: users.bio },
76+
{ plaintext: 'c@d.com', table: users, column: users.email },
77+
])
78+
79+
// The entire reason this method exists: 3 values, 1 round trip.
80+
expect(ffi.encryptBulk).toHaveBeenCalledTimes(1)
81+
expect(ffi.encrypt).not.toHaveBeenCalled()
82+
})
83+
84+
it('builds the FFI payload as { plaintext, table, column } with no id', async () => {
85+
const c = await client()
86+
await c.bulkEncrypt([
87+
{ plaintext: 'a@b.com', table: users, column: users.email },
88+
{ plaintext: 'hello', table: users, column: users.bio },
89+
])
90+
91+
const [, opts] = ffi.encryptBulk.mock.calls[0]
92+
expect(opts.plaintexts).toEqual([
93+
{ plaintext: 'a@b.com', table: 'users', column: 'email' },
94+
{ plaintext: 'hello', table: 'users', column: 'bio' },
95+
])
96+
// protect-ffi's EncryptPayload has no `id` — the native entry's is
97+
// dropped at the boundary, so sending one would be dead weight.
98+
for (const p of opts.plaintexts as Array<Record<string, unknown>>) {
99+
expect(p).not.toHaveProperty('id')
100+
}
101+
})
102+
103+
it('mixes tables and columns in a single batch', async () => {
104+
const c = await client()
105+
await c.bulkEncrypt([
106+
{ plaintext: 'a@b.com', table: users, column: users.email },
107+
{
108+
plaintext: 'x@y.com',
109+
table: accounts,
110+
column: accounts.emailAddress,
111+
},
112+
])
113+
114+
const [, opts] = ffi.encryptBulk.mock.calls[0]
115+
expect(opts.plaintexts).toEqual([
116+
{ plaintext: 'a@b.com', table: 'users', column: 'email' },
117+
// The DECLARED column name, not the property key.
118+
{ plaintext: 'x@y.com', table: 'accounts', column: 'email_address' },
119+
])
120+
})
121+
122+
it('is position-stable across null and undefined plaintexts', async () => {
123+
const c = await client()
124+
const out = await c.bulkEncrypt([
125+
{ plaintext: null, table: users, column: users.email },
126+
{ plaintext: 'a@b.com', table: users, column: users.email },
127+
{ plaintext: undefined, table: users, column: users.email },
128+
{ plaintext: 'c@d.com', table: users, column: users.email },
129+
])
130+
131+
expect(out).toHaveLength(4)
132+
expect(out[0]).toBeNull()
133+
expect(out[2]).toBeNull()
134+
// Live values keep their ORIGINAL indices, not their compacted ones.
135+
expect(out[1]).toEqual({ v: 3, i: {}, c: 'ct-0' })
136+
expect(out[3]).toEqual({ v: 3, i: {}, c: 'ct-1' })
137+
138+
// Nulls never reach ZeroKMS.
139+
const [, opts] = ffi.encryptBulk.mock.calls[0]
140+
expect(opts.plaintexts).toHaveLength(2)
141+
})
142+
143+
it('short-circuits an all-null batch without calling the FFI', async () => {
144+
const c = await client()
145+
const out = await c.bulkEncrypt([
146+
{ plaintext: null, table: users, column: users.email },
147+
{ plaintext: undefined, table: users, column: users.email },
148+
])
149+
150+
expect(out).toEqual([null, null])
151+
expect(ffi.encryptBulk).not.toHaveBeenCalled()
152+
})
153+
154+
it('returns an empty array for an empty batch, with no FFI call', async () => {
155+
const c = await client()
156+
expect(await c.bulkEncrypt([])).toEqual([])
157+
expect(ffi.encryptBulk).not.toHaveBeenCalled()
158+
})
159+
})
160+
161+
describe('WasmEncryptionClient.bulkDecrypt', () => {
162+
beforeEach(() => {
163+
vi.clearAllMocks()
164+
})
165+
166+
it('sends one FFI call for the whole batch', async () => {
167+
const c = await client()
168+
await c.bulkDecrypt([ct('a'), ct('b'), ct('c')])
169+
170+
expect(ffi.decryptBulkFallible).toHaveBeenCalledTimes(1)
171+
expect(ffi.decrypt).not.toHaveBeenCalled()
172+
})
173+
174+
it('builds the FFI payload as { ciphertext }', async () => {
175+
const c = await client()
176+
const a = ct('a')
177+
const b = ct('b')
178+
await c.bulkDecrypt([a, b])
179+
180+
const [, opts] = ffi.decryptBulkFallible.mock.calls[0]
181+
expect(opts.ciphertexts).toEqual([{ ciphertext: a }, { ciphertext: b }])
182+
})
183+
184+
it('is position-stable across null and undefined ciphertexts', async () => {
185+
const c = await client()
186+
const out = await c.bulkDecrypt([null, ct('a'), undefined, ct('b')])
187+
188+
expect(out).toEqual([null, 'plain-0', null, 'plain-1'])
189+
const [, opts] = ffi.decryptBulkFallible.mock.calls[0]
190+
expect(opts.ciphertexts).toHaveLength(2)
191+
})
192+
193+
it('short-circuits an all-null batch without calling the FFI', async () => {
194+
const c = await client()
195+
expect(await c.bulkDecrypt([null, undefined])).toEqual([null, null])
196+
expect(ffi.decryptBulkFallible).not.toHaveBeenCalled()
197+
})
198+
199+
it('reports EVERY failing index, not just the first', async () => {
200+
ffi.decryptBulkFallible.mockResolvedValueOnce([
201+
{ data: 'ok' },
202+
{ error: 'boom-one' },
203+
{ error: 'boom-two' },
204+
] as never)
205+
206+
const c = await client()
207+
// Indices are into the INPUT array, so the leading null shifts them:
208+
// input 1/2/3 map to live 0/1/2, and the two failures are inputs 2 and 3.
209+
await expect(
210+
c.bulkDecrypt([null, ct('a'), ct('b'), ct('c')]),
211+
).rejects.toThrow(/failed for 2 of 3 payload\(s\)/)
212+
213+
ffi.decryptBulkFallible.mockResolvedValueOnce([
214+
{ data: 'ok' },
215+
{ error: 'boom-one' },
216+
{ error: 'boom-two' },
217+
] as never)
218+
const err = await c
219+
.bulkDecrypt([null, ct('a'), ct('b'), ct('c')])
220+
.catch((e: Error) => e)
221+
222+
expect((err as Error).message).toContain('[2]: boom-one')
223+
expect((err as Error).message).toContain('[3]: boom-two')
224+
})
225+
226+
it('succeeds when every item decrypts', async () => {
227+
const c = await client()
228+
await expect(c.bulkDecrypt([ct('a'), ct('b')])).resolves.toEqual([
229+
'plain-0',
230+
'plain-1',
231+
])
232+
})
233+
})
234+
235+
// Results are matched to inputs BY POSITION — the FFI payloads carry no
236+
// correlation id. A short response would otherwise leave trailing slots null,
237+
// which a caller cannot tell apart from "this row had no value".
238+
describe('bulk result/input length mismatch', () => {
239+
beforeEach(() => {
240+
vi.clearAllMocks()
241+
})
242+
243+
it('bulkEncrypt throws rather than returning a partially-null batch', async () => {
244+
ffi.encryptBulk.mockResolvedValueOnce([{ v: 3, i: {}, c: 'only-one' }])
245+
246+
const c = await client()
247+
await expect(
248+
c.bulkEncrypt([
249+
{ plaintext: 'a', table: users, column: users.email },
250+
{ plaintext: 'b', table: users, column: users.email },
251+
]),
252+
).rejects.toThrow(/sent 2 payload\(s\).*received 1 back/s)
253+
})
254+
255+
it('bulkDecrypt throws rather than returning a partially-null batch', async () => {
256+
ffi.decryptBulkFallible.mockResolvedValueOnce([
257+
{ data: 'only-one' },
258+
] as never)
259+
260+
const c = await client()
261+
await expect(c.bulkDecrypt([ct('a'), ct('b')])).rejects.toThrow(
262+
/sent 2 payload\(s\).*received 1 back/s,
263+
)
264+
})
265+
})

0 commit comments

Comments
 (0)