Commit 508f1d5
authored
feat(stack)!: align wasm-inline to the Result contract, and add bulkEncrypt/bulkDecrypt (#741)
* feat(stack): bulkEncrypt/bulkDecrypt on the wasm-inline entry (#737)
The WASM entry exposed no bulk operations at all — only encrypt/decrypt —
so rendering an N-row list on Deno, Cloudflare Workers, or Supabase Edge
Functions cost N sequential ZeroKMS round trips. Combined with the ~6.6s
cold start that made list endpoints impractical on the edge, and it left
the entry with no access to the bulk path AGENTS.md names as the way to
exercise ZeroKMS bulk speed. Surfaced as M1 in the rc.3 skilltester run.
The FFI already had the primitives — `encryptBulk`, `decryptBulkFallible`
are exported from protect-ffi's WASM build (identical in 0.29 and 0.30).
This wires them up; no new capability was needed underneath.
Signature deliberately differs from the native entry's bulk methods. It
follows `encryptQueryBulk`, the bulk primitive already on this client: an
index-aligned array with per-item table/column routing, throwing rather
than a `{ data } | { failure }` envelope. No `{ id, plaintext }` payload
envelopes either — protect-ffi's `EncryptPayload` has no `id` field, so
the native one is dropped at the boundary and buys nothing once positions
are stable. Consistency within this surface beats parity with a different
entry point, since a WASM caller is reading this module's API.
Per-item routing is what makes the saving real: one call covers several
columns across many rows, where a single-column batch would still cost one
round trip per column.
Two failure modes are handled rather than assumed away:
- `bulkDecrypt` builds on the FALLIBLE primitive, so when items fail it
throws once naming every failing index and reason, instead of surfacing
the first and discarding work the caller already paid for.
- Both methods assert the FFI returned as many results as were sent.
Matching is positional (no correlation id), so a short response would
silently leave trailing slots null — indistinguishable from "this row
had no value". That is silent wrong data, so it throws.
Model helpers stay Node-only: this entry has no single-model operation to
build them on, so adding bulk-model alone would be incoherent. Noted in
the class docblock as a separate port.
14 new unit tests: one-FFI-call-per-batch, payload shape, mixed
tables/columns with declared-vs-property column names, position stability
across null/undefined, all-null and empty short-circuits, per-index
failure reporting, and both length-mismatch guards. 823 tests pass;
typecheck and build clean; 0 biome errors.
Skill updated — `stash-encryption` documented only the native bulk shape,
which would not compile on the edge.
* feat(stack)!: align the wasm-inline entry to the Result contract
The WASM entry threw on every fallible method — `encrypt`, `decrypt`,
`encryptQuery`, `encryptQueryBulk` — while every other surface in the repo
returns `{ data } | { failure }`. AGENTS.md states that as a contract, not
a preference: "Operations return `{ data }` or `{ failure }`. Preserve
this shape and error `type` values in `EncryptionErrorTypes`."
This was drift, not a design decision. Nothing about WASM prevents it:
`@byteslice/result` is ALREADY bundled into dist/wasm-inline.js
(tsup.config.ts `noExternal`). The cost was that edge code had to be
written in a different shape from every other surface, with failures that
were easy to miss.
Fixed now because it is breaking and 1.0.0 has not shipped —
`@cipherstash/stack@latest` is still 0.19.0, so this surface has only ever
been published under the `rc` tag. After GA it would have waited for a
major. The earlier commit on this branch made the problem worse by adding
two more throwing methods; this corrects both.
All six fallible methods now return `Result<T, EncryptionError>`, with
`failure.type` distinguishing encrypt-side (`EncryptionError`) from
decrypt-side (`DecryptionError`) and `failure.code` carrying the FFI code.
`isEncrypted` stays a bare boolean — a pure predicate with nothing to fail
at, as on the native entry.
Consumers updated: the supabase-worker example (now returns a 500 with the
failure message rather than throwing past the handler), and the WASM
integration adapter via an `unwrap` helper that aborts the harness loudly.
A new `wasm-inline-result-contract.test.ts` pins the contract on both
paths for all six methods so it cannot drift back.
One thing found while testing: `withResult`'s `ensureError` REPLACES any
non-Error throw with `new Error('Something went wrong')` before the error
mapper runs, discarding the original value (@byteslice/result@0.2.0). That
is library-wide — the native entry has it too — so it is pinned as a test
rather than worked around, and will surface if the #532 bump to 0.5.0
changes it.
831 tests pass. Zero new tsc errors (verified by diffing the error set
against main — the pre-existing 107 are unchanged). 0 biome errors.
* fix(stack): preserve non-Error rejection detail on the wasm-inline entry
`withResult`'s default `ensureError` REPLACES any non-Error throw with
`new Error('Something went wrong')`, discarding the original value
(@byteslice/result@0.2.0, dist/result.js:27). But that is only the
FALLBACK — the library prefers an `onException` hook:
const error = hooks?.onException?.(ex) ?? ensureError(ex)
Nothing in this repo passes it. On the native entry that rarely bites,
because the FFI throws real `ProtectError` instances. On WASM it is a
live hazard: wasm-bindgen rejects with the raw `JsValue` the Rust side
produced (`throw takeFromExternrefTable0(...)` in the generated glue),
and the WASM build exports no `ProtectError` class at all — so a genuine
FFI failure can arrive as a bare string or object.
That made the Result conversion in the previous commit a partial
REGRESSION: the old throwing behaviour at least propagated the raw value
to the caller, whereas `withResult` was silently swallowing it. Fixed by
supplying `onException`.
The six call sites now go through one `wasmResult()` seam that binds both
the failure shape and the hook, so a method added later cannot omit
either — an omission would not fail a build, it would just quietly
degrade failure messages, which is the whole bug.
`toError` preserves strings as-is, JSON-serializes objects (so
`{code, detail}` survives rather than becoming "[object Object]"), and
falls back to `String()` for cycles and other unserializable values.
+2 tests covering string and object preservation, plus the cyclic
fallback. 833 pass; 0 biome errors; zero new tsc errors (diffed against
main's error set, not counted).
* fix(stack): address review — native FFI leak, Result plumbing, batch bugs
Review found 15 issues. This commit takes the source and test ones.
CRITICAL (1/15) — the entry pulled in the NATIVE protect-ffi.
`@/encryption/helpers/error-code` value-imports `ProtectError` for an
`instanceof` narrow, and protect-ffi is not in tsup `noExternal`, so
`dist/wasm-inline.js` carried a bare `@cipherstash/protect-ffi` import —
the NAPI entry, in the one bundle that exists to avoid it. On Workers /
Edge the non-`node` condition resolves it to a module exporting no
`ProtectError`; under Deno it resolves to the NAPI loader.
I had previously dismissed this as pre-existing. That was wrong: I ran
`git stash -u` on a clean tree, so it stashed nothing and I rebuilt my
own branch and compared it against itself. Re-verified properly by
swapping in main's `wasm-inline.ts` and rebuilding: main emits only the
`/wasm-inline` specifier, mine emitted both. Replaced with a structural
`readErrorCode` — which is the only thing that could ever work here
anyway, since the WASM build ships no error class for `instanceof` to
match. GH #744 is filed on my faulty premise and needs correcting.
New `wasm-inline-bundle-isolation.test.ts` asserts the built bundle's
external imports, so this class of regression fails CI. Verified it
catches the exact regression by reintroducing it (2 of 3 assertions
fail), rather than assuming.
3/15 — `failure.code` could never be populated: `withResult` runs
`onException` first, so the mapper only ever saw a fresh Error. `toError`
now carries a structural `code` onto the synthesized Error, and
`bulkDecrypt` puts each item's FFI code in its per-index message (a batch
has no single code, but dropping it lost the only machine-readable part).
9/15 — `toError`'s `String(ex)` fallback could itself throw on a
null-prototype object, escaping the Result contract entirely and
rejecting the call. Guarded with `safeString`.
10/15 — sparse inputs. `items.map(() => null)` SKIPS holes, so a hole
came back `undefined` rather than the documented `null` while lengths
still matched, which the length guard cannot see. Now `Array.from`.
15/15 — the positional-batch scaffolding was hand-rolled three times, so
a fourth batch method could omit `assertBatchLength` with no build
failure. One `runBatch` helper owns compaction, short-circuit, length
assert and scatter; the sparse fix lives there once.
5/15, 6/15, 7/15 — docs and surface: removed a duplicate stale `@returns`,
migrated the module-header and all five `encryptQuery` examples off the
pre-Result contract (they interpolated the envelope straight into SQL),
corrected the pending `.changeset/wasm-encrypt-query.md` sentence that
still said errors throw, and exported `EncryptionError` /
`EncryptionErrorTypes` so an edge consumer can discriminate
`failure.type` from the same import.
`WasmResult<T>` is now declared locally rather than re-exporting
`@byteslice/result`'s. Re-exporting put that specifier in the emitted
`.d.ts`, which Deno cannot resolve — the same reason `WasmPlaintext`
re-declares protect-ffi's `JsPlaintext`.
14/15 — test gaps: falsy-but-live values (`0` / `''` / `false`) per batch
method, the sparse-hole case, `encryptQueryBulk`'s previously untested
length guard, and the two stale "throws" titles.
840 tests pass. Zero new tsc errors, diffed against main's error set.
0 biome errors. Bundle verified free of the native FFI.
* fix(stack): review round 2 — e2e, example, skill, and adapter bulk usage
2/15 — the Deno e2e suites still consumed bare values, so with live
credentials `isEncrypted(encrypted)` received a Result, `decrypt` was
handed the envelope as ciphertext, and every `assertV3Term` saw
`term.v === undefined`. Secret-gated, so it would have landed silently.
Both suites now unwrap at each boundary. `roundtrip` also gains the first
live coverage of `bulkEncrypt`/`bulkDecrypt` anywhere in the repo —
including the index-aligned null hole — which the unit tests' "live
coverage runs in the Deno e2e" claim previously had no backing for.
(`deno check` reports pre-existing column-brand errors on these files;
the task runs `--no-check` deliberately for that reason, documented in
`e2e/wasm/deno.json`. Confirmed no NEW error kinds were introduced.)
4/15 — the supabase-worker example could not run against any published
version: it pinned `@^0.18.0` (bare-value entry, so `encryptResult.data`
was undefined and `decrypt` threw), and bumping the pin broke it earlier
still because it authored a v2 schema the v3-only entry rejects. Pinned
to `^1.0.0-rc.3` and migrated to `types.TextEq`. README's surface claim
updated with it.
8/15 — the skill's WASM bulk section sat in a file where the running
`client` is the NATIVE one, so an agent would apply the per-item shape to
`bulkEncrypt(plaintexts, { table, column })` and fail in a customer repo.
It now constructs the WASM client explicitly, with a callout that it is a
different client, and `@cipherstash/stack/wasm-inline` is listed in the
subpath table.
13/15 — the WASM integration adapter encrypted one field per request
inside `Promise.all`: concurrency hides latency, not request count, so a
100-row × 5-field family was 500 ZeroKMS requests. Now one `bulkEncrypt`
per row, which also makes the harness live coverage for the path.
840 tests pass; zero new tsc errors; 0 biome errors.1 parent cddb182 commit 508f1d5
15 files changed
Lines changed: 1547 additions & 151 deletions
File tree
- .changeset
- e2e/wasm
- examples/supabase-worker
- supabase/functions/cipherstash-roundtrip
- packages/stack
- __tests__
- helpers
- integration/wasm
- src
- skills/stash-encryption
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
17 | | - | |
18 | | - | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
72 | 72 | | |
73 | 73 | | |
74 | 74 | | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
75 | 95 | | |
76 | 96 | | |
77 | 97 | | |
| |||
119 | 139 | | |
120 | 140 | | |
121 | 141 | | |
122 | | - | |
123 | | - | |
124 | | - | |
125 | | - | |
126 | | - | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
127 | 150 | | |
128 | 151 | | |
129 | 152 | | |
130 | 153 | | |
131 | 154 | | |
132 | | - | |
133 | | - | |
134 | | - | |
135 | | - | |
136 | | - | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
137 | 163 | | |
138 | 164 | | |
139 | 165 | | |
140 | 166 | | |
141 | 167 | | |
142 | | - | |
143 | | - | |
144 | | - | |
145 | | - | |
146 | | - | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
147 | 176 | | |
148 | 177 | | |
149 | 178 | | |
150 | 179 | | |
151 | 180 | | |
152 | 181 | | |
153 | | - | |
154 | | - | |
155 | | - | |
156 | | - | |
157 | | - | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
158 | 190 | | |
159 | 191 | | |
160 | 192 | | |
| |||
171 | 203 | | |
172 | 204 | | |
173 | 205 | | |
174 | | - | |
175 | | - | |
176 | | - | |
177 | | - | |
178 | | - | |
179 | | - | |
180 | | - | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
181 | 216 | | |
182 | 217 | | |
183 | 218 | | |
184 | 219 | | |
185 | 220 | | |
186 | 221 | | |
187 | 222 | | |
188 | | - | |
189 | | - | |
190 | | - | |
191 | | - | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
192 | 230 | | |
193 | 231 | | |
194 | 232 | | |
195 | 233 | | |
196 | 234 | | |
197 | | - | |
| 235 | + | |
198 | 236 | | |
199 | 237 | | |
200 | 238 | | |
| |||
225 | 263 | | |
226 | 264 | | |
227 | 265 | | |
| 266 | + | |
228 | 267 | | |
229 | 268 | | |
230 | 269 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
102 | 102 | | |
103 | 103 | | |
104 | 104 | | |
105 | | - | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
106 | 109 | | |
107 | 110 | | |
108 | 111 | | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
109 | 118 | | |
110 | 119 | | |
111 | 120 | | |
| |||
138 | 147 | | |
139 | 148 | | |
140 | 149 | | |
141 | | - | |
142 | | - | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
143 | 157 | | |
144 | 158 | | |
145 | 159 | | |
146 | 160 | | |
147 | | - | |
| 161 | + | |
148 | 162 | | |
149 | 163 | | |
150 | 164 | | |
151 | | - | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
152 | 172 | | |
153 | 173 | | |
154 | 174 | | |
| |||
158 | 178 | | |
159 | 179 | | |
160 | 180 | | |
161 | | - | |
| 181 | + | |
162 | 182 | | |
163 | 183 | | |
164 | 184 | | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
165 | 191 | | |
166 | 192 | | |
167 | 193 | | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
168 | 224 | | |
169 | 225 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
62 | | - | |
| 62 | + | |
63 | 63 | | |
64 | 64 | | |
65 | 65 | | |
Lines changed: 26 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
18 | | - | |
19 | 18 | | |
20 | 19 | | |
21 | | - | |
| 20 | + | |
| 21 | + | |
22 | 22 | | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
23 | 27 | | |
24 | | - | |
| 28 | + | |
25 | 29 | | |
26 | 30 | | |
27 | 31 | | |
| |||
61 | 65 | | |
62 | 66 | | |
63 | 67 | | |
64 | | - | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
65 | 71 | | |
66 | 72 | | |
67 | 73 | | |
68 | | - | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
69 | 90 | | |
70 | 91 | | |
71 | 92 | | |
| |||
0 commit comments