fix(v3): resolve column domains when the client is built, not per row - #133
fix(v3): resolve column domains when the client is built, not per row#133coderdan wants to merge 1 commit into
Conversation
|
Warning Review limit reached
Next review available in: 59 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (6)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
96e8c87 to
f4909ca
Compare
Every encrypted row re-derived its column's eql_v3 domain name and re-scanned the eql-bindings domain inventory. Resolve every configured column once, when the client is built, and hand the encrypt seams the resolved target. Resolving up front also makes an unrepresentable column a configuration error rather than a per-row one: it now fails newClient, naming the offending table.column, instead of on the first encrypt to it — which left a configured-but-never-written column silently broken. This is deliberately fatal to the whole client, so it is a BREAKING change for a v3 config containing such a column that the app only decrypts or never touches: that client built before and does not now. CHANGELOG carries the note and the remedy (read-only services can build with eqlVersion 2 — decrypt is version-agnostic and reads v3 payloads regardless). ResolvedEncryptConfig owns the whole fail-fast sequence (validate eqlVersion, parse the config, resolve each column) so the Neon and wasm clients cannot drift: both now hold one Arc and share the sequence rather than hand-assembling the same three fields behind identical resolver() methods. Each column's config and target live in one entry, so resolve() serves both from a single lookup and no second map's keys can disagree with the first. Config parsing now runs after the auth strategy is built, before any network I/O — the strategy is built first on both targets, so a broken strategy reports the same way on each. Config errors now precede auth/network errors for all clients, both wire versions. - errors name table.column: a whole-config sweep has no encrypt(table, column) call site to disambiguate the bare name, and two tables may configure the same column - encrypt_bulk carries the Copy OutputTarget out of its first loop on both targets, as encrypt_query_bulk already did, dropping a per-row re-lookup and an unreachable error arm after rows were encrypted - the conversion tests resolve through ResolvedEncryptConfig itself rather than a hand-rolled match their doc claimed was the real seam - new tests: the resolved target's variant and domain (nothing asserted either), unknown-column, table disambiguation, and the wasm build's rejection wiring, which no test reached
f4909ca to
9bcec22
Compare
|
Rebased onto latest main and addressed a full review pass. Summary of what changed since the last push: Behaviour decision (the load-bearing one)The review confirmed that resolving domains at Decision: keep the fail-fast rejection. A config declaring a column v3 cannot store is a configuration error, and reporting it at build — once, naming the column — is the point of the change. But it is a breaking change, so it is now documented as one under the repo's Fixes
Also fixed a changelog misfiling the rebase introduced: the entry had context-matched into the released Not changed: no integration test covers an encrypt-time v3 conversion error, because no user input can reach those paths on a validly-built client — they're fail-closed invariant guards, unit-tested in 218 unit tests pass (+2), both targets compile, fmt/lint/typecheck clean. |
There was a problem hiding this comment.
Pull request overview
This PR moves EQL v3 column-domain resolution from per-encrypted-row (and per-query payload) to client construction time, so invalid v3 column configurations fail fast in newClient and bulk encryption avoids repeated domain parsing/allocation work.
Changes:
- Introduces a pre-resolved encrypt configuration (
ResolvedEncryptConfig) and per-columnOutputTarget(v2 vs v3 + resolvedTargetDomain) and threads this through Neon + wasm encrypt/query output seams. - Updates integration tests to assert the new error timing (v3 config rejection at
newClient, while v2 still accepts the same config) and adds wasm wiring tests for the same seam. - Documents the behavior change and performance improvement in
CHANGELOG.md.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| integration-tests/tests/wasm-round-trip.test.ts | Adds wasm-side validation tests to ensure v3 domain resolution (and its failures) happen during newClient without requiring valid credentials. |
| integration-tests/tests/eql-v3.test.ts | Updates v3 configuration error expectations to occur at newClient rather than first encrypt. |
| crates/protect-ffi/src/wasm.rs | Switches wasm client to store a resolved config and passes OutputTarget into output shaping to avoid per-row domain derivation. |
| crates/protect-ffi/src/lib.rs | Switches Neon client to store a resolved config; updates encrypt/query entry points to use OutputTarget and resolves config before ZeroKMS setup. |
| crates/protect-ffi/src/eql_v3.rs | Adds OutputTarget + ResolvedEncryptConfig, resolves v3 TargetDomain once at build time, and updates storage/query output helpers and tests accordingly. |
| CHANGELOG.md | Adds breaking-change note and documents domain-resolution timing/performance change. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| - **An `eqlVersion: 3` client no longer builds when ANY configured column is | ||
| one EQL v3 cannot represent** — even a column you only decrypt, query by | ||
| selector, or never touch. `newClient` now maps every configured column onto |
Follow-up to #132, addressing findings 2, 4 and 5 from the re-review.
Findings 2 and 5 turned out to have one shared fix, so they land together.
The docs were describing behaviour that didn't exist (#2)
target_domain_for_columnwas only reached fromstorage_output/query_output, so the EQL v3 column-domain check ran at encrypt time. ButCHANGELOG.md:91,README.md:98anddocs/jsonb-api-reference.md:319all said it happened "at configuration time", and the comment on the ste_vec guard said it fired "instead of at encrypt time" while sitting on the encrypt path.The repo's own test proved the docs wrong —
newClientaccepted an invalid v3 config and onlyencryptrejected it:Rather than reword three documents, this makes the claim true.
newClientnow resolves every configured column onto itseql_v3domain up front — before any network I/O, alongside the existingeqlVersionfail-fast — and stores the result on the client.This is a behaviour change. An unrepresentable v3 column now throws
EQL_V3_UNSUPPORTED_COLUMNfromnewClientinstead of on the firstencryptto that column. That closes a real gap: a column that was configured but never written to previously never errored at all. v2 clients are untouched — their payloads pass through unconverted, so no domain is needed, and the same configs stay valid there.The two tests at
eql-v3.test.ts:696-740are updated to assert againstnewClient, and three Rust tests pin the new seam directly (v2 resolves nothing; v3 resolves every column; v3 rejects and names the offending one).Per-row domain re-derivation on the bulk path (#5)
Falls out of the above.
storage_outputran once per payload inside both bulk loops (lib.rs:1408,wasm.rs:515), and each call allocated the domainStringand then calledTargetDomain::parse, which rebuilds a ~52-elementVec<Box<dyn DomainType>>and linear-scans it comparing strings. The resolvedTargetDomainisCopyand depends only on the column config, so it now comes from a map built once at client construction.Two supporting types keep this honest rather than bolted on:
OutputTargetpairs the wire version with the resolved domain (V2 | V3(TargetDomain)), so the v2/v3 split is a single exhaustivematchat each output seam instead of a version check followed by a fallible lookup that "can't fail".ColumnResolverbundles the three client fields the encrypt entry points need. The Neon and wasm clients share no type but hand exactly these three to the same seams. It also keepsprepare_query_plaintextunder clippy's argument limit (it would otherwise have hit 8) and removes the duplicated.ok_or_else(|| Error::UnknownColumn(...))blocks from four call sites.Columns resolve in identifier order.
HashMapiteration order varies per map instance, so a config with two unrepresentable columns would otherwise name a different one on each run — an error that moves between identical runs reads like a flake. There's a test for it, and it's a real guard: removing the sort fails it on every run.The missing inventory case (#4)
every_selected_domain_resolves_in_the_v3_inventorynever exercisedunique + ope + match→eql_v3_text_search, the only case reaching the_searcharm added in #132, and the only domain name the_search_orecase didn't already cover. A typo in that arm's suffix would have slipped past the one test written to catch exactly that, and surfaced at encrypt time as anInvariantViolation. One line.Not included
Findings 6, 7 and 8 (the per-arm
v3_domainprefix obligation, the redundantsv/sv_modestate, and the double allocation ateql_v3.rs:374) are untouched — pure internal quality, no user-visible behaviour, and better kept out of a diff that already changes when an error fires.Verification
cargo fmt --check,cargo clippy --no-deps --tests --all-features --all-targets -- -D warnings(the CI invocation), 211 Rust unit tests, thewasm32-unknown-unknowntarget,npm test(typecheck + 56 unit + lint + format), and a separatetsc --noEmitoverintegration-tests/, which reports the same three pre-existing@cipherstash/auth/wasm-inlinemodule-resolution errors asmainand nothing new.The integration tests themselves were not run — they need Docker, Postgres and CipherStash credentials. The
eql-v3.test.tschanges here are exactly the two tests whose behaviour this PR moves, so they are unproven end to end.