feat: upgrade protect-ffi to 0.23.0 (split storage/query types)

protect-ffi 0.23.0 is a types-only release that splits the previously-
conflated `Encrypted` type into:

- `Encrypted` — storage-only payload returned by encrypt/encryptBulk,
  the only shape decrypt accepts. `c` is now required (was `c?`).
- `EncryptedQuery` (new) — query payload returned by encryptQuery /
  encryptQueryBulk for scalar unique/match/ore lookups and ste_vec
  selector path queries. Carries no ciphertext (`c?: never`) so the
  union discriminates cleanly via `'c' in payload`. JSON containment
  queries (ste_vec_term) still come back as a storage-shaped
  `Encrypted`, hence the union return.
- `isEncrypted` arg widened to `unknown`.
- `EncryptedSteVecStorage` collapsed into `EncryptedSteVec`.

No runtime change; consumers that lied about the loose union now have
to narrow properly.

Stack-side updates:

- `EncryptedSearchTerm` and `EncryptedQueryResult` (in protect and
  stack) widen to `Encrypted | EncryptedQuery | string [| null]`.
- `formatEncryptedResult`, `encryptedToCompositeLiteral`, and
  `encryptedToEscapedCompositeLiteral` accept the union via a new
  internal `EncryptedQueryTerm = CipherStashEncrypted |
  CipherStashEncryptedQuery` alias.
- `BatchEncryptQueryOperation.assembleResults` takes the union for the
  FFI's bulk return shape.
- `protect-dynamodb`'s `SearchTermsOperation` narrows via
  `'hm' in term && typeof term.hm === 'string'` so the new
  `EncryptedScalarQuery & { hm }` / `& { bf }` / `& { ob }` discriminator
  unions narrow cleanly without reading off the loose union.

Verified: pnpm test (1876 passed | 20 skipped, 14/14 packages) and
prisma-next e2e (36/36 across the seven shape suites). No DB-side
changes needed since the runtime is unchanged from 0.22.0 → 0.23.0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: 2 people

.changeset/protect-ffi-0-22-0.md .changeset/protect-ffi-0-23-0.md.changeset/protect-ffi-0-22-0.md renamed to .changeset/protect-ffi-0-23-0.md

@@ -7,11 +7,12 @@
 "stash": minor
 ---
 
-Upgrade `@cipherstash/protect-ffi` to `0.22.0` and the bundled CipherStash EQL extension to `eql-2.3.1`.
+Upgrade `@cipherstash/protect-ffi` to `0.23.0` and the bundled CipherStash EQL extension to `eql-2.3.1`.
 
 Breaking upstream changes adopted in this release:
 
-- **Encrypt-config schema version**: `buildEncryptConfig` now emits `{ v: 1, ... }` (was `{ v: 2, ... }`). protect-ffi `0.22.0` validates this field and rejects any value other than `1` with the new `UNSUPPORTED_CONFIG_VERSION` error code.
+- **Encrypt-config schema version**: `buildEncryptConfig` now emits `{ v: 1, ... }` (was `{ v: 2, ... }`). protect-ffi `0.22.0` started validating this field and rejects any value other than `1` with the new `UNSUPPORTED_CONFIG_VERSION` error code.
+- **Storage and query payloads are now distinct types** (protect-ffi `0.23.0`): the previously-conflated `Encrypted` type splits into `Encrypted` (storage-only, `c` required) and a new `EncryptedQuery` (search terms — scalar `unique`/`match`/`ore` lookups and `ste_vec_selector` JSON path queries; no `c`). JSON containment queries (`ste_vec_term`) still return a storage-shaped `Encrypted` payload. `encryptQuery` / `encryptQueryBulk` now return `Encrypted | EncryptedQuery`, and the stack's `EncryptedSearchTerm` / `EncryptedQueryResult` unions widen to match. `decrypt` rejects query payloads at the type level. The DynamoDB `SearchTermsOperation` narrows via `'hm' in term` rather than `term.hm`.
 - **SteVec encoding default flipped**: protect-ffi's default `mode` for `ste_vec` indexes changed from `compat` to `standard`. The two encodings are not cross-compatible. Existing JSON-searchable data that was indexed under `compat` will need to be re-encrypted to be queryable. The stack adopts the new `standard` default — there is no longer a way to pin `compat` from the SDK.
 - **EQL extension bumped to `eql-2.3.1`**: the new SteVec `standard` encoding requires matching support in the database EQL extension. The CLI's bundled SQL (`packages/cli/src/sql/*.sql`) and the `@cipherstash/prisma-next` install bundle (`migrations/20260601T0000_install_eql_bundle/ops.json` + `eql-install.generated.ts`) are updated to `eql-2.3.1`. Databases installed with an older EQL extension must be reinstalled (`stash db install`) before containment / contained-by queries against SteVec columns will work. `eql-2.3.1` ships the `_encrypted_check_c` fix for SteVec storage payloads ([cipherstash/encrypt-query-language#232](https://github.com/cipherstash/encrypt-query-language/issues/232)).
 - **New error codes**: `ProtectErrorCode` (re-exported from `@cipherstash/protect-ffi`) gains `MATCH_REQUIRES_TEXT` and `UNSUPPORTED_CONFIG_VERSION`. Exhaustive switches over `ProtectErrorCode` will need additional cases.

packages/protect-dynamodb/src/operations/search-terms.ts

@@ -53,7 +53,10 @@ export class SearchTermsOperation extends DynamoDBOperation<string[]> {
             )
           }
 
-          if (term?.k !== 'ct' || !term.hm) {
+          // DynamoDB lookups go through equality queries → the FFI returns an
+          // EncryptedScalarQuery carrying `hm`. Anything else (scalar storage,
+          // a `bf`/`ob` query, or a SteVec payload) is a misconfiguration.
+          if (term?.k !== 'ct' || !('hm' in term) || typeof term.hm !== 'string') {
             throw new Error('expected encrypted search term to have an HMAC')
           }
 

packages/protect/package.json

@@ -69,7 +69,7 @@
 	},
 	"dependencies": {
 		"@byteslice/result": "^0.2.0",
-		"@cipherstash/protect-ffi": "0.22.0",
+		"@cipherstash/protect-ffi": "0.23.0",
 		"@cipherstash/schema": "workspace:*",
 		"@stricli/core": "^1.2.7",
 		"dotenv": "17.4.2",

packages/protect/src/ffi/operations/batch-encrypt-query.ts

@@ -4,7 +4,10 @@ import {
   type QueryPayload,
   encryptQueryBulk as ffiEncryptQueryBulk,
 } from '@cipherstash/protect-ffi'
-import type { Encrypted as CipherStashEncrypted } from '@cipherstash/protect-ffi'
+import type {
+  Encrypted as CipherStashEncrypted,
+  EncryptedQuery as CipherStashEncryptedQuery,
+} from '@cipherstash/protect-ffi'
 import { type ProtectError, ProtectErrorTypes } from '../..'
 import { logger } from '../../../../utils/logger'
 import { formatEncryptedResult } from '../../helpers'
@@ -83,7 +86,7 @@ function buildQueryPayload(
  */
 function assembleResults(
   totalLength: number,
-  encryptedValues: CipherStashEncrypted[],
+  encryptedValues: (CipherStashEncrypted | CipherStashEncryptedQuery)[],
   nonNullTerms: { term: ScalarQueryTerm; originalIndex: number }[],
 ): EncryptedQueryResult[] {
   const results: EncryptedQueryResult[] = new Array(totalLength).fill(null)

packages/protect/src/helpers/index.ts

@@ -1,5 +1,6 @@
 import type {
   Encrypted as CipherStashEncrypted,
+  EncryptedQuery as CipherStashEncryptedQuery,
   KeysetIdentifier as KeysetIdentifierFfi,
 } from '@cipherstash/protect-ffi'
 import type {
@@ -8,6 +9,14 @@ import type {
   KeysetIdentifier,
 } from '../types'
 
+/**
+ * The shape `encryptQuery` / `encryptQueryBulk` can return: a full storage
+ * payload (`Encrypted`, returned for `ste_vec_term` containment queries) or a
+ * query-only payload with no ciphertext (`EncryptedQuery`, returned for
+ * scalar `unique`/`match`/`ore` lookups and `ste_vec_selector` path queries).
+ */
+type EncryptedQueryTerm = CipherStashEncrypted | CipherStashEncryptedQuery
+
 export type EncryptedPgComposite = {
   data: Encrypted
 }
@@ -43,7 +52,7 @@ export function encryptedToPgComposite(obj: Encrypted): EncryptedPgComposite {
  * await supabase.from('table').select().eq('column', searchTerm)
  * ```
  */
-export function encryptedToCompositeLiteral(obj: CipherStashEncrypted): string {
+export function encryptedToCompositeLiteral(obj: EncryptedQueryTerm): string {
   if (obj === null) {
     throw new Error('encryptedToCompositeLiteral: obj cannot be null')
   }
@@ -71,7 +80,7 @@ export function encryptedToCompositeLiteral(obj: CipherStashEncrypted): string {
  * ```
  */
 export function encryptedToEscapedCompositeLiteral(
-  obj: CipherStashEncrypted,
+  obj: EncryptedQueryTerm,
 ): string {
   if (obj === null) {
     throw new Error('encryptedToEscapedCompositeLiteral: obj cannot be null')
@@ -80,7 +89,7 @@ export function encryptedToEscapedCompositeLiteral(
 }
 
 export function formatEncryptedResult(
-  encrypted: CipherStashEncrypted,
+  encrypted: EncryptedQueryTerm,
   returnType?: string,
 ): EncryptedQueryResult {
   if (returnType === 'composite-literal') {

packages/protect/src/types.ts

@@ -1,5 +1,6 @@
 import type {
   Encrypted as CipherStashEncrypted,
+  EncryptedQuery as CipherStashEncryptedQuery,
   JsPlaintext,
   QueryOpName,
   newClient,
@@ -17,10 +18,22 @@ import type {
 export type Client = Awaited<ReturnType<typeof newClient>> | undefined
 
 /**
- * Type to represent an encrypted payload
+ * Type to represent an encrypted payload stored in the database. Always carries
+ * a ciphertext — scalar payloads at the root (`c`), SteVec payloads at
+ * `sv[0].c`. For search-term payloads returned by `encryptQuery`, see
+ * {@link EncryptedQuery}.
  */
 export type Encrypted = CipherStashEncrypted | null
 
+/**
+ * Type to represent an encrypted query term (search needle) returned by
+ * `encryptQuery` / `encryptQueryBulk` for scalar (`unique` / `match` / `ore`)
+ * lookups and `ste_vec_selector` JSON path queries. Carries no ciphertext — it
+ * is matched against stored values, never decrypted. JSON containment queries
+ * (`ste_vec_term`) return a storage-shaped {@link Encrypted} payload instead.
+ */
+export type EncryptedQuery = CipherStashEncryptedQuery | null
+
 /**
  * Represents an encrypted payload in the database
  * @deprecated Use `Encrypted` instead
@@ -52,18 +65,21 @@ export type KeysetIdentifier =
     }
 
 /**
- * The return type of the search term based on the return type specified in the `SearchTerm` type
- * If the return type is `eql`, the return type is `Encrypted`
- * If the return type is `composite-literal`, the return type is `string` where the value is a composite literal
- * If the return type is `escaped-composite-literal`, the return type is `string` where the value is an escaped composite literal
+ * The return type of the search term based on the return type specified in the `SearchTerm` type.
+ * - `eql` → an `Encrypted` storage payload (for `ste_vec_term` containment) or an
+ *   {@link EncryptedQuery} term (for scalar lookups and `ste_vec_selector` queries).
+ * - `composite-literal` → `string` where the value is a composite literal.
+ * - `escaped-composite-literal` → `string` where the value is an escaped composite literal.
  */
-export type EncryptedSearchTerm = Encrypted | string
+export type EncryptedSearchTerm = Encrypted | EncryptedQuery | string
 
 /**
  * Result type for encryptQuery batch operations.
- * Can be Encrypted (default), string (for composite-literal formats), or null.
+ * Can be an `Encrypted` storage payload (e.g. for `ste_vec_term`), an
+ * {@link EncryptedQuery} term (for scalar lookups and `ste_vec_selector`),
+ * a `string` (for composite-literal formats), or `null` (for null inputs).
  */
-export type EncryptedQueryResult = Encrypted | string | null
+export type EncryptedQueryResult = Encrypted | EncryptedQuery | string | null
 
 /**
  * Represents a payload to be encrypted using the `encrypt` function

packages/stack/package.json

@@ -203,8 +203,13 @@
 	},
 	"dependencies": {
 		"@byteslice/result": "0.2.0",
+<<<<<<< HEAD
 		"@cipherstash/protect-ffi": "0.21.4",
 		"evlog": "1.11.0",
+=======
+		"@cipherstash/protect-ffi": "0.23.0",
+		"evlog": "1.9.0",
+>>>>>>> 4c2af98 (feat: upgrade protect-ffi to 0.23.0 (split storage/query types))
 		"uuid": "14.0.0",
 		"zod": "3.25.76"
 	},

packages/stack/src/encryption/helpers/index.ts

@@ -1,9 +1,18 @@
 import type { Encrypted, EncryptedQueryResult, KeysetIdentifier } from '@/types'
 import type {
   Encrypted as CipherStashEncrypted,
+  EncryptedQuery as CipherStashEncryptedQuery,
   KeysetIdentifier as KeysetIdentifierFfi,
 } from '@cipherstash/protect-ffi'
 
+/**
+ * The shape `encryptQuery` / `encryptQueryBulk` can return: a full storage
+ * payload (returned for `ste_vec_term` containment queries) or a query-only
+ * payload with no ciphertext (scalar `unique`/`match`/`ore` lookups and
+ * `ste_vec_selector` path queries).
+ */
+type EncryptedQueryTerm = CipherStashEncrypted | CipherStashEncryptedQuery
+
 export type EncryptedPgComposite = {
   data: Encrypted
 }
@@ -28,7 +37,7 @@ export function encryptedToPgComposite(obj: Encrypted): EncryptedPgComposite {
  * await supabase.from('table').select().eq('column', literal)
  * ```
  */
-export function encryptedToCompositeLiteral(obj: CipherStashEncrypted): string {
+export function encryptedToCompositeLiteral(obj: EncryptedQueryTerm): string {
   return `(${JSON.stringify(JSON.stringify(obj))})`
 }
 
@@ -42,7 +51,7 @@ export function encryptedToCompositeLiteral(obj: CipherStashEncrypted): string {
  * ```
  */
 export function encryptedToEscapedCompositeLiteral(
-  obj: CipherStashEncrypted,
+  obj: EncryptedQueryTerm,
 ): string {
   return JSON.stringify(encryptedToCompositeLiteral(obj))
 }
@@ -55,7 +64,7 @@ export function encryptedToEscapedCompositeLiteral(
  * - default (`'eql'` or omitted) → raw encrypted object
  */
 export function formatEncryptedResult(
-  encrypted: CipherStashEncrypted,
+  encrypted: EncryptedQueryTerm,
   returnType?: string,
 ): EncryptedQueryResult {
   if (returnType === 'composite-literal') {

packages/stack/src/encryption/operations/batch-encrypt-query.ts

@@ -9,7 +9,10 @@ import {
   type QueryPayload,
   encryptQueryBulk as ffiEncryptQueryBulk,
 } from '@cipherstash/protect-ffi'
-import type { Encrypted as CipherStashEncrypted } from '@cipherstash/protect-ffi'
+import type {
+  Encrypted as CipherStashEncrypted,
+  EncryptedQuery as CipherStashEncryptedQuery,
+} from '@cipherstash/protect-ffi'
 import { resolveIndexType } from '../helpers/infer-index-type'
 import {
   assertValidNumericValue,
@@ -58,7 +61,7 @@ function buildQueryPayload(
  */
 function assembleResults(
   terms: readonly ScalarQueryTerm[],
-  encryptedValues: CipherStashEncrypted[],
+  encryptedValues: (CipherStashEncrypted | CipherStashEncryptedQuery)[],
 ): EncryptedQueryResult[] {
   return terms.map((term, i) =>
     formatEncryptedResult(encryptedValues[i], term.returnType),

packages/stack/src/types.ts

@@ -6,6 +6,7 @@ import type {
 } from '@/schema'
 import type {
   Encrypted as CipherStashEncrypted,
+  EncryptedQuery as CipherStashEncryptedQuery,
   JsPlaintext,
   QueryOpName,
   newClient,
@@ -30,9 +31,20 @@ export type Client = Awaited<ReturnType<typeof newClient>> | undefined
 /** A branded type representing encrypted data. Cannot be accidentally used as plaintext. */
 export type EncryptedValue = Brand<CipherStashEncrypted, 'encrypted'>
 
-/** Structural type representing encrypted data. See also `EncryptedValue` for branded nominal typing. */
+/** Structural type representing encrypted data stored in the database. Always
+ * carries a ciphertext. See also `EncryptedValue` for branded nominal typing,
+ * and {@link EncryptedQuery} for the search-term shape returned by
+ * `encryptQuery`. */
 export type Encrypted = CipherStashEncrypted
 
+/** Structural type representing an encrypted query term (search needle)
+ * returned by `encryptQuery` / `encryptQueryBulk` for scalar
+ * (`unique` / `match` / `ore`) lookups and `ste_vec_selector` JSON path
+ * queries. Carries no ciphertext — matched against stored values, never
+ * decrypted. JSON containment queries (`ste_vec_term`) return a
+ * storage-shaped {@link Encrypted} payload instead. */
+export type EncryptedQuery = CipherStashEncryptedQuery
+
 // ---------------------------------------------------------------------------
 // Client configuration
 // ---------------------------------------------------------------------------
@@ -112,11 +124,16 @@ export type SearchTerm = {
   returnType?: EncryptedReturnType
 }
 
-/** Encrypted search term result: EQL object or composite literal string */
-export type EncryptedSearchTerm = Encrypted | string
+/** Encrypted search term result. `eql` return type yields either a storage
+ * payload (`Encrypted`, for `ste_vec_term`) or a query-only term
+ * (`EncryptedQuery`, for scalar lookups and `ste_vec_selector`); the
+ * `composite-literal` return types yield a string. */
+export type EncryptedSearchTerm = Encrypted | EncryptedQuery | string
 
-/** Result of encryptQuery (single or batch): EQL or composite literal string */
-export type EncryptedQueryResult = Encrypted | string
+/** Result of encryptQuery (single or batch). `eql` return type yields either a
+ * storage payload (`Encrypted`) or a query-only term (`EncryptedQuery`); the
+ * `composite-literal` return types yield a string. */
+export type EncryptedQueryResult = Encrypted | EncryptedQuery | string
 
 // ---------------------------------------------------------------------------
 // Model field types (encrypted vs decrypted views)