Skip to content

Commit e9a4a75

Browse files
grrowlclaude
andcommitted
fix(server): align filtered-sub predicate floor across SQL and JS evaluators (ADR-0013)
A filtered subscription's membership was decided by two evaluators that disagreed: the SQL snapshot (sql-compiler) and the JS delta/catch-up path (@tanstack/db's compileSingleRowExpression). Two clients could see different rows for the same `where` depending on connection timing. - ne: the SQL floor accepted `ne` but @tanstack/db has no `ne` (not-equal is `not(eq(...))`). Its compile error escaped handleSub past the UnsupportedPredicateError catch (which only wraps compileSubsetQuery), so no `reset` was sent and the client hung forever. Drop `ne` from the floor → rejected with `reset`; and make compilePredicate rethrow any @tanstack/db compile failure as UnsupportedPredicateError, caught around subs.add → reset (defensive: fail loud, never a hang, for any future floor skew). - like: SQLite LIKE is ASCII case-insensitive; @tanstack/db's `like` is case-sensitive. Set PRAGMA case_sensitive_like = ON in the DO constructor so the snapshot matches the delta path on every wake (connection-scoped; the IR→SQL compiler is the only LIKE producer). Also restrict the `like` pattern to a string literal — evaluateLike returns false for non-strings while SQL would coerce, so a numeric pattern desynced the paths. Floor is now eq, gt, gte, lt, lte, like, in, and, or, not — exactly the set the two evaluators agree on row-for-row. Pinned by tests/predicate-parity.test.ts (6 cases, both paths), tests/subscriptions.test.ts (JS-floor guard), and the sql-compiler ne/like-pattern rejections. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 7f573ed commit e9a4a75

11 files changed

Lines changed: 449 additions & 15 deletions

CHANGELOG.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,31 @@ While pre-1.0, the public API may change between 0.x releases.
1010

1111
_Nothing yet._
1212

13+
## [0.3.3] — 2026-06-13
14+
15+
### Fixed
16+
17+
- **A filtered subscription's membership no longer depends on which path
18+
decided it (ADR-0013).** The SQL snapshot and the JS delta/catch-up evaluators
19+
disagreed on two operators, so two clients could see different rows for the
20+
same `where` depending on connection timing:
21+
- **`ne` crashed the delta path and hung the client.** The SQL floor accepted
22+
`ne` but `@tanstack/db`'s evaluator has no `ne` (not-equal is `not(eq(...))`);
23+
its compile error escaped `handleSub` uncaught, so no `reset` was sent. `ne`
24+
is now off the floor and rejected with `reset` like any unsupported operator,
25+
and a defensive guard turns any predicate-compile failure into a `reset`
26+
rather than a hang. Use `not(eq(...))` for not-equal (unchanged for real
27+
clients).
28+
- **`like` was case-insensitive in the snapshot but case-sensitive in deltas.**
29+
The DO now sets `PRAGMA case_sensitive_like = ON`, making SQLite `LIKE` match
30+
`@tanstack/db`'s case-sensitive `like` on every path. (#17)
31+
32+
### Changed
33+
34+
- **Operator floor for server-side filtering dropped `ne`** — it is now
35+
`eq, gt, gte, lt, lte, like, in, and, or, not`, exactly the set the SQL and JS
36+
evaluators agree on row-for-row. `like` is now case-sensitive. (#17)
37+
1338
## [0.3.2] — 2026-06-13
1439

1540
### Added
Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# ADR-0013: Filtered-subscription membership — one evaluator is the source of truth; the floor is the verified-agreeing set
2+
3+
**Status**: Accepted
4+
**Date**: 2026-06-13
5+
**Characterization**: `tests/predicate-parity.test.ts` (originally plan 003)
6+
7+
## Context
8+
9+
A filtered subscription's membership is decided by **two different evaluators**
10+
depending on the path:
11+
12+
- The **initial snapshot** filters in SQLite: the `where` IR is lowered to SQL
13+
by `src/server/sql-compiler.ts` (`"col" = ?`, `"col" LIKE ?`, …).
14+
- **Live deltas and reconnect catch-up** filter in JavaScript: the same IR is
15+
compiled by `@tanstack/db`'s `compileSingleRowExpression` + `toBooleanPredicate`
16+
in `src/server/subscriptions.ts` (`sync-do.ts` delta/catch-up paths). The client's
17+
eager-write preflight (`do-collection.ts`) uses that same JS evaluator, and so do
18+
the client's own live queries.
19+
20+
If the two disagree on any row, a client's view of a filtered subset depends on
21+
*when it connected* — a row excluded by the snapshot can be pushed in by a later
22+
delta, and two clients silently diverge. Characterization found two real divergences:
23+
24+
1. **`ne` — crash + hang.** `sql-compiler.ts` accepted `ne` (lowered to `!=`), but
25+
`@tanstack/db`'s evaluator has no `ne` in its registry (not-equal is expressed as
26+
`not(eq(...))`). `compileSingleRowExpression` threw a `QueryCompilationError` from
27+
inside `subs.add`, which runs in `handleSub` **after** the `try/catch` that wraps
28+
`compileSubsetQuery`. The throw escaped uncaught: no `reset` was sent and the
29+
subscriber hung until timeout.
30+
2. **`like` — case divergence.** SQLite `LIKE` is ASCII case-*insensitive* by default;
31+
`@tanstack/db`'s `like` is case-*sensitive* (its case-insensitive variant is
32+
`ilike`, which is off-floor). A row `"HELLO"` matched `like "hello%"` in the SQL
33+
snapshot but not in the JS deltas.
34+
35+
## Decisions
36+
37+
### D1: `@tanstack/db`'s evaluator is the source of truth for membership
38+
39+
It already decides membership on three of the four sites (delta, catch-up, client
40+
preflight), and the client's live queries use the same library. The SQL snapshot is
41+
an *optimization* that must reproduce exactly the rows that evaluator accepts. So when
42+
the two disagree, **SQL conforms to `@tanstack/db`** — never the reverse.
43+
44+
### D2: The operator floor is the verified-agreeing set
45+
46+
Floor = `{ eq, gt, gte, lt, lte, like, in, and, or, not }`. **`ne` is removed** — it
47+
is not in `@tanstack/db`'s evaluator, and a real client emits `not(eq(...))` (which
48+
both paths handle identically). Anything outside the floor is rejected with
49+
`UnsupportedPredicateError``reset`, as before. `tests/predicate-parity.test.ts`
50+
pins row-for-row agreement for each floor operator across both paths; **any operator
51+
added to `COMPARATORS` must add a parity case** (enforced by review).
52+
53+
### D3: `LIKE` is made case-sensitive to match
54+
55+
The DO sets `PRAGMA case_sensitive_like = ON` in the `SyncDurableObject` constructor.
56+
The pragma is connection-scoped, and the constructor runs on every instantiation —
57+
including a hibernation wake — so it is always in force before any query (the same
58+
lifecycle the existing `setWebSocketAutoResponse` registration relies on). It is
59+
contained: the IR→SQL compiler is the only producer of `LIKE` in the codebase.
60+
(`case_sensitive_like` is a *setter-only* pragma — there is no getter — so it is
61+
verified behaviorally in tests, not by readback.) Case-*insensitive* matching is
62+
`ilike`, which stays off-floor (see deferred).
63+
64+
The `like` **pattern must be a string literal**: `@tanstack/db`'s `evaluateLike`
65+
returns `false` unless both operands are strings, whereas SQLite `LIKE` would
66+
coerce a non-string pattern (`123``'123'`) and match rows the JS path rejects.
67+
The compiler rejects a non-string `like` pattern with `UnsupportedPredicateError`
68+
(→ `reset`), keeping the floor *verified*-agreeing rather than merely usually so.
69+
70+
### D4: A defensive fail-loud guard at predicate compile
71+
72+
`compilePredicate` (`subscriptions.ts`) now wraps the `@tanstack/db` compile and
73+
re-throws any failure as `UnsupportedPredicateError`; `handleSub` catches it around
74+
`subs.add` and answers with `reset`. With the floors aligned this is belt-and-
75+
suspenders, but it guarantees fail-loud (a `reset`, never a hang) for any future
76+
operator that lands in one floor and not the other.
77+
78+
## Consequences
79+
80+
- **Behavior change (observable):** a `ne` subscription is now rejected with `reset`
81+
(was: an indefinite hang). `like` on a filtered subscription is now case-sensitive
82+
on every path (was: case-insensitive in the snapshot only, and divergent from
83+
deltas) — `"HELLO"` no longer matches `like "hello%"`. This aligns snapshot, delta,
84+
catch-up, and client-side semantics. Real `@tanstack/db` clients are unaffected by
85+
the `ne` removal — they emit `not(eq(...))`, which is fully supported.
86+
- **No idle timers / hibernation impact:** the pragma is set synchronously in the
87+
constructor; no polling introduced.
88+
- **Deferred:** `ilike` (case-insensitive `LIKE`) remains off-floor. Adding it would
89+
lower to `lower("col") LIKE lower(?)` (or equivalent) plus a parity case — a
90+
separate decision.
91+
- **Test coverage:** `tests/predicate-parity.test.ts` (6 cases, both paths) pins
92+
floor-operator agreement; `tests/subscriptions.test.ts` pins the JS-floor guard at
93+
the unit level; `tests/sql-compiler.test.ts` pins `ne` rejection and `not(eq)`
94+
lowering.

docs/adr/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,3 +19,4 @@ explains the displacement.
1919
| [0009](./0009-changelog-time-retention.md) | Changelog time-based retention; reset stale reconnects | Accepted |
2020
| [0010](./0010-typed-mutations-collection-manifest.md) | Typed mutations via a collection-row manifest on `SyncRegistry` | Accepted |
2121
| [0012](./0012-wire-input-hardening.md) | Wire-input hardening: frame-shape guards, inbound limits, sanitized execute errors | Accepted |
22+
| [0013](./0013-predicate-floor-one-evaluator.md) | Filtered-subscription membership: one evaluator is the source of truth; the floor is the verified-agreeing set | Accepted |

package-lock.json

Lines changed: 2 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "tanstack-do-db-collection",
3-
"version": "0.3.2",
3+
"version": "0.3.3",
44
"description": "Sync a TanStack DB collection to a Cloudflare Durable Object over WebSockets — optimistic mutations, live queries, and single-ordered-stream write confirmation.",
55
"type": "module",
66
"license": "MIT",

src/server/sql-compiler.ts

Lines changed: 26 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,18 @@
22
// `where` plus orderBy/limit/offset into a parameterised SELECT, so subset
33
// shaping runs in SQLite rather than scanning every row in memory.
44
//
5-
// Supported operator floor (D6): eq, ne, gt, gte, lt, lte, like, in, and, or,
6-
// not. Anything outside it — ilike, isNull, functional/nested-path predicates —
7-
// throws UnsupportedPredicateError. The caller rejects such a subscription
8-
// rather than silently falling back to a full scan (fail loud).
5+
// Supported operator floor (D6): eq, gt, gte, lt, lte, like, in, and, or,
6+
// not. Anything outside it — ne, ilike, isNull, functional/nested-path
7+
// predicates — throws UnsupportedPredicateError. The caller rejects such a
8+
// subscription rather than silently falling back to a full scan (fail loud).
9+
//
10+
// The floor is exactly the set of operators that the SQL snapshot path and
11+
// @tanstack/db's JS evaluator (delta/catch-up path) agree on, row-for-row —
12+
// see ADR-0013. `ne` is excluded because @tanstack/db has no `ne` (it only
13+
// knows `not(eq(...))`); accepting it here desynced the two paths and crashed
14+
// the delta path. `like` stays in the floor but is only correct because the DO
15+
// sets `PRAGMA case_sensitive_like = ON` (SyncDurableObject constructor), which
16+
// makes SQLite LIKE case-sensitive to match @tanstack/db's case-sensitive `like`.
917

1018
export class UnsupportedPredicateError extends Error {
1119
constructor(message: string) {
@@ -17,11 +25,12 @@ export class UnsupportedPredicateError extends Error {
1725
const IDENT = /^[A-Za-z_][A-Za-z0-9_]*$/
1826
const COMPARATORS: Record<string, string> = {
1927
eq: "=",
20-
ne: "!=",
2128
gt: ">",
2229
gte: ">=",
2330
lt: "<",
2431
lte: "<=",
32+
// LIKE is case-sensitive only because the DO sets PRAGMA case_sensitive_like
33+
// = ON; see the floor note above and ADR-0013.
2534
like: "LIKE",
2635
}
2736

@@ -60,6 +69,17 @@ function compileExpr(node: unknown, params: Array<unknown>): string {
6069

6170
if (name in COMPARATORS) {
6271
if (args.length !== 2) throw new UnsupportedPredicateError(`'${name}' expects 2 arguments`)
72+
if (name === "like") {
73+
// @tanstack/db's evaluateLike returns false unless BOTH operands are
74+
// strings, whereas SQLite LIKE coerces a non-string pattern (e.g. 123 →
75+
// '123') and could match rows the JS delta/catch-up path rejects. Restrict
76+
// the pattern to a string literal so the two paths stay in lockstep — the
77+
// floor must be verified-agreeing, not merely "usually agreeing" (ADR-0013).
78+
const lit = args[1]
79+
if (!isNode(lit) || lit.type !== "val" || typeof lit.value !== "string") {
80+
throw new UnsupportedPredicateError("'like' pattern must be a string literal")
81+
}
82+
}
6383
return `${column(args[0])} ${COMPARATORS[name]} ${literal(args[1], params)}`
6484
}
6585
if (name === "and" || name === "or") {
@@ -84,7 +104,7 @@ function compileExpr(node: unknown, params: Array<unknown>): string {
84104
}
85105
throw new UnsupportedPredicateError(
86106
`operator '${name}' is not supported for server-side filtering ` +
87-
`(floor: eq, ne, gt, gte, lt, lte, like, in, and, or, not)`,
107+
`(floor: eq, gt, gte, lt, lte, like, in, and, or, not)`,
88108
)
89109
}
90110

src/server/subscriptions.ts

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
// client's operator semantics exactly (no second predicate implementation).
88

99
import { compileSingleRowExpression, toBooleanPredicate } from "@tanstack/db"
10+
import { UnsupportedPredicateError } from "./sql-compiler.ts"
1011

1112
export interface Sub {
1213
subId: string
@@ -19,9 +20,20 @@ export interface Sub {
1920

2021
function compilePredicate(where: unknown): (row: Record<string, unknown>) => boolean {
2122
if (where === undefined || where === null) return () => true
22-
const evaluate = compileSingleRowExpression(where as never) as (
23-
row: Record<string, unknown>,
24-
) => boolean | null
23+
let evaluate: (row: Record<string, unknown>) => boolean | null
24+
try {
25+
evaluate = compileSingleRowExpression(where as never) as (row: Record<string, unknown>) => boolean | null
26+
} catch (e) {
27+
// @tanstack/db's evaluator rejects any operator outside its registry (e.g. a
28+
// hand-built `ne`, which it does not know — only `not(eq(...))`). The SQL
29+
// snapshot floor (sql-compiler.ts) and this JS delta floor MUST be the same
30+
// set, or a sub's membership depends on which path decided it. Re-throw as
31+
// UnsupportedPredicateError so handleSub rejects the sub with a `reset`
32+
// (fail loud) instead of letting this escape uncaught and hang the client. (ADR-0013)
33+
throw new UnsupportedPredicateError(
34+
`predicate not compilable by @tanstack/db evaluator: ${e instanceof Error ? e.message : String(e)}`,
35+
)
36+
}
2537
// toBooleanPredicate collapses SQL 3-valued null to false, matching SQL.
2638
return (row) => toBooleanPredicate(evaluate(row))
2739
}

src/server/sync-do.ts

Lines changed: 24 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ import { Broadcaster } from "./broadcast.ts"
3333
import { decodeResult, encodeResult, lookupTx, recordTx, type SeenTx, sweepDedup } from "./dedup.ts"
3434
import type { SyncRegistry } from "./registry.ts"
3535
import { andPredicates, compileSubsetQuery, UnsupportedPredicateError } from "./sql-compiler.ts"
36-
import { SubscriptionRegistry } from "./subscriptions.ts"
36+
import { SubscriptionRegistry, type Sub } from "./subscriptions.ts"
3737

3838
export abstract class SyncDurableObject<Env = unknown, TUser = unknown> extends DurableObject<Env> {
3939
/** Set by `registerSync` — the collections/mutations/commands this DO serves. */
@@ -88,6 +88,12 @@ export abstract class SyncDurableObject<Env = unknown, TUser = unknown> extends
8888
super(ctx, env)
8989
// Auto-pong via the runtime: survives hibernation, no per-message billing.
9090
this.ctx.setWebSocketAutoResponse(new WebSocketRequestResponsePair("ping", "pong"))
91+
// Make SQLite LIKE case-sensitive so the SQL snapshot path matches
92+
// @tanstack/db's case-sensitive `like` evaluator on the delta path — the
93+
// single source of truth for filtered-subscription membership (ADR-0013).
94+
// Connection-scoped pragma; re-applied on every instantiation, including a
95+
// hibernation wake (same lifecycle as the auto-response registration above).
96+
this.sql.exec("PRAGMA case_sensitive_like = ON")
9197
// Restore the live-socket set after a hibernation wake.
9298
for (const ws of this.ctx.getWebSockets()) this.liveWs.add(ws)
9399
this.broadcaster = new Broadcaster((ws, frame) => this.send(ws, frame), this.tickMs)
@@ -609,7 +615,23 @@ export abstract class SyncDurableObject<Env = unknown, TUser = unknown> extends
609615
// before the tick loses the write (reconnect resumes past it).
610616
this.broadcaster.flushOne(ws)
611617

612-
const sub = this.subs.add(ws, frame.subId, frame.collection, frame.where)
618+
// Registering compiles the predicate in @tanstack/db's evaluator. If the
619+
// predicate is outside the JS floor (e.g. an operator the SQL floor somehow
620+
// let through), that throws UnsupportedPredicateError — reject with `reset`
621+
// rather than letting it escape uncaught and hang the client (ADR-0013).
622+
// With the floors aligned this is belt-and-suspenders, but it must hold for
623+
// any future operator added to one floor and not the other.
624+
let sub: Sub
625+
try {
626+
sub = this.subs.add(ws, frame.subId, frame.collection, frame.where)
627+
} catch (e) {
628+
if (e instanceof UnsupportedPredicateError) {
629+
console.error(`sub '${frame.subId}' on '${frame.collection}' rejected: ${e.message}`)
630+
this.send(ws, { t: "reset", sub: frame.subId })
631+
return
632+
}
633+
throw e
634+
}
613635
const seq = String(currentSeq(this.sql))
614636

615637
// Reconnect catch-up: a `since` cursor asks for changes after that point

0 commit comments

Comments
 (0)