|
| 1 | +# ADR-0013: Filtered-subscription membership — one evaluator is the source of truth; the floor is the verified-agreeing set |
| 2 | + |
| 3 | +**Status**: Accepted |
| 4 | +**Date**: 2026-06-13 |
| 5 | +**Characterization**: `tests/predicate-parity.test.ts` (originally plan 003) |
| 6 | + |
| 7 | +## Context |
| 8 | + |
| 9 | +A filtered subscription's membership is decided by **two different evaluators** |
| 10 | +depending on the path: |
| 11 | + |
| 12 | +- The **initial snapshot** filters in SQLite: the `where` IR is lowered to SQL |
| 13 | + by `src/server/sql-compiler.ts` (`"col" = ?`, `"col" LIKE ?`, …). |
| 14 | +- **Live deltas and reconnect catch-up** filter in JavaScript: the same IR is |
| 15 | + compiled by `@tanstack/db`'s `compileSingleRowExpression` + `toBooleanPredicate` |
| 16 | + in `src/server/subscriptions.ts` (`sync-do.ts` delta/catch-up paths). The client's |
| 17 | + eager-write preflight (`do-collection.ts`) uses that same JS evaluator, and so do |
| 18 | + the client's own live queries. |
| 19 | + |
| 20 | +If the two disagree on any row, a client's view of a filtered subset depends on |
| 21 | +*when it connected* — a row excluded by the snapshot can be pushed in by a later |
| 22 | +delta, and two clients silently diverge. Characterization found two real divergences: |
| 23 | + |
| 24 | +1. **`ne` — crash + hang.** `sql-compiler.ts` accepted `ne` (lowered to `!=`), but |
| 25 | + `@tanstack/db`'s evaluator has no `ne` in its registry (not-equal is expressed as |
| 26 | + `not(eq(...))`). `compileSingleRowExpression` threw a `QueryCompilationError` from |
| 27 | + inside `subs.add`, which runs in `handleSub` **after** the `try/catch` that wraps |
| 28 | + `compileSubsetQuery`. The throw escaped uncaught: no `reset` was sent and the |
| 29 | + subscriber hung until timeout. |
| 30 | +2. **`like` — case divergence.** SQLite `LIKE` is ASCII case-*insensitive* by default; |
| 31 | + `@tanstack/db`'s `like` is case-*sensitive* (its case-insensitive variant is |
| 32 | + `ilike`, which is off-floor). A row `"HELLO"` matched `like "hello%"` in the SQL |
| 33 | + snapshot but not in the JS deltas. |
| 34 | + |
| 35 | +## Decisions |
| 36 | + |
| 37 | +### D1: `@tanstack/db`'s evaluator is the source of truth for membership |
| 38 | + |
| 39 | +It already decides membership on three of the four sites (delta, catch-up, client |
| 40 | +preflight), and the client's live queries use the same library. The SQL snapshot is |
| 41 | +an *optimization* that must reproduce exactly the rows that evaluator accepts. So when |
| 42 | +the two disagree, **SQL conforms to `@tanstack/db`** — never the reverse. |
| 43 | + |
| 44 | +### D2: The operator floor is the verified-agreeing set |
| 45 | + |
| 46 | +Floor = `{ eq, gt, gte, lt, lte, like, in, and, or, not }`. **`ne` is removed** — it |
| 47 | +is not in `@tanstack/db`'s evaluator, and a real client emits `not(eq(...))` (which |
| 48 | +both paths handle identically). Anything outside the floor is rejected with |
| 49 | +`UnsupportedPredicateError` → `reset`, as before. `tests/predicate-parity.test.ts` |
| 50 | +pins row-for-row agreement for each floor operator across both paths; **any operator |
| 51 | +added to `COMPARATORS` must add a parity case** (enforced by review). |
| 52 | + |
| 53 | +### D3: `LIKE` is made case-sensitive to match |
| 54 | + |
| 55 | +The DO sets `PRAGMA case_sensitive_like = ON` in the `SyncDurableObject` constructor. |
| 56 | +The pragma is connection-scoped, and the constructor runs on every instantiation — |
| 57 | +including a hibernation wake — so it is always in force before any query (the same |
| 58 | +lifecycle the existing `setWebSocketAutoResponse` registration relies on). It is |
| 59 | +contained: the IR→SQL compiler is the only producer of `LIKE` in the codebase. |
| 60 | +(`case_sensitive_like` is a *setter-only* pragma — there is no getter — so it is |
| 61 | +verified behaviorally in tests, not by readback.) Case-*insensitive* matching is |
| 62 | +`ilike`, which stays off-floor (see deferred). |
| 63 | + |
| 64 | +The `like` **pattern must be a string literal**: `@tanstack/db`'s `evaluateLike` |
| 65 | +returns `false` unless both operands are strings, whereas SQLite `LIKE` would |
| 66 | +coerce a non-string pattern (`123` → `'123'`) and match rows the JS path rejects. |
| 67 | +The compiler rejects a non-string `like` pattern with `UnsupportedPredicateError` |
| 68 | +(→ `reset`), keeping the floor *verified*-agreeing rather than merely usually so. |
| 69 | + |
| 70 | +### D4: A defensive fail-loud guard at predicate compile |
| 71 | + |
| 72 | +`compilePredicate` (`subscriptions.ts`) now wraps the `@tanstack/db` compile and |
| 73 | +re-throws any failure as `UnsupportedPredicateError`; `handleSub` catches it around |
| 74 | +`subs.add` and answers with `reset`. With the floors aligned this is belt-and- |
| 75 | +suspenders, but it guarantees fail-loud (a `reset`, never a hang) for any future |
| 76 | +operator that lands in one floor and not the other. |
| 77 | + |
| 78 | +## Consequences |
| 79 | + |
| 80 | +- **Behavior change (observable):** a `ne` subscription is now rejected with `reset` |
| 81 | + (was: an indefinite hang). `like` on a filtered subscription is now case-sensitive |
| 82 | + on every path (was: case-insensitive in the snapshot only, and divergent from |
| 83 | + deltas) — `"HELLO"` no longer matches `like "hello%"`. This aligns snapshot, delta, |
| 84 | + catch-up, and client-side semantics. Real `@tanstack/db` clients are unaffected by |
| 85 | + the `ne` removal — they emit `not(eq(...))`, which is fully supported. |
| 86 | +- **No idle timers / hibernation impact:** the pragma is set synchronously in the |
| 87 | + constructor; no polling introduced. |
| 88 | +- **Deferred:** `ilike` (case-insensitive `LIKE`) remains off-floor. Adding it would |
| 89 | + lower to `lower("col") LIKE lower(?)` (or equivalent) plus a parity case — a |
| 90 | + separate decision. |
| 91 | +- **Test coverage:** `tests/predicate-parity.test.ts` (6 cases, both paths) pins |
| 92 | + floor-operator agreement; `tests/subscriptions.test.ts` pins the JS-floor guard at |
| 93 | + the unit level; `tests/sql-compiler.test.ts` pins `ne` rejection and `not(eq)` |
| 94 | + lowering. |
0 commit comments