Skip to content

feat(expr): ADR-0058 P1–P4 — canonical CEL→FilterCondition compiler, RLS/sharing cutover, check enforcement, conformance ledger#2090

Merged
xuyushun441-sys merged 7 commits into
mainfrom
feat/expression-pushdown-compiler
Jun 21, 2026
Merged

feat(expr): ADR-0058 P1–P4 — canonical CEL→FilterCondition compiler, RLS/sharing cutover, check enforcement, conformance ledger#2090
xuyushun441-sys merged 7 commits into
mainfrom
feat/expression-pushdown-compiler

Conversation

@xuyushun441-sys

Copy link
Copy Markdown
Contributor

Implements ADR-0058 (expression & predicate surface) P1–P4. The ADR was merged in #2087; this is the implementation.

The problem (ADR-0058 audit)

ObjectStack had ONE authoring language (CEL) + ONE good interpreter, but a fragmented compile-to-filter path: three disconnected front-ends (plugin-security's 4-form regex, plugin-sharing's celToFilter, the ObjectUI array-AST path). That divergence is the root of #1887 — a sharing condition the interpreter understands but no compiler lowers, so it never enforces.

What landed

P1 — canonical compiler (@objectstack/formula)
compileCelToFilter lowers the same @marcbachmann/cel-js AST the interpreter uses to a Mongo-style FilterCondition — the one shape both backends already consume (ObjectQL where + analytics read-scope-sql). Supported subset (D2): == != > < >= <=, in, && || !, == null, string methods. Hard boundaries (ADR-0055): single-column field paths only; arithmetic / functions / cross-object / subqueries are an authoring compile error, never silently dropped. isPushdownableCel (shape gate) + lowerCelAst (one-AST-two-backends, D6). 52-case matrix.

P2 — cut over the two front-ends
rls-compiler.ts (compileExpression + isSupportedRlsExpression) and plugin-sharing's celToFilter now delegate to the canonical compiler. A thin sqlPredicateToCel bridges the legacy SQL-ish RLS using subset (===, INin). compileFilter orchestration (membership merge, null→deny aggregation) and the empty-membership parity are preserved. Output parity verified: plugin-security 119, plugin-sharing 65, service-analytics 137.

P3/D3 — close #1887
Sharing condition now compiles to a compound criteria_json and enforces e2e: a record.amount >= 100000 && record.name == "Big1" shares ONLY the full-AND match. The ⚠️ EXPERIMENTAL — NOT ENFORCED block on SharingRuleSchema is removed; only owner-type + group/guest remain experimental.

P3/D4 — enforce RLS check
New matchesFilterCondition (formula) — the single-record backend for the canonical filter shape (D6). security-plugin step 3.6 validates the post-image (insert → new row; by-id update → pre-image ∪ change) against the compiled check; violating writes fail closed (D5). Scoped to policies that explicitly declare check → zero blast radius (no seeded policy declares one). 7 proof cases.

P4/D7 — conformance ledger + CI ratchet
expression-conformance.ledger.ts classifies all 27 expression surfaces (dialect / mode / state / fail-policy / site / proof). The companion test re-discovers every ExpressionInputSchema field in packages/spec/src (+ RLS using/check) and fails the build if any surface is unclassified — the #1887 class can't recur silently. Compile-mode rows must be compiler-reachable + carry an existing proof.

Verification (local, all green)

Full workspace build 75/75 · formula 201 · plugin-security 119 · plugin-sharing 65 · service-analytics 137 · spec 6599 · dogfood 130.

Closes #1887.

🤖 Generated with Claude Code

os-zhuang and others added 5 commits June 21, 2026 12:35
…ompiler

Build the single, canonical lowering from the @marcbachmann/cel-js AST (the
SAME AST the interpreter uses) to a Mongo-style FilterCondition — the one shape
both backends already consume (ObjectQL engine `where` + analytics read-scope
SQL). This replaces the THREE disconnected compile-to-filter front-ends ADR-0058
identified (plugin-security's 4-form regex, plugin-sharing's celToFilter, the
ObjectUI array-AST path) and is the root fix for #1887.

Supported subset (D2): == != > < >= <=, in (→$in), && || !, == null / != null
(→$null), string methods startsWith/endsWith/contains. `not in` = !(x in y).
Hard boundaries (ADR-0055 stands): single-column field paths only — a nested
relation path (record.account.region), arithmetic, function calls, ternary, and
any other non-pushdown shape is an authoring-time COMPILE ERROR, never silently
dropped (failing closed is the security-correct outcome, ADR-0049/0056 D4).

Value resolution: variableRoot leaves (current_user.*) resolve against a context
to literals — current_user.org_user_ids → a pre-resolved membership array for
$in (no subquery; the runtime pre-resolves the set per ADR-0055). An undefined/
null variable yields unresolved-variable (the "no active org" fail-closed path).

Public API: compileCelToFilter (resolve), isPushdownableCel (shape-only gate),
lowerCelAst (pre-parsed AST → one-AST-two-backends, D6). P1 = parity + full
operator/variable test matrix (52 cases); NOT yet wired into the call sites
(that is P2). Full formula suite green (181).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…he one compiler

Replace the two bespoke compile-to-filter front-ends with delegation to the
canonical @objectstack/formula CEL→FilterCondition compiler (P1):

- plugin-security/rls-compiler.ts: compileExpression + isSupportedRlsExpression
  now delegate to the compiler. A thin sqlPredicateToCel() bridges the legacy
  SQL-ish `using` subset (`=`→`==`, `IN`→`in`, quoted literals untouched) so
  authored policies keep working while the bespoke 4-form regex is retired.
  compileFilter's orchestration (userCtx build, §7.3.1 membership merge, null→
  RLS_DENY aggregation) is unchanged; an isEmptyMembershipFilter() guard
  preserves the "empty pre-resolved set drops the policy" parity so the single-
  policy path still yields the deny sentinel, not a permissive `$in: []`.
- plugin-sharing celToFilter: delegates to the compiler. A sharing condition is
  a pure record predicate, so compound criteria (AND/OR, comparisons, null, in)
  now lower to criteria_json instead of being skipped — the substance of #1887
  (proven e2e in P3).

Security contract unchanged: non-pushdownable shapes (arithmetic, functions,
subqueries, cross-object, SQL AND) and unresolved current_user.* still fail
closed. The shape gate (isSupportedRlsExpression) legitimately BROADENS — `==`,
comparisons and CEL compound predicates now report supported because the runtime
genuinely enforces them; the gate stays shape-only (variable exposure is a
separate concern). Tests updated to the new, larger enforceable set with
rationale.

Parity verified green: plugin-security 112, plugin-sharing 63, service-analytics
137 (the RLS→SQL read-scope bridge).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… enforced e2e

The sharing-rule CEL `condition` is now compiled to a compound `criteria_json`
by the canonical compiler (P2) and enforced end-to-end: records matching the
full predicate materialise sys_record_share grants. Drop the "⚠️ EXPERIMENTAL —
NOT ENFORCED" block on SharingRuleSchema; only `owner`-type rules and
`group`/`guest` recipients remain experimental (no static/runtime mapping), and
a non-pushdownable condition is skipped + logged (never a permissive match-all).

Proof (plugin-sharing): a compound `record.amount >= 100000 && record.name ==
"Big1"` lowers to `{$and:[…]}`, persists as criteria_json, and shares ONLY opp1
(the full-AND match) — not opp2 (right amount, wrong name). Also proves `||` +
`== null` lower and that a non-pushdownable condition returns null.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The RLS `check` clause (the PostgreSQL WITH CHECK analog) was declared in the
schema but never read — a declared-but-unenforced gap (ADR-0049). Close it:

- @objectstack/formula gains matchesFilterCondition(record, filter) — the
  single-record backend for the canonical FilterCondition shape (ADR-0058 D6),
  completing the round-trip: compile CEL → filter, run as `where`, lower to SQL,
  and now match one in-memory record for write validation. Fail-closed on any
  unknown operator / malformed node (20-case test).
- RLSCompiler.compileFilter gains a `clause` selector; the write pass sources the
  predicate from `check ?? using`. Also fixes a latent bug surfaced by check-only
  policies: when no applicable policy carries a predicate for the pass, return
  null (no filter) instead of the deny sentinel — a check-only policy no longer
  spuriously denies the `using` read/pre-image path.
- security-plugin step 3.6: on insert/update, compile the declared `check`
  clauses and match the POST-IMAGE (insert → new row; by-id update → pre-image ∪
  change set) against them; a violating write is denied (fail closed, D5). Scoped
  to policies that explicitly declare `check`, so objects governed only by
  `using` are unaffected (zero blast radius — no seeded policy declares check).
  Bulk updates are governed by the using-scoped where and logged as not
  post-image-validated.

Proof: 7 cases — compliant/violating/missing-field insert, post-image
satisfy/violate/unchanged update, delete-unaffected. Suites green: security 119,
formula 201, analytics 137, dogfood 21.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…CI ratchet

One classification per expression-holding declaration across the spec (27
surfaces): dialect, mode (compile|interpret), state, fail-policy (D5), evaluator/
compiler site, and — for the security-critical COMPILE surfaces (RLS using/check,
sharing condition) — a proof. The companion test makes it a CHECKED artifact:

- every COMPILE row must be security fail-closed, name the canonical compiler
  (compileCelToFilter / celToFilter / matchesFilterCondition), and carry a proof
  file that exists on disk;
- every surface is covered by exactly one row (no double classification);
- THE RATCHET: the test re-discovers every `ExpressionInputSchema` field in
  packages/spec/src (plus the RLS using/check string predicates) and fails if any
  is unclassified or any `covers` entry is stale. A new declared-but-unwired
  predicate — the #1887 class — now breaks the build until it is classified.

Verified the ratchet has teeth (dropping a covers entry fails CI with an
actionable message). 7 assertions green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@vercel

vercel Bot commented Jun 21, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
spec Ready Ready Preview, Comment Jun 21, 2026 6:23am

Request Review

@github-actions github-actions Bot added dependencies Pull requests that update a dependency file tests size/xl labels Jun 21, 2026
@github-actions

github-actions Bot commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 5 package(s): @objectstack/dogfood, @objectstack/formula, @objectstack/plugin-security, @objectstack/plugin-sharing, @objectstack/spec.

89 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/concepts/architecture.mdx (via @objectstack/spec)
  • content/docs/concepts/cloud-artifact-api.mdx (via packages/spec)
  • content/docs/concepts/cluster-semantics.mdx (via @objectstack/spec)
  • content/docs/concepts/design-principles.mdx (via packages/spec)
  • content/docs/concepts/implementation-status.mdx (via @objectstack/plugin-security, @objectstack/spec)
  • content/docs/concepts/index.mdx (via @objectstack/spec)
  • content/docs/concepts/metadata-driven.mdx (via @objectstack/spec)
  • content/docs/concepts/metadata-lifecycle.mdx (via packages/spec)
  • content/docs/concepts/north-star.mdx (via packages/spec)
  • content/docs/concepts/packages.mdx (via @objectstack/formula, @objectstack/plugin-security, @objectstack/plugin-sharing, @objectstack/spec)
  • content/docs/concepts/setup-app.mdx (via @objectstack/spec)
  • content/docs/concepts/skills.mdx (via @objectstack/spec)
  • content/docs/concepts/webhook-delivery.mdx (via @objectstack/spec)
  • content/docs/getting-started/architecture.mdx (via @objectstack/spec)
  • content/docs/getting-started/cli.mdx (via @objectstack/plugin-security, @objectstack/spec)
  • content/docs/getting-started/core-concepts.mdx (via @objectstack/spec)
  • content/docs/getting-started/examples.mdx (via @objectstack/spec)
  • content/docs/getting-started/quick-start.mdx (via @objectstack/spec)
  • content/docs/guides/adding-a-metadata-type.mdx (via @objectstack/spec)
  • content/docs/guides/ai-capabilities.mdx (via @objectstack/spec)
  • content/docs/guides/airtable-dashboard-analysis.mdx (via @objectstack/spec)
  • content/docs/guides/analytics-datasets.mdx (via @objectstack/spec)
  • content/docs/guides/api-reference.mdx (via @objectstack/spec)
  • content/docs/guides/business-logic.mdx (via @objectstack/spec)
  • content/docs/guides/cheatsheets/backward-compatibility.mdx (via @objectstack/spec)
  • content/docs/guides/cheatsheets/error-catalog.mdx (via @objectstack/spec)
  • content/docs/guides/cheatsheets/field-type-gallery.mdx (via @objectstack/spec)
  • content/docs/guides/cheatsheets/field-validation-rules.mdx (via @objectstack/spec)
  • content/docs/guides/cheatsheets/permissions-matrix.mdx (via packages/plugins/plugin-security, packages/plugins/plugin-sharing, @objectstack/spec)
  • content/docs/guides/cheatsheets/protocol-diagram.mdx (via packages/spec)
  • content/docs/guides/cheatsheets/query-cheat-sheet.mdx (via @objectstack/spec)
  • content/docs/guides/cheatsheets/quick-reference.mdx (via @objectstack/spec)
  • content/docs/guides/client-sdk.mdx (via @objectstack/spec)
  • content/docs/guides/common-patterns.mdx (via @objectstack/spec)
  • content/docs/guides/contracts/auth-service.mdx (via packages/spec)
  • content/docs/guides/contracts/cache-service.mdx (via packages/spec)
  • content/docs/guides/contracts/data-engine.mdx (via @objectstack/spec)
  • content/docs/guides/contracts/index.mdx (via @objectstack/spec)
  • content/docs/guides/contracts/metadata-service.mdx (via packages/spec)
  • content/docs/guides/contracts/storage-service.mdx (via packages/spec)
  • content/docs/guides/data-modeling.mdx (via @objectstack/spec)
  • content/docs/guides/deployment-vercel.mdx (via @objectstack/spec)
  • content/docs/guides/driver-configuration.mdx (via @objectstack/spec)
  • content/docs/guides/error-handling-client.mdx (via @objectstack/spec)
  • content/docs/guides/error-handling-server.mdx (via @objectstack/spec)
  • content/docs/guides/formula.mdx (via @objectstack/formula, @objectstack/spec)
  • content/docs/guides/hook-bodies.mdx (via packages/spec)
  • content/docs/guides/kernel-services.mdx (via @objectstack/spec)
  • content/docs/guides/metadata/dashboard.mdx (via @objectstack/plugin-security, @objectstack/spec)
  • content/docs/guides/metadata/field.mdx (via @objectstack/spec)
  • content/docs/guides/metadata/flow.mdx (via @objectstack/spec)
  • content/docs/guides/metadata/index.mdx (via @objectstack/spec)
  • content/docs/guides/metadata/object.mdx (via @objectstack/spec)
  • content/docs/guides/metadata/validation.mdx (via @objectstack/formula, @objectstack/spec)
  • content/docs/guides/metadata/workflow.mdx (via @objectstack/spec)
  • content/docs/guides/packages.mdx (via @objectstack/formula, @objectstack/plugin-security, @objectstack/plugin-sharing, @objectstack/spec)
  • content/docs/guides/plugin-development.mdx (via @objectstack/spec)
  • content/docs/guides/plugins.mdx (via @objectstack/plugin-security, @objectstack/spec)
  • content/docs/guides/project-scoping.mdx (via @objectstack/spec)
  • content/docs/guides/public-forms.mdx (via @objectstack/spec)
  • content/docs/guides/runtime-services/email-service.mdx (via packages/spec)
  • content/docs/guides/runtime-services/index.mdx (via packages/spec)
  • content/docs/guides/runtime-services/queue-service.mdx (via packages/spec)
  • content/docs/guides/runtime-services/sharing-service.mdx (via packages/spec)
  • content/docs/guides/runtime-services/storage-service.mdx (via packages/spec)
  • content/docs/guides/security.mdx (via @objectstack/plugin-security, @objectstack/plugin-sharing, @objectstack/spec)
  • content/docs/guides/seed-data.mdx (via @objectstack/spec)
  • content/docs/guides/skills.mdx (via @objectstack/spec)
  • content/docs/guides/standards.mdx (via @objectstack/spec)
  • content/docs/guides/troubleshooting.mdx (via @objectstack/spec)
  • content/docs/protocol/knowledge.mdx (via @objectstack/spec)
  • content/docs/protocol/objectos/config-resolution.mdx (via @objectstack/spec)
  • content/docs/protocol/objectos/i18n-standard.mdx (via @objectstack/spec)
  • content/docs/protocol/objectos/lifecycle.mdx (via @objectstack/spec)
  • content/docs/protocol/objectos/plugin-spec.mdx (via @objectstack/spec)
  • content/docs/protocol/objectos/runtime-capabilities.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/index.mdx (via packages/spec)
  • content/docs/protocol/objectql/query-syntax.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/schema.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/security.mdx (via packages/plugins/plugin-sharing, packages/spec)
  • content/docs/protocol/objectql/state-machine.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/actions.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/concept.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/index.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/layout-dsl.mdx (via packages/spec)
  • content/docs/protocol/objectui/record-alert.mdx (via @objectstack/formula, @objectstack/spec)
  • content/docs/protocol/objectui/widget-contract.mdx (via @objectstack/spec)
  • content/docs/releases/index.mdx (via @objectstack/spec)
  • content/docs/releases/v9.mdx (via @objectstack/spec)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@github-actions github-actions Bot added documentation Improvements or additions to documentation tooling labels Jun 21, 2026
Comment thread packages/dogfood/test/expression-conformance.test.ts Fixed
Comment thread packages/formula/src/cel-to-filter.ts Fixed
- expression-conformance.test.ts (HIGH js/file-system-race): the ratchet's spec
  walk did statSync(p) then readFileSync(p) — a check-then-use TOCTOU window.
  Read the entry type from the single readdir syscall via withFileTypes instead.
- cel-to-filter.ts coerceLiteral (warning, comparison between inconvertible
  types): `v !== null` is dead after the preceding line returns for null — drop
  it; the early return already guarantees non-null.

No behavior change; cel-to-filter 52 + expression-conformance 7 still green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@xuyushun441-sys
xuyushun441-sys merged commit cfd86ce into main Jun 21, 2026
18 checks passed
@xuyushun441-sys
xuyushun441-sys deleted the feat/expression-pushdown-compiler branch June 21, 2026 06:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation size/xl tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P0][security] SharingRuleSchema disconnected from the live engine

3 participants