Skip to content

Commit a2316e6

Browse files
os-zhuangclaude
andcommitted
feat(dogfood): owner-isolated RLS fixture — hard, revert-provable #1994 gate
The cross-owner RLS proof runner (src/rls.ts, #2032) was unit-proven but had nothing live to exercise it: the harness boots single-tenant, so the org tenant_isolation policy is stripped and a fresh member falls back to member_default (broad read) — every object reports `member-visible`, so the #1994 by-id-write invariant ("can't read ⇒ can't write") never fired live. Two faithful ways to create real isolation, both now live: 1. Owner-scoped fixture (test/rls-fixture.dogfood.test.ts + fixtures/): one object `rls_note` whose member fallback permission set carries RLS.ownerPolicy('rls_note','created_by'). Keyed on created_by/current_user.id (not organization_id), so it survives single-tenant stripping — a fresh member genuinely can't read the admin's note. - GREEN: owner policy on `all` ops → rls-consistent (pre-image check denies the by-id write). - AUTOMATED RED: owner policy on `select` only → rls-hole (read scoped, no write policy applies, by-id write lands) — #1994 hole class every CI run. - MANUAL REVERT PROOF (README): disabling the pre-image check flips the green fixture to "GET 404 yet MUTATED it by id (PATCH 200)". Verified, restored. 2. Multi-tenant org-scoped (test/rls-multitenant.dogfood.test.ts): bootDogfoodStack gains `{ multiTenant: true }` → registers plugin-org-scoping before SecurityPlugin so organization_id policies APPLY. CRM flips from every-object-member-visible (single-tenant) to 4 consistent / 0 holes / 0 member-visible — confirming member-visible was single-tenant RLS-stripping, not a broad-read default (the model real apps like hotcrm rely on). harness: BootOptions gains `security?` (SecurityPlugin override for the fixture fallback) and `multiTenant?`. README documents both paths + the revert proof. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent db02bd5 commit a2316e6

8 files changed

Lines changed: 410 additions & 10 deletions

File tree

packages/dogfood/README.md

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,4 +63,89 @@ the spec's zod surface; then run the AI **template corpus** through the harness.
6363
The binding policy — every authorable+live primitive must carry a runtime proof
6464
— is the subject of a dedicated ADR.
6565

66+
## Authorization gate (RLS / #1994)
67+
68+
The capability matrix above proves *data* round-trips. The authorization
69+
dimension proves a record the caller must not touch stays untouched. The
70+
app-agnostic invariant (`src/rls.ts`, `runRlsProofs`):
71+
72+
> **A user who cannot READ a record must not be able to WRITE it.**
73+
74+
[#1994](https://github.com/objectstack-ai/framework/pull/1994) was exactly this
75+
hole: a single-id `update`/`delete` goes straight to `driver.update(object, id)`
76+
and builds no query AST, so the row-level `where` filter the middleware injects
77+
on the *read* path was never applied to *by-id writes*. Any member could PATCH a
78+
record they couldn't even see. The fix is a **pre-image check** in
79+
`plugin-security` (re-read the target row under the write-op RLS filter before
80+
mutating; deny if invisible).
81+
82+
### The runner, and why it needs a fixture
83+
84+
Per object: admin creates a record → a fresh member (`signUp`, no grants) tries
85+
to read it, then mutate it by id → re-read as admin decides if the row actually
86+
changed. Verdicts: `rls-consistent` (can't read **and** can't write — good),
87+
`rls-hole` (can't read **yet** wrote — the #1994 bug), `member-visible`
88+
(member *can* read it — inconclusive, not a cross-owner scenario).
89+
90+
`auto-verify-rls.dogfood.test.ts` runs this over the example apps, but they boot
91+
**single-tenant**, where every object comes back `member-visible` — so the
92+
by-id-write path is never actually exercised. Two ways to create real isolation:
93+
94+
### 1. Owner-scoped fixture — `test/rls-fixture.dogfood.test.ts` (hard gate)
95+
96+
`fixtures/rls-owner-fixture.ts` is a one-object app (`rls_note`) whose member
97+
permission set carries `RLS.ownerPolicy('rls_note', 'created_by')`. The predicate
98+
is `created_by = current_user.id` — keyed on the column the engine stamps on
99+
every record and referencing `current_user.id`, **not**
100+
`current_user.organization_id`, so it survives single-tenant policy stripping. A
101+
fresh member genuinely can't read the admin's note. `bootDogfoodStack` takes a
102+
`security:` override so the fixture's permission set is the member's fallback:
103+
104+
```ts
105+
bootDogfoodStack(rlsFixtureStack, { security: rlsFixtureSecurity(ownerScopedMemberSet) })
106+
```
107+
108+
- **Green gate** (owner policy on `all` ops) → `rls-consistent`. Safe *only*
109+
because the pre-image check enforces the by-id write.
110+
- **Automated red proof** (owner policy on `select` only) → `rls-hole`. Read is
111+
owner-scoped but no write policy applies, so the by-id write lands — the #1994
112+
hole class reproduced at the policy layer, on every CI run. A gate that can't
113+
go red isn't a gate.
114+
115+
**Manual revert proof** (confirms the *fix*, not just the hole class, is what
116+
keeps the green gate green):
117+
118+
```sh
119+
# 1. In plugin-security/src/security-plugin.ts, disable the pre-image check:
120+
# change `if (` to `if ( false &&` at the `(opCtx.operation === 'update' …` block.
121+
pnpm --filter @objectstack/plugin-security build # package resolves to dist
122+
cd packages/dogfood && npx vitest run test/rls-fixture.dogfood.test.ts -t "owner-scoped"
123+
# → rls_note flips to [rls-hole]: "GET 404 yet MUTATED it by id (PATCH 200)".
124+
git checkout -- ../plugins/plugin-security/src/security-plugin.ts && pnpm --filter @objectstack/plugin-security build
125+
```
126+
127+
### 2. Org-scoped / multi-tenant — `test/rls-multitenant.dogfood.test.ts`
128+
129+
Why the single-tenant example-app run is all `member-visible`: `member_default`
130+
scopes rows with a wildcard `tenant_isolation` policy
131+
(`organization_id = current_user.organization_id`), and
132+
`SecurityPlugin.collectRLSPolicies` **strips** every `current_user.organization_id`
133+
policy when the org-scoping plugin is absent — while `member_default` carries no
134+
owner-scoped *read* policy. So the member reads everything. That is the harness
135+
booting single-tenant, **not** a broad-read default of the app (hotcrm's 9
136+
sharing files / `requires: ['sharing']` rely on exactly this org boundary).
137+
138+
Faithful fix — boot multi-tenant so `@objectstack/plugin-org-scoping` registers
139+
before `SecurityPlugin` and the `organization_id` policies apply:
140+
141+
```ts
142+
bootDogfoodStack(crmStack, { multiTenant: true })
143+
```
144+
145+
The dev admin is bound to the seeded default org; a fresh `signUp` member is not,
146+
so the admin's org-scoped records are invisible to them. Empirically CRM flips
147+
from *every object `member-visible`* (single-tenant) to **`4 consistent, 0
148+
holes, 0 member-visible`** (multi-tenant) — the runner now exercises the #1994
149+
by-id-write invariant over org-scoped (not just owner-scoped) RLS.
150+
66151
Runs in CI as the `Dogfood Regression Gate` job (and under `turbo run test`).

packages/dogfood/package.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,8 @@
2222
"@objectstack/service-analytics": "workspace:*",
2323
"@objectstack/example-crm": "workspace:*",
2424
"@objectstack/example-showcase": "workspace:*",
25-
"@objectstack/plugin-sharing": "workspace:*"
25+
"@objectstack/plugin-sharing": "workspace:*",
26+
"@objectstack/plugin-org-scoping": "workspace:*"
2627
},
2728
"devDependencies": {
2829
"@types/node": "^25.9.3",

packages/dogfood/src/harness.ts

Lines changed: 29 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,24 @@ export interface DogfoodStack {
5757
export interface BootOptions {
5858
/** Override the dev admin credentials the harness signs in with. */
5959
admin?: { email: string; password: string };
60+
/**
61+
* Override the SecurityPlugin instance. Pass a `new SecurityPlugin({...})`
62+
* to carry a custom `fallbackPermissionSet` / extra permission sets — this
63+
* is how the owner-isolated RLS fixture makes a fresh member fall back to a
64+
* permission set that carries `RLS.ownerPolicy(...)` instead of the broad-read
65+
* `member_default`. Defaults to a vanilla `new SecurityPlugin()`.
66+
*/
67+
security?: SecurityPlugin;
68+
/**
69+
* Boot multi-tenant: register `@objectstack/plugin-org-scoping` BEFORE the
70+
* SecurityPlugin so the wildcard `organization_id` RLS policies that ship in
71+
* the default permission sets actually apply (SecurityPlugin probes the
72+
* `org-scoping` service once at start and otherwise STRIPS them — see
73+
* `collectRLSPolicies`). This exercises the org-scoped isolation real apps
74+
* (e.g. hotcrm) rely on, rather than the single-tenant default where every
75+
* tenant policy is stripped and a member sees every row. Default `false`.
76+
*/
77+
multiTenant?: boolean;
6078
}
6179

6280
/**
@@ -90,7 +108,17 @@ export async function bootDogfoodStack(
90108
await kernel.use(new SettingsServicePlugin());
91109
await kernel.use(new AnalyticsServicePlugin());
92110
await kernel.use(new AuthPlugin({ secret: 'dogfood-regression-secret' }));
93-
await kernel.use(new SecurityPlugin());
111+
112+
// Multi-tenant: org-scoping MUST register BEFORE SecurityPlugin — the latter
113+
// probes the `org-scoping` service exactly once at start and caches it, then
114+
// keeps (vs strips) the wildcard `organization_id` RLS policies accordingly.
115+
// Mirrors `plugin-dev`'s ordering for `OS_MULTI_ORG_ENABLED`.
116+
if (opts.multiTenant) {
117+
const { OrgScopingPlugin } = await import('@objectstack/plugin-org-scoping');
118+
await kernel.use(new OrgScopingPlugin());
119+
}
120+
121+
await kernel.use(opts.security ?? new SecurityPlugin());
94122
// Sharing service — apps that declare `requires: ['sharing']` rely on it for
95123
// record-share grants; without it their RLS/sharing rules are inert and the
96124
// verifier would under-report authorization.

packages/dogfood/test/auto-verify-rls.dogfood.test.ts

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,17 @@
44
// real second user. The runner's hole-detection logic is unit-proven in
55
// `rls-runner.test.ts`; this exercises it end-to-end against real apps.
66
//
7-
// HONEST CURRENT STATE: the harness boots single-tenant, so org-scoped RLS is
8-
// stripped and a fresh member falls back to `member_default` (broad read) — so
9-
// every object reports `member-visible` and the #1994 by-id-write path isn't
10-
// exercised here. A hard, revert-provable gate needs an owner-scoped fixture
11-
// (a private-default object + a member permission set carrying RLS.ownerPolicy
12-
// + SecurityPlugin.fallbackPermissionSet) — tracked as the next step. The
13-
// invariant asserted now (zero holes) still guards against a regression that
14-
// makes a member able to mutate a record it cannot read.
7+
// SCOPE: this file is the single-tenant SMOKE over real apps. Single-tenant
8+
// strips the org `tenant_isolation` policy and a fresh member falls back to
9+
// `member_default` (broad read), so every object reports `member-visible` and
10+
// the by-id-write path isn't exercised HERE. That gap is now closed by two
11+
// sibling tests, so the hard gate lives there, not here:
12+
// • rls-fixture.dogfood.test.ts — owner-scoped fixture; green gate +
13+
// automated red proof + a documented manual #1994 revert proof (README).
14+
// • rls-multitenant.dogfood.test.ts — `{ multiTenant: true }`; org-scoped
15+
// (organization_id) isolation, the model real apps like hotcrm use.
16+
// The invariant asserted here (zero holes) still guards against a regression
17+
// that makes a member able to mutate a record it cannot read.
1518

1619
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
1720
import crmStack from '@objectstack/example-crm';
Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
//
3+
// Owner-isolated RLS fixture — the live, revert-provable #1994 gate.
4+
//
5+
// The example apps and hotcrm boot single-tenant, where the wildcard
6+
// `organization_id` tenant policy is stripped and a fresh member falls back to
7+
// `member_default` (broad read). Result: every object reports `member-visible`,
8+
// so the #1994 cross-owner write invariant ("a user who cannot READ a record
9+
// must not be able to WRITE it") is never actually exercised.
10+
//
11+
// This fixture creates the missing precondition with ZERO dependence on
12+
// org-scoping: a single object `rls_note` whose member permission set carries an
13+
// OWNER policy keyed on `created_by` (`RLS.ownerPolicy`). `created_by` is stamped
14+
// on every record by the engine and the predicate references `current_user.id`
15+
// (not `current_user.organization_id`), so it survives single-tenant stripping.
16+
// A fresh member therefore genuinely CANNOT read a note the admin created — the
17+
// exact cross-owner scenario the runner needs.
18+
//
19+
// Two member permission sets, identical except for the scope of the owner
20+
// policy, drive the green gate and the automated red proof:
21+
//
22+
// ownerScopedMemberSet (operation: 'all') → reads AND writes owner-scoped.
23+
// The #1994 pre-image check enforces the by-id write → `rls-consistent`.
24+
// readOnlyScopedMemberSet (operation: 'select') → reads owner-scoped, but NO
25+
// write policy applies, so the pre-image check has nothing to enforce and
26+
// the by-id write lands → `rls-hole`. This reproduces the #1994 hole CLASS
27+
// at the policy layer ("can't read, yet can write") without touching
28+
// engine code, so the gate's red path is proven on every CI run.
29+
30+
import { defineStack } from '@objectstack/spec';
31+
import { ObjectSchema, Field } from '@objectstack/spec/data';
32+
import { PermissionSetSchema, RLS, type PermissionSet } from '@objectstack/spec/security';
33+
import { SecurityPlugin, securityDefaultPermissionSets } from '@objectstack/plugin-security';
34+
35+
/** The one object under test: a private note, owner-scoped via `created_by`. */
36+
export const RlsNote = ObjectSchema.create({
37+
name: 'rls_note',
38+
label: 'RLS Note',
39+
pluralLabel: 'RLS Notes',
40+
fields: {
41+
name: Field.text({ label: 'Name', required: true }),
42+
body: Field.text({ label: 'Body' }),
43+
},
44+
});
45+
46+
/** A minimal, self-contained app config the dogfood harness can boot. */
47+
export const rlsFixtureStack = defineStack({
48+
manifest: {
49+
id: 'com.dogfood.rls_fixture',
50+
namespace: 'rls',
51+
version: '0.0.0',
52+
type: 'app',
53+
name: 'RLS Owner Fixture',
54+
description: 'Owner-isolated single-object app exercising the #1994 by-id-write invariant.',
55+
},
56+
objects: [RlsNote],
57+
});
58+
59+
/**
60+
* The fallback permission set a fresh member resolves to. Both variants grant
61+
* CRUD on `rls_note` (so the request reaches the RLS layer rather than being
62+
* denied by RBAC) and carry an owner RLS policy keyed on `created_by`. They
63+
* SHARE a name so each can be the `fallbackPermissionSet` of its own boot.
64+
*/
65+
const FIXTURE_MEMBER_SET = 'rls_fixture_member';
66+
67+
const noteCrud = {
68+
rls_note: { allowRead: true, allowCreate: true, allowEdit: true, allowDelete: true },
69+
} as const;
70+
71+
/**
72+
* GREEN. Owner policy on ALL operations — reads and writes are both
73+
* owner-scoped. A member cannot read another user's note, and the #1994
74+
* pre-image check (security-plugin.ts) re-reads the target row under the
75+
* write-op owner filter before a by-id update/delete, so the write is denied.
76+
* Expected runner verdict: `rls-consistent`.
77+
*/
78+
export const ownerScopedMemberSet: PermissionSet = PermissionSetSchema.parse({
79+
name: FIXTURE_MEMBER_SET,
80+
label: 'RLS Fixture Member — owner-scoped (all ops)',
81+
isProfile: true,
82+
objects: noteCrud,
83+
rowLevelSecurity: [RLS.ownerPolicy('rls_note', 'created_by')],
84+
});
85+
86+
/**
87+
* RED. Owner policy on SELECT only — reads stay owner-scoped (member still
88+
* can't see others' notes) but no UPDATE/DELETE policy applies, so
89+
* `computeRlsFilter` returns null for the write op and the pre-image check is
90+
* skipped → the by-id write lands. The member mutated a row it could not read:
91+
* the #1994 hole class. Expected runner verdict: `rls-hole`.
92+
*/
93+
export const readOnlyScopedMemberSet: PermissionSet = PermissionSetSchema.parse({
94+
name: FIXTURE_MEMBER_SET,
95+
label: 'RLS Fixture Member — owner-scoped reads only (#1994 hole)',
96+
isProfile: true,
97+
objects: noteCrud,
98+
rowLevelSecurity: [{ ...RLS.ownerPolicy('rls_note', 'created_by'), operation: 'select' }],
99+
});
100+
101+
/**
102+
* Build a SecurityPlugin whose fallback (for a fresh, grant-less member) is the
103+
* given fixture permission set, layered on top of the real platform defaults so
104+
* the seeded admin keeps `admin_full_access`.
105+
*/
106+
export function rlsFixtureSecurity(memberSet: PermissionSet): SecurityPlugin {
107+
return new SecurityPlugin({
108+
defaultPermissionSets: [...securityDefaultPermissionSets, memberSet],
109+
fallbackPermissionSet: memberSet.name,
110+
});
111+
}
Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
//
3+
// The HARD, revert-provable #1994 gate.
4+
//
5+
// `auto-verify-rls.dogfood.test.ts` runs the cross-owner runner over the real
6+
// apps, but single-tenant boot strips the tenant policy so every object is
7+
// `member-visible` — the by-id-write path is never exercised. This test boots a
8+
// purpose-built owner-isolated fixture (see `fixtures/rls-owner-fixture.ts`) so
9+
// a fresh member genuinely cannot read an admin-created record, then asserts the
10+
// runner's verdict in BOTH directions:
11+
//
12+
// • owner policy on ALL ops → `rls-consistent` (green gate). Safe ONLY
13+
// because the #1994 pre-image check enforces the by-id write — revert that
14+
// fix and this flips to `rls-hole` (see README for the manual revert proof,
15+
// and the RED block below for the automated analogue).
16+
// • owner policy on SELECT only → `rls-hole` (automated red proof). The read
17+
// is owner-scoped but no write policy applies, so the by-id write lands —
18+
// the #1994 hole class, reproduced without touching engine code. This proves
19+
// the gate can actually go red.
20+
21+
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
22+
import { bootDogfoodStack, type DogfoodStack } from '../src/harness.js';
23+
import { runRlsProofs, formatRlsReport, type RlsReport } from '../src/rls.js';
24+
import {
25+
rlsFixtureStack,
26+
ownerScopedMemberSet,
27+
readOnlyScopedMemberSet,
28+
rlsFixtureSecurity,
29+
} from './fixtures/rls-owner-fixture.js';
30+
31+
describe('objectstack verify RLS: owner-isolated fixture (#1994 hard gate)', () => {
32+
// ── GREEN: the gate that must stay consistent ──────────────────────────────
33+
describe('owner-scoped member set (all ops)', () => {
34+
let stack: DogfoodStack;
35+
let report: RlsReport;
36+
let adminToken: string;
37+
let memberToken: string;
38+
39+
beforeAll(async () => {
40+
stack = await bootDogfoodStack(rlsFixtureStack, {
41+
security: rlsFixtureSecurity(ownerScopedMemberSet),
42+
});
43+
adminToken = await stack.signIn();
44+
memberToken = await stack.signUp('owner-green@verify.test');
45+
report = await runRlsProofs(stack, adminToken, memberToken, rlsFixtureStack);
46+
// eslint-disable-next-line no-console
47+
console.error(formatRlsReport(report));
48+
}, 60_000);
49+
50+
afterAll(async () => {
51+
await stack?.stop();
52+
});
53+
54+
it('precondition: a fresh member CANNOT read an admin-created note (owner RLS reaches the member)', async () => {
55+
const created = await stack.apiAs(adminToken, 'POST', '/data/rls_note', {
56+
name: 'admin note',
57+
body: 'admin-only secret',
58+
});
59+
expect(created.status).toBeLessThan(300);
60+
const cj = (await created.json()) as { id?: string; record?: { id?: string } };
61+
const id = cj.id ?? cj.record?.id;
62+
expect(id, 'admin create should return an id').toBeTruthy();
63+
64+
const bRead = await stack.apiAs(memberToken, 'GET', `/data/rls_note/${id}`);
65+
// Owner-scoped: the member is not the creator, so the row is invisible.
66+
expect(bRead.status, 'member B must not be able to read the admin note').not.toBe(200);
67+
});
68+
69+
it('rls_note is rls-consistent — member can neither read nor mutate it by id', () => {
70+
const note = report.results.find((r) => r.object === 'rls_note');
71+
expect(note?.status, formatRlsReport(report)).toBe('rls-consistent');
72+
});
73+
74+
it('the report has ZERO holes and ZERO member-visible objects (real isolation)', () => {
75+
expect(report.summary.holes, formatRlsReport(report)).toBe(0);
76+
expect(report.summary.memberVisible, formatRlsReport(report)).toBe(0);
77+
expect(report.summary.consistent).toBeGreaterThanOrEqual(1);
78+
});
79+
});
80+
81+
// ── RED: proof the gate can go red on the #1994 hole class ──────────────────
82+
describe('read-only-scoped member set (select only) — #1994 hole reproduced', () => {
83+
let stack: DogfoodStack;
84+
let report: RlsReport;
85+
86+
beforeAll(async () => {
87+
stack = await bootDogfoodStack(rlsFixtureStack, {
88+
security: rlsFixtureSecurity(readOnlyScopedMemberSet),
89+
});
90+
const adminToken = await stack.signIn();
91+
const memberToken = await stack.signUp('owner-red@verify.test');
92+
report = await runRlsProofs(stack, adminToken, memberToken, rlsFixtureStack);
93+
// eslint-disable-next-line no-console
94+
console.error(formatRlsReport(report));
95+
}, 60_000);
96+
97+
afterAll(async () => {
98+
await stack?.stop();
99+
});
100+
101+
it('rls_note is rls-hole — member cannot read it yet mutated it by id', () => {
102+
const note = report.results.find((r) => r.object === 'rls_note');
103+
expect(note?.status, formatRlsReport(report)).toBe('rls-hole');
104+
expect(report.summary.holes).toBe(1);
105+
});
106+
});
107+
});

0 commit comments

Comments
 (0)