feat(auth)!: retire the api.requireAuth opt-out — anonymous data access is always denied (#3963 step 2) (#4043)

`api.requireAuth: false` let a deployment open the entire data plane with one boolean. Retired: auth is a kernel concern, and anonymous callers are now denied unconditionally on every HTTP surface that touches object data. Every surface that legitimately serves a session-less caller derives its own narrow authorization from a declaration (control-plane allowlist, publicFormGrant, share-link token, book.audience:'public', MCP token) — none needs the global switch.

Retirement ran the full repo process: retiredKey() tombstones on both schemas, protocol-18 conversion + migration step18, regenerated artifacts. Auth-less stacks that would serve data now fail at boot (A1). shouldDenyAnonymous no longer accepts a requireAuth input.

Closes #3963.

Author: os-zhuang

.changeset/drop-require-auth.md

@@ -0,0 +1,49 @@
+---
+"@objectstack/spec": major
+"@objectstack/rest": major
+"@objectstack/runtime": major
+"@objectstack/core": minor
+"@objectstack/cli": minor
+"@objectstack/plugin-hono-server": minor
+"@objectstack/plugin-dev": minor
+---
+
+feat(auth)!: retire the `api.requireAuth` opt-out — anonymous access to object data is always denied (#3963)
+
+`api.requireAuth: false` let a deployment open its ENTIRE data plane with one
+config key. It is removed. Auth is a kernel concern, not a deployment posture:
+anonymous callers are denied on every HTTP surface that reaches object data,
+unconditionally.
+
+Every surface that legitimately serves a session-less caller already derives its
+own narrow authorization from a DECLARATION, so none of them needed the global
+switch:
+
+- control plane (`/auth/*`, `/health`, `/ready`, `/discovery`, ADR-0069
+  remediation) — the auth-gate allowlist;
+- public form submission — `publicFormGrant` (ADR-0056 Option A);
+- share links — the capability token, validated then read as SYSTEM;
+- a `book.audience: 'public'` read — the ADR-0046 §6.7 audience gate (#3995);
+- MCP — an OAuth token or API key.
+
+**Breaking changes.**
+
+- `api.requireAuth` is a retired key. It is tombstoned (`retiredKey`) in both
+  `RestApiConfigSchema` and the stack `api` block, so authoring it now fails with
+  a fix-it message rather than being silently stripped (the ADR-0104 / #3733
+  quiet-failure this whole line of work has been closing). `os migrate meta`
+  drops it via the protocol-18 conversion `stack-api-require-auth-removed`.
+- `shouldDenyAnonymous` (@objectstack/core) no longer takes a `requireAuth`
+  input; it denies any anonymous, non-system caller outside the control-plane
+  allowlist.
+- A stack that mounts **no auth at all** now FAILS AT BOOT when it would serve a
+  data API (`objectstack serve`, plugin-dev), instead of getting an explicit
+  fail-open. Enable auth (the `auth` tier or AuthPlugin), or run without the data
+  API. There is no anonymous-data carve-out any more — publishing a public
+  surface is done by declaration (see above).
+
+**Migration.** Delete `api.requireAuth` from the stack config (or run
+`os migrate meta`). If you were serving data publicly with `requireAuth: false`,
+replace it with the declaration that fits: a public form view, a share link, or
+`book.audience: 'public'`. If you have an auth-less stack that intentionally
+served data, it must now mount auth or stop serving the data API.

content/docs/references/api/rest-server.mdx

@@ -192,7 +192,7 @@ const result = BatchEndpointsConfig.parse(data);
 | **enableOpenApi** | `boolean` | ✅ | Enable OpenAPI 3.1 spec & docs viewer endpoints |
 | **enableProjectScoping** | `boolean` | ✅ | Enable project-scoped routing for data/meta/AI APIs |
 | **projectResolution** | `Enum<'required' \| 'optional' \| 'auto'>` | ✅ | Project ID resolution strategy |
-| **requireAuth** | `boolean` | ✅ | Reject anonymous requests on ALL HTTP surfaces that reach object data (REST /data, /meta, raw-hono /data) with HTTP 401 (secure-by-default; set false to serve them publicly) |
+| **requireAuth** | `any` | optional | [REMOVED] `api.requireAuth` was removed in @objectstack/spec 18 (#3963). Anonymous access to object data is now always denied — auth is a kernel concern, not a deployment posture. Delete the key. To publish something publicly, declare it: a public form view (`sharing.allowAnonymous`), a share link, or `book.audience: 'public'` — each derives its own narrow authorization instead of opening the whole data plane. |
 | **documentation** | `{ enabled: boolean; title: string; description?: string; version?: string; … }` | optional | OpenAPI/Swagger documentation config |
 | **responseFormat** | `{ envelope: boolean; includeMetadata: boolean; includePagination: boolean }` | optional | Response format options |
 

packages/cli/src/commands/serve.ts

@@ -1728,26 +1728,34 @@ export default class Serve extends Command {
         // (env-native auth IS the membership — ADR-0024 D9) by setting
         // `api.enforceProjectMembership: false`. Undefined → dispatcher default.
         const enforceProjectMembership = apiConfig.enforceProjectMembership;
-        // `requireAuth: true` rejects anonymous requests on `/api/v1/data/*`
-        // with HTTP 401 before they reach ObjectQL. The platform default is
-        // now secure-by-default (ADR-0056 D2): deny anonymous. The CLI keeps
-        // one deliberate carve-out — a stack with NO `auth` tier has no way
-        // to authenticate anyone, so denying would brick its data API
-        // entirely; such auth-less playgrounds get an EXPLICIT `false`
-        // (fail-open), which the REST plugin surfaces with a boot warning.
-        // Apps can always override via stack `api.requireAuth`.
-        // Auth availability = tier auto-registers it OR the stack mounts
-        // AuthPlugin explicitly (hasAuthPlugin, computed above). Keying only on
-        // the tier would hand an explicit fail-open to a stack that ships auth
-        // via `plugins:` under a minimal tier set — re-opening the very hole
-        // the flip closes. Only a stack with NO auth at all gets the carve-out.
-        const requireAuth = apiConfig.requireAuth
-          ?? ((tierEnabled('auth') || hasAuthPlugin) ? true : false);
+        // [#3963] Anonymous access to object data is denied unconditionally —
+        // there is no `api.requireAuth` opt-out any more (auth is a kernel
+        // concern; every legitimately session-less surface derives its own narrow
+        // authorization from a declaration instead).
+        //
+        // The CLI used to hand an EXPLICIT fail-open to a stack with no auth at
+        // all, reasoning that nobody could authenticate against it so denying
+        // would brick its data API. Under A1 that inverts the conclusion: a stack
+        // with no auth has no security model, so it must not serve a data API —
+        // and it should say so at boot instead of quietly serving object data to
+        // the internet. Auth availability = the tier auto-registers it OR the
+        // stack mounts AuthPlugin explicitly.
+        if (flags.server && !(tierEnabled('auth') || hasAuthPlugin)) {
+          throw new Error(
+            'This stack mounts no auth, so no caller can authenticate — and anonymous access to object '
+            + 'data is always denied (#3963), which would leave the data API unusable.\n'
+            + 'Fix it one of two ways:\n'
+            + `  • enable auth — add the 'auth' tier (or mount AuthPlugin in \`plugins\`);\n`
+            + '  • or serve without the data API — run with --no-server, or drop the REST/dispatcher plugins.\n'
+            + "Publishing a genuinely public surface does not need anonymous data access: use a public form "
+            + "view, a share link, or `book.audience: 'public'`.",
+          );
+        }
 
         try {
           const { createRestApiPlugin } = await import('@objectstack/rest');
           await kernel.use(
-            createRestApiPlugin({ api: { api: { enableProjectScoping, projectResolution, requireAuth } } as any }),
+            createRestApiPlugin({ api: { api: { enableProjectScoping, projectResolution } } as any }),
           );
           trackPlugin('RestAPI');
         } catch (e: any) {
@@ -1767,9 +1775,6 @@ export default class Serve extends Command {
             createDispatcherPlugin({
               scoping: { enableProjectScoping, projectResolution },
               enforceProjectMembership,
-              // Keep the dispatcher's `auth: true` service routes (AI) in
-              // lockstep with the REST `/data` gate above — same `requireAuth`.
-              requireAuth,
               observability,
             }),
           );

packages/cli/src/utils/merge-boot-config.test.ts

@@ -7,28 +7,27 @@ import { mergeBootConfig } from './merge-boot-config.js';
 const BOOT_API = { enableProjectScoping: false, projectResolution: 'none' } as const;
 
 describe('mergeBootConfig (#4002)', () => {
-    it('keeps the authored api keys the boot result does not set', () => {
+    it('keeps an authored api key the boot result does not set', () => {
         const merged: any = mergeBootConfig(
-            { api: { requireAuth: false, enforceProjectMembership: false } },
+            { api: { enforceProjectMembership: false } },
             { api: { ...BOOT_API }, plugins: [] },
         );
 
-        // The two live knobs the CLI reads a few lines later — both were dropped
-        // by the old shallow spread.
-        expect(merged.api.requireAuth).toBe(false);
+        // `enforceProjectMembership` is a live knob the CLI reads a few lines
+        // later — dropped by the old shallow spread, kept by the per-key merge.
         expect(merged.api.enforceProjectMembership).toBe(false);
     });
 
     it('lets the boot result win on the keys it decides', () => {
         // Environment scoping is not the author's call on a standalone host.
         const merged: any = mergeBootConfig(
-            { api: { enableProjectScoping: true, projectResolution: 'auto', requireAuth: false } },
+            { api: { enableProjectScoping: true, projectResolution: 'auto', enforceProjectMembership: false } },
             { api: { ...BOOT_API } },
         );
 
         expect(merged.api.enableProjectScoping).toBe(false);
         expect(merged.api.projectResolution).toBe('none');
-        expect(merged.api.requireAuth).toBe(false); // untouched by boot → survives
+        expect(merged.api.enforceProjectMembership).toBe(false); // untouched by boot → survives
     });
 
     it('still replaces every other top-level key wholesale', () => {
@@ -49,8 +48,8 @@ describe('mergeBootConfig (#4002)', () => {
     });
 
     it('carries an api block through when only one side has one', () => {
-        expect((mergeBootConfig({ api: { requireAuth: false } }, {}) as any).api)
-            .toEqual({ requireAuth: false });
+        expect((mergeBootConfig({ api: { enforceProjectMembership: false } }, {}) as any).api)
+            .toEqual({ enforceProjectMembership: false });
         expect((mergeBootConfig({}, { api: { ...BOOT_API } }) as any).api)
             .toEqual({ ...BOOT_API });
     });
@@ -60,16 +59,16 @@ describe('mergeBootConfig (#4002)', () => {
         // character-indexed keys from a string spread.
         expect((mergeBootConfig({ api: 'nonsense' as any }, { api: { ...BOOT_API } }) as any).api)
             .toEqual({ ...BOOT_API });
-        expect((mergeBootConfig({ api: { requireAuth: false } }, { api: null as any }) as any).api)
-            .toEqual({ requireAuth: false });
+        expect((mergeBootConfig({ api: { enforceProjectMembership: false } }, { api: null as any }) as any).api)
+            .toEqual({ enforceProjectMembership: false });
     });
 
     it('does not mutate either input', () => {
-        const authored = { api: { requireAuth: false } };
+        const authored = { api: { enforceProjectMembership: false } };
         const boot = { api: { ...BOOT_API } };
         mergeBootConfig(authored, boot);
 
-        expect(authored).toEqual({ api: { requireAuth: false } });
+        expect(authored).toEqual({ api: { enforceProjectMembership: false } });
         expect(boot).toEqual({ api: { ...BOOT_API } });
     });
 });

packages/client/src/client.batch-transaction.test.ts

@@ -37,6 +37,17 @@ describe('data.batchTransaction (live Hono, #1604)', () => {
     beforeAll(async () => {
         kernel = new LiteKernel();
         kernel.use(new ObjectQLPlugin());
+        // [#3963] The anonymous-deny gate is unconditional now, so the live-server
+        // client suites need an authenticated session. Register a minimal auth
+        // service that resolves a fixed user for every request.
+        kernel.use({
+            metadata: { name: 'test-auth', version: '1.0.0' },
+            async init(ctx: any) {
+                ctx.registerService('auth', {
+                    api: { getSession: async () => ({ user: { id: 'test-user' } }) },
+                });
+            },
+        } as any);
 
         kernel.use(
             new HonoServerPlugin({

packages/client/src/client.environment-scoping.test.ts

@@ -28,6 +28,17 @@ describe('Project-scoped REST routing (live Hono)', () => {
     beforeAll(async () => {
         kernel = new LiteKernel();
         kernel.use(new ObjectQLPlugin());
+        // [#3963] The anonymous-deny gate is unconditional now, so the live-server
+        // client suites need an authenticated session. Register a minimal auth
+        // service that resolves a fixed user for every request.
+        kernel.use({
+            metadata: { name: 'test-auth', version: '1.0.0' },
+            async init(ctx: any) {
+                ctx.registerService('auth', {
+                    api: { getSession: async () => ({ user: { id: 'test-user' } }) },
+                });
+            },
+        } as any);
 
         const honoPlugin = new HonoServerPlugin({
             port: 0,

packages/client/src/client.hono.test.ts

@@ -13,6 +13,17 @@ describe('ObjectStackClient (with Hono Server)', () => {
         // 1. Setup Kernel
         kernel = new LiteKernel();
         kernel.use(new ObjectQLPlugin());
+        // [#3963] The anonymous-deny gate is unconditional now, so the live-server
+        // client suites need an authenticated session. Register a minimal auth
+        // service that resolves a fixed user for every request.
+        kernel.use({
+            metadata: { name: 'test-auth', version: '1.0.0' },
+            async init(ctx: any) {
+                ctx.registerService('auth', {
+                    api: { getSession: async () => ({ user: { id: 'test-user' } }) },
+                });
+            },
+        } as any);
 
         // 2. Setup Hono Plugin
         // This suite exercises the CLIENT's data operations over the raw-hono
@@ -25,7 +36,6 @@ describe('ObjectStackClient (with Hono Server)', () => {
         const honoPlugin = new HonoServerPlugin({
             port: 0,
             registerStandardEndpoints: true,
-            restConfig: { api: { requireAuth: false } } as any,
         });
         kernel.use(honoPlugin);
 

packages/core/src/security/anonymous-deny.test.ts

@@ -3,49 +3,48 @@
 // #2567 Phase 2 — the shared anonymous-deny decision. These lock the exact
 // contract every HTTP seam now delegates to, including the load-bearing
 // `undefined`-path trap (a naive allowlist call would reopen GraphQL).
+//
+// [#3963] The `requireAuth` opt-out is gone: an anonymous, non-system caller
+// outside the control-plane allowlist is denied unconditionally. There is no
+// longer a posture that turns the decision off.
 
 import { describe, it, expect } from 'vitest';
 import { shouldDenyAnonymous, ANONYMOUS_DENY_BODY, ANONYMOUS_DENY_STATUS } from './anonymous-deny.js';
 
-describe('shouldDenyAnonymous — the shared HTTP anonymous-deny decision (#2567)', () => {
-  it('no-op when requireAuth is off (demo / single-tenant)', () => {
-    expect(shouldDenyAnonymous({ requireAuth: false })).toBe(false);
-    expect(shouldDenyAnonymous({ requireAuth: undefined })).toBe(false);
-  });
-
-  it('denies an anonymous caller under requireAuth', () => {
-    expect(shouldDenyAnonymous({ requireAuth: true })).toBe(true);
+describe('shouldDenyAnonymous — the shared HTTP anonymous-deny decision (#2567, #3963)', () => {
+  it('denies an anonymous caller (unconditionally — no opt-out)', () => {
+    expect(shouldDenyAnonymous({})).toBe(true);
   });
 
   it('passes an authenticated caller', () => {
-    expect(shouldDenyAnonymous({ requireAuth: true, userId: 'u1' })).toBe(false);
+    expect(shouldDenyAnonymous({ userId: 'u1' })).toBe(false);
   });
 
   it('passes an internal system context', () => {
-    expect(shouldDenyAnonymous({ requireAuth: true, isSystem: true })).toBe(false);
+    expect(shouldDenyAnonymous({ isSystem: true })).toBe(false);
   });
 
   it('passes an OPTIONS preflight even when anonymous', () => {
-    expect(shouldDenyAnonymous({ requireAuth: true, method: 'OPTIONS' })).toBe(false);
-    expect(shouldDenyAnonymous({ requireAuth: true, method: 'options' })).toBe(false);
+    expect(shouldDenyAnonymous({ method: 'OPTIONS' })).toBe(false);
+    expect(shouldDenyAnonymous({ method: 'options' })).toBe(false);
   });
 
   it('exempts a real control-plane path (auth / health)', () => {
-    expect(shouldDenyAnonymous({ requireAuth: true, path: '/api/v1/auth/login' })).toBe(false);
-    expect(shouldDenyAnonymous({ requireAuth: true, path: '/api/v1/health' })).toBe(false);
+    expect(shouldDenyAnonymous({ path: '/api/v1/auth/login' })).toBe(false);
+    expect(shouldDenyAnonymous({ path: '/api/v1/health' })).toBe(false);
   });
 
   it('denies a real data path', () => {
-    expect(shouldDenyAnonymous({ requireAuth: true, path: '/api/v1/data/sys_user' })).toBe(true);
+    expect(shouldDenyAnonymous({ path: '/api/v1/data/sys_user' })).toBe(true);
   });
 
   // The trap: isAuthGateAllowlisted(undefined) === true. A body-routed seam
   // (GraphQL) passes no path; it MUST still deny anonymous, not fall through to
   // the allowlist. Guards against silently reopening #2567.
   it('denies when path is undefined/empty (body-routed seam — GraphQL trap guard)', () => {
-    expect(shouldDenyAnonymous({ requireAuth: true, path: undefined })).toBe(true);
-    expect(shouldDenyAnonymous({ requireAuth: true, path: null })).toBe(true);
-    expect(shouldDenyAnonymous({ requireAuth: true, path: '' })).toBe(true);
+    expect(shouldDenyAnonymous({ path: undefined })).toBe(true);
+    expect(shouldDenyAnonymous({ path: null })).toBe(true);
+    expect(shouldDenyAnonymous({ path: '' })).toBe(true);
   });
 
   it('exposes a stable 401 body + status for seams to return', () => {