fix(rest,runtime): make api-exposure metadata fail-open observable (#3545) (#3567)

#3545 evaluated the residual risk of the API-exposure gate failing OPEN when
object metadata can't be resolved. Conclusion: acceptable — the gate is a
surface-area control, not the authz boundary (auth + CRUD/FLS/RLS enforce on the
data call regardless), and failing closed would 405 every request during the
cold-start window for no security gain. The one gap was that the fail-open was
SILENT.

- rest: loadObjectItems logs a THROWN metadata read (a real fault) while leaving
  a legitimately-empty registry silent — so a persistent outage, during which
  the gate allows every op unchecked, is diagnosable without cold-start false
  alarms. Behavior unchanged (still returns [] → gate abstains).
- runtime: api-exposure.ts records the #3545 tiered decision in its contract doc
  (keep fail-open when the whole metadata service is unavailable; defer the
  narrow present-but-unreadable-policy widen — unreachable via Zod-validated
  registration — to the exposure-semantics window #3543).

Tests: rest gate suite gains three #3545 cases (thrown read → fail-open + logged;
empty registry → fail-open + silent; enforceApiAccess does not block on throw).
Full rest.test.ts 171 pass; rest + runtime build (CJS/ESM/DTS) clean.


Claude-Session: https://claude.ai/code/session_012L8EfEa157Pe6C73qRnaJH

Co-authored-by: Claude <noreply@anthropic.com>

Author: os-zhuang

.changeset/api-exposure-failopen-observability.md

@@ -0,0 +1,33 @@
+---
+"@objectstack/rest": patch
+"@objectstack/runtime": patch
+---
+
+fix(rest): make the API-exposure gate's metadata fail-open observable (#3545, #3391 follow-up)
+
+The object API-exposure gate (`apiEnabled` / `apiMethods`) fails OPEN when object
+metadata can't be resolved, so a transient metadata outage doesn't 405 every
+request. #3545 evaluated the residual risk of that path and confirmed it is
+acceptable — the gate is a **surface-area control, not the authorization
+boundary**: every request still passes auth and the ObjectQL security middleware
+(CRUD / FLS / RLS) on the data call regardless of the gate's outcome, so a
+fail-open can never bypass data authorization.
+
+The one gap was that the fail-open was **silent** — a persistent metadata fault
+(store down / corrupt schema doc), during which the gate allows every operation
+unchecked, looked identical to healthy operation.
+
+- **rest** `loadObjectItems` now LOGS a *thrown* metadata read (a real fault)
+  while leaving a legitimately-empty registry (a cold-start `[]`) silent — so a
+  genuine outage is diagnosable without false alarms during normal startup. The
+  behavior is unchanged (still returns `[]` → gate abstains → data path + security
+  enforce).
+- **runtime** `api-exposure.ts` records the #3545 tiered decision in its
+  contract doc: keep fail-open when the whole metadata service is unavailable
+  (failing closed would break the cold-start window for no security gain); the
+  narrow "object resolvable but its `enable` policy is present-yet-unreadable"
+  widen (unreachable through Zod-validated registration) is deferred to the
+  exposure-semantics window (#3543).
+
+No contract or behavior change to the gate itself — observability + decision
+record only.

packages/rest/src/rest-server.ts

@@ -41,6 +41,7 @@ import { prepareImportRequest, isMetaEnvelope } from './import-prepare.js';
 
 // Node-safe logger — avoids importing 'console' which is absent from ES2020 lib typings.
 const logError = (...args: unknown[]) => (globalThis as any).console?.error(...args);
+const logWarn = (...args: unknown[]) => ((globalThis as any).console?.warn ?? (globalThis as any).console?.error)?.(...args);
 
 /**
  * Metadata types whose user-facing labels are localized at the REST boundary
@@ -1159,7 +1160,23 @@ export class RestServer {
                 ...(environmentId ? { environmentId } : {}),
             });
             return Array.isArray(r?.items) ? r.items : Array.isArray(r) ? r : [];
-        } catch {
+        } catch (err) {
+            // [#3545] The API-exposure gate fails OPEN when object metadata can't
+            // be read: the exposure whitelist is a SURFACE-AREA control, not the
+            // authorization boundary (auth + CRUD/FLS/RLS still enforce on the
+            // data call, which needs the same metadata and surfaces the real
+            // error), and failing closed here would 405 every request during the
+            // normal cold-start window. But a THROWN read is a real fault
+            // (metadata store down / corrupt schema doc), NOT a legitimately-empty
+            // registry (a `[]` return, e.g. a fresh deployment) — so LOG it. Left
+            // silent, a persistent metadata outage, during which the gate allows
+            // every operation unchecked, is indistinguishable from healthy
+            // operation. Still returns `[]` (fail-open preserved). See #3545.
+            logWarn(
+                '[REST] api-exposure gate: object metadata read failed — failing open ' +
+                    '(auth + CRUD/FLS/RLS still enforce on the data call)',
+                (err as Error)?.message ?? err,
+            );
             return [];
         }
     }

packages/rest/src/rest.test.ts

@@ -2827,4 +2827,56 @@ describe('RestServer — object API exposure (apiEnabled / apiMethods)', () => {
     expect(res.statusCode).toBe(404);
     expect(protocol.createManyData).not.toHaveBeenCalled();
   });
+
+  // [#3545] fail-open residual-risk decision: the exposure gate is a
+  // surface-area control (not the authz boundary — auth + CRUD/FLS/RLS still
+  // enforce on the data call), so an unresolvable metadata read fails OPEN to
+  // avoid 405ing every request during the cold-start window. The one change is
+  // OBSERVABILITY: a THROWN metadata read (a real fault) is logged, while a
+  // legitimately-empty registry stays silent — so a persistent outage, during
+  // which the gate silently allows every op, is no longer invisible.
+  describe('metadata-unavailable fail-open observability (#3545)', () => {
+    it('loadObjectItems fails OPEN (returns []) AND logs a THROWN metadata read', async () => {
+      const { rest, protocol } = setup(undefined);
+      protocol.getMetaItems = vi.fn().mockRejectedValue(new Error('metadata store down'));
+      const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
+      try {
+        const items = await (rest as any).loadObjectItems(protocol, undefined);
+        expect(items).toEqual([]); // fail-open preserved: gate abstains
+        expect(warnSpy).toHaveBeenCalled();
+        expect(String(warnSpy.mock.calls[0]?.[0] ?? '')).toContain('api-exposure gate');
+      } finally {
+        warnSpy.mockRestore();
+      }
+    });
+
+    it('loadObjectItems stays SILENT on a legitimately-empty registry (no false alarm)', async () => {
+      const { rest, protocol } = setup(undefined);
+      protocol.getMetaItems = vi.fn().mockResolvedValue([]); // empty, not thrown
+      const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
+      try {
+        const items = await (rest as any).loadObjectItems(protocol, undefined);
+        expect(items).toEqual([]);
+        expect(warnSpy).not.toHaveBeenCalled(); // distinguishes real fault from empty
+      } finally {
+        warnSpy.mockRestore();
+      }
+    });
+
+    it('enforceApiAccess does NOT block when the metadata read throws (fail-open)', async () => {
+      const { rest, protocol } = setup(undefined);
+      protocol.getMetaItems = vi.fn().mockRejectedValue(new Error('metadata store down'));
+      const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
+      try {
+        const res = makeRes();
+        const blocked = await (rest as any).enforceApiAccess(
+          { params: { object: 'widget' } }, res, protocol, undefined, 'list',
+        );
+        expect(blocked).toBe(false); // request proceeds — data path + security enforce
+        expect(res.status).not.toHaveBeenCalled();
+      } finally {
+        warnSpy.mockRestore();
+      }
+    });
+  });
 });

packages/runtime/src/api-exposure.ts

@@ -27,6 +27,28 @@
  * defaults (`apiEnabled` true, no `apiMethods`) and avoids breaking traffic when
  * metadata is briefly unavailable. The gate is a no-op for system/internal
  * contexts (callers pass `isSystem` and skip this check entirely).
+ *
+ * ## #3545 — fail-open residual-risk decision
+ *
+ * This gate is a SURFACE-AREA control (which API operations an object exposes),
+ * NOT the authorization boundary — every request still passes auth and the
+ * ObjectQL security middleware (CRUD / FLS / RLS) on the data call regardless of
+ * the outcome here. So a fail-open on unresolvable metadata cannot bypass data
+ * authorization; at worst an operation the author meant to HIDE from the API is
+ * transiently reachable (still fully access-controlled). Given that, the tiered
+ * decision is:
+ *   • Metadata service not ready / whole registry unavailable (cold start,
+ *     registration race, scoped kernel warming) → KEEP fail-open. Failing closed
+ *     would 405 every request during the normal startup window for no security
+ *     gain (the data call fails or is authorized independently). Callers that
+ *     read metadata (e.g. the REST `loadObjectItems`) LOG a thrown read so a
+ *     PERSISTENT outage is observable instead of a silent blanket-allow.
+ *   • Object resolvable but its `enable`/`apiMethods` is present-yet-unreadable
+ *     (a non-array policy) → `resolveEffectiveApiMethods` currently treats it as
+ *     `unrestricted` (silent widen). This path is unreachable through the
+ *     Zod-validated registration flow (only a raw/out-of-band metadata write
+ *     could produce it), so tightening it to fail-CLOSED is deferred to the
+ *     exposure-semantics window (#3543) rather than changed here unilaterally.
  */
 
 import {