fix(spec,metadata-protocol,runtime): stop advertising routes for the kernel-internal cache/queue/job slots (#4318) (#4448)

`SERVICE_CONFIG` declared `/api/v1/cache`, `/api/v1/queue` and `/api/v1/jobs` —
three paths that existed nowhere else in the repository: no dispatcher domain, no
adapter mount, no plugin registration. Nor was one pending: the slots' shipped
providers (`service-cache` / `-queue` / `-job`) are in-process contracts that
mount no HTTP surface. The kernel pre-injects self-describing in-memory fallbacks
into all three on every default boot, so every default deployment emitted a
`ServiceInfo` whose `route` said "call me here" next to its own
`handlerReady: false` saying "there is no handler".

The route is removed at the root rather than suppressed per-occupant: these slots
are route-less now, structurally, the way `realtime` already was. What differs
from `realtime` is the unmarked case, so each route-less entry states it —
`realtime`'s advertised capability IS the missing HTTP/WS surface, so an
in-process bus there is `degraded`; a cache/queue/job slot's contract is
in-process to begin with, so a real (unmarked) implementation stays `available`.

The dispatcher builder had the same defect one field over: `svcAvailable` gave an
unmarked occupant `handlerReady: true`, a handler that does not exist. Those
slots report through `svcInProcess` now, with `handlerReady` pinned `false`. The
explanatory message is written once, as `inProcessServiceMessage()` in
`@objectstack/spec/system`, so the two builders cannot drift.

Tests pin both builders and, for cache/queue/job, pin them against each other
across both occupant shapes. The two service READMEs advertised REST endpoint
tables for surfaces that were never mounted; replaced with what is true.

Closes #4318.

Author: os-zhuang

.changeset/discovery-cache-queue-job-no-route.md

@@ -0,0 +1,25 @@
+---
+"@objectstack/spec": patch
+"@objectstack/metadata-protocol": patch
+"@objectstack/runtime": patch
+---
+
+fix(spec,metadata-protocol,runtime): discovery stops advertising routes for the kernel-internal cache/queue/job slots (#4318)
+
+The metadata-protocol discovery builder declared `/api/v1/cache`, `/api/v1/queue`
+and `/api/v1/jobs` — three paths that existed nowhere else in the repository: no
+dispatcher domain, no adapter mount, no plugin registration, and the shipped
+providers (`service-cache`/`-queue`/`-job`) are in-process contracts that will
+never mount one. Every default boot therefore advertised a route inside the same
+`ServiceInfo` whose `handlerReady: false` said the opposite — a single record
+contradicting itself (ADR-0076 D12).
+
+These slots are route-less now, like `realtime` — but unlike `realtime` an
+unmarked real implementation stays `available`: the slot's contract is
+in-process, so "no HTTP surface" is not reduced capability for it. `handlerReady`
+is reported `false` on both discovery builders — for a route-less slot it is not
+a proxy for anything, it is the fact itself (the dispatcher used to claim
+`handlerReady: true` here for an unmarked occupant, a handler that does not
+exist). The explanatory message is written once, as
+`inProcessServiceMessage(slot)` in `@objectstack/spec/system`, so the two
+builders cannot drift apart.

packages/metadata-protocol/src/protocol.ts

@@ -29,7 +29,7 @@ import {
 import { PLURAL_TO_SINGULAR, SINGULAR_TO_PLURAL } from '@objectstack/spec/shared';
 import { applyConversionsToStoredItem } from '@objectstack/spec';
 import { type FormView, isAggregatedViewContainer } from '@objectstack/spec/ui';
-import { METADATA_FORM_REGISTRY, CORE_SERVICE_PROVIDER, serviceUnavailableMessage } from '@objectstack/spec/system';
+import { METADATA_FORM_REGISTRY, CORE_SERVICE_PROVIDER, serviceUnavailableMessage, inProcessServiceMessage } from '@objectstack/spec/system';
 import { DEFAULT_METADATA_TYPE_REGISTRY, getMetadataTypeSchema, getMetadataTypeActions, getMetadataCreateSeed } from '@objectstack/spec/kernel';
 import {
     extractProtection,
@@ -1194,9 +1194,16 @@ function suggestFieldName(name: string, knownFields: readonly string[]): string
  * Service Configuration for Discovery
  * Maps service names to their routes and plugin providers.
  *
- * `route: undefined` means the service has NO HTTP surface — discovery must
+ * A missing `route` means the service has NO HTTP surface — discovery must
  * not advertise a route for it (ADR-0076 D12, #2462: an advertised route
- * with no mounted handler 404s and misleads consumers).
+ * with no mounted handler 404s and misleads consumers). Such entries carry
+ * `noHttpSurface` instead, stating how to report an occupant that does not
+ * self-describe: `realtime`'s advertised capability IS the missing HTTP/WS
+ * surface, so an in-process bus is `degraded`; `cache`/`queue`/`job` are
+ * kernel-internal contracts fully served in-process (#4318), so an unmarked
+ * real implementation stays `available`. Either way `handlerReady` is
+ * reported `false` — for a route-less slot it is not a proxy for anything,
+ * it is the fact itself.
  */
 /**
  * [#4093 follow-up] `plugin` is no longer written here. It named the package a
@@ -1212,22 +1219,36 @@ function suggestFieldName(name: string, knownFields: readonly string[]): string
  * registers each slot and guarded by `scripts/check-service-providers.mjs`.
  * Only the ROUTE stays local — that is this builder's own knowledge.
  */
-const SERVICE_CONFIG: Record<string, { route?: string }> = {
+const SERVICE_CONFIG: Record<string, {
+    route?: string;
+    /** Route-less slots only: status + message for an occupant with no self-description. */
+    noHttpSurface?: { statusWhenUnmarked: 'available' | 'degraded'; message: string };
+}> = {
     // Plugin-provided like every other optional service since the degraded
     // ObjectQL fallback was retired (#3891): advertised iff the real engine
     // is registered — never hardcoded 'available' (the pre-#2462 lie the
     // fallback existed to paper over).
     analytics:    { route: '/api/v1/analytics' },
     auth:         { route: '/api/v1/auth' },
     automation:   { route: '/api/v1/automation' },
-    cache:        { route: '/api/v1/cache' },
-    queue:        { route: '/api/v1/queue' },
-    job:          { route: '/api/v1/jobs' },
+    // Kernel-internal slots (#4318): their providers (service-cache/-queue/
+    // -job) mount no HTTP routes — these are in-process contracts, not HTTP
+    // capabilities, so there is no route to advertise and never will be. The
+    // /api/v1/cache|queue|jobs paths this table used to declare existed
+    // nowhere else in the repository; every default boot advertised them next
+    // to the fallbacks' own `handlerReady: false` — a single ServiceInfo
+    // contradicting itself.
+    cache:        { noHttpSurface: { statusWhenUnmarked: 'available', message: inProcessServiceMessage('cache') } },
+    queue:        { noHttpSurface: { statusWhenUnmarked: 'available', message: inProcessServiceMessage('queue') } },
+    job:          { noHttpSurface: { statusWhenUnmarked: 'available', message: inProcessServiceMessage('job') } },
     ui:           { route: '/api/v1/ui' },
     workflow:     { route: '/api/v1/workflow' },
     // service-realtime is an in-process pub/sub bus; nothing mounts
-    // /api/v1/realtime, so no route is advertised (D12, #2462).
-    realtime:     {},
+    // /api/v1/realtime, so no route is advertised (D12, #2462). Unlike the
+    // kernel-internal slots above, the capability this slot advertises is
+    // realtime push to clients — without a surface that IS reduced, so an
+    // unmarked bus reports degraded. Message matches the dispatcher builder.
+    realtime:     { noHttpSurface: { statusWhenUnmarked: 'degraded', message: 'In-process event bus only — no HTTP/WS realtime surface is mounted' } },
     notification: { route: '/api/v1/notifications' },
     ai:           { route: '/api/v1/ai' },
     i18n:         { route: '/api/v1/i18n' },
@@ -2077,21 +2098,23 @@ export class ObjectStackProtocolImplementation implements
                 // Registered — but honor a stub/dev/fallback self-description
                 // instead of blindly reporting 'available' (ADR-0076 D12).
                 const self = readServiceSelfInfo(registeredServices.get(serviceName));
-                // No HTTP surface at all (e.g. realtime): the handler can never
-                // be ready and 'available' would overstate it — report degraded.
+                // No HTTP surface at all: the handler can never be ready, and
+                // the entry's own `noHttpSurface` declaration says whether that
+                // also degrades the slot (realtime) or not (cache/queue/job —
+                // in-process contracts, #4318).
                 const noHttpSurface = !config.route;
                 services[serviceName] = {
                     enabled: true,
-                    status: self?.status ?? (noHttpSurface ? ('degraded' as const) : ('available' as const)),
+                    status: self?.status ?? (config.noHttpSurface?.statusWhenUnmarked ?? ('available' as const)),
                     route: advertisedRoute(serviceName, config.route),
                     provider: CORE_SERVICE_PROVIDER[serviceName] ?? undefined,
                     ...(noHttpSurface || self?.handlerReady !== undefined
                         ? { handlerReady: noHttpSurface ? false : self?.handlerReady }
                         : {}),
                     ...(self?.message
                         ? { message: self.message }
-                        : noHttpSurface
-                            ? { message: 'In-process service only — no HTTP surface is mounted' }
+                        : config.noHttpSurface
+                            ? { message: config.noHttpSurface.message }
                             : {}),
                 };
             } else {

packages/objectql/src/protocol-discovery.test.ts

@@ -333,10 +333,12 @@ describe('ObjectStackProtocolImplementation - Dynamic Service Discovery', () =>
   });
 
   // Not every SERVICE_CONFIG entry is dispatcher-owned: `search` (REST layer),
-  // `workflow`, `graphql` and the queue/job/cache families have their routes
-  // mounted by the plugin that registers the service, so `handlerReady` there
-  // says nothing about whether THAT route is mounted and the advertisement
-  // stays presence-gated. Suppressing it would be a guess, not honesty.
+  // `workflow` and `graphql` have their routes mounted by the plugin that
+  // registers the service, so `handlerReady` there says nothing about whether
+  // THAT route is mounted and the advertisement stays presence-gated.
+  // Suppressing it would be a guess, not honesty. (cache/queue/job used to be
+  // named here on the same theory, but nothing mounts routes for them at all —
+  // they are route-less kernel-internal slots since #4318.)
   it('should leave non-dispatcher-owned routes presence-gated', async () => {
     const mockServices = new Map<string, any>();
     mockServices.set('search', { __serviceInfo: { status: 'stub', message: 'dev fake' } });
@@ -348,6 +350,49 @@ describe('ObjectStackProtocolImplementation - Dynamic Service Discovery', () =>
     expect(discovery.services.search.route).toBe('/api/v1/search');
   });
 
+  // ── Kernel-internal slots advertise no route, ever (#4318) ────────────────
+  // SERVICE_CONFIG used to declare /api/v1/cache, /api/v1/queue and
+  // /api/v1/jobs — three paths that existed nowhere else in the repository:
+  // no dispatcher domain, no adapter mount, no plugin registration, and the
+  // shipped providers (service-cache/-queue/-job) are in-process contracts
+  // that will never mount one. Every default boot therefore advertised a
+  // route inside the same ServiceInfo whose `handlerReady: false` said the
+  // opposite. The slots are route-less now, like realtime — but unlike
+  // realtime an unmarked real implementation stays `available`, because the
+  // slot's contract is in-process and "no HTTP surface" is not reduced
+  // capability for it.
+  it('reports an unmarked cache/queue/job occupant available with no route and handlerReady false (#4318)', async () => {
+    const mockServices = new Map<string, any>();
+    for (const slot of ['cache', 'queue', 'job']) mockServices.set(slot, { /* real, unmarked */ });
+
+    protocol = new ObjectStackProtocolImplementation(engine, () => mockServices);
+    const discovery = await protocol.getDiscovery();
+
+    for (const slot of ['cache', 'queue', 'job']) {
+      const reported = discovery.services[slot];
+      expect(reported.enabled, `${slot}.enabled`).toBe(true);
+      expect(reported.status, `${slot}.status`).toBe('available');
+      expect(reported.handlerReady, `${slot}.handlerReady`).toBe(false);
+      expect(reported.route, `${slot}.route`).toBeUndefined();
+      expect(reported.message, `${slot}.message`).toContain('no HTTP surface');
+    }
+  });
+
+  it('never advertises a route for a cache/queue/job fallback either (#4318)', async () => {
+    for (const slot of ['cache', 'queue', 'job']) {
+      const mockServices = new Map<string, any>();
+      mockServices.set(slot, CORE_FALLBACK_FACTORIES[slot]());
+
+      protocol = new ObjectStackProtocolImplementation(engine, () => mockServices);
+      const reported = (await protocol.getDiscovery()).services[slot];
+
+      // Self-description wins for status/message (the class-wide #3898 gate
+      // pins `degraded`); the route stays gone and handlerReady stays false.
+      expect(reported.route, `${slot}.route`).toBeUndefined();
+      expect(reported.handlerReady, `${slot}.handlerReady`).toBe(false);
+    }
+  });
+
   it('should map file-storage service to storage route', async () => {
     const mockServices = new Map<string, any>();
     mockServices.set('file-storage', {});

packages/runtime/src/http-dispatcher.test.ts

@@ -2668,8 +2668,9 @@ describe('HttpDispatcher', () => {
         // `available`. Table-driven so the next fallback added to the table is
         // gated the day it lands; this class of hole recurs with every new
         // fallback. cache/queue/job had no per-slot pin before this — dropping
-        // their `svcAvailable(…, svc)` third argument, the exact #4130
-        // regression shape, was test-invisible.
+        // their occupant argument (`svcAvailable(…, svc)` then,
+        // `svcInProcess(slot, svc)` since #4318), the exact #4130 regression
+        // shape, was test-invisible.
 
         it('reports every CORE_FALLBACK_FACTORIES product as degraded, never available (#3898)', async () => {
             const { CORE_FALLBACK_FACTORIES } = await import('@objectstack/core');
@@ -2688,6 +2689,56 @@ describe('HttpDispatcher', () => {
                 expect(reported.message, `services.${slot}.message`).toBeTruthy();
             }
         });
+
+        // ── Kernel-internal slots (#4318): no route, handlerReady is the fact ──
+        //
+        // service-cache/-queue/-job mount no HTTP routes — the slots are
+        // in-process contracts, so no route is ever advertised for them and
+        // `handlerReady` is `false` as a fact, not a proxy. `svcAvailable`
+        // used to claim `handlerReady: true` for an unmarked occupant here — a
+        // handler that does not exist. The status stays `available` for an
+        // unmarked real implementation: "no HTTP surface" is not reduced
+        // capability for an in-process contract (contrast realtime).
+        it('reports unmarked cache/queue/job occupants available with no route and handlerReady false (#4318)', async () => {
+            for (const slot of ['cache', 'queue', 'job']) {
+                const svc = { /* real, unmarked */ };
+                (kernel as any).getService = vi.fn().mockImplementation((n: string) => (n === slot ? svc : null));
+                (kernel as any).services = new Map([[slot, svc]]);
+
+                const info = await dispatcher.getDiscoveryInfo('/api/v1');
+                const reported = (info.services as Record<string, any>)[slot];
+                expect(reported.enabled, `services.${slot}.enabled`).toBe(true);
+                expect(reported.status, `services.${slot}.status`).toBe('available');
+                expect(reported.handlerReady, `services.${slot}.handlerReady`).toBe(false);
+                expect(reported.route, `services.${slot}.route`).toBeUndefined();
+                expect(reported.message, `services.${slot}.message`).toContain('no HTTP surface');
+            }
+        });
+
+        it('answers the cache/queue/job slots identically to the metadata-protocol builder (#4318)', async () => {
+            const { ObjectStackProtocolImplementation } = await import('@objectstack/metadata-protocol');
+            const { CORE_FALLBACK_FACTORIES } = await import('@objectstack/core');
+
+            for (const slot of ['cache', 'queue', 'job']) {
+                // Both shapes an occupant can take: a real (unmarked) service
+                // and the kernel's self-describing in-memory fallback.
+                for (const svc of [{}, CORE_FALLBACK_FACTORIES[slot]()]) {
+                    (kernel as any).getService = vi.fn().mockImplementation((n: string) => (n === slot ? svc : null));
+                    (kernel as any).services = new Map([[slot, svc]]);
+
+                    const fromDispatcher = ((await dispatcher.getDiscoveryInfo('/api/v1')).services as Record<string, any>)[slot];
+                    const fromProtocol = (await new ObjectStackProtocolImplementation(
+                        mockObjectQL as any,
+                        () => new Map<string, any>([[slot, svc]]),
+                    ).getDiscovery()).services[slot];
+
+                    expect(fromDispatcher.status, `${slot}.status`).toBe(fromProtocol.status);
+                    expect(fromDispatcher.handlerReady, `${slot}.handlerReady`).toBe(fromProtocol.handlerReady);
+                    expect(fromDispatcher.message, `${slot}.message`).toBe(fromProtocol.message);
+                    expect(fromDispatcher.route, `${slot}.route`).toBe(fromProtocol.route);
+                }
+            }
+        });
     });
 
     // ═══════════════════════════════════════════════════════════════

packages/runtime/src/http-dispatcher.ts

@@ -5,7 +5,7 @@ import {
 } from '@objectstack/core';
 import { isMcpServerEnabled, looksLikeInternalErrorLeak, INTERNAL_ERROR_MESSAGE } from '@objectstack/types';
 import { measureServerTiming, allowPerfDisclosure, isPerfDisclosurePrincipal } from '@objectstack/observability';
-import { CoreServiceName, serviceUnavailableMessage } from '@objectstack/spec/system';
+import { CoreServiceName, serviceUnavailableMessage, inProcessServiceMessage } from '@objectstack/spec/system';
 import type { IDataEngine, IObjectQLEngine } from '@objectstack/spec/contracts';
 import { readServiceSelfInfo, DispatcherErrorCode } from '@objectstack/spec/api';
 import { apiErrorResponse } from './error-envelope.js';
@@ -1037,6 +1037,23 @@ export class HttpDispatcher {
             enabled: false, status: 'unavailable' as const, handlerReady: false,
             message: serviceUnavailableMessage(name),
         });
+        // [#4318] Kernel-internal slots (cache/queue/job): their providers
+        // mount no HTTP routes, so no route is advertised and `handlerReady`
+        // is `false` as a fact, not a proxy — `svcAvailable` would claim a
+        // handler that does not exist. An unmarked occupant stays `available`:
+        // the slot's contract is in-process, so "no HTTP surface" is not
+        // reduced capability (contrast `realtime` below, whose advertised
+        // capability IS the missing surface). Message written once in
+        // `@objectstack/spec/system` so both discovery builders agree.
+        const svcInProcess = (name: string, svc: unknown) => {
+            const self = svc ? readServiceSelfInfo(svc) : undefined;
+            return {
+                enabled: true,
+                status: self?.status ?? ('available' as const),
+                handlerReady: false,
+                message: self?.message ?? inProcessServiceMessage(name),
+            };
+        };
 
         // Self-description of the registered realtime service, if any (D12).
         const realtimeSelf = realtimeSvc ? readServiceSelfInfo(realtimeSvc) : undefined;
@@ -1148,9 +1165,9 @@ export class HttpDispatcher {
                 // "install a plugin" would say strictly less.
                 automation:     automationRegistered ? svcAvailable(routes.automation, undefined, automationSvc) : svcUnavailable('automation'),
                 analytics:      analyticsRegistered ? svcAvailable(routes.analytics, undefined, analyticsSvc) : svcUnavailable('analytics'),
-                cache:          hasCache ? svcAvailable(undefined, undefined, cacheSvc) : svcUnavailable('cache'),
-                queue:          hasQueue ? svcAvailable(undefined, undefined, queueSvc) : svcUnavailable('queue'),
-                job:            hasJob ? svcAvailable(undefined, undefined, jobSvc) : svcUnavailable('job'),
+                cache:          hasCache ? svcInProcess('cache', cacheSvc) : svcUnavailable('cache'),
+                queue:          hasQueue ? svcInProcess('queue', queueSvc) : svcUnavailable('queue'),
+                job:            hasJob ? svcInProcess('job', jobSvc) : svcUnavailable('job'),
                 // [#4093] Reported from what serves it, like the route above:
                 // `/ui` is a dispatcher domain answered by the `protocol`
                 // service, so its self-description (none today — MetadataPlugin