fix(objectql,cli): the integrity audit stops when the engine does (#4747)

Every `os migrate` subcommand ended a SUCCESSFUL run with two
`ERROR Find operation failed` lines and a #4551 report naming
`sys_metadata` / `sys_view_definition` as `unreadableObjects`. That
bucket exists to separate "I could not check" from "I checked and it
was fine"; non-empty on every healthy run, it separated nothing.

Two silent no-ops stacked up:

- `ObjectQLPlugin` put its teardown in `stop()`, but the kernel's Plugin
  contract is `init`/`start`/`destroy` — `stop()` is never called by
  anyone, so the ADR-0057 sweep timers were never disarmed on any host.
- `bootSchemaStack().shutdown()` called `(runtime as any).stop?.()` and
  `Runtime` has no `stop`. The optional call made "no teardown" look
  exactly like "teardown performed": the kernel stayed running with
  every timer armed while the command closed its driver.

60s after boot the sweep woke inside the still-alive process and read a
pool its own host had already disconnected.

- ObjectQLPlugin: `stop` -> `destroy`, the hook the kernel actually calls.
- bootSchemaStack: tear down via `kernel.shutdown()` — the same path
  `os serve` takes on SIGTERM, so one-shot and server exit alike.
- LifecycleService.stop(): raise an abort bit the in-flight sweep holds,
  not just clear timers; refuse to start a sweep once stopped.
- The audit takes that bit as `signal` and stops issuing reads. A read
  that fails *because* the run was called off is dropped rather than
  filed — it is not evidence about the datasource. The new
  `DanglingReferenceReport.aborted` keeps the incompleteness loud
  without spending the finding bucket on it.

A genuine datasource fault still lands in `unreadableObjects`, and the
audit still runs on CLI hosts: this is a liveness boundary, not the
"one-shot commands skip the audit" switch the issue rejected.

Fixes #4747

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Br2xsJsczFsTR9bvbh2Ny

Author: claude

.changeset/hungry-donkeys-repeat.md

@@ -0,0 +1,28 @@
+---
+'@objectstack/objectql': patch
+'@objectstack/cli': patch
+---
+
+修复:每个 `os migrate` 子命令关停后,#4551 悬空引用巡检都会把 `sys_metadata` / `sys_view_definition` 报成 `unreadableObjects`(#4747)
+
+一条**成功**的命令过去会在返回 JSON 之后打出两行 `ERROR Find operation failed` 和一份
+`unreadableObjects` 非空的巡检报告 —— 对抓 ERROR 的 CI 流水线是直接误报源,更要命的是它把
+`unreadableObjects` 变成了恒为真的告警:那个桶存在的意义正是区分「我没能检查」和「我检查了,
+没问题」,一个每次健康运行都非空的桶不再携带任何信息。
+
+两处静默空转叠出了这个结果:
+
+- `ObjectQLPlugin` 的关停逻辑写在 `stop()` 里,而内核的插件契约是 `init`/`start`/`destroy` ——
+  `stop()` 从来没有被任何人调用过,ADR-0057 巡检定时器因此在任何宿主上都不会被解除。改为
+  `destroy()`(与 `DefaultDatasourcePlugin` 一致)。
+- `bootSchemaStack().shutdown()` 调的是 `(runtime as any).stop?.()`,而 `Runtime` 根本没有
+  `stop` —— 可选调用把「没有关停」伪装成了「关停过了」。改为走内核自己的 `kernel.shutdown()`,
+  与 `os serve` 收到 SIGTERM 时同一条路径。
+
+同时 `LifecycleService.stop()` 不再只是清定时器:它还会把「引擎正在拆」这一位交给正在飞行中的
+sweep,巡检据此在读之前停手。因关停而失败的读**不再进入** `unreadableObjects` —— 那不是关于
+数据源的证据;报告改用新增的 `DanglingReferenceReport.aborted` 记录「这次没跑完」,所以不完整
+依然是响的,只是不再占用发现桶。
+
+**真正读不出来的对象(数据源故障)照旧进 `unreadableObjects`**,巡检在 CLI 场景也照旧运行 ——
+这里没有「一次性命令不跑巡检」的开关,只有「引擎活着才读」的生命周期边界。

packages/cli/src/utils/schema-migrate.teardown.integration.test.ts

@@ -0,0 +1,120 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * [#4747] A one-shot CLI stack tears down through the KERNEL, and the ADR-0057
+ * sweep stops with it.
+ *
+ * The bug this pins, end to end on the real `bootSchemaStack` path:
+ *
+ *   $ os migrate recorded-by --json      # exits 0, prints valid JSON …
+ *   ERROR Find operation failed {"object":"sys_metadata", …}
+ *   WARN  [integrity] dangling-reference audit could not list an object …
+ *   WARN  [integrity] stored references that resolve to nothing (#4551)
+ *         {"unreadableObjects":["sys_metadata","sys_view_definition"], …}
+ *
+ * Two silent no-ops stacked up to produce it. `shutdown()` called
+ * `(runtime as any).stop?.()` and `Runtime` has no `stop`; the one thing that
+ * would have disarmed the sweep was `ObjectQLPlugin.stop()`, a hook the kernel
+ * never calls (the Plugin contract is `init`/`start`/`destroy`). So the kernel
+ * stayed "running" with every timer armed while the command closed its driver,
+ * and 60s later the sweep read a pool that was gone — filing both objects as
+ * `unreadableObjects` on a completely healthy run.
+ *
+ * Two assertions matter here and they pull in opposite directions on purpose:
+ *
+ *  - while the engine is LIVE, a CLI-booted stack really does audit (this is
+ *    not #4747's rejected option C — "one-shot commands skip the audit" would
+ *    make it permanently blind exactly where an operator has no other tool);
+ *  - once the stack is torn down, the sweep issues no reads at all.
+ */
+
+import { describe, it, expect, beforeAll, afterAll } from 'vitest';
+import { mkdtempSync, writeFileSync, mkdirSync, rmSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { bootSchemaStack } from './schema-migrate.js';
+
+interface LifecycleServiceLike {
+  stopped: boolean;
+  sweep(): Promise<{ danglingReferences?: { unreadableObjects: string[]; aborted?: boolean } }>;
+}
+
+describe('[#4747] bootSchemaStack teardown disarms the ADR-0057 sweep', () => {
+  let dir: string;
+  let dbFile: string;
+  const savedEnv: Record<string, string | undefined> = {};
+
+  beforeAll(() => {
+    dir = mkdtempSync(join(tmpdir(), 'os-teardown-'));
+    mkdirSync(join(dir, 'dist'), { recursive: true });
+    mkdirSync(join(dir, 'data'), { recursive: true });
+    dbFile = join(dir, 'data', 'app.db');
+    writeFileSync(
+      join(dir, 'dist', 'objectstack.json'),
+      JSON.stringify({
+        id: 'teardown_smoke',
+        name: 'Teardown Smoke',
+        objects: [
+          {
+            name: 'td_note',
+            fields: {
+              title: { type: 'text', required: true },
+              // A real reference field, so the audit has something to read
+              // rather than skipping the object outright.
+              owner: { type: 'lookup', reference: 'td_person' },
+            },
+          },
+          { name: 'td_person', fields: { name: { type: 'text' } } },
+        ],
+      }),
+    );
+    savedEnv.OS_ARTIFACT_PATH = process.env.OS_ARTIFACT_PATH;
+    process.env.OS_ARTIFACT_PATH = join(dir, 'dist', 'objectstack.json');
+  });
+
+  afterAll(() => {
+    process.env.OS_ARTIFACT_PATH = savedEnv.OS_ARTIFACT_PATH;
+    try { rmSync(dir, { recursive: true, force: true }); } catch { /* ignore */ }
+  });
+
+  it('audits while the engine is live, and reads nothing once the stack is down', async () => {
+    const stack = await bootSchemaStack({ databaseUrl: `file:${dbFile}`, projectRoot: dir });
+    // Resolved BEFORE teardown — the point is what this same instance does
+    // afterwards, and service resolution post-shutdown is not the subject.
+    const lifecycle = stack.kernel.getService('lifecycle') as LifecycleServiceLike;
+    expect(lifecycle).toBeTruthy();
+
+    // ── While the engine is live: the audit runs for real ─────────────────
+    // Not "the CLI skips the audit" — it reads, and reports a clean, COMPLETE
+    // run. An empty `unreadableObjects` here is a fact about the database,
+    // which is precisely what it stopped being before this fix.
+    expect(lifecycle.stopped).toBe(false);
+    const live = await lifecycle.sweep();
+    expect(live.danglingReferences).toBeDefined();
+    expect(live.danglingReferences!.unreadableObjects).toEqual([]);
+    expect(live.danglingReferences!.aborted).toBe(false);
+
+    // ── Teardown ──────────────────────────────────────────────────────────
+    await stack.shutdown();
+
+    // The kernel really shut down. `(runtime as any).stop?.()` left it running
+    // and every plugin undestroyed, which is how a missing teardown managed to
+    // look exactly like a performed one.
+    expect(stack.kernel.isRunning()).toBe(false);
+    expect(lifecycle.stopped).toBe(true);
+
+    // The sweep the timer would have fired 60s later: no engine reads, so no
+    // `ERROR Find operation failed` on a successful command, and no object
+    // filed as unreadable for the crime of being asked after closing time.
+    const afterDown = await lifecycle.sweep();
+    expect(afterDown.danglingReferences).toBeUndefined();
+
+    // …and this is not vacuous: the pool really is closed, so a read issued
+    // here really would fail. The silence above is the fix, not an absence of
+    // anything to read.
+    const engine = stack.kernel.getService('objectql') as {
+      find(object: string, options: Record<string, unknown>): Promise<unknown[]>;
+    };
+    await expect(engine.find('td_note', { limit: 1, context: { isSystem: true } })).rejects.toThrow();
+  }, 120_000);
+});

packages/cli/src/utils/schema-migrate.ts

@@ -209,8 +209,28 @@ export async function bootSchemaStack(
     flushSchemaDdl: async () => (defer && driver?.flushDeferredSchemaDdl
       ? await driver.flushDeferredSchemaDdl()
       : []),
+    /**
+     * Tear the one-shot stack down through the kernel's own teardown — the
+     * same `kernel.shutdown()` `os serve` runs on SIGTERM, so a one-shot
+     * command and a server take ONE path out (#4747).
+     *
+     * It used to call `(runtime as any).stop?.()`. `Runtime` has no `stop` —
+     * the optional-call swallowed that fact, so every `os migrate` subcommand
+     * closed its driver while leaving the kernel fully "running": no plugin
+     * ever got `destroy()`, and the ADR-0057 lifecycle sweep stayed armed. 60s
+     * later it woke inside the still-alive process and read through the pool
+     * this line had already closed, which is why a successful command ended in
+     * `ERROR Find operation failed` and a #4551 report naming `sys_metadata` /
+     * `sys_view_definition` as unreadable. A cast plus `?.` is how a missing
+     * teardown looks exactly like a performed one; there is no version of that
+     * call that could ever have worked.
+     *
+     * The explicit `disconnect()` stays as the backstop for a driver this
+     * kernel did not register through `DefaultDatasourcePlugin` (whose own
+     * `destroy()` closes the ones it owns); a second disconnect is a no-op.
+     */
     shutdown: async () => {
-      try { await (runtime as any).stop?.(); } catch { /* ignore */ }
+      try { await kernel.shutdown(); } catch { /* teardown is best-effort */ }
       try { await driver?.disconnect?.(); } catch { /* ignore */ }
     },
   };

packages/objectql/src/index.ts

@@ -138,6 +138,9 @@ export type {
   DanglingReferenceAuditOptions,
   DanglingReferenceAuditPort,
   AuditableObject,
+  // [#4747] The teardown bit the audit reads — exported alongside the options
+  // type that names it, so a caller can spell the parameter it passes.
+  AuditAbortSignal,
 } from './integrity/dangling-reference-audit.js';
 
 // Export MetadataFacade

packages/objectql/src/integrity/dangling-reference-audit.test.ts

@@ -372,3 +372,162 @@ describe('[#4551] dangling stored references are reported, never rewritten', ()
     ]);
   });
 });
+
+/**
+ * [#4747] "The datasource refused" and "nobody asked" are different facts.
+ *
+ * The audit reads through a live engine, and the engine outlives it only until
+ * the host closes the pool. Before this, every `os migrate` invocation ended
+ * with `unreadableObjects: ['sys_metadata', 'sys_view_definition']` — the sweep
+ * fired 60s after boot, inside a process whose datasource had already been
+ * disconnected. An `unreadableObjects` that is non-empty on every healthy run
+ * is not a cautious report; it is a broken alarm, and it costs exactly the
+ * signal #4551 built the bucket for.
+ *
+ * The pair of tests that opens this block is the whole point: BEFORE the fix
+ * the two runs were indistinguishable, and after it they must never again be
+ * confusable. Deleting the abort handling makes the second one fail; deleting
+ * the `unreadableObjects` push makes the first one fail.
+ */
+describe('[#4747] a run that was called off is not a finding about the data', () => {
+  /** A port whose listing always fails — the two runs below differ ONLY in why. */
+  const unreadablePort = () => makePort({
+    objects: [binding],
+    rows: {},
+    unreadable: new Set(['sys_position_permission_set']),
+  });
+
+  it('a REAL datasource fault still lands in `unreadableObjects`, loudly', async () => {
+    // The case the bucket exists for, and the one that must survive the fix:
+    // the audit tried, the store would not answer, and the report must not read
+    // as a clean bill of health on an object nothing could look at.
+    const port = unreadablePort();
+
+    const out = await auditDanglingReferences(port);
+
+    expect(out.unreadableObjects).toEqual(['sys_position_permission_set']);
+    expect(out.aborted).toBe(false);
+    expect(port.warnings.map((w) => w[0])).toContain(
+      '[integrity] dangling-reference audit could not list an object',
+    );
+  });
+
+  it('the SAME failure, raced by a teardown, is dropped instead of filed', async () => {
+    // Identical throw from identical code — the only difference is that the
+    // caller had called the run off, which is what closing a connection pool on
+    // purpose looks like from in here. It is not evidence about the datasource,
+    // so it must not spend the bucket that only holds evidence.
+    const signal = { aborted: false };
+    const port = unreadablePort();
+    const failing = port.find.bind(port);
+    port.find = async (o, opts) => {
+      signal.aborted = true;          // the pool closes mid-query
+      return failing(o, opts);        // …and the query fails because of it
+    };
+
+    const out = await auditDanglingReferences(port, { signal });
+
+    expect(out.unreadableObjects).toEqual([]);
+    // Not silence either: the run says it did not finish, so nothing can read
+    // `dangling: []` as "everything is fine".
+    expect(out.aborted).toBe(true);
+    expect(port.warnings).toEqual([]);
+  });
+
+  it('called off BEFORE a read: no query is issued at all', async () => {
+    // This is what removes the `ERROR Find operation failed` line from a
+    // SUCCESSFUL command — the engine never gets asked, so it never logs.
+    const reads: string[] = [];
+    const port = makePort({
+      objects: [binding, task],
+      rows: { sys_position_permission_set: [{ id: 'ppr_1', permission_set_id: 'ps_gone' }] },
+    });
+    const findSpy = port.find.bind(port);
+    port.find = async (o, opts) => { reads.push(o); return findSpy(o, opts); };
+
+    const out = await auditDanglingReferences(port, { signal: { aborted: true } });
+
+    expect(reads).toEqual([]);
+    expect(port.probes).toEqual([]);
+    expect(out.aborted).toBe(true);
+    expect(out.unreadableObjects).toEqual([]);
+  });
+
+  it('called off MID-run keeps what it already proved and stops there', async () => {
+    // A finding is a finding whenever it was made; only the SILENCE about the
+    // rest of the run becomes unreliable, which is what `aborted` records. So
+    // an abort must not discard the first object's verdict — and must not turn
+    // the second object into a report about the datasource.
+    const signal = { aborted: false };
+    const reads: string[] = [];
+    const port = makePort({
+      objects: [binding, task],   // binding is scanned first (security surface)
+      rows: {
+        sys_position_permission_set: [{ id: 'ppr_1', permission_set_id: 'ps_gone' }],
+        showcase_task: [{ id: 't1', title: 'T', project: 'proj_gone' }],
+      },
+      unreadable: new Set(['showcase_task']),
+    });
+    const findSpy = port.find.bind(port);
+    port.find = async (o, opts) => {
+      reads.push(o);
+      // Teardown lands as the second listing is issued — so that listing fails
+      // BECAUSE the run was called off, which is the production race exactly.
+      if (o === 'showcase_task') signal.aborted = true;
+      return findSpy(o, opts);
+    };
+
+    const out = await auditDanglingReferences(port, { signal });
+
+    expect(reads).toEqual(['sys_position_permission_set', 'showcase_task']);
+    expect(out.dangling).toHaveLength(1);
+    expect(out.dangling[0]).toMatchObject({ recordId: 'ppr_1', value: 'ps_gone' });
+    expect(out.aborted).toBe(true);
+    // The object the teardown cut short is NOT reported as unreadable.
+    expect(out.unreadableObjects).toEqual([]);
+    // The real finding is still reported, and the summary line carries the
+    // incompleteness so the log cannot read as a finished run either.
+    const summary = port.warnings.find((w) => w[0].includes('#4551'));
+    expect(summary).toBeDefined();
+    expect((summary![1] as any).aborted).toBe(true);
+  });
+
+  it('a probe that fails under a teardown is not `undetermined` either', async () => {
+    // `undetermined` means "the probe RAN and could not tell". A withdrawn
+    // question did not run, so counting it there would repeat the same category
+    // error one bucket over.
+    const signal = { aborted: false };
+    const port = makePort({
+      objects: [binding],
+      rows: { sys_position_permission_set: [{ id: 'ppr_1', permission_set_id: 'ps_x' }] },
+      throwingTargets: new Set(['sys_permission_set']),
+    });
+    const probeSpy = port.probe.bind(port);
+    port.probe = async (target, id) => {
+      signal.aborted = true;
+      return probeSpy(target, id);
+    };
+
+    const out = await auditDanglingReferences(port, { signal });
+
+    expect(out.undetermined).toBe(0);
+    expect(out.dangling).toEqual([]);
+    expect(out.aborted).toBe(true);
+  });
+
+  it('a run nobody called off says so explicitly — `aborted: false`, not absent', async () => {
+    // The flag is a positive statement about completeness, so a consumer never
+    // has to guess whether `undefined` meant "finished" or "old report shape".
+    const port = makePort({
+      objects: [binding],
+      rows: { sys_position_permission_set: [{ id: 'ppr_1', permission_set_id: 'ps_real' }] },
+      existing: new Set(['sys_permission_set ps_real']),
+    });
+
+    const out = await auditDanglingReferences(port, { signal: { aborted: false } });
+    expect(out.aborted).toBe(false);
+
+    const noSignal = await auditDanglingReferences(port);
+    expect(noSignal.aborted).toBe(false);
+  });
+});