fix(runtime,cloud-connection): multi-tenant seed replay covers every source, not just the first (#3453) (#3478)

In multi-tenant mode a new org replays the kernel's `seed-datasets` list on the
`sys_organization` insert, so that list must hold the union of every seed source
(every config app + every marketplace package). Two framework traps — the same
pair #3444 fixed for seed-summary — shrank it to just the first source:

- the standard PluginContext has no `.kernel` handle, so
  `(ctx as any).kernel?.getService('seed-datasets')` always read undefined and
  each source clobbered the list with only its own datasets; and
- `registerService` throws on a duplicate name, so the second source's
  re-register was swallowed and its datasets (and, for a config app, its
  replayer) were silently lost.

`seed-datasets` is now a single shared array, registered once and mutated in
place by every source via a new `mergeSeedDatasets` helper that reads through the
context's own resolver first. AppPlugin's per-org replayer reads that live list
at invoke time — not a captured snapshot — and is itself registered once and
reused by later config apps, so a new org replays the full union.

Seam-level unit tests only (accumulation across app + marketplace sources; the
replayer reads the live union); true multi-tenant e2e needs the enterprise
`@objectstack/organizations` plugin, which lives in the cloud repo.

Co-authored-by: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

Author: os-zhuang

.changeset/seed-datasets-multitenant-replay-union.md

@@ -0,0 +1,34 @@
+---
+"@objectstack/runtime": patch
+"@objectstack/cloud-connection": patch
+---
+
+fix(runtime,cloud-connection): multi-tenant seed replay covers every source, not just the first (#3453)
+
+In multi-tenant deployments (enterprise `@objectstack/organizations`) a brand-new org
+gets its own private copy of demo data by replaying the kernel's `seed-datasets` list
+on the `sys_organization` insert. That list is meant to hold the union of every seed
+source — every config-declared app AND every marketplace package — but two framework
+traps (the same pair #3444 fixed for seed-summary) shrank it to just the first source:
+
+- The standard `PluginContext` exposes `getService`/`registerService` but has NO
+  `.kernel` handle, so `(ctx as any).kernel?.getService('seed-datasets')` always read
+  `undefined`. Each source then saw "nothing registered" and overwrote the list with
+  only its own datasets instead of extending it.
+- `registerService` throws on a duplicate name, so the second source's re-register was
+  swallowed by the surrounding try/catch — its datasets (and, for a config app, its
+  replayer) silently lost.
+
+Net effect: with two config apps, or a config app plus marketplace packages, a new org
+replayed only the first app's seeds.
+
+The fix mirrors #3444's seed-summary hardening: `seed-datasets` is now a single shared
+array, registered once and mutated in place by every source through a new
+`mergeSeedDatasets` helper that reads via the context's own resolver first. AppPlugin's
+per-org replayer reads that live list at invoke time instead of a captured snapshot, so
+it replays the full union — including datasets merged after its closure was built — and
+the replayer itself is registered once and reused by later config apps.
+
+Covered by seam-level unit tests (accumulation across app + marketplace sources; the
+replayer reads the live union). True multi-tenant end-to-end coverage requires the
+enterprise `@objectstack/organizations` plugin, which lives in the cloud repo.

packages/cloud-connection/src/marketplace-install-local-plugin.ts

@@ -320,6 +320,57 @@ export class MarketplaceInstallLocalPlugin implements Plugin {
         } catch { /* banner summary is best-effort — never break the heal */ }
     };
 
+    /**
+     * Merge `datasets` onto the SHARED `seed-datasets` service via the runtime's
+     * register-once-then-mutate helper (#3453), so THIS package's seeds accumulate
+     * alongside every config app's and every other package's rather than clobbering
+     * them — the per-org replayer (AppPlugin) replays the whole union on the next
+     * `sys_organization` insert. Resolved lazily through `@objectstack/runtime` and
+     * guarded exactly like {@link recordSeedSummary}: a runtime that predates the
+     * helper — or a test that mocks the module without it — falls back to an
+     * equivalent inline merge. Returns the post-merge total for the log line.
+     */
+    private mergeSeedDatasetsIntoKernel = async (ctx: PluginContext, datasets: any[]): Promise<number> => {
+        try {
+            const mod: any = await import('@objectstack/runtime');
+            if (typeof mod?.mergeSeedDatasets === 'function') {
+                const list = mod.mergeSeedDatasets(ctx, datasets);
+                return Array.isArray(list) ? list.length : datasets.length;
+            }
+        } catch { /* fall through to the inline merge below */ }
+        return this.mergeSeedDatasetsInline(ctx, datasets);
+    };
+
+    /**
+     * Fallback for {@link mergeSeedDatasetsIntoKernel} when the runtime helper is
+     * unavailable (older build / mocked module). Same register-once-then-mutate:
+     * read the live list through the context's OWN resolver first — a standard
+     * PluginContext has no `.kernel`, which is precisely why the old
+     * `(ctx as any).kernel` read clobbered instead of accumulating — push in place,
+     * and register only when the service does not yet exist so a second source
+     * cannot trip the duplicate-register throw. Returns the post-merge total.
+     */
+    private mergeSeedDatasetsInline = (ctx: PluginContext, datasets: any[]): number => {
+        const c = ctx as any;
+        const read = (): any[] | undefined => {
+            if (typeof c?.getService === 'function') {
+                try { const v = c.getService('seed-datasets'); if (Array.isArray(v)) return v; } catch { /* absent */ }
+            }
+            if (typeof c?.kernel?.getService === 'function') {
+                try { const v = c.kernel.getService('seed-datasets'); if (Array.isArray(v)) return v; } catch { /* absent */ }
+            }
+            return undefined;
+        };
+        const current = read();
+        const list: any[] = Array.isArray(current) ? current : [];
+        list.push(...datasets);
+        if (!Array.isArray(current)) {
+            if (typeof c?.kernel?.registerService === 'function') c.kernel.registerService('seed-datasets', list);
+            else if (typeof c?.registerService === 'function') c.registerService('seed-datasets', list);
+        }
+        return list.length;
+    };
+
     private handleInstall = async (c: any, ctx: PluginContext): Promise<Response> => {
         const userId = await this.requireAuthenticatedUser(c, ctx);
         if (!userId) {
@@ -905,16 +956,8 @@ export class MarketplaceInstallLocalPlugin implements Plugin {
 
         if (datasets.length > 0) {
             try {
-                const kernel: any = (ctx as any).kernel;
-                let existing: any[] = [];
-                try {
-                    const v = kernel?.getService?.('seed-datasets');
-                    if (Array.isArray(v)) existing = v;
-                } catch { /* unset */ }
-                const merged = [...existing, ...datasets];
-                if (kernel?.registerService) kernel.registerService('seed-datasets', merged);
-                else (ctx as any).registerService?.('seed-datasets', merged);
-                ctx.logger?.info?.(`[MarketplaceInstallLocal] merged ${datasets.length} seed dataset(s) into kernel (total: ${merged.length})`);
+                const total = await this.mergeSeedDatasetsIntoKernel(ctx, datasets);
+                ctx.logger?.info?.(`[MarketplaceInstallLocal] merged ${datasets.length} seed dataset(s) into kernel (total: ${total})`);
             } catch (err: any) {
                 ctx.logger?.warn?.(`[MarketplaceInstallLocal] failed to merge seed-datasets: ${err?.message ?? err}`);
             }

packages/runtime/src/app-plugin.ts

@@ -5,6 +5,7 @@ import { assertProtocolCompat } from '@objectstack/metadata-core';
 import { resolveMultiOrgEnabled } from '@objectstack/types';
 import { SeedLoaderService } from './seed-loader.js';
 import { recordSeedOutcome } from './seed-summary.js';
+import { mergeSeedDatasets, readSeedDatasets, registerSeedReplayerOnce } from './seed-datasets.js';
 import { loadDisabledPackageIds } from './package-state-store.js';
 import type { IMetadataService, II18nService } from '@objectstack/spec/contracts';
 import { QuickJSScriptRunner } from './sandbox/quickjs-runner.js';
@@ -750,31 +751,29 @@ export class AppPlugin implements Plugin {
              // captures the SeedLoaderService closure and exposes a
              // narrow `(orgId) => Promise<summary>` surface.
              try {
-                 const kernel: any = (ctx as any).kernel;
-                 const existing = (() => {
-                     try { return kernel?.getService?.('seed-datasets'); } catch { return undefined; }
-                 })();
-                 const merged = Array.isArray(existing)
-                     ? [...existing, ...normalizedDatasets]
-                     : normalizedDatasets;
-                 const registerSvc = (name: string, value: any) => {
-                     if (kernel?.registerService) kernel.registerService(name, value);
-                     else if (typeof (ctx as any).registerService === 'function') (ctx as any).registerService(name, value);
-                 };
-                 registerSvc('seed-datasets', merged);
+                 // #3453: append this app's datasets onto the SHARED `seed-datasets`
+                 // array (register-once-then-mutate). Reading through the context's
+                 // own resolver — NOT the non-existent `(ctx as any).kernel`, which was
+                 // always undefined — means a SECOND config app (or a marketplace
+                 // install) extends the list instead of clobbering it or tripping the
+                 // duplicate-register throw. The per-org replayer below re-reads this
+                 // live list on every call, so a new org replays the full union.
+                 const sharedDatasets = mergeSeedDatasets(ctx, normalizedDatasets);
 
-                 const metadataNow = ctx.getService('metadata') as IMetadataService | undefined;
                  const loggerRef = ctx.logger;
                  const replayer = async (organizationId: string) => {
                      if (!organizationId) return { inserted: 0, updated: 0, skipped: 0, errors: [] as any[] };
-                     const md = metadataNow ?? (ctx.getService('metadata') as IMetadataService | undefined);
+                     const md = ctx.getService('metadata') as IMetadataService | undefined;
                      if (!md) {
                          loggerRef.warn('[seed-replayer] metadata service unavailable');
                          return { inserted: 0, updated: 0, skipped: 0, errors: [] as any[] };
                      }
-                     const datasetsNow = (() => {
-                         try { return kernel?.getService?.('seed-datasets'); } catch { return merged; }
-                     })() ?? merged;
+                     // Read the LIVE shared list on every replay — NOT the
+                     // `sharedDatasets` snapshot captured when this closure was built.
+                     // An org created after a later app/marketplace install must still
+                     // replay their seeds (the #3453 fix). Fall back to the snapshot
+                     // only if the service somehow vanished.
+                     const datasetsNow = readSeedDatasets(ctx) ?? sharedDatasets;
                      if (!Array.isArray(datasetsNow) || datasetsNow.length === 0) {
                          return { inserted: 0, updated: 0, skipped: 0, errors: [] as any[] };
                      }
@@ -806,8 +805,15 @@ export class AppPlugin implements Plugin {
                          errors: result.errors,
                      };
                  };
-                 registerSvc('seed-replayer', replayer);
-                 ctx.logger.info(`[Seeder] Registered ${normalizedDatasets.length} datasets + replayer on kernel (total seeds: ${merged.length})`);
+                 // Register the replayer once; a later config app reuses the first
+                 // one, which reads the now-extended shared list on every call — so
+                 // there is no duplicate-register throw and no lost datasets (#3453).
+                 const replayerRegistered = registerSeedReplayerOnce(ctx, replayer);
+                 ctx.logger.info(
+                     replayerRegistered
+                         ? `[Seeder] Registered ${normalizedDatasets.length} datasets + replayer (total seeds: ${sharedDatasets.length})`
+                         : `[Seeder] Appended ${normalizedDatasets.length} datasets to shared registry; reused existing replayer (total seeds: ${sharedDatasets.length})`,
+                 );
              } catch (e: any) {
                  ctx.logger.warn('[Seeder] Failed to register seed-datasets/seed-replayer service', { error: e?.message });
              }

packages/runtime/src/index.ts

@@ -23,6 +23,10 @@ export { SeedLoaderService } from './seed-loader.js';
 // contract shared by AppPlugin and the marketplace rehydrate/heal path.
 export { recordSeedOutcome } from './seed-summary.js';
 export type { SeedSourceOutcome } from './seed-summary.js';
+// Multi-tenant seed-replay registry (#3453) — the register-once-then-mutate
+// contract shared by AppPlugin and the marketplace install path so a new org
+// replays the UNION of every seed source, not just the first one.
+export { mergeSeedDatasets, readSeedDatasets, registerSeedReplayerOnce } from './seed-datasets.js';
 // External Datasource Federation — boot-validation gate (ADR-0015, Gate 2)
 export { ExternalValidationPlugin, createExternalValidationPlugin } from './external-validation-plugin.js';
 export type { ExternalSchemaDriftEvent } from './external-validation-plugin.js';