Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
match the current monorepo layout.

### Fixed
- **ObjectQLPlugin: cold-start metadata restoration** — `ObjectQLPlugin.start()` now calls
`protocol.loadMetaFromDb()` after driver initialization and before schema sync, restoring
all persisted metadata (objects, views, apps, etc.) from the `sys_metadata` table into the
in-memory `SchemaRegistry`. Previously, user-created custom objects were lost after kernel
cold starts or redeployments because the hydration step was missing. The fix gracefully
degrades in in-memory-only or first-run scenarios where `sys_metadata` does not yet exist.
- **Studio Vercel API routes returning HTML instead of JSON** — Adopted the
same Vercel deployment pattern used by `hotcrm`: committed
`api/[[...route]].js` catch-all route so Vercel detects it pre-build,
Expand Down
170 changes: 170 additions & 0 deletions packages/objectql/src/plugin.integration.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -822,4 +822,174 @@ describe('ObjectQLPlugin - Metadata Service Integration', () => {
expect(singleCalls).toContain('nb__item');
});
});

describe('Cold-Start Metadata Restoration', () => {
it('should restore metadata from sys_metadata via protocol.loadMetaFromDb on start', async () => {
// Arrange — a driver whose find() returns persisted metadata records
const findCalls: Array<{ object: string; query: any }> = [];
const mockDriver = {
name: 'restore-driver',
version: '1.0.0',
connect: async () => {},
disconnect: async () => {},
find: async (object: string, query: any) => {
findCalls.push({ object, query });
if (object === 'sys_metadata') {
return [
{
id: '1',
type: 'apps',
name: 'custom_crm',
state: 'active',
metadata: JSON.stringify({ name: 'custom_crm', label: 'Custom CRM' }),
},
{
id: '2',
type: 'object',
name: 'invoice',
state: 'active',
metadata: JSON.stringify({
name: 'invoice',
label: 'Invoice',
fields: { amount: { name: 'amount', label: 'Amount', type: 'number' } },
}),
packageId: 'user_pkg',
},
];
}
return [];
},
findOne: async () => null,
create: async (_o: string, d: any) => d,
update: async (_o: string, _i: any, d: any) => d,
delete: async () => true,
syncSchema: async () => {},
};

await kernel.use({
name: 'mock-restore-driver',
type: 'driver',
version: '1.0.0',
init: async (ctx) => {
ctx.registerService('driver.restore', mockDriver);
},
});

const plugin = new ObjectQLPlugin();
await kernel.use(plugin);

// Act
await kernel.bootstrap();

// Assert — sys_metadata should have been queried
const metaQuery = findCalls.find((c) => c.object === 'sys_metadata');
expect(metaQuery).toBeDefined();
expect(metaQuery!.query.where).toEqual({ state: 'active' });

// Assert — items should be restored into the registry
const registry = (kernel.getService('objectql') as any).registry;
expect(registry.getAllApps()).toContainEqual({
name: 'custom_crm',
label: 'Custom CRM',
});
});

it('should not throw when protocol.loadMetaFromDb fails (graceful degradation)', async () => {
// Arrange — driver that throws on find('sys_metadata')
const mockDriver = {
name: 'failing-db-driver',
version: '1.0.0',
connect: async () => {},
disconnect: async () => {},
find: async (object: string) => {
if (object === 'sys_metadata') {
throw new Error('SQLITE_ERROR: no such table: sys_metadata');
}
return [];
},
findOne: async () => null,
create: async (_o: string, d: any) => d,
update: async (_o: string, _i: any, d: any) => d,
delete: async () => true,
syncSchema: async () => {},
};

await kernel.use({
name: 'mock-fail-driver',
type: 'driver',
version: '1.0.0',
init: async (ctx) => {
ctx.registerService('driver.faildb', mockDriver);
},
});

const plugin = new ObjectQLPlugin();
await kernel.use(plugin);

// Act & Assert — should not throw
await expect(kernel.bootstrap()).resolves.not.toThrow();
});

it('should restore metadata before syncRegisteredSchemas so restored objects get table sync', async () => {
// Arrange — track the order of operations
const operations: string[] = [];
const mockDriver = {
name: 'order-driver',
version: '1.0.0',
connect: async () => {},
disconnect: async () => {},
find: async (object: string) => {
if (object === 'sys_metadata') {
operations.push('loadMetaFromDb');
return [
{
id: '1',
type: 'object',
name: 'restored_obj',
state: 'active',
metadata: JSON.stringify({
name: 'restored_obj',
label: 'Restored Object',
fields: { title: { name: 'title', label: 'Title', type: 'text' } },
}),
packageId: 'user_pkg',
},
];
}
return [];
},
findOne: async () => null,
create: async (_o: string, d: any) => d,
update: async (_o: string, _i: any, d: any) => d,
delete: async () => true,
syncSchema: async (object: string) => {
operations.push(`syncSchema:${object}`);
},
};

await kernel.use({
name: 'mock-order-driver',
type: 'driver',
version: '1.0.0',
init: async (ctx) => {
ctx.registerService('driver.order', mockDriver);
},
});

const plugin = new ObjectQLPlugin();
await kernel.use(plugin);

// Act
await kernel.bootstrap();

// Assert — loadMetaFromDb must appear before any syncSchema call
const loadIdx = operations.indexOf('loadMetaFromDb');
expect(loadIdx).toBeGreaterThanOrEqual(0);

const firstSync = operations.findIndex((op) => op.startsWith('syncSchema:'));
if (firstSync >= 0) {
expect(loadIdx).toBeLessThan(firstSync);
}
});
});
});
72 changes: 72 additions & 0 deletions packages/objectql/src/plugin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,25 @@ import { Plugin, PluginContext } from '@objectstack/core';

export type { Plugin, PluginContext };

/**
* Protocol extension for DB-based metadata hydration.
* `loadMetaFromDb` is implemented by ObjectStackProtocolImplementation but
* is NOT (yet) part of the canonical ObjectStackProtocol wire-contract in
* `@objectstack/spec`, since it is a server-side bootstrap concern only.
*/
interface ProtocolWithDbRestore {
loadMetaFromDb(): Promise<{ loaded: number; errors: number }>;
}

/** Type guard — checks whether the service exposes `loadMetaFromDb`. */
function hasLoadMetaFromDb(service: unknown): service is ProtocolWithDbRestore {
return (
typeof service === 'object' &&
service !== null &&
typeof (service as Record<string, unknown>)['loadMetaFromDb'] === 'function'
);
}

export class ObjectQLPlugin implements Plugin {
name = 'com.objectstack.engine.objectql';
type = 'objectql';
Expand Down Expand Up @@ -99,6 +118,12 @@ export class ObjectQLPlugin implements Plugin {
// Initialize drivers (calls driver.connect() which sets up persistence)
await this.ql?.init();

// Restore persisted metadata from sys_metadata table.
// This hydrates SchemaRegistry with objects/views/apps that were saved
// via protocol.saveMetaItem() in a previous session, ensuring custom
// schemas survive cold starts and redeployments.
await this.restoreMetadataFromDb(ctx);

// Sync all registered object schemas to database
// This ensures tables/collections are created or updated for every
// object registered by plugins (e.g., sys_user from plugin-auth).
Expand Down Expand Up @@ -332,6 +357,53 @@ export class ObjectQLPlugin implements Plugin {
}
}

/**
* Restore persisted metadata from the database (sys_metadata) on startup.
*
* Calls `protocol.loadMetaFromDb()` to bulk-load all active metadata
* records (objects, views, apps, etc.) into the in-memory SchemaRegistry.
* This closes the persistence loop so that user-created schemas survive
* kernel cold starts and redeployments.
*
* Gracefully degrades when:
* - The protocol service is unavailable (e.g., in-memory-only mode).
* - `loadMetaFromDb` is not implemented by the protocol shim.
* - The underlying driver/table does not exist yet (first-run scenario).
*/
private async restoreMetadataFromDb(ctx: PluginContext): Promise<void> {
// Phase 1: Resolve protocol service (separate from DB I/O for clearer diagnostics)
let protocol: ProtocolWithDbRestore;
try {
const service = ctx.getService('protocol');
if (!service || !hasLoadMetaFromDb(service)) {
ctx.logger.debug('Protocol service does not support loadMetaFromDb, skipping DB restore');
return;
}
protocol = service;
} catch (e: unknown) {
ctx.logger.debug('Protocol service unavailable, skipping DB restore', {
error: e instanceof Error ? e.message : String(e),
});
return;
}

// Phase 2: DB hydration
try {
const { loaded, errors } = await protocol.loadMetaFromDb();

Comment on lines +373 to +393
Copy link

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

restoreMetadataFromDb relies on ctx.getService('protocol') as any to access loadMetaFromDb, but loadMetaFromDb is not part of the typed protocol contract (and the as any defeats type-safety). Consider introducing a local ProtocolWithDbRestore type + type guard (or extending the protocol contract with an optional loadMetaFromDb?) so this call is type-checked and discoverable.

Copilot uses AI. Check for mistakes.
if (loaded > 0 || errors > 0) {
ctx.logger.info('Metadata restored from database', { loaded, errors });
} else {
ctx.logger.debug('No persisted metadata found in database');
}
} catch (e: unknown) {
// Non-fatal: first-run or in-memory driver may not have sys_metadata yet
ctx.logger.debug('DB metadata restore failed (non-fatal)', {
error: e instanceof Error ? e.message : String(e),
});
}
Comment on lines +373 to +404
Copy link

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The catch-all log message DB metadata restore skipped (non-fatal) can be misleading because this try also covers non-DB failures (e.g., getService('protocol') throwing, permission enforcement, async-service error). Consider narrowing the try/catch (separate service resolution from DB hydration) or adjusting the message/fields (e.g., include a structured error object/stack) so operators can distinguish missing protocol vs DB/table issues.

Copilot uses AI. Check for mistakes.
}

/**
* Load metadata from external metadata service into ObjectQL registry
* This enables ObjectQL to use file-based or remote metadata
Expand Down