Skip to content

Commit 0feea92

Browse files
xuyushun441-sysos-zhuangclaude
authored
feat(app-showcase): add an end-to-end external datasource + federated object example (#2111) (#2139)
* feat(app-showcase): add an end-to-end external datasource + federated object example (#2111) A code-defined EXTERNAL datasource (ADR-0015) over a second, read-only SQLite file — no external server required, so `os dev` just works. Demonstrates the full federation path the runtime datasource-admin UI exposes. - src/datasources/showcase-external.datasource.ts — schemaMode:'external', read-only, onMismatch:'warn' (a fixture hiccup never bricks the showcase boot). Includes a commented Postgres+secret variant for the masked-secret widget. - src/objects/external/{customer,order}.object.ts — federated objects whose names (showcase_ext_customer/order) deliberately differ from their remote tables (customers/orders), bound via external.remoteName — exercises the remote-table remap fixed in the companion driver change. - src/datasources/external-fixture.ts — the onEnable hook: idempotently provisions the fixture sqlite (tables + seed rows), registers a live read-only driver under the datasource name, and registers the federated objects' read metadata (DDL-free, via ql.syncObjectSchema). - objectstack.config.ts wires the datasource + objects + onEnable; package.json adds @objectstack/driver-sql; README documents the flow and the Setup → Datasources sync UI; test/external-datasource.test.ts smoke-asserts the binding. Depends on the ADR-0015 federation read-path fix (honoring external.remoteName). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(verify): skip read-only federated (external) objects in CRUD verification `objectstack verify` probe-inserts into every object; a read-only external object (ADR-0015) correctly rejects the insert via the write gate, which verify reported as a create-failed runtime failure (breaking the Dogfood Regression Gate once the showcase gained federated objects). deriveCrudCases now marks external objects that aren't fully write-opted-in (datasource.external.allowWrites && object.external.writable) as `blocked`/skipped. Adds derive.test.ts covering the skip + the write-opted-in and managed pass-through cases. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 7108ff3 commit 0feea92

13 files changed

Lines changed: 384 additions & 2 deletions

File tree

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
"@objectstack/verify": patch
3+
---
4+
5+
fix(verify): skip read-only federated (external) objects in CRUD verification.
6+
7+
`objectstack verify` probe-inserts a record into every object. A federated object
8+
on an external datasource is read-only unless BOTH the datasource and the object
9+
opt into writes (ADR-0015 write gate), so that insert is correctly rejected —
10+
which `verify` was reporting as a `create-failed` runtime failure. `deriveCrudCases`
11+
now marks such objects `blocked` (skipped), matching the write gate's double
12+
opt-in rule, so the dogfood gate stays honest while supporting external datasource
13+
example apps.

examples/app-showcase/README.md

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -122,3 +122,31 @@ app-showcase/
122122
When you add a new variant to the platform, the coverage test will go red and
123123
point at the gap. Add a field/view/widget that uses it, reference it in
124124
`COVERAGE`, and the test goes green again.
125+
126+
## External datasource federation (ADR-0015)
127+
128+
The showcase ships a **code-defined external datasource** that demonstrates the
129+
full federation path with **no external server**`os dev` just works.
130+
131+
- `src/datasources/showcase-external.datasource.ts` — a second, read-only SQLite
132+
database (`schemaMode: 'external'`), separate from the managed standalone DB.
133+
- `src/objects/external/{customer,order}.object.ts` — federated objects
134+
(`showcase_ext_customer`, `showcase_ext_order`) bound to the remote tables
135+
`customers` / `orders` via `external.remoteName`. The object names deliberately
136+
differ from the table names to show the remote-table remap.
137+
- `src/datasources/external-fixture.ts` — the stack's `onEnable` hook. It
138+
idempotently provisions the fixture SQLite file (tables + seed rows), registers
139+
a live read-only driver under the datasource name, and registers the federated
140+
objects' read metadata.
141+
142+
**See it:**
143+
144+
- `GET /api/v1/datasources` and `GET /api/v1/meta/datasource` include
145+
`showcase_external`.
146+
- `GET /api/v1/data/showcase_ext_customer` returns the seeded rows — queried live
147+
from the external table `customers`.
148+
- Sign in as a platform admin → **Setup → Integrations → Datasources** to see it
149+
listed, and run the runtime **Sync objects** wizard against it.
150+
151+
To showcase a masked **secret** credential field, uncomment the Postgres variant
152+
in `showcase-external.datasource.ts` and point it at a real warehouse.

examples/app-showcase/objectstack.config.ts

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,9 @@ import {
1212
} from '@objectstack/cloud-connection';
1313

1414
import * as objects from './src/objects/index.js';
15+
import { ShowcaseExternalDatasource } from './src/datasources/showcase-external.datasource.js';
16+
import { ExternalCustomer, ExternalOrder } from './src/objects/external/index.js';
17+
import { setupShowcaseExternalDatasource } from './src/datasources/external-fixture.js';
1518
import { TaskViews, ProjectViews, InquiryViews } from './src/views/index.js';
1619
import { ShowcaseApp } from './src/apps/index.js';
1720
import { ChartGalleryDashboard, OpsDashboard } from './src/dashboards/index.js';
@@ -124,6 +127,13 @@ export default defineStack({
124127
// database at `<project>/.objectstack/data/standalone.db`, so data and
125128
// AI-authored metadata survive restarts (a `:memory:` datasource would wipe
126129
// everything on every restart, which makes local app-building unusable).
130+
//
131+
// External-datasource federation demo (ADR-0015): a second, read-only SQLite
132+
// file declared as a code-defined external datasource. It appears in
133+
// Setup → Datasources and its federated objects (below) are queryable via
134+
// REST. The fixture file + live driver are wired by `onEnable` (bottom of
135+
// this file). `os dev` needs no extra setup.
136+
datasources: [ShowcaseExternalDatasource],
127137

128138
// i18n
129139
translations: [ShowcaseTranslationBundle],
@@ -137,7 +147,7 @@ export default defineStack({
137147
},
138148

139149
// Data
140-
objects: Object.values(objects),
150+
objects: [...Object.values(objects), ExternalCustomer, ExternalOrder],
141151

142152
// UI
143153
apps: [ShowcaseApp],
@@ -172,3 +182,14 @@ export default defineStack({
172182
// Seed data
173183
data: ShowcaseSeedData,
174184
});
185+
186+
/**
187+
* Runtime wiring for the external-datasource federation demo (ADR-0015).
188+
*
189+
* Code (fixture provisioning + live external-driver registration) can't live in
190+
* the declarative artifact, so it runs here. The AppPlugin invokes `onEnable` at
191+
* boot (Phase 2), before the external-validation gate fires on `kernel:ready`.
192+
*/
193+
export const onEnable = async (ctx: unknown): Promise<void> => {
194+
await setupShowcaseExternalDatasource(ctx as Parameters<typeof setupShowcaseExternalDatasource>[0]);
195+
};

examples/app-showcase/package.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"name": "@objectstack/example-showcase",
33
"version": "0.2.0",
4-
"description": "Kitchen-sink showcase workspace exercises every metadata type, every view type, every chart type, and the major end-to-end capability chains (security, automation, AI). Built for demonstration, debugging, and coverage-driven verification.",
4+
"description": "Kitchen-sink showcase workspace \u2014 exercises every metadata type, every view type, every chart type, and the major end-to-end capability chains (security, automation, AI). Built for demonstration, debugging, and coverage-driven verification.",
55
"license": "Apache-2.0",
66
"private": true,
77
"main": "./objectstack.config.ts",
@@ -24,6 +24,7 @@
2424
"@objectstack/cloud-connection": "workspace:*",
2525
"@objectstack/connector-rest": "workspace:*",
2626
"@objectstack/connector-slack": "workspace:*",
27+
"@objectstack/driver-sql": "workspace:*",
2728
"@objectstack/runtime": "workspace:*",
2829
"@objectstack/spec": "workspace:*"
2930
},
Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
import { SqlDriver } from '@objectstack/driver-sql';
4+
5+
/**
6+
* Runtime wiring for the external-datasource federation demo (ADR-0015).
7+
*
8+
* This is CODE, so it can't live in the declarative artifact — it runs from the
9+
* stack's `onEnable` hook, which the AppPlugin invokes at boot (Phase 2), before
10+
* the external-validation gate fires on `kernel:ready` (Phase 3).
11+
*
12+
* It does three things, all zero-config so `os dev` "just works":
13+
* 1. Idempotently provisions the "remote" SQLite file with `customers` /
14+
* `orders` tables + a little seed data (a MANAGED driver — DDL allowed).
15+
* 2. Registers a read-only `schemaMode: 'external'` driver for that same file
16+
* under the datasource name `showcase_external`, so ObjectQL can route
17+
* queries to it. (Declared datasources are surfaced in the metadata
18+
* registry for Setup → Datasources, but a live driver must be registered
19+
* for the federated objects to be queryable in standalone mode.)
20+
* 3. Registers the federated objects' read metadata (DDL-free) so coercion +
21+
* the remote-table mapping exist immediately.
22+
*/
23+
24+
// Same relative path as the datasource config — resolved against the project
25+
// cwd by better-sqlite3. `connect()` creates the parent dir if missing.
26+
const EXTERNAL_DB_FILE = '.objectstack/data/showcase_external.db';
27+
28+
const CUSTOMER_TABLE = {
29+
name: 'customers',
30+
fields: {
31+
id: { type: 'text' },
32+
name: { type: 'text' },
33+
email: { type: 'text' },
34+
region: { type: 'text' },
35+
lifetime_value: { type: 'number' },
36+
},
37+
};
38+
39+
const ORDER_TABLE = {
40+
name: 'orders',
41+
fields: {
42+
id: { type: 'text' },
43+
customer_id: { type: 'text' },
44+
amount: { type: 'number' },
45+
status: { type: 'text' },
46+
placed_on: { type: 'date' },
47+
},
48+
};
49+
50+
const CUSTOMER_ROWS = [
51+
{ id: 'c1', name: 'Aurora Labs', email: 'ap@aurora.example', region: 'NA', lifetime_value: 480000 },
52+
{ id: 'c2', name: 'Borealis GmbH', email: 'billing@borealis.example', region: 'EU', lifetime_value: 312000 },
53+
{ id: 'c3', name: 'Cyan Pacific', email: 'accounts@cyan.example', region: 'APAC', lifetime_value: 95000 },
54+
];
55+
56+
const ORDER_ROWS = [
57+
{ id: 'o1', customer_id: 'c1', amount: 12000, status: 'paid', placed_on: '2026-01-12' },
58+
{ id: 'o2', customer_id: 'c1', amount: 8400, status: 'paid', placed_on: '2026-02-03' },
59+
{ id: 'o3', customer_id: 'c2', amount: 21500, status: 'pending', placed_on: '2026-02-20' },
60+
{ id: 'o4', customer_id: 'c3', amount: 3300, status: 'paid', placed_on: '2026-03-01' },
61+
];
62+
63+
/** Stack `onEnable` payload (subset we use). */
64+
interface OnEnableContext {
65+
ql: { syncObjectSchema?: (name: string) => Promise<void> };
66+
drivers: { register: (driver: unknown) => void };
67+
logger?: { info?: (msg: string, meta?: unknown) => void; warn?: (msg: string, meta?: unknown) => void };
68+
}
69+
70+
export async function setupShowcaseExternalDatasource(ctx: OnEnableContext): Promise<void> {
71+
// 1. Provision the remote fixture with a MANAGED driver (DDL allowed). Idempotent.
72+
const fixture = new SqlDriver({
73+
client: 'better-sqlite3',
74+
connection: { filename: EXTERNAL_DB_FILE },
75+
useNullAsDefault: true,
76+
}) as unknown as {
77+
name: string;
78+
connect: () => Promise<void>;
79+
disconnect: () => Promise<void>;
80+
initObjects: (objs: unknown[]) => Promise<void>;
81+
count: (object: string, query: unknown) => Promise<number>;
82+
bulkCreate: (object: string, rows: unknown[]) => Promise<unknown>;
83+
};
84+
fixture.name = 'showcase_external_fixture';
85+
await fixture.connect();
86+
try {
87+
await fixture.initObjects([CUSTOMER_TABLE, ORDER_TABLE]);
88+
if ((await fixture.count('customers', {})) === 0) {
89+
await fixture.bulkCreate('customers', CUSTOMER_ROWS);
90+
await fixture.bulkCreate('orders', ORDER_ROWS);
91+
}
92+
} finally {
93+
await fixture.disconnect();
94+
}
95+
96+
// 2. Register the read-only EXTERNAL driver under the datasource name.
97+
const ext = new SqlDriver({
98+
client: 'better-sqlite3',
99+
connection: { filename: EXTERNAL_DB_FILE },
100+
useNullAsDefault: true,
101+
schemaMode: 'external',
102+
} as never) as unknown as { name: string; connect: () => Promise<void> };
103+
ext.name = 'showcase_external'; // MUST equal the datasource name (ObjectQL routes by driver name).
104+
await ext.connect();
105+
ctx.drivers.register(ext);
106+
107+
// 3. Register read metadata for the federated objects (DDL-free; ADR-0015).
108+
// Needed because the driver is registered after the boot schema-sync.
109+
await ctx.ql.syncObjectSchema?.('showcase_ext_customer');
110+
await ctx.ql.syncObjectSchema?.('showcase_ext_order');
111+
112+
ctx.logger?.info?.('[showcase] external datasource "showcase_external" ready — federation demo (ADR-0015)');
113+
}
Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
import { defineDatasource } from '@objectstack/spec/data';
4+
5+
/**
6+
* A code-defined EXTERNAL datasource (ADR-0015 federation) — a *second*,
7+
* read-only SQLite file separate from the managed standalone DB. It exists so
8+
* the showcase can demonstrate the full federation path with **no external
9+
* server**: `os dev` provisions the fixture file at boot (see
10+
* `external-fixture.ts`), then federated objects bound to its tables are
11+
* queryable through the normal ObjectQL/REST surface.
12+
*
13+
* `schemaMode: 'external'` ⇒ ObjectStack never runs DDL here (it's a guest in a
14+
* database it does not own). `onMismatch: 'warn'` keeps a fixture hiccup from
15+
* bricking the whole showcase boot — the rest of the demo still loads.
16+
*
17+
* It also shows up in **Setup → Integrations → Datasources** and via
18+
* `GET /api/v1/meta/datasource`, where an admin can run the runtime
19+
* "Sync objects" wizard against it.
20+
*/
21+
export const ShowcaseExternalDatasource = defineDatasource({
22+
name: 'showcase_external',
23+
label: 'External Analytics (SQLite)',
24+
driver: 'sqlite',
25+
schemaMode: 'external',
26+
// Relative path → resolved against the project cwd by better-sqlite3, the
27+
// same place the fixture writes it. Sits next to the managed standalone.db.
28+
config: { filename: '.objectstack/data/showcase_external.db' },
29+
external: {
30+
label: 'External Analytics DB — read-only federation demo (ADR-0015)',
31+
allowWrites: false,
32+
validation: { onMismatch: 'warn', checkOnBoot: true },
33+
},
34+
active: true,
35+
});
36+
37+
// ── Optional secret demo (commented) ────────────────────────────────────────
38+
// A Postgres variant exercising a `secret` credential field. Uncomment and
39+
// point at a real warehouse to see the MASKED secret widget in
40+
// Setup → Datasources (objectui#1853). Left commented so the default example
41+
// needs no external server.
42+
//
43+
// export const ShowcaseWarehouseDatasource = defineDatasource({
44+
// name: 'showcase_warehouse',
45+
// label: 'Analytics Warehouse (Postgres)',
46+
// driver: 'postgres',
47+
// schemaMode: 'external',
48+
// config: { host: 'localhost', port: 5432, database: 'analytics', user: 'readonly' },
49+
// external: {
50+
// allowWrites: false,
51+
// credentialsRef: 'secret:warehouse/password',
52+
// validation: { onMismatch: 'fail', checkOnBoot: true },
53+
// },
54+
// active: false,
55+
// });
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
import { ObjectSchema, Field } from '@objectstack/spec/data';
4+
5+
/**
6+
* A customer row federated LIVE from the external SQLite database
7+
* (`showcase_external`). The object name (`showcase_ext_customer`) deliberately
8+
* differs from the remote table (`customers`) — the binding is expressed by
9+
* `external.remoteName`, exercising ADR-0015's remote-table remap on the read
10+
* path. Read-only: the datasource sets `allowWrites: false`.
11+
*/
12+
export const ExternalCustomer = ObjectSchema.create({
13+
name: 'showcase_ext_customer',
14+
label: 'External Customer',
15+
pluralLabel: 'External Customers',
16+
icon: 'database',
17+
description: 'A customer federated from the external analytics DB (ADR-0015). Bound to remote table `customers` via external.remoteName.',
18+
datasource: 'showcase_external',
19+
external: { remoteName: 'customers' },
20+
fields: {
21+
name: Field.text({ label: 'Name', searchable: true }),
22+
email: Field.text({ label: 'Email' }),
23+
region: Field.text({ label: 'Region' }),
24+
lifetime_value: Field.currency({ label: 'Lifetime Value', scale: 2 }),
25+
},
26+
});
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
export { ExternalCustomer } from './customer.object.js';
4+
export { ExternalOrder } from './order.object.js';
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
import { ObjectSchema, Field } from '@objectstack/spec/data';
4+
5+
/**
6+
* An order row federated from the external SQLite database. Object name
7+
* (`showcase_ext_order`) ≠ remote table (`orders`); bound via
8+
* `external.remoteName`. Read-only.
9+
*/
10+
export const ExternalOrder = ObjectSchema.create({
11+
name: 'showcase_ext_order',
12+
label: 'External Order',
13+
pluralLabel: 'External Orders',
14+
icon: 'shopping-cart',
15+
description: 'An order federated from the external analytics DB (ADR-0015). Bound to remote table `orders` via external.remoteName.',
16+
datasource: 'showcase_external',
17+
external: { remoteName: 'orders' },
18+
fields: {
19+
customer_id: Field.text({ label: 'Customer ID' }),
20+
amount: Field.currency({ label: 'Amount', scale: 2 }),
21+
status: Field.text({ label: 'Status' }),
22+
placed_on: Field.date({ label: 'Placed On' }),
23+
},
24+
});
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* Smoke test for the external-datasource federation example (ADR-0015).
5+
* Asserts the datasource + federated objects are declared correctly — the
6+
* remote-table remap (`object.name !== external.remoteName`) is the whole point.
7+
* The live read path is covered by the driver-level integration test
8+
* (packages/plugins/driver-sql/src/sql-driver-external-remote-name.test.ts).
9+
*/
10+
11+
import { describe, it, expect } from 'vitest';
12+
import { ShowcaseExternalDatasource } from '../src/datasources/showcase-external.datasource.js';
13+
import { ExternalCustomer, ExternalOrder } from '../src/objects/external/index.js';
14+
15+
describe('showcase external datasource (ADR-0015 federation)', () => {
16+
it('is an external, read-only datasource that degrades gracefully', () => {
17+
expect(ShowcaseExternalDatasource.name).toBe('showcase_external');
18+
expect(ShowcaseExternalDatasource.schemaMode).toBe('external');
19+
expect(ShowcaseExternalDatasource.external?.allowWrites).toBe(false);
20+
// A fixture hiccup must not brick the whole showcase boot.
21+
expect(ShowcaseExternalDatasource.external?.validation?.onMismatch).toBe('warn');
22+
});
23+
24+
it('federated objects bind to remote tables via remoteName (name != table)', () => {
25+
expect(ExternalCustomer.datasource).toBe('showcase_external');
26+
expect(ExternalCustomer.external?.remoteName).toBe('customers');
27+
expect(ExternalCustomer.name).not.toBe(ExternalCustomer.external?.remoteName);
28+
29+
expect(ExternalOrder.datasource).toBe('showcase_external');
30+
expect(ExternalOrder.external?.remoteName).toBe('orders');
31+
expect(ExternalOrder.name).not.toBe(ExternalOrder.external?.remoteName);
32+
});
33+
});

0 commit comments

Comments
 (0)