feat(datasource): DatasourceConnectionService + declared auto-connect (ADR-0062 Phase 1) (#2198)

* feat(datasource): DatasourceConnectionService + declared auto-connect (ADR-0062 Phase 1)

Declare an external datasource → it auto-connects to a live ObjectQL driver and
its federated objects are queryable with ZERO app code (no onEnable). Implements
ADR-0062 Phase 1 (D1/D2/D5) toward epic #2163.

D1 — one connect path: new DatasourceConnectionService owns the single
"definition → live driver" path (factory build → credentialsRef resolve →
connect → registerDriver under the datasource name → registerDatasourceDef →
DDL-free syncObjectSchema per bound object). The runtime-admin registerPool now
delegates to it; AppPlugin auto-connects code-defined datasources. Exposed as the
'datasource-connection' kernel service.

D2 — opt-in-safe gate: connect only when external, an object explicitly binds via
object.datasource, or autoConnect:true. Managed datasources referenced only by a
datasourceMapping rule (e.g. app-crm's :memory: datasources) stay metadata-only —
existing apps byte-for-byte unchanged. Adds datasource.autoConnect to the spec.

D5 — lifecycle/ordering/policy: connect in AppPlugin.start() before the
kernel:ready validation gate (init-all-then-start-all). Fail-fast for declared
external + onMismatch:'fail'; degrade otherwise (always for runtime-admin/
rehydrate). New host-injectable DatasourceConnectPolicy (open-core default allows;
multi-tenant host binds a stricter fail-closed policy) consulted before connect.

Tests: 15 connection-service unit tests + 5 runtime integration tests (auto-connect,
managed-unrouted stays metadata-only, queryable end-to-end, deny policy). onEnable +
ctx.drivers.register remains a supported, idempotent escape hatch.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(runtime): use single-string logger.error in auto-connect catch (DTS build)

The context logger interface types error(message, error?: Error) — passing a meta
object {appId, error} tripped the tsup DTS build (TS2353) even though tsc --noEmit
passed. Switch to a single interpolated message; the rethrow still surfaces the
real cause to the kernel.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

Author: 3 people

.changeset/adr-0062-datasource-connection-service.md

@@ -0,0 +1,38 @@
+---
+"@objectstack/service-datasource": minor
+"@objectstack/runtime": minor
+"@objectstack/spec": minor
+---
+
+feat(datasource): auto-connect declared external datasources (ADR-0062 Phase 1, D1/D2/D5)
+
+A declared external datasource is now connected to a live ObjectQL driver and its
+federated objects are queryable **with zero app code** — no `onEnable` driver
+wiring. Implements ADR-0062 Phase 1.
+
+- **D1 — one connect path.** New `DatasourceConnectionService` in
+  `@objectstack/service-datasource` owns the single "definition → live driver"
+  path: build via the injected driver factory → resolve `external.credentialsRef`
+  via the `SecretBinder` → connect → `engine.registerDriver` under the datasource
+  name → register the datasource def → sync each bound federated object's read
+  metadata (DDL-free). Both origins converge on it: the runtime-admin
+  `registerPool` now delegates here, and `AppPlugin` auto-connects code-defined
+  datasources. Exposed as the `'datasource-connection'` kernel service.
+- **D2 — opt-in-safe gate.** A declared datasource auto-connects only when it is
+  `external`, an object **explicitly** binds to it via `object.datasource`, or it
+  sets the new `autoConnect: true` flag. A managed datasource that nothing
+  explicitly binds (incl. ones referenced only by a `datasourceMapping` rule, e.g.
+  `examples/app-crm`'s `:memory:` datasources) stays metadata-only — existing apps
+  are byte-for-byte unchanged. See the ADR-0062 D2 implementation note.
+- **D5 — lifecycle, ordering & policy.** Connect happens in `AppPlugin.start()`
+  (before the `kernel:ready` validation gate, relying on the kernel's
+  init-all-then-start-all ordering). Fail-fast for a declared `external` datasource
+  with `validation.onMismatch: 'fail'`; degrade-with-warning otherwise (and always
+  for runtime-admin/rehydrate, so a UI action or replica blip never bricks the
+  server). Adds a host-injectable `DatasourceConnectPolicy` (open-core default
+  allows; a multi-tenant host binds a stricter fail-closed policy for egress
+  isolation) consulted before every connect — one connect path, no cloud fork.
+
+Adds `datasource.autoConnect` to the spec. The legacy `onEnable` +
+`ctx.drivers.register` bridge remains supported as an escape hatch (idempotent vs.
+auto-connect). No behavior change for managed apps.

content/docs/references/ai/agent.mdx

@@ -76,6 +76,7 @@ const result = AIKnowledge.parse(data);
 | **instructions** | `string` | ✅ | System Prompt / Prime Directives |
 | **model** | `Object` | optional |  |
 | **lifecycle** | `Object` | optional | State machine defining the agent conversation follow and constraints |
+| **surface** | `Enum<'ask' \| 'build'>` | ✅ | Product surface this agent binds ('ask' | 'build') — ADR-0063 §1 |
 | **skills** | `string[]` | optional | Skill names to attach (Agent→Skill→Tool architecture) |
 | **tools** | `Object[]` | optional | Direct tool references (legacy fallback) |
 | **knowledge** | `Object` | optional | RAG access |

content/docs/references/ai/skill.mdx

@@ -36,6 +36,7 @@ const result = Skill.parse(data);
 | **name** | `string` | ✅ | Skill unique identifier (snake_case) |
 | **label** | `string` | ✅ | Skill display name |
 | **description** | `string` | optional | Skill description |
+| **surface** | `Enum<'ask' \| 'build' \| 'both'>` | ✅ | Agent surface this skill binds to ('ask' | 'build' | 'both') — ADR-0063 §3 |
 | **instructions** | `string` | optional | LLM instructions when skill is active |
 | **tools** | `string[]` | ✅ | Tool names belonging to this skill (supports trailing wildcard, e.g. `action_*`) |
 | **triggerPhrases** | `string[]` | optional | Phrases that activate this skill |

content/docs/references/api/contract.mdx

@@ -154,7 +154,7 @@ const result = ApiError.parse(data);
 | **having** | `[__schema2](./__schema2)` | optional | HAVING clause for aggregation filtering |
 | **windowFunctions** | `Object[]` | optional | Window functions with OVER clause |
 | **distinct** | `boolean` | optional | SELECT DISTINCT flag |
-| **expand** | `Record<string, [__schema0](./__schema0)>` | optional | Recursive relation loading map. Keys are lookup/master_detail field names; values are nested QueryAST objects that control select, filter, sort, and further expansion on the related object. The engine resolves expand via batch $in queries (driver-agnostic) with a default max depth of 3. |
+| **expand** | `Record<string, [__schema0](./__schema0)>` | optional | Recursive relation loading map. Keys are lookup/master_detail field names; values are nested QueryAST objects that control select (`fields`) and filter (`where`, AND-merged with the batch $in), plus further expansion on the related object. The engine resolves expand via batch $in queries (driver-agnostic) with a default max depth of 3; per-parent `limit`/`offset`/`orderBy` are NOT applied on this path. |
 
 
 ---

content/docs/references/data/data-engine.mdx

@@ -481,6 +481,7 @@ QueryAST-aligned options for DataEngine.aggregate operations
 | **where** | `Record<string, any> \| [__schema0](./__schema0)` | optional |  |
 | **groupBy** | `string[]` | optional |  |
 | **aggregations** | `Object[]` | optional |  |
+| **timezone** | `string` | optional |  |
 
 
 ---

content/docs/references/data/datasource.mdx

@@ -43,6 +43,7 @@ const result = Datasource.parse(data);
 | **retryPolicy** | `Object` | optional | Connection retry policy for transient failures |
 | **description** | `string` | optional | Internal description |
 | **active** | `boolean` | ✅ | Is datasource enabled |
+| **autoConnect** | `boolean` | ✅ | Force a live driver connection at boot even when managed + unrouted (ADR-0062 D2). |
 | **schemaMode** | `Enum<'managed' \| 'external' \| 'validate-only'>` | ✅ | Schema ownership mode |
 | **external** | `Object` | optional | External datasource federation settings (schemaMode != "managed") |
 | **origin** | `Enum<'code' \| 'runtime'>` | ✅ | Datasource provenance (server-managed, read-only) |

content/docs/references/data/driver.mdx

@@ -95,6 +95,7 @@ const result = DriverCapabilities.parse(data);
 | **skipCache** | `boolean` | optional | Bypass cache |
 | **traceContext** | `Record<string, string>` | optional | OpenTelemetry context or request ID |
 | **tenantId** | `string` | optional | Tenant Isolation identifier |
+| **timezone** | `string` | optional | Business reference timezone (IANA) for date-dependent generation, e.g. autonumber date tokens |
 
 
 ---

content/docs/references/data/hook-body.mdx

@@ -15,6 +15,14 @@ a capability it did not declare, the call throws at invocation time.
 
 - `api.write`  — `ctx.api.object(...).insert / update / delete`
 
+- `api.transaction` — `ctx.api.transaction(async () => \{ … \})` — runs the
+
+callback's `ctx.api` writes/reads inside one driver transaction, committed
+
+on return and rolled back if the callback throws. Requires `api.write`
+
+alongside it to be useful (the transaction body still needs write access).
+
 - `crypto.uuid` — `ctx.crypto.randomUUID()`
 
 - `crypto.hash` — `ctx.crypto.hash(algo, data)`
@@ -86,7 +94,7 @@ L2 sandboxed JS body — runs inside an isolated VM with declared capabilities
 | :--- | :--- | :--- | :--- |
 | **language** | `string` | ✅ |  |
 | **source** | `string` | ✅ | Function body source |
-| **capabilities** | `Enum<'api.read' \| 'api.write' \| 'crypto.uuid' \| 'crypto.hash' \| 'log'>[]` | ✅ | Granted capability tokens |
+| **capabilities** | `Enum<'api.read' \| 'api.write' \| 'api.transaction' \| 'crypto.uuid' \| 'crypto.hash' \| 'log'>[]` | ✅ | Granted capability tokens |
 | **timeoutMs** | `integer` | optional | Per-invocation timeout (ms) |
 | **memoryMb** | `integer` | optional | Per-invocation memory cap (MB) |
 
@@ -101,6 +109,7 @@ L2 sandboxed JS body — runs inside an isolated VM with declared capabilities
 
 * `api.read`
 * `api.write`
+* `api.transaction`
 * `crypto.uuid`
 * `crypto.hash`
 * `log`
@@ -118,7 +127,7 @@ L2 sandboxed JS body — runs inside an isolated VM with declared capabilities
 | :--- | :--- | :--- | :--- |
 | **language** | `string` | ✅ |  |
 | **source** | `string` | ✅ | Function body source |
-| **capabilities** | `Enum<'api.read' \| 'api.write' \| 'crypto.uuid' \| 'crypto.hash' \| 'log'>[]` | ✅ | Granted capability tokens |
+| **capabilities** | `Enum<'api.read' \| 'api.write' \| 'api.transaction' \| 'crypto.uuid' \| 'crypto.hash' \| 'log'>[]` | ✅ | Granted capability tokens |
 | **timeoutMs** | `integer` | optional | Per-invocation timeout (ms) |
 | **memoryMb** | `integer` | optional | Per-invocation memory cap (MB) |
 

content/docs/references/data/object.mdx

@@ -14,8 +14,8 @@ API Operations Enum
 ## TypeScript Usage
 
 ```typescript
-import { ApiMethod, CDCConfig, Index, ObjectCapabilities, ObjectExternalBinding, ObjectOwnershipEnum, PartitioningConfig, SoftDeleteConfig, TenancyConfig, VersioningConfig } from '@objectstack/spec/data';
-import type { ApiMethod, CDCConfig, Index, ObjectCapabilities, ObjectExternalBinding, ObjectOwnershipEnum, PartitioningConfig, SoftDeleteConfig, TenancyConfig, VersioningConfig } from '@objectstack/spec/data';
+import { ApiMethod, Index, ObjectCapabilities, ObjectExternalBinding, ObjectOwnershipEnum, PartitioningConfig, SoftDeleteConfig, TenancyConfig, VersioningConfig } from '@objectstack/spec/data';
+import type { ApiMethod, Index, ObjectCapabilities, ObjectExternalBinding, ObjectOwnershipEnum, PartitioningConfig, SoftDeleteConfig, TenancyConfig, VersioningConfig } from '@objectstack/spec/data';
 
 // Validate data
 const result = ApiMethod.parse(data);
@@ -43,19 +43,6 @@ const result = ApiMethod.parse(data);
 * `export`
 
 
----
-
-## CDCConfig
-
-### Properties
-
-| Property | Type | Required | Description |
-| :--- | :--- | :--- | :--- |
-| **enabled** | `boolean` | ✅ | Enable Change Data Capture |
-| **events** | `Enum<'insert' \| 'update' \| 'delete'>[]` | ✅ | Event types to capture |
-| **destination** | `string` | ✅ | Destination endpoint (e.g., "kafka://topic", "webhook://url") |
-
-
 ---
 
 ## Index

content/docs/references/data/query.mdx

@@ -202,7 +202,7 @@ Type: `string`
 | **having** | `[__schema1](./__schema1)` | optional | HAVING clause for aggregation filtering |
 | **windowFunctions** | `Object[]` | optional | Window functions with OVER clause |
 | **distinct** | `boolean` | optional | SELECT DISTINCT flag |
-| **expand** | `Record<string, [#](./#)>` | optional | Recursive relation loading map. Keys are lookup/master_detail field names; values are nested QueryAST objects that control select, filter, sort, and further expansion on the related object. The engine resolves expand via batch $in queries (driver-agnostic) with a default max depth of 3. |
+| **expand** | `Record<string, [#](./#)>` | optional | Recursive relation loading map. Keys are lookup/master_detail field names; values are nested QueryAST objects that control select (`fields`) and filter (`where`, AND-merged with the batch $in), plus further expansion on the related object. The engine resolves expand via batch $in queries (driver-agnostic) with a default max depth of 3; per-parent `limit`/`offset`/`orderBy` are NOT applied on this path. |
 
 
 ---