Skip to content

Commit cdf4d9a

Browse files
os-zhuangclaude
andauthored
feat(spec,service-datasource): datasource.config 按驱动契约校验 (#4410) (#4465)
* feat(spec,service-datasource): datasource.config is parsed against its driver's contract (#4410) `config` was the one authorable slot on a datasource with no gate at all. The module comment justified the hole by saying "the driver's own `configSchema` is what validates it". Nothing did: both bundled driver specs set `configSchema: {}`, no code read the field, and the per-driver zod schemas were not exported from the package — `data/driver/` was reachable only from its own tests. So `config: { hostname: 'db.internal' }` (the key is `host`) was accepted in silence and the datasource connected to localhost while the parse, the save and the connection probe all reported success. That is #4001's original bug verbatim, one level down, and #4001's own fix pointed authors straight into it. The frontend question the issue raised has an answer, and it is not a third false claim: objectui's DatasourceResourcePage really does render the connection form from a driver `configSchema` (`GET /api/v1/datasources/drivers`, reading properties/required/title/format). It reads DRIVER_CATALOG — a SECOND set of hand-written JSON-Schema literals in service-datasource, never checked against the spec's zod schemas and never validating anything. One live copy, one dead copy, no gate between them. So: `packages/spec/src/data/driver/` becomes the one contract, and three consumers read it. `DatasourceSchema` parses `config` — and each `readReplicas` entry — against the schema for the declared driver; `DriverDefinitionSchema .configSchema` publishes its JSON-Schema projection; the catalog serves that same projection, so the form offers exactly the fields the validator accepts. `mysql` and `sqlite` / `sqlite-wasm` had no config shape anywhere, though both were offered by the form and buildable by the factory. The wizard is the other authoring door and does not reach DatasourceSchema: createDatasource writes through `metadata.register`, whose validation is a structural name/label check. DatasourceAdminService create/update/test now consults the same registry — testConnection BEFORE probing, or a green "connection successful" gets reported against localhost. Enforcing the contract forced honouring it. A gate over `config` means every key inside it claims to be read, so each was audited against the code that reads it: - `datasource.pool` reaches every SQL driver. It was declared, strict, carried into the connection spec — then overwritten with a hardcoded { min: 0, max: 5 }. Maps onto minPoolSize/maxPoolSize for mongo. - `datasource.schemaMode` reaches the driver. It was dropped between the record and the spec, so the factory looked for it in two places that could never hold it and an `external` database — one ObjectStack must never run DDL against — was constructed as `managed`. - `datasource.ssl` reaches the SQL clients, certificates and all. It stopped at the record, so a TLS block configured nothing: the failure its own schema comment warns about. - postgres `schema` (knex searchPath), `applicationName`, `statementTimeout`. - mongo `password`, `authSource`, `options`. A mongo datasource carrying a `config.password` composed its URL with an EMPTY password. Two memory keys had nothing to wire to — `InMemoryDriverConfig` has no field for `indexes` or `maxRecordsPerObject`, the driver keeps no indexes and evicts nothing — so they are removed under ADR-0049 with the rejection carrying why. `config.ssl` is the boolean shorthand only, deliberately. A `boolean | object` union is honest about what the client accepts, but the form turns anything that is not boolean/enum/number into a TEXT INPUT: the wizard would have produced a string the new gate rejects. Certificates go in the datasource-level block, which this change makes live. One table for driver ids, in the spec. The factory kept its own copy, which meant the id selecting a DRIVER and the id selecting that driver's CONFIG CONTRACT could disagree — the same silent acceptance, reintroduced as a lookup miss. Also fixes the docs generator's one-level-deep source walk, which filed the new schemas onto a `misc` page whose "Source" line named a file that does not exist (the identical bug the strictness ledger's own coverage gate had). The recursive walk gives `data/driver/`, `integration/connector/` and `kernel/events/` real per-file pages; no documented schema was lost, six more are now covered. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WsgTqRF58HsQYKLsrZ5pQY * test(spec): pin the ssl split the config gate forced `config.ssl` narrowed to the on/off shorthand when the connection form turned out to render a non-boolean/enum/number prop as a TEXT INPUT — a `boolean | object` union there would have produced a wizard whose every `ssl` value the new gate rejects. These fixtures still passed the object form, so they asserted a shape the contract no longer has. They now assert the prescription instead: the certificate-bearing form is rejected and named toward the datasource-level `ssl` block, which #4410 wired through to the client. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WsgTqRF58HsQYKLsrZ5pQY * docs(spec): regenerate driver-sqlite reference after the comment fix The generated page carried the pre-fix wording of the module comment — the `check:docs` gate caught it on the post-merge re-run, which is what that gate is for. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WsgTqRF58HsQYKLsrZ5pQY --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent ebb209c commit cdf4d9a

90 files changed

Lines changed: 4292 additions & 1704 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
---
2+
'@objectstack/spec': minor
3+
'@objectstack/service-datasource': minor
4+
---
5+
6+
`datasource.config` is now validated against its driver's contract (#4410)
7+
8+
`config` was the one authorable slot on a datasource with no gate at all. The
9+
schema's own comment claimed "the driver's own `configSchema` is what validates
10+
it" — nothing did: both bundled driver specs set `configSchema: {}`, no code read
11+
the field, and the per-driver zod schemas were not even exported from the
12+
package. So `config: { hostname: 'db.internal' }` (the key is `host`) was
13+
accepted in silence and the datasource connected to `localhost` while the parse,
14+
the save and the connection probe all reported success.
15+
16+
`DatasourceSchema` now parses `config` — and each `readReplicas` entry — against
17+
the contract for the declared driver, and `DatasourceAdminService`
18+
(create/update/test, the Setup wizard's path) applies the same check. Both read
19+
one registry in `@objectstack/spec/data`, which also projects each contract to
20+
JSON Schema for `DriverDefinitionSchema.configSchema` and the Studio connection
21+
form, so the form offers exactly the fields the validator accepts.
22+
23+
New exports from `@objectstack/spec/data`: `PostgresConfigSchema`,
24+
`MysqlConfigSchema`, `SqliteConfigSchema`, `SqliteWasmConfigSchema`,
25+
`MongoConfigSchema`, `MemoryConfigSchema`, plus `resolveDriverId`,
26+
`getDriverConfigSchema`, `getDriverConfigJsonSchemaById` and
27+
`validateDriverConfig`. A driver the platform ships no contract for (a plugin's
28+
`com.vendor.snowflake`) keeps an unvalidated `config`.
29+
30+
**Migration.** A config that was silently ignored now fails with the correction
31+
in the message. The renames:
32+
33+
| Wrote | Write instead | Driver |
34+
| --- | --- | --- |
35+
| `user` | `username` | postgres, mysql, mongo |
36+
| `connectionString` / `dsn` | `url` | postgres, mysql, mongo |
37+
| `uri` | `url` | mongo |
38+
| `file` / `path` / `database` | `filename` | sqlite, sqlite-wasm |
39+
| `hostname` | `host` | postgres, mysql, mongo |
40+
| `searchPath` | `schema` | postgres |
41+
42+
And the relocations — keys that were never driver config:
43+
44+
| Wrote in `config` | Write instead |
45+
| --- | --- |
46+
| `min` / `max` / `idleTimeoutMillis` / `connectionTimeoutMillis` | the datasource's own `pool` block |
47+
| `schemaMode` | next to `driver`, on the datasource |
48+
| `readOnly` | `capabilities: { readOnly: true }` |
49+
| `ssl: { ca, cert, key, rejectUnauthorized }` | the datasource's own `ssl` block — inside `config`, `ssl` is the on/off boolean shorthand |
50+
51+
Two memory-driver keys are **removed**: `indexes` and `maxRecordsPerObject`.
52+
`InMemoryDriverConfig` has no field for either — the driver keeps no indexes and
53+
evicts nothing — so both were inert. Drop them; for real indexing use a driver
54+
that indexes.
55+
56+
A postgres, mysql or mongo datasource must now name a connection target
57+
(`database`, or a `url` that carries it). An empty `config` used to mean "the
58+
client's own localhost default", which is the same defect in its most complete
59+
form.
60+
61+
**Also fixed, because the contract can only be enforced where it is honoured.**
62+
These keys were declared and read by nothing; they now reach the driver:
63+
64+
- `datasource.pool` is honoured by every SQL driver (it was declared, carried
65+
into the connection spec, then overwritten with a hardcoded `{ min: 0, max: 5 }`),
66+
and maps onto the Mongo client's `minPoolSize` / `maxPoolSize`.
67+
- `datasource.schemaMode` reaches the driver. It was dropped between the
68+
datasource record and the connection spec, so a `schemaMode: 'external'`
69+
database — one ObjectStack must never run DDL against — was constructed as
70+
`managed`.
71+
- `datasource.ssl` reaches the SQL clients, certificates and all. It stopped at
72+
the record — nothing put it on the connection spec — so a TLS block configured
73+
nothing, which is exactly what its own schema comment warns about ("a TLS
74+
setting that never took effect looked identical to one that did").
75+
- postgres `schema` (knex `searchPath`), `applicationName` and `statementTimeout`.
76+
- mongo `password`, `authSource` and `options`. A mongo datasource carrying a
77+
`config.password` previously composed its URL with an **empty** password.

content/docs/data-modeling/drivers.mdx

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,56 @@ actually connect to Turso.
7575
> Knex client name (`pg` / `mysql2` / `better-sqlite3`) when you instantiate
7676
> `SqlDriver`.
7777
78+
## `config` is validated per driver
79+
80+
A datasource's `config` is driver-specific — a SQLite `filename` and a Postgres
81+
`host` share no shape — so the datasource schema keeps that slot open at the top
82+
level and parses it against the contract for the driver you named. Each built-in
83+
driver ships that contract as a zod schema, exported from `@objectstack/spec/data`:
84+
85+
| `driver` | Contract | Keys |
86+
| :--- | :--- | :--- |
87+
| `postgres` \| `postgresql` \| `pg` | `PostgresConfigSchema` | `url`, `host`, `port`, `database`, `username`, `password`, `ssl`, `schema`, `applicationName`, `statementTimeout`, `autoMigrate` |
88+
| `mysql` \| `mysql2` \| `mariadb` | `MysqlConfigSchema` | `url`, `host`, `port`, `database`, `username`, `password`, `ssl`, `autoMigrate` |
89+
| `sqlite` \| `sqlite3` | `SqliteConfigSchema` | `filename`, `autoMigrate` |
90+
| `sqlite-wasm` \| `wasm-sqlite` | `SqliteWasmConfigSchema` | `filename`, `persist` |
91+
| `mongo` \| `mongodb` | `MongoConfigSchema` | `url`, `host`, `port`, `database`, `username`, `password`, `authSource`, `options` |
92+
| `memory` \| `in-memory` | `MemoryConfigSchema` | `initialData`, `strictMode`, `persistence` |
93+
94+
An unrecognised key is rejected with its correction, at authoring time and in the
95+
Setup → Datasources wizard alike:
96+
97+
```text
98+
Unrecognized key(s) on this postgres datasource's config: `hostname`.
99+
Did you mean `hostname` → `host`?
100+
```
101+
102+
This matters more than a typical typo check, because the failure it replaces was
103+
silent: a misspelled key was dropped, the driver fell back to its own defaults,
104+
and the datasource connected to `localhost` while every signal — the parse, the
105+
save, the connection probe — reported success.
106+
107+
Two things live **outside** `config`, because they are not driver-specific:
108+
109+
- **Pool sizing** — the `pool` block on the datasource (`min`, `max`,
110+
`idleTimeoutMillis`, `connectionTimeoutMillis`), honoured for every SQL driver
111+
and mapped onto the Mongo client's `minPoolSize` / `maxPoolSize`.
112+
- **TLS certificates** — the `ssl` block on the datasource (`enabled`,
113+
`rejectUnauthorized`, `ca`, `cert`, `key`). Inside `config`, `ssl` is the
114+
on/off boolean shorthand.
115+
- **`schemaMode`** — the ADR-0015 ownership mode, declared next to `driver`.
116+
117+
A plugin-contributed driver (`com.vendor.snowflake`) has no contract in this
118+
repo, so its `config` is left unvalidated rather than judged against a shape the
119+
platform does not have.
120+
121+
<Callout type="info">
122+
The same schemas are projected to JSON Schema for
123+
`DriverDefinitionSchema.configSchema` and for `GET /api/v1/datasources/drivers`,
124+
which the Studio connection form renders — so the form offers exactly the fields
125+
the validator accepts.
126+
</Callout>
127+
78128
## Startup: a driver that cannot connect aborts the boot
79129

80130
`ObjectQLEngine.init()` connects every registered driver during kernel

content/docs/data-modeling/external-datasources.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ export const Warehouse = defineDatasource({
3535
label: 'Analytics Warehouse (Postgres)',
3636
driver: 'postgres',
3737
schemaMode: 'external', // ObjectStack never runs DDL here
38-
config: { host: 'db.internal', port: 5432, database: 'analytics', user: 'readonly' },
38+
config: { host: 'db.internal', port: 5432, database: 'analytics', username: 'readonly' },
3939
external: {
4040
allowWrites: false, // read-only (the default)
4141
credentialsRef: 'sys_secret:9f2c…', // opaque handle minted by the secret store

content/docs/references/api/connector.mdx

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,6 @@ description: Connector protocol schemas
55

66
{/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */}
77

8-
<Callout type="info">
9-
**Source:** `packages/spec/src/api/connector.zod.ts`
10-
</Callout>
11-
128
## TypeScript Usage
139

1410
```typescript

content/docs/references/api/core-services.mdx

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,6 @@ description: Core Services protocol schemas
55

66
{/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */}
77

8-
<Callout type="info">
9-
**Source:** `packages/spec/src/api/core-services.zod.ts`
10-
</Callout>
11-
128
## TypeScript Usage
139

1410
```typescript

content/docs/references/api/http.mdx

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,6 @@ description: Http protocol schemas
55

66
{/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */}
77

8-
<Callout type="info">
9-
**Source:** `packages/spec/src/api/http.zod.ts`
10-
</Callout>
11-
128
## TypeScript Usage
139

1410
```typescript

content/docs/references/api/identity.mdx

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,6 @@ description: Identity protocol schemas
55

66
{/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */}
77

8-
<Callout type="info">
9-
**Source:** `packages/spec/src/api/identity.zod.ts`
10-
</Callout>
11-
128
## TypeScript Usage
139

1410
```typescript

content/docs/references/api/metadata-plugin.mdx

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,6 @@ description: Metadata Plugin protocol schemas
55

66
{/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */}
77

8-
<Callout type="info">
9-
**Source:** `packages/spec/src/api/metadata-plugin.zod.ts`
10-
</Callout>
11-
128
## TypeScript Usage
139

1410
```typescript

content/docs/references/api/notification.mdx

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,6 @@ description: Notification protocol schemas
55

66
{/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */}
77

8-
<Callout type="info">
9-
**Source:** `packages/spec/src/api/notification.zod.ts`
10-
</Callout>
11-
128
## TypeScript Usage
139

1410
```typescript

content/docs/references/api/package-registry.mdx

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,6 @@ description: Package Registry protocol schemas
55

66
{/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */}
77

8-
<Callout type="info">
9-
**Source:** `packages/spec/src/api/package-registry.zod.ts`
10-
</Callout>
11-
128
## TypeScript Usage
139

1410
```typescript

0 commit comments

Comments
 (0)