Skip to content

Commit 85b818a

Browse files
committed
feat: update documentation and tests to reflect removal of metadata DB bridge in ObjectOS
1 parent 82173f4 commit 85b818a

10 files changed

Lines changed: 145 additions & 108 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1717
- **M1 — Project Artifact envelope schema (`@objectstack/spec`)** — Introduced the v0 `ProjectArtifactSchema` in `packages/spec/src/system/project-artifact.zod.ts`, the immutable envelope that `objectstack compile` will produce and ObjectOS will consume at boot. Required fields: `schemaVersion` (literal `'0.1'`), `projectId`, `commitId`, `checksum` (`{ algorithm, value }`), `metadata` (per-category arrays, `passthrough()` for forward compatibility), `functions` (inlined source with optional language/source/hash), and `manifest` (plugin / driver / engine requirements). Optional `builtAt`, `builtWith`, and a reserved `payloadRef` (`{ url, expiresAt, checksum }`) for future S3 indirection without an envelope bump. 14 new tests in `project-artifact.test.ts`. Resolves ROADMAP M1; unblocks M3 (Artifact API) and M4 (ObjectOS artifact loader).
1818

1919
### Changed
20+
- **D1 — ObjectOS metadata DB bridge removed**`MetadataPlugin` no longer registers `sys_metadata` / `sys_metadata_history` into the ObjectOS manifest and no longer auto-bridges ObjectQL to `DatabaseLoader` during `start()`. Runtime metadata is now file/artifact backed; database-backed metadata persistence remains an explicit `MetadataManager` capability for control-plane services. Updated ROADMAP, North Star, Metadata Service docs, and ObjectStack skills to reflect the boundary.
2021
- **D5 — `ManifestSchema.scope` enum trimmed to `'cloud' | 'system' | 'project'`** — Removed the deprecated `'platform'` and `'environment'` aliases from `packages/spec/src/kernel/manifest.zod.ts`. No call site in the workspace was setting the deprecated values, so this is a clean break. Resolves ROADMAP D5.
2122
- **D4 — `ObjectSchemaBase.namespace` removed** — Object identity is now single-sourced on `name`. The deprecated `namespace` field has been removed from `ObjectSchemaBase` (`packages/spec/src/data/object.zod.ts`); legacy inputs that still set `namespace` are silently stripped by Zod's default object behavior. Package-level namespace (FQN computation, marketplace publishing, `DatasourceRoutingRule.namespace`) is intentionally retained as an internal mechanic. Five legacy `ObjectSchema namespace` test blocks were rewritten as a single `name-as-identity` block. Resolves ROADMAP D4.
2223
- **D7 — Plugin manifest header + objects unified per package** — Each plugin / service now exposes a single canonical `src/manifest.ts` file that both `objectstack.config.ts` (compile-time) and the runtime `*-plugin.ts` (`manifest.register()`) import from. This eliminates a real divergence in `plugin-auth` and `plugin-security` whose configs imported from a non-existent `./src/objects/` directory and silently shipped empty object lists at compile time while their runtime registrations were intact. Affected packages: `@objectstack/plugin-auth`, `@objectstack/plugin-security`, `@objectstack/service-tenant`. Resolves ROADMAP D7.

ROADMAP.md

Lines changed: 8 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# ObjectStack - Road Map
22

3-
> **Last Updated:** 2026-04-26 (Phase 1 foundation: M1 + D4 + D5 + D7 landed)
3+
> **Last Updated:** 2026-04-26 (Phase 1 foundation: M1 + D1 + D4 + D5 + D7 landed)
44
> **Authoritative Spec:** [content/docs/concepts/north-star.mdx](content/docs/concepts/north-star.mdx) - §7 Alignment Check is the single source of truth for Built / Drift / Missing.
55
> This file is the **actionable checklist** derived from that ledger. When north-star §7 changes, update this file too.
66
@@ -49,33 +49,14 @@ Code that exists and matches the intended architecture. Do not regress these.
4949
| JSON-payload metadata column (`sys_metadata.metadata` textarea) | [packages/metadata/src/objects/sys-metadata.object.ts](packages/metadata/src/objects/sys-metadata.object.ts) |
5050
| CLI `publish` - local JSON -> remote server wire (endpoint shape still wrong, see D2) | [packages/cli/src/commands/publish.ts](packages/cli/src/commands/publish.ts) |
5151
| **M1** Project Artifact envelope schema (`schemaVersion / projectId / commitId / checksum / metadata / functions / manifest`) | [packages/spec/src/system/project-artifact.zod.ts](packages/spec/src/system/project-artifact.zod.ts) |
52+
| **D1** ObjectOS metadata DB bridge removed - `MetadataPlugin` no longer registers `sys_metadata` / `sys_metadata_history` or auto-bridges ObjectQL to `DatabaseLoader` | [packages/metadata/src/plugin.ts](packages/metadata/src/plugin.ts) |
5253

5354
---
5455

5556
## 🟡 Drift (Needs Cleanup)
5657

5758
Existing code that contradicts the intended Phase 1 architecture. Fix these before building new surface area that depends on them.
5859

59-
### D1 - 🟡 MetadataPlugin maintains a DB bridge in ObjectOS (Biggest Drift)
60-
61-
**Priority: P0.** The primary startup read has been partially fixed — `local-file` mode (M1.x) lets ObjectOS boot from `dist/objectstack.json` without touching any database. However, [packages/metadata/src/plugin.ts](packages/metadata/src/plugin.ts) still:
62-
63-
1. Registers `SysMetadataObject` and `SysMetadataHistoryObject` into the **ObjectOS** manifest — these tables belong only in `apps/cloud` (control plane).
64-
2. Bridges ObjectQL to a `DatabaseLoader` via `setDataEngine()` after startup — subsequent `metadata.register()` writes persist metadata to `sys_metadata` in the project DB, and `metadata.list()` / `get()` aggregate from it.
65-
66-
Under Phase 1 the project DB contains **business rows only**. Metadata lives in the control plane and is delivered to ObjectOS as an immutable artifact. The DB bridge breaks that boundary and undermines the `commitId` + `checksum` immutability guarantee — `sys_metadata` rows in the project DB may diverge from the published artifact.
67-
68-
**What is already done:**
69-
- `local-file` artifact mode — ObjectOS boots from `dist/objectstack.json` without any DB read (M1.x). ✅
70-
71-
**Remaining fix path:**
72-
1. Remove `SysMetadataObject` / `SysMetadataHistoryObject` registration from `MetadataPlugin.init()` — these tables belong in `apps/cloud` only.
73-
2. Remove the `setDataEngine()` / `DatabaseLoader` bridge from `MetadataPlugin.start()` — ObjectOS metadata is read-only (artifact → kernel); history and versioning are control-plane concerns.
74-
3. Implement the Artifact API endpoint (M3) and swap `MetadataPlugin`'s source to HTTP fetch (M4).
75-
4. Confirm `sys_metadata` is absent from the project DB schema after the D8 split into `apps/cloud` + `apps/server`.
76-
77-
**Depends on:** M3, M4. **Prerequisite for:** D8 clean split.
78-
7960
### D2 - `objectstack publish` uses legacy `/api/v1/packages` endpoint
8061

8162
[packages/cli/src/commands/publish.ts](packages/cli/src/commands/publish.ts) POSTs a "package" payload that is not the Phase 1 project metadata endpoint.
@@ -204,7 +185,7 @@ Tests: [packages/spec/src/system/project-artifact.test.ts](packages/spec/src/sys
204185
- [ ] Data migration script for existing installations.
205186
- [ ] Keep project DBs for business rows only.
206187

207-
**Prerequisite for:** M3, D1 fix, D3.
188+
**Prerequisite for:** M3, D3.
208189

209190
### M3 - Project Artifact API endpoint
210191

@@ -214,16 +195,16 @@ Tests: [packages/spec/src/system/project-artifact.test.ts](packages/spec/src/sys
214195
- [ ] Response includes `commitId` and `checksum`.
215196
- [ ] Reserve response shape for future `{ url, expiresAt, checksum }` indirection, but do not build S3 yet.
216197

217-
**Prerequisite for:** M4, D1 fix.
198+
**Prerequisite for:** M4.
218199

219200
### M4 - ObjectOS artifact loader
220201

221-
- [ ] Swap `MetadataPlugin` data source: project-DB reads -> HTTP fetch against Artifact API.
202+
- [ ] Add `MetadataPlugin` production source: HTTP fetch against Artifact API.
222203
- [ ] Validate artifact with Zod before hydrating kernel.
223204
- [ ] Local artifact cache with durability across control-plane outages.
224205
- [ ] Cache key by `projectId` + `commitId`/`checksum`.
225206

226-
**Resolves:** D1.
207+
**Completes:** production ObjectOS artifact source.
227208

228209
### M5 - Project publish endpoint
229210

@@ -306,7 +287,7 @@ M1 Artifact format v0
306287
├── M1.x Runtime Inputs 边界化 (Artifact + Deployment Config 分离)
307288
└── M2 Metadata migration to control plane
308289
├── M3 Project Artifact API
309-
│ └── M4 ObjectOS artifact loader -> resolves D1
290+
│ └── M4 ObjectOS artifact loader
310291
└── M5 Project publish endpoint -> resolves D2
311292
└── M6 Studio metadata/artifact viewer
312293
@@ -327,4 +308,4 @@ D8 split apps/cloud + apps/server(after M3/M4 make ObjectOS standalone)
327308
| [.github/copilot-instructions.md](.github/copilot-instructions.md) | Mirror of CLAUDE.md for Copilot |
328309
| [packages/cli/src/commands/compile.ts](packages/cli/src/commands/compile.ts) | TS -> JSON compile (Built anchor) |
329310
| [packages/cli/src/commands/publish.ts](packages/cli/src/commands/publish.ts) | Publish command (Drift D2 target) |
330-
| [packages/metadata/src/plugin.ts](packages/metadata/src/plugin.ts) | MetadataPlugin (Drift D1 target) |
311+
| [packages/metadata/src/plugin.ts](packages/metadata/src/plugin.ts) | MetadataPlugin (artifact/local-file metadata loader; D1 resolved anchor) |

content/docs/concepts/implementation-status.mdx

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -65,12 +65,13 @@ This matrix is generated from actual codebase analysis and represents the curren
6565
### Metadata Framework
6666

6767
<Callout type="info">
68-
**Degraded Status**Metadata is an in-memory registry framework. Database persistence for metadata storage is not yet implemented. Discovery reports this service as `degraded`.
68+
**Runtime Boundary**ObjectOS metadata is file/artifact-backed and read-only at boot. Database-backed metadata persistence exists for explicitly configured control-plane services, not as an automatic ObjectOS project-DB bridge.
6969
</Callout>
7070

7171
- Metadata API (list types, list items, get item) is fully functional
72-
- Metadata is loaded from config files at startup and held in memory
73-
- Future: metadata persistence to database for runtime modifications
72+
- Metadata is loaded from config files or `dist/objectstack.json` at startup and held in memory
73+
- `MetadataPlugin` no longer registers `sys_metadata` / `sys_metadata_history` into ObjectOS or auto-bridges ObjectQL to `DatabaseLoader`
74+
- Future: production Artifact API loader and durable local artifact cache
7475

7576
### System Services
7677

content/docs/concepts/north-star.mdx

Lines changed: 8 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -169,9 +169,10 @@ data rows.
169169

170170
**Status:** Multi-project kernel + hostname routing built. `local-file` artifact
171171
boot (M1.x) implemented — ObjectOS can boot from `dist/objectstack.json` without
172-
any DB read. Remaining drift: `MetadataPlugin` still bridges ObjectQL to a
173-
`DatabaseLoader` after startup, persisting metadata to and reading from the
174-
project DB. This is D1. See [packages/runtime/src/project-kernel-factory.ts](packages/runtime/src/project-kernel-factory.ts)
172+
any DB read. D1 is resolved: `MetadataPlugin` no longer registers
173+
`sys_metadata` / `sys_metadata_history` into the ObjectOS manifest and no longer
174+
auto-bridges ObjectQL to `DatabaseLoader` after startup. See
175+
[packages/runtime/src/project-kernel-factory.ts](packages/runtime/src/project-kernel-factory.ts)
175176
and [packages/metadata/src/plugin.ts](packages/metadata/src/plugin.ts).
176177

177178
## 4. Core Entity Hierarchy
@@ -201,7 +202,7 @@ Organization <- control plane
201202
| :----- | :--------------- | :----- |
202203
| Organization | Control plane `sys_organization` | Built |
203204
| Project | Control plane `sys_project` (hostname binding) | Built |
204-
| Metadata State | Control-plane metadata payloads + project revision (`commit_id`, checksum), scoped by `organization_id` + `project_id` | Partially fixed: startup read now artifact-based (M1.x); `DatabaseLoader` bridge still persists metadata to project DB post-startup (D1) |
205+
| Metadata State | Control-plane metadata payloads + project revision (`commit_id`, checksum), scoped by `organization_id` + `project_id` | ObjectOS DB bridge removed (D1); control-plane migration still tracked by M2 |
205206
| Artifact | `GET /api/v1/apps/:projectId/artifact` response | To build |
206207
| Business DB | Per-project dedicated DB (Turso / SQL / memory) | Built (driver provisioning), but currently shares space with metadata |
207208
| Branch / Release | Future control-plane entities | Deferred |
@@ -326,7 +327,7 @@ identity would require federated auth, which is out of scope.
326327
- Per-project kernel factory: [packages/runtime/src/project-kernel-factory.ts](packages/runtime/src/project-kernel-factory.ts)
327328
- HTTP dispatch: [packages/runtime/src/http-dispatcher.ts](packages/runtime/src/http-dispatcher.ts)
328329
- Org-scoped proxy: [packages/runtime/src/control-plane-proxy-driver.ts](packages/runtime/src/control-plane-proxy-driver.ts)
329-
- Metadata loading (startup read fixed for `local-file` mode; DB bridge still persists to project DB post-startup — D1): [packages/metadata/src/plugin.ts](packages/metadata/src/plugin.ts)
330+
- Metadata loading (artifact/local-file backed; no automatic ObjectQL -> `DatabaseLoader` bridge in ObjectOS): [packages/metadata/src/plugin.ts](packages/metadata/src/plugin.ts)
330331
- App catalog sync: [packages/services/service-tenant/src/services/app-catalog.service.ts](packages/services/service-tenant/src/services/app-catalog.service.ts)
331332

332333
## 6. Key Lifecycles
@@ -457,23 +458,12 @@ reduce Drift, and advance Missing.
457458
- **TS -> JSON compile pipeline.** `objectstack compile` validates a local TS stack against `ObjectStackDefinitionSchema` and writes `dist/objectstack.json` - [packages/cli/src/commands/compile.ts](packages/cli/src/commands/compile.ts).
458459
- **Zod -> JSON Schema publishing pipeline.** The bridge that makes TS/JSON interchangeable - [packages/spec/scripts/build-schemas.ts](packages/spec/scripts/build-schemas.ts).
459460
- **Scaffolded TS file tree.** `create-objectstack` emits a `defineStack()` entry point plus split-out `src/objects/*.ts`, `src/views/*.ts` etc. - [packages/create-objectstack/src/index.ts](packages/create-objectstack/src/index.ts).
461+
- **ObjectOS metadata DB bridge removed.** `MetadataPlugin` no longer registers `sys_metadata` / `sys_metadata_history` into the ObjectOS manifest and no longer auto-connects ObjectQL to a `DatabaseLoader`; runtime metadata is hydrated from files or artifacts only - [packages/metadata/src/plugin.ts](packages/metadata/src/plugin.ts).
460462
- **JSON-payload metadata storage.** `sys_metadata.metadata` column is already a textarea-shaped JSON payload column - [packages/metadata/src/objects/sys-metadata.object.ts](packages/metadata/src/objects/sys-metadata.object.ts). (The *location* of the table still needs to move to the control plane - see Drift.)
461463
- **CLI `publish` link.** The end-to-end "local JSON -> remote server" wire is alive, even though the endpoint shape is wrong - [packages/cli/src/commands/publish.ts](packages/cli/src/commands/publish.ts).
462464

463465
### <AlertTriangle className="inline" /> Drift (Needs Cleanup)
464466

465-
- **Biggest drift: `MetadataPlugin` maintains a DB bridge in ObjectOS.**
466-
The primary startup read has been partially fixed — `local-file` mode (M1.x)
467-
lets ObjectOS boot from `dist/objectstack.json` without any DB read. However,
468-
[packages/metadata/src/plugin.ts](packages/metadata/src/plugin.ts) still
469-
registers `sys_metadata`/`sys_metadata_history` tables into the ObjectOS
470-
manifest and bridges ObjectQL to a `DatabaseLoader` after startup. This means
471-
subsequent `metadata.register()` writes persist to `sys_metadata` in the
472-
project DB, and `metadata.list()` / `get()` aggregate from it. Under Phase 1,
473-
the project DB must contain **business rows only** — metadata lives in the
474-
control plane, delivered via the Artifact API as an immutable artifact. The DB
475-
bridge breaks the `commitId` / `checksum` immutability guarantee: project DB
476-
rows may silently diverge from the published artifact.
477467
- **`env_id` should be removed from metadata storage.** In Phase 1 metadata is
478468
scoped by `organization_id` + `project_id`; deployment target differences are
479469
runtime/deployment configuration, not metadata row partitioning. Branch-like
@@ -502,13 +492,11 @@ reduce Drift, and advance Missing.
502492

503493
### <Globe className="inline" /> Missing (Not Started)
504494

505-
- **Artifact format v0** - Zod schema for `{ schemaVersion, projectId, commitId, checksum, metadata, functions, manifest }`.
506495
- **Metadata migration to control plane** - move user metadata out of project DBs into the control-plane DB, scoped by `organization_id` + `project_id`.
507496
- **Project Artifact API** - `GET /api/v1/apps/:projectId/artifact` assembles the current project metadata + inlined function code into a single consumable blob, with content hash/ETag for cache validation.
508-
- **ObjectOS artifact loader** - swap `MetadataPlugin`'s data source from project-DB reads to HTTP fetch against the Artifact API; add local artifact cache with durability across control-plane outages.
497+
- **ObjectOS Artifact API loader** - add the production HTTP fetch source for `MetadataPlugin` and local artifact cache durability across control-plane outages.
509498
- **Project publish endpoint** - `POST /api/v1/apps/:projectId/metadata` receives compiled JSON, validates with Zod, writes the current project metadata state, and returns `commitId` + checksum.
510499
- **Studio metadata/artifact viewer** - browse published metadata, artifact envelope, commit id, checksum, publish history, logs, and runtime health. No metadata editing.
511-
- **`objectstack dev` offline boot path** - `from-local-file` kernel boot. ObjectOS reads `dist/objectstack.json` (or an in-memory TS definition) and runs fully without a control-plane connection.
512500
- **UI auto-generation** - artifact schemas -> Amis/React components without hand-wiring.
513501

514502
## 8. Non-Goals and Anti-Patterns

0 commit comments

Comments
 (0)