Skip to content

Commit 33d1842

Browse files
hotlonghotlong
committed
feat: implement package and package version schemas; enhance installation protocols
1 parent a762b39 commit 33d1842

5 files changed

Lines changed: 728 additions & 73 deletions

File tree

‎docs/adr/0003-package-as-first-class-citizen.md‎

Lines changed: 197 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,197 @@
1+
# ADR-0003: Package as First-Class Citizen with Versioned Releases
2+
3+
**Status**: Accepted
4+
**Date**: 2026-04-20
5+
**Deciders**: ObjectStack Protocol Architects
6+
**Supersedes**: The flat `sys_package_installation (package_id + version string)` model introduced alongside ADR-0002
7+
**Consumers**: `@objectstack/spec/cloud`, `@objectstack/service-tenant`, `@objectstack/metadata`, future `service-marketplace`, `service-solution-history`, `service-subscription`
8+
9+
---
10+
11+
## Context
12+
13+
ADR-0002 established the Control Plane / Data Plane split and introduced `sys_package_installation` to track which packages are installed in each environment. That model stores a `package_id` (reverse-domain string) and a `version` (semver string) on the installation row.
14+
15+
Operating this design reveals four structural problems:
16+
17+
1. **Packages have no identity of their own.** There is no `sys_package` row. The platform cannot answer "what packages exist?", "who published them?", or "what is the latest stable version?" without scanning installation records.
18+
19+
2. **Versions are strings, not references.** A version like `"1.2.3"` carries no payload. The metadata objects, views, flows, and migrations that constitute that release live outside the model — there is no atomic snapshot to deploy, validate, or roll back.
20+
21+
3. **Metadata ownership is wrong.** `sys_metadata` currently carries `env_id` to scope schema definitions. But a CRM object definition (`account`, `contact`) belongs to *a specific package version*, not to an environment. Environments only need to record *which version is active* — they should not own the schema.
22+
23+
4. **Upgrade and rollback are not atomic.** "Upgrade env from v1.2.3 to v1.3.0" should be a single pointer swap (`package_version_id`). With the string model it degenerates into multi-row writes with no transactional boundary.
24+
25+
Meanwhile, every mature low-code platform treats packages/solutions as first-class versioned artifacts:
26+
27+
| Platform | Package | Version artifact | Install record |
28+
|---|---|---|---|
29+
| Salesforce | Unlocked Package (`0Ho…`) | Package Version (`04t…`) | Subscriber org row |
30+
| Power Platform | Solution | Solution Version | Solution in Environment |
31+
| ServiceNow | Application | App Version | Installed Application |
32+
| npm / pip / cargo | Package | Published version tarball | `node_modules` / venv |
33+
34+
The common invariant: **a published version is an immutable snapshot**. Installing means pointing an environment at a snapshot; upgrading means pointing at a newer snapshot.
35+
36+
---
37+
38+
## Decision
39+
40+
We introduce a three-layer package model in the Control Plane:
41+
42+
```
43+
Control Plane DB
44+
│
45+
├── sys_package — Package identity (one row per logical package)
46+
├── sys_package_version — Immutable release snapshot (one row per published version)
47+
└── sys_package_installation — Environment ↔ version pairing (replaces old install row)
48+
```
49+
50+
`sys_metadata` gains a `package_version_id` foreign key to express that a metadata record *belongs to a package version*, not to an environment directly.
51+
52+
### sys_package — Package Identity
53+
54+
| Field | Type | Notes |
55+
|---|---|---|
56+
| `id` | UUID | Stable identifier |
57+
| `manifest_id` | text UNIQUE | Reverse-domain e.g. `com.acme.crm` |
58+
| `owner_org_id` | text | Organization that publishes this package |
59+
| `display_name` | text | Human label |
60+
| `description` | text | Short description |
61+
| `visibility` | enum | `private` / `org` / `marketplace` |
62+
| `created_at` | datetime | |
63+
| `updated_at` | datetime | |
64+
65+
### sys_package_version — Immutable Release
66+
67+
| Field | Type | Notes |
68+
|---|---|---|
69+
| `id` | UUID | Stable, never reused |
70+
| `package_id` | FK → sys_package | |
71+
| `version` | text | semver e.g. `1.2.3` |
72+
| `status` | enum | `draft` / `published` / `deprecated` |
73+
| `release_notes` | text | Optional changelog |
74+
| `manifest_json` | JSON | Full package manifest snapshot at publish time |
75+
| `checksum` | text | SHA-256 of `manifest_json` for integrity checks |
76+
| `min_platform_version` | text | Minimum ObjectStack version required |
77+
| `published_at` | datetime | Null while `draft` |
78+
| `published_by` | text | User ID |
79+
| `created_at` | datetime | |
80+
81+
Unique constraint: `(package_id, version)`.
82+
83+
Once `status = 'published'`, `manifest_json` and `checksum` are **immutable**.
84+
85+
### sys_package_installation — Environment ↔ Version Pairing
86+
87+
| Field | Type | Notes |
88+
|---|---|---|
89+
| `id` | UUID | |
90+
| `environment_id` | FK → sys_environment | |
91+
| `package_version_id` | FK → sys_package_version | **replaces** `package_id + version` string pair |
92+
| `status` | enum | `installed` / `installing` / `upgrading` / `disabled` / `error` |
93+
| `enabled` | boolean | Whether metadata is loaded into this env |
94+
| `settings` | JSON | Per-installation config overrides |
95+
| `installed_at` | datetime | |
96+
| `installed_by` | text | |
97+
| `updated_at` | datetime | |
98+
| `error_message` | text | Set when `status = 'error'` |
99+
100+
Unique constraint: `(environment_id, package_id)` — derived via `package_version_id.package_id`. Only one version of a given package may be active per environment at a time.
101+
102+
**Upgrade** = UPDATE `package_version_id` to new version's UUID. The old version row remains intact (audit trail). `upgradeHistory` is removed from the installation row — the history is implicit in the sequence of `updated_at` snapshots and an optional `sys_package_installation_history` log table.
103+
104+
### sys_metadata — Ownership Clarification
105+
106+
`sys_metadata` gains one new optional foreign key:
107+
108+
```
109+
package_version_id FK → sys_package_version nullable
110+
```
111+
112+
Effective query for "what metadata is active in environment E?":
113+
114+
```sql
115+
-- 1. All package-owned metadata from installed versions
116+
SELECT m.*
117+
FROM sys_metadata m
118+
JOIN sys_package_installation i ON i.package_version_id = m.package_version_id
119+
WHERE i.environment_id = :env_id
120+
AND i.enabled = true
121+
122+
UNION ALL
123+
124+
-- 2. Environment-level overrides / customizations
125+
SELECT m.*
126+
FROM sys_metadata m
127+
WHERE m.env_id = :env_id
128+
129+
-- Result: overlay env overrides on top of package metadata (same type+name → env wins)
130+
```
131+
132+
Three ownership tiers:
133+
134+
| `package_version_id` | `env_id` | Meaning |
135+
|---|---|---|
136+
| set | NULL | Belongs to a package version (deployed with the package) |
137+
| NULL | set | Environment-level override or custom metadata |
138+
| NULL | NULL | Platform-built-in / global (e.g. `sys_user` object) |
139+
140+
---
141+
142+
## Migration from the Old Model
143+
144+
1. **Create `sys_package` and `sys_package_version` tables** (additive, non-breaking).
145+
2. **Backfill**: For each distinct `(package_id, version)` string pair found in the old `sys_package_installation`, create one `sys_package` row and one `sys_package_version` row. The `manifest_json` field can be populated lazily (null until the package is re-published through the new flow).
146+
3. **Add `package_version_id` column** to `sys_package_installation`. Populate from the backfill mapping.
147+
4. **Drop** old `package_id` (string) and `version` (string) columns from `sys_package_installation` — in v5.0 after a deprecation window.
148+
5. **Add `package_version_id` column** to `sys_metadata`. Populate for any metadata rows that were installed by a known package version.
149+
150+
The migration is non-destructive and idempotent. Steps 1–4 ship in v4.x as an opt-in; step 4 (column drop) is a v5.0 hard cut.
151+
152+
---
153+
154+
## Consequences
155+
156+
### Positive
157+
158+
- **Package identity is a first-class query.** `GET /cloud/packages` returns the catalog. `GET /cloud/packages/:id/versions` lists all releases.
159+
- **Atomic deploys and rollbacks.** Upgrading or rolling back is a single `UPDATE package_version_id`. No row-level copy jobs.
160+
- **Schema ownership is unambiguous.** An `account` object lives in `sys_metadata` with `package_version_id = <crm-1.2.3>`. It does not belong to any environment — environments only install the version.
161+
- **Marketplace / App Store foundation.** `sys_package.visibility = 'marketplace'` is the hook for the public registry (ADR-0004, future).
162+
- **Integrity guarantees.** `manifest_json + checksum` on a published version means the platform can verify nothing has been tampered with at install time.
163+
- **Clean upgrade audit trail.** The history of `package_version_id` changes on an installation row (plus an optional history table) is authoritative.
164+
165+
### Negative / Trade-offs
166+
167+
- **More join hops** for the effective-schema query (env → installations → versions → metadata). Mitigated by the metadata cache layer in `MetadataManager`.
168+
- **Backfill cost** for existing deployments — `manifest_json` is not available for legacy string-version installations. Lazy population is acceptable for most cases.
169+
- **Draft versions** must not be accidentally installed in production. Enforcement: install API rejects `status != 'published'` unless `allowDraft = true` flag is set (dev/sandbox envs only).
170+
- **One version per package per environment** is a hard constraint. Side-loaded / multi-version installs are explicitly out of scope (same trade-off as npm's `peerDependencies` model).
171+
172+
### Neutral
173+
174+
- No change to `sys_environment`, `sys_environment_member`, or `sys_database_credential`.
175+
- No change to the business data in environment DBs.
176+
- No change to `env_id = NULL` meaning "platform-global" for metadata without a package owner.
177+
- `better-auth` session shape is unchanged.
178+
179+
---
180+
181+
## Alternatives Considered
182+
183+
1. **Keep `package_id + version` strings, add a separate version catalog table but don't FK it.** Rejected — without a hard FK the catalog can drift from installations, defeating the integrity argument.
184+
2. **Embed the full manifest in each installation row.** Rejected — N environments × M packages = N×M copies of the same JSON. The version table is the single source of truth.
185+
3. **Move package versioning entirely to the filesystem / Git.** Rejected — query-ability (list installed packages, filter by status, detect conflicts) requires a database-backed model.
186+
4. **Allow multiple active versions of the same package per environment.** Rejected — conflict resolution between overlapping metadata definitions is intractable. One version per package per env, same as every comparable platform.
187+
188+
---
189+
190+
## References
191+
192+
- `packages/spec/src/cloud/environment-package.zod.ts` — current installation schema (to be updated)
193+
- `packages/services/service-tenant/src/objects/sys-package-installation.object.ts` — DB object (to be updated)
194+
- ADR-0002: `docs/adr/0002-environment-database-isolation.md` — Control Plane / Data Plane split
195+
- Salesforce Unlocked Packages: <https://developer.salesforce.com/docs/atlas.en-us.pkg2_dev.meta/pkg2_dev/>
196+
- Power Platform Solution Layers: <https://learn.microsoft.com/power-platform/alm/solution-layers-alm>
197+
- ServiceNow Application Management: <https://docs.servicenow.com/bundle/washingtondc-application-development/page/build/applications/concept/application-management.html>

0 commit comments

Comments
 (0)