|
1 | 1 | # Spec liveness ledger |
2 | 2 |
|
3 | 3 | For a metadata-driven platform, **the spec is the product surface**: authors write |
4 | | -metadata against these Zod schemas. A property that is parsed but has no runtime |
5 | | -consumer is a silent no-op — and for a *security* property, a silent no-op is |
6 | | -**false compliance** (e.g. `forceMfa: true` accepted and ignored). The |
7 | | -metadata-liveness audits (`docs/audits/2026-06-*-property-liveness.md`) found that |
8 | | -large swaths of the declared surface are DEAD. |
| 4 | +metadata against these schemas. A property that is parsed but has no runtime consumer |
| 5 | +is a silent no-op — and for a *security* property, a silent no-op is **false |
| 6 | +compliance** (e.g. `forceMfa: true` accepted and ignored). The metadata-liveness audits |
| 7 | +(`docs/audits/2026-06-*-property-liveness.md`) found that large swaths of the declared |
| 8 | +surface are DEAD. |
9 | 9 |
|
10 | | -This ledger makes that classification **explicit and regression-proof**: in a |
11 | | -*governed* category, every authorable property must declare a liveness status with |
12 | | -evidence, or CI fails (the ratchet — you can't add new undeclared surface). |
| 10 | +This ledger makes that classification **explicit and regression-proof**: every property |
| 11 | +of a governed metadata type must declare a liveness status with evidence, or CI fails |
| 12 | +(the ratchet — you can't add new undeclared surface). |
13 | 13 |
|
14 | | -## Status vocabulary |
15 | | - |
16 | | -| Status | Meaning | |
17 | | -|---|---| |
18 | | -| `live` | Has a runtime consumer. Cite it in `evidence` (`file:line`, or a test). | |
19 | | -| `experimental` / `planned` | Declared, intentionally not enforced yet. Also recognised from a spec `.describe()` marker like `[EXPERIMENTAL — not enforced]`. | |
20 | | -| `dead` | Parsed, no consumer. Tracked for **enforce-or-remove** (cite the audit/grep). | |
21 | | -| `internal` *(schema-level)* | Not authorable metadata (runtime result/DTO, context, enum). Exempt. | |
22 | | - |
23 | | -Resolution order per property: **ledger entry → spec `.describe()` marker → UNCLASSIFIED**. |
24 | | -A schema-level `"_schema": "<status>"` applies to all its properties (used for |
25 | | -wholesale-dead subtrees like `PasswordPolicy`, or `internal` runtime types). |
26 | | -Caveat: a `_schema`-classified schema also absorbs *new* properties at that status, |
27 | | -so the ratchet does not flag additions to a wholesale-dead/internal subtree — only |
28 | | -additions to per-property schemas (the mixed ones like `ObjectPermission`, |
29 | | -`PermissionSet`). Use `_schema` only for subtrees that are genuinely all-one-status. |
30 | | - |
31 | | -## Two governance modes |
| 14 | +## Source of truth = the metadata-type registry |
32 | 15 |
|
33 | | -A category's ledger picks how the gate scopes it: |
| 16 | +The gate reads `BUILTIN_METADATA_TYPE_SCHEMAS` (`packages/spec/src/kernel/metadata-type-schemas.ts`) |
| 17 | +via `listMetadataTypeSchemaTypes()` / `getMetadataTypeSchema()` — **the same registry the |
| 18 | +runtime `/api/v1/meta/types/:type` endpoint and the Studio metadata-admin forms use**, |
| 19 | +i.e. exactly the set of *authorable* metadata types. It walks each type's Zod schema |
| 20 | +directly (not `z.toJSONSchema`, which throws on `object`/`action`). |
34 | 21 |
|
35 | | -- **default** — *every* authorable object schema in the category must be classified |
36 | | - (`"_schema": "internal"` exempts non-authorable ones). Right for clean, |
37 | | - fully-authorable categories: `security`, `identity`. |
38 | | -- **allowlist** (`"mode": "allowlist"` + `"governed": ["Agent","Tool","Skill"]`) — |
39 | | - only the named schemas are checked; the rest of the category is out of scope. |
40 | | - Right for categories dominated by protocol/engine/runtime DTOs where the |
41 | | - authorable types are a small subset: `ai` (Agent/Tool/Skill among embedder/ |
42 | | - knowledge/model DTOs). A `governed` name that no longer resolves to a schema is |
43 | | - reported (catches renames that would silently drop coverage). |
| 22 | +This matters: the older gate read the generated `json-schema/` directory, which omits |
| 23 | +most top-level authorable types (object/field/flow/action/...) — so it was blind to the |
| 24 | +core surface. The registry is complete. |
44 | 25 |
|
45 | | -## Framework fields (auto-classified) |
46 | | - |
47 | | -The ADR-0010 provenance/lock overlay fields — `_lock`, `_lockReason`, `_lockSource`, |
48 | | -`_lockDocsUrl`, `_provenance`, `_packageId`, `_packageVersion`, `protection` — appear |
49 | | -on every authorable type and are system-stamped, not type-specific surface. The gate |
50 | | -auto-classifies them `live`, so ledgers don't repeat them. |
| 26 | +## Status vocabulary |
51 | 27 |
|
52 | | -## Files |
| 28 | +| Status | Meaning | |
| 29 | +|---|---| |
| 30 | +| `live` | Has a runtime consumer. Cite it in `evidence` (`file:line`; objectui-repo paths as prose to avoid false stale-flags). | |
| 31 | +| `experimental` / `planned` | Declared, intentionally not enforced yet. Also read from a spec `.describe()` marker like `[EXPERIMENTAL — not enforced]`. | |
| 32 | +| `dead` | Parsed, no consumer. Tracked for **enforce-or-remove** (ADR-0049). | |
| 33 | + |
| 34 | +Resolution per property: **ledger entry → spec `.describe()` marker → UNCLASSIFIED**. |
| 35 | +Framework provenance/lock fields (`_lock*`, `_provenance`, `_packageId/Version`, |
| 36 | +`protection` — ADR-0010) are auto-classified `live`. |
| 37 | + |
| 38 | +## Granularity — drill one level |
| 39 | + |
| 40 | +A property is classified at the top level by default. A **container** property (object / |
| 41 | +record / array-of-object) may be drilled one level via `"children"` to keep sub-properties |
| 42 | +distinguishable — e.g. `permission.objects.allowCreate` (live) vs `allowTransfer` (experimental), |
| 43 | +or `flow.errorHandling.fallbackNodeId` (dead) vs the rest (live). Drill only where the |
| 44 | +audit gives divergent sub-statuses; otherwise the top-level entry covers the whole subtree. |
| 45 | + |
| 46 | +```jsonc |
| 47 | +// packages/spec/liveness/permission.json |
| 48 | +{ "type": "permission", "props": { |
| 49 | + "name": { "status": "live", "evidence": "packages/plugins/plugin-security/src/permission-evaluator.ts" }, |
| 50 | + "objects": { "children": { |
| 51 | + "allowCreate": { "status": "live", "evidence": "permission-evaluator.ts:8" } |
| 52 | + // allowTransfer/Restore/Purge omitted → resolved 'experimental' via spec marker |
| 53 | + } } |
| 54 | +} } |
| 55 | +``` |
53 | 56 |
|
54 | | -- `<category>.json` — the ledger for a governed category (`security`, `identity`, `ai`). |
55 | | -- `../scripts/liveness/check-liveness.mjs` — the gate. Reads the generated |
56 | | - `packages/spec/json-schema/<category>/*.json`, resolves each authorable |
57 | | - property's status, and exits non-zero on any UNCLASSIFIED property. |
| 57 | +## Files & usage |
58 | 58 |
|
59 | | -## Usage |
| 59 | +- `<type>.json` — the ledger for a governed metadata type. |
| 60 | +- `../scripts/liveness/check-liveness.mts` — the gate (tsx; imports the registry). |
60 | 61 |
|
61 | 62 | ```bash |
62 | | -pnpm --filter @objectstack/spec gen:schema # produce json-schema/ (the source of truth) |
63 | | -pnpm --filter @objectstack/spec check:liveness # run the gate |
64 | | -node packages/spec/scripts/liveness/check-liveness.mjs --dump security # inventory a category (seeding aid) |
| 63 | +pnpm --filter @objectstack/spec check:liveness # run the gate |
| 64 | +tsx packages/spec/scripts/liveness/check-liveness.mts --dump field # inventory a type (seeding aid) |
65 | 65 | ``` |
66 | 66 |
|
67 | | -CI: `.github/workflows/spec-liveness-check.yml` runs the gate on PRs touching |
68 | | -`packages/spec/**`. |
69 | | - |
70 | | -## Rolling out the next category |
71 | | - |
72 | | -Governed categories are listed in `GOVERNED` at the top of `check-liveness.mjs`, |
73 | | -rolled out **highest-risk-first**. To add one (e.g. `automation`, `ui`, `data`): |
74 | | - |
75 | | -1. `--dump <category>` to inventory its authorable properties. **The json-schema |
76 | | - categories do NOT map to "authorable types"** — most (`data`, `automation`, `ui`, |
77 | | - `kernel`) are dominated by ObjectQL/engine/protocol DTOs, and some authorable |
78 | | - types live elsewhere (Agent/Tool/Skill in `ai`, Dataset in `ui`). Decide the |
79 | | - handful of authorable schemas and use **allowlist mode** unless the whole |
80 | | - category is authorable. |
81 | | -2. Seed `<category>.json` from that category's liveness audit (file:line evidence) |
82 | | - and targeted greps for anything the audit didn't cover. **Classify only with |
83 | | - evidence** — `live` needs a cited consumer; `dead` needs a confirmed absence. |
84 | | -3. Add the category to `GOVERNED` and confirm the gate is green. |
85 | | - |
86 | | -## Current state |
87 | | - |
88 | | -| Category | Mode | Properties | Notes | |
89 | | -|---|---|---|---| |
90 | | -| `security` | default | 93 (26 live / 1 exp / 66 dead)* | ~71% parsed-but-unenforced; enforce-or-remove worklist | |
91 | | -| `identity` | default | 4 (3 live / 1 dead) | `Role` authorable; rest internal (SCIM/auth runtime) | |
92 | | -| `ai` | allowlist | 63 (46 live / 5 exp / 12 dead) | Agent/Tool/Skill; `Tool` is write-only, agent access-control dead | |
93 | | - |
94 | | -\* security numbers shift to 26 / 35 / 32 once the PolicySchema experimental |
95 | | -markers (ADR-0049 #1882) land. |
96 | | - |
97 | | -The `dead` entries are the cross-category enforce-or-remove worklist (ADR-0049). |
98 | | -Highest-signal: the destructive `ObjectPermission.allow{Transfer,Restore,Purge}` |
99 | | -(ungated), the entirely-dead `Policy` tree, and **agent `access`/`permissions`/ |
100 | | -`visibility`** — "who can chat with this agent" is a no-op (the chat route hardcodes |
101 | | -`['ai:chat','ai:agents']`). |
| 67 | +CI: `.github/workflows/spec-liveness-check.yml` runs on PRs touching `packages/spec/**`. |
| 68 | + |
| 69 | +## Adding a type |
| 70 | + |
| 71 | +The governed set is `GOVERNED` at the top of `check-liveness.mts`. To add a type: |
| 72 | + |
| 73 | +1. `--dump <type>` to inventory its properties (containers auto-expand so you can see |
| 74 | + drill-down candidates). |
| 75 | +2. Seed `<type>.json` from that type's liveness audit (file:line evidence) + targeted |
| 76 | + greps. **Classify only with evidence** — `live` needs a cited consumer; `dead` needs a |
| 77 | + confirmed absence. |
| 78 | +3. Add the type to `GOVERNED`; confirm the gate is green. |
| 79 | + |
| 80 | +## Current state — 10 governed types (~295 properties) |
| 81 | + |
| 82 | +| Type | live | exp | dead | Notes | |
| 83 | +|---|---|---|---|---| |
| 84 | +| object | 31 | – | 17 | `enable`/ObjectCapabilities + versioning/partitioning/cdc tier dead; `apiEnabled` unenforced | |
| 85 | +| field | 34 | – | 39 | ~half dead — aspirational enhanced-type + governance config; naming-drift props server-live/client-snake | |
| 86 | +| flow | 29 | 1 | 7 | `runAs` experimental (unenforced identity switch); status/active gate nothing; FlowNodeAction enum out of sync | |
| 87 | +| action | 26 | – | 5 | `disabled` CEL ignored (renderers read non-spec `enabled`); type:'form'/shortcut/bulkEnabled dead | |
| 88 | +| hook | 11 | – | 2 | model-healthy — near-total liveness; only label/description dead | |
| 89 | +| permission | 23 | 3 | 2 | CRUD/FLS/RLS live; allow{Transfer,Restore,Purge} experimental; isProfile/contextVariables dead | |
| 90 | +| role | 3 | – | 1 | `parent` dead (org hierarchy uses sys_department) | |
| 91 | +| agent | 18 | 4 | 5 | access/permissions/visibility dead (chat route hardcodes perms); autonomy experimental | |
| 92 | +| tool | 13 | 1 | 5 | write-only metadata; runtime uses a parallel AIToolDefinition | |
| 93 | +| skill | 15 | – | 2 | triggerPhrases dead (no matcher); permissions dead | |
| 94 | + |
| 95 | +The `dead` set across types is the enforce-or-remove worklist (ADR-0049). Not yet governed |
| 96 | +(rollout): view, page, dashboard, app, report, dataset, job, datasource, translation, |
| 97 | +email_template, doc, book, validation, seed. |
0 commit comments