Skip to content

Commit deb0f0f

Browse files
styk-tvstyk-tv
committed
parked: v3.7-external-deltas.md — actionable briefs for #14, #15, #16, #24
Consolidated artefact for the four tasks that fall outside the v3.7 documentation overhaul itself: #14 — `static` kernel type missing from external SPEC.CK.NEW #15 — 10-step reconciliation lifecycle: external SPEC missing deploy.security and deploy.graph #16 — 18+12 module enumeration: external SPEC treats both libraries as opaque #24 — CK.Operator code drift: RO-FS action.save(), hostname synthesis, dual kopf handlers Each task section is structured for direct use: - "v3.7 spec position" with file:line citations from this site so the external doc can quote the canonical statement - "Discrepancy" naming the gap exactly - "Recommended insertion into SPEC.CK.NEW" — paste-ready text blocks for #14, #15, #16 - "Code-side fix brief" — paste-ready dev ticket bodies for the three #24 sub-items, each citing the v3.7 contract being violated Bonus: a small internal-drift note in #15 about changelog.md showing a 12-step versioned breakdown without deploy.security while reconciliation.md is the canonical 10-step lifecycle. Flagged for a follow-up clarification pass on this site, kept separate from the external delta work. The file lives at parked/v3.7-external-deltas.md (repo root, outside docs/), with frontmatter ingest:false / visibility:internal / status:external. VitePress never builds it, sitemap.xml never indexes it, llms.txt never references it. It is a working artefact for the user's external SPEC.CK.NEW reconciliation and for the operator-dev briefs.
1 parent ed21140 commit deb0f0f

1 file changed

Lines changed: 278 additions & 0 deletions

File tree

‎parked/v3.7-external-deltas.md‎

Lines changed: 278 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,278 @@
1+
---
2+
title: v3.7 External Deltas — for SPEC.CK.NEW reconciliation and operator-dev briefs
3+
status: external
4+
ingest: false
5+
visibility: internal
6+
revisit: pending
7+
---
8+
9+
# v3.7 External Deltas
10+
11+
Tasks **#14, #15, #16, #24** from the v3.7 documentation overhaul. These items are not changes to the published v3.7 spec docs — they are notes and recommendations targeted at:
12+
13+
- the user's external `SPEC.CK.NEW` (#14, #15, #16)
14+
- the `CK.Operator` codebase (#24)
15+
16+
Each section is self-contained and citation-ready. Paste blocks of "Recommended insertion" text directly into the target document; paste "Code-side fix brief" blocks into a developer ticket.
17+
18+
This file is held outside `docs/v3.7/` and `docs/public/` so it is never served by VitePress, never indexed by `sitemap.xml`, and never referenced by `llms.txt`.
19+
20+
---
21+
22+
## #14 — `static` kernel type missing from external SPEC.CK.NEW
23+
24+
### v3.7 spec position
25+
26+
`static` is one of five kernel types in the v3.7 normative `qualities.type` enum. It is used by at least one system kernel and described consistently across the spec:
27+
28+
| Citation (v3.7) | Statement |
29+
|---|---|
30+
| `docs/v3.7/crd.md:49` | ConceptKernel CRD `spec.type` enum: `["node:hot", "node:cold", "inline", "static", "agent"]` |
31+
| `docs/v3.7/conformance.md:137` | "One of four deployment profiles: `node:hot` (always-on), `node:cold` (on-demand), `inline` (browser-side), `static` (filer-served, no process)." |
32+
| `docs/v3.7/bfo-grounding.md:80` | `static` ↔ `ckp:StaticKernel`, "Filer-served web only — No process — gateway serves files — No NATS (read-only web surface)" |
33+
| `docs/v3.7/auth.md:48` | Kernel Type Authentication Requirements: `static` row, "None / None / No NATS connection" |
34+
| `docs/v3.7/message-envelope.md:267` | `static` row, "No NATS connection / None / None / None" |
35+
| `docs/v3.7/taxonomy.md:24` | Declarator archetype example: `CK.Project` is `qualities.type: static` |
36+
| `docs/v3.7/project.md:14,19` | "CK.Project is a `static` kernel because it holds only declarations; it has no processor that executes actions at runtime." |
37+
| `docs/public/ontology/v3.7/core.ttl` | `ckp:StaticKernel` is a subClassOf `ckp:Kernel` |
38+
39+
Internal note: `conformance.md:137` says **four** deployment profiles, omitting `agent`. That sentence is itself a v3.7 internal drift bug (the enum has five). It should be fixed independently to: "One of five deployment profiles: `node:hot`, `node:cold`, `inline`, `static`, `agent`." — but it does *include* `static`.
40+
41+
### Discrepancy
42+
43+
External `SPEC.CK.NEW §1.2` and the §7 manifest template list four kernel types (no `static`). v3.7 needs five.
44+
45+
### Recommended insertion into SPEC.CK.NEW
46+
47+
Add a fifth kernel type to the §1.2 enum and the §7 manifest template:
48+
49+
> ```yaml
50+
> qualities:
51+
> type: static # filer-served, no process; gateway serves storage/web/ directly
52+
> ```
53+
>
54+
> A `static` kernel:
55+
> - has **no** processor pod and **no** NATS connection
56+
> - is served by the project gateway from its `tool/web/` (or equivalent) DATA-organ path
57+
> - is the canonical type for **declaration-only** kernels (e.g., CK.Project, which materialises a `.ckproject` manifest but does not execute actions at runtime)
58+
> - is mapped to `ckp:StaticKernel` in `core.ttl`
59+
>
60+
> Conformance: implementations that omit `static` cannot represent declaration-only kernels. CKP requires support for all five types.
61+
62+
---
63+
64+
## #15 — 10-step reconciliation lifecycle: external SPEC missing `deploy.security` and `deploy.graph`
65+
66+
### v3.7 spec position (canonical 10-step lifecycle)
67+
68+
`docs/v3.7/reconciliation.md` defines the lifecycle. `docs/v3.7/operator.md:67-76` repeats it as the high-level overview. Both agree on 10 steps in this order:
69+
70+
| # | Step | Purpose | v3.7 citation |
71+
|---|---|---|---|
72+
| 1 | `deploy.namespace` | Create namespace `ck-{subdomain}` | `reconciliation.md:33-37` |
73+
| 2 | **`deploy.security`** | ServiceAccount + 4 NetworkPolicies (default-deny-all, allow-nats, allow-dns, allow-gateway); `automountServiceAccountToken: false` | `reconciliation.md:39-51` |
74+
| 3 | `deploy.storage` | PV/PVC per three-loop spec (CK ReadOnlyMany + DATA ReadWriteMany) | `reconciliation.md:53-62` |
75+
| 4 | `deploy.processors` | ConfigMap (`boot.py`) + Deployment for processor pods | `reconciliation.md:64-68` |
76+
| 5 | `deploy.web` | Generated `index.html` + web Deployment + Service | `reconciliation.md:70-81` |
77+
| 6 | `deploy.routing` | HTTPRoute via Gateway API `parentRef` | `reconciliation.md:83-94` |
78+
| 7 | `deploy.ck_resources` | ConceptKernel CRs per kernel | `reconciliation.md:96-99` |
79+
| 8 | `deploy.auth` | Keycloak realm reuse / `KeycloakRealmImport` create | `reconciliation.md:~100`, `auth.md` Step-by-Step Execution |
80+
| 9 | **`deploy.graph`** | RDF materialisation to Jena Fuseki `/ckp` dataset, named graph per project (`urn:ckp:fleet:{hostname}`) | `reconciliation.md:124-148`, `graph.md` |
81+
| 10 | `deploy.endpoint` | External endpoint verification (HTTP 200) | `reconciliation.md:~150` |
82+
83+
### Internal v3.7 drift (bonus finding)
84+
85+
`docs/v3.7/changelog.md:73-86` shows a "Reconciliation Lifecycle (Versioned -- 12 Steps)" that splits storage into `deploy.materialise` + `deploy.storage.ck` + `deploy.storage.tool` + `deploy.storage.data`, and **omits `deploy.security`** entirely from the listing. This is an internal documentation drift bug independent of the external-spec reconciliation:
86+
87+
- `reconciliation.md` is the canonical lifecycle (10 logical steps)
88+
- `changelog.md` shows a versioned-implementation breakdown (12 implementation steps for the v3.7 multi-version materialisation feature)
89+
- The two should cross-reference each other and clarify which is the protocol-level lifecycle vs. which is the implementation breakdown — currently a reader hits both and cannot tell
90+
91+
Suggest adding to `changelog.md:73` a sentence: "This 12-step breakdown is the implementation expansion of the canonical 10-step lifecycle from [Reconciliation](./reconciliation), with `deploy.materialise` introduced and `deploy.storage` split per organ. The canonical lifecycle remains 10 steps." (Tracked as a candidate Pass-3 follow-up; not part of this externals delta.)
92+
93+
### Discrepancy with external SPEC
94+
95+
External `SPEC.CK.NEW §4` defines a 5-check contract. The two steps missing relative to v3.7's 10:
96+
97+
- **`deploy.security`** — default-deny NetworkPolicies per project namespace
98+
- **`deploy.graph`** — RDF materialisation to Jena Fuseki at `/ckp`
99+
100+
### Recommended insertion into SPEC.CK.NEW
101+
102+
Extend §4 from the 5-check minimum to the v3.7 10-step lifecycle (table above). The two new entries:
103+
104+
> **`deploy.security`** (between `deploy.namespace` and `deploy.storage`)
105+
>
106+
> Per-project Kubernetes namespace isolation via:
107+
> - `ServiceAccount: ckp-runtime` with `automountServiceAccountToken: false` (kernel pods cannot reach the Kubernetes API)
108+
> - `NetworkPolicy: ckp-default-deny` denying all ingress and egress
109+
> - `NetworkPolicy: ckp-allow-nats` permitting egress to NATS service on port 4222
110+
> - `NetworkPolicy: ckp-allow-dns` permitting egress to kube-dns
111+
> - `NetworkPolicy: ckp-allow-gateway` permitting ingress from the gateway namespace on port 80
112+
>
113+
> Rationale: defence in depth. A compromised kernel pod cannot reach services outside the allowed egress list, cannot inspect cluster state via the Kubernetes API, and cannot accept traffic from outside its declared gateway.
114+
>
115+
> **`deploy.graph`** (between `deploy.auth` and `deploy.endpoint`)
116+
>
117+
> RDF materialisation. After all kernel resources exist, CK.Operator publishes kernel metadata, edges, and instance pointers as RDF triples to a SPARQL endpoint (reference implementation: Jena Fuseki, dataset `/ckp`). Each project's fleet lives in a named graph identified as `urn:ckp:fleet:{hostname}`. Graph materialisation is best-effort: `deploy.endpoint` verification MUST succeed even if the SPARQL endpoint is unreachable.
118+
>
119+
> Rationale: the ontological graph is the queryable index of fleet state alongside the Kubernetes API.
120+
121+
---
122+
123+
## #16 — 18-module CK.Lib.Py + 12-module CK.Lib.Js: external SPEC treats as opaque
124+
125+
### v3.7 spec position (precise enumeration)
126+
127+
**CK.Lib.Py — 18 modules** (from `docs/v3.7/project.md:144-165`):
128+
129+
| # | Module | Ontology Class | Purpose |
130+
|---|---|---|---|
131+
| 1 | `processor.py` | `ckp:Kernel` | Base class for all kernel processors |
132+
| 2 | `events.py` | `ckp:Action` | Action handler decorator (`@on`) and typed event emission |
133+
| 3 | `instance.py` | `ckp:InstanceManifest`, `ckp:SealedInstance` | Instance lifecycle: create, seal, ledger, proof |
134+
| 4 | `ledger.py` | `ckp:LedgerEntry` | Kernel-wide append-only JSONL action log |
135+
| 5 | `schema.py` | `ckp:SealedInstance` validation | LinkML `ontology.yaml` loading and validation |
136+
| 6 | `context.py` | N/A | Three-loop-aware context builder |
137+
| 7 | `capacity.py` | N/A | Concurrency control via file-based capacity management |
138+
| 8 | `occurrent.py` | `ckp:Occurrent` (BFO:0000015) | Action substeps with PROV-O proof and hash chains |
139+
| 9 | `edges.py` | `ckp:Edge`, `ckp:RelationshipType` | Edge subscription materialisation for NATS |
140+
| 10 | `actions.py` | `ckp:Action`, `rbac.ttl` | Action type resolution, composition, RBAC |
141+
| 11 | `dispatch.py` | `ckp:EdgeCommunication` | Local inter-kernel action dispatch and FIFO queue |
142+
| 12 | `serve.py` | `ckp:WebServing`, `ckp:APIServing` | FastAPI HTTP server wrapping `@on` handlers |
143+
| 13 | `auth.py` | `rbac.ttl` | Keycloak JWT verification + API token fallback |
144+
| 14 | `urn.py` | CKP URN scheme | URN parser, validator, builder (17 entity types) |
145+
| 15 | `prov.py` | `prov:Activity` | PROV-O session recording for any kernel |
146+
| 16 | `execution.py` | `ckp:ProofRecord`, `ckp:ProofCheck` | Multi-step execution proofs with hash chains |
147+
| 17 | `entities.py` | N/A | In-memory entity store with code-based lookup |
148+
| 18 | `nats_loop.py` | `ckp:NATSListening` | NATS subscriber loop implementing CK processing cycle |
149+
150+
**CK.Lib.Js — 12 modules** (from `docs/v3.7/project.md:249-262`):
151+
152+
| # | Module | Export Path | Ontology Grounding | Purpose |
153+
|---|---|---|---|---|
154+
| 1 | `ck-client.js` | `.` / `./client` | `ckp:NATSBrowserClient` | Core NATS WSS client; connection, auth, send, subscribe |
155+
| 2 | `ck-page.js` | `./page` | `ckp:InlineKernel` | Page harness; auto-detects kernel, renders chrome |
156+
| 3 | `ck-bus.js` | `./bus` | `ckp:EdgeCommunication` | In-browser event bus for decoupled component communication |
157+
| 4 | `ck-kernel.js` | `./kernel` | — | Kernel-specific CSS variables and theming |
158+
| 5 | `ck-registry.js` | `./registry` | `ckp:Project` | Kernel registry for multi-kernel page composition |
159+
| 6 | `ck-runtime.js` | `./runtime` | — | Client-side runtime utilities |
160+
| 7 | `ck-materializer.js` | `./materializer` | — | Client-side resource materialisation |
161+
| 8 | `ck-store.js` | `./store` | — | Client-side state persistence (localStorage + server filer) |
162+
| 9 | `ck-shapes.js` | `./shapes` | `sh:NodeShape` | SHACL shape rendering and validation UI |
163+
| 10 | `ck-anim.js` | `./anim` | — | Animation engine for kernel visualisations |
164+
| 11 | `ck-anim-grammar.js` | (internal) | — | Animation grammar parser |
165+
| 12 | `ck-sound.js` | `./sound` | — | Web Audio API integration |
166+
167+
### Discrepancy with external SPEC
168+
169+
External `SPEC.CK.NEW` treats both `CK.Lib.Py` and `CK.Lib.Js` as opaque — referring to them by name without enumerating modules. This means any divergence in module count or naming between v3.7 and `SPEC.CK.NEW` is silent: an implementer cannot conform to one and validate against the other.
170+
171+
### Recommended insertion into SPEC.CK.NEW
172+
173+
Add (or replace) the libraries section with citation-style references back to v3.7:
174+
175+
> **CK.Lib.Py** — the canonical Python runtime library; npm-equivalent package `conceptkernel` on PyPI. Conformant implementations MUST provide all 18 modules listed in the v3.7 specification, with the ontology class bindings declared per `docs/v3.7/project.md:144-165`. The 18 modules are: `processor`, `events`, `instance`, `ledger`, `schema`, `context`, `capacity`, `occurrent`, `edges`, `actions`, `dispatch`, `serve`, `auth`, `urn`, `prov`, `execution`, `entities`, `nats_loop`.
176+
>
177+
> **CK.Lib.Js** — the canonical JavaScript client library; npm package `@conceptkernel/cklib`. Conformant implementations MUST provide all 12 modules listed in the v3.7 specification, with export paths and ontology grounding per `docs/v3.7/project.md:249-262`. The 12 modules are: `ck-client`, `ck-page`, `ck-bus`, `ck-kernel`, `ck-registry`, `ck-runtime`, `ck-materializer`, `ck-store`, `ck-shapes`, `ck-anim`, `ck-anim-grammar`, `ck-sound`.
178+
179+
### Verification step (do this on the actual code, separate from the spec edit)
180+
181+
The 18+12 module list above is taken from `project.md`. Before relying on it, verify against the live packages:
182+
183+
```bash
184+
# Python (in a checkout that has cklib/)
185+
python -c "import cklib, pkgutil; print(sorted(m.name for m in pkgutil.iter_modules(cklib.__path__)))"
186+
187+
# JS (in a checkout with @conceptkernel/cklib installed)
188+
node -e "console.log(Object.keys(require('@conceptkernel/cklib/package.json').exports))"
189+
```
190+
191+
If the actual exports differ, treat that as a separate bug and reconcile both v3.7 docs and the external spec to the live truth.
192+
193+
---
194+
195+
## #24 — Operator-implementation drift (code briefs, not spec changes)
196+
197+
These three items are runtime/code bugs in `CK.Operator`. Each violates a clear v3.7 spec contract; the spec is correct, the code disagrees. Hand these as discrete dev tickets.
198+
199+
### #24.a — RO-FS violation: `action.save()` writes to a ReadOnly organ
200+
201+
**Reported**: `action.save()` writes proof artefacts to `/ck/CK.Operator/storage/proof/`. `storage/` is also a stale path (renamed `data/` in v3.6); the more important defect is that proofs are landing in the **CK** organ (ReadOnlyMany at runtime) rather than the **DATA** organ (ReadWriteMany).
202+
203+
**Spec contract violated**:
204+
205+
| Citation | Statement |
206+
|---|---|
207+
| `docs/v3.7/proof.md:256` | "Action proofs MUST be stored in the kernel's `data/proof/` directory as `proof-{action}-{timestamp}.json`." |
208+
| `docs/v3.7/proof.md:281` | Conformance: "Proofs MUST be stored in `data/proof/` — REQUIRED" |
209+
| `docs/v3.7/three-loops.md`, `docs/v3.7/isolation.md` | CK organ is ReadOnlyMany at runtime, no exceptions (post-v3.7 retirement of `serving.json`). |
210+
211+
**Code-side fix brief**:
212+
213+
> In CK.Operator, change `action.save()` to write to `/ck/CK.Operator/data/proof/proof-{action}-{timestamp}.json` instead of `/ck/CK.Operator/storage/proof/`. The current path is wrong on both axes:
214+
> - It targets the **CK organ**, which is mounted ReadOnlyMany at runtime (post-v3.7 retirement of `serving.json` there are no write-through exceptions). The current `EROFS` failure is the kernel's defence working correctly.
215+
> - It uses the legacy `storage/` directory name; v3.7 renamed this to `data/`.
216+
>
217+
> The DATA organ is mounted ReadWriteMany; `data/proof/` is the canonical proof location per `docs/v3.7/proof.md:256,281`. After the fix, proofs are sealed instances and inherit the standard sealed-instance immutability contract documented in `docs/v3.7/data-loop.md`.
218+
219+
### #24.b — Hostname synthesis bug
220+
221+
**Reported**: a code path ignores `serving.hostname` (or `spec.hostname`) and synthesises a hostname from other fields.
222+
223+
**Spec contract violated**:
224+
225+
| Citation | Statement |
226+
|---|---|
227+
| `docs/v3.7/project.md` Required Fields table | `domain` (string, REQUIRED), `serving.subdomain` (string, REQUIRED). The `.ckproject` manifest declares serving identity. |
228+
| `docs/v3.7/operator.md:107` | CKProject CR example shows `spec.hostname: hello.tech.games` as a top-level required field. |
229+
| `docs/v3.7/crd.md:449` | CKProject CRD conformance: "`spec.hostname`, `spec.storage`, `spec.gateway.parentRef`, and `spec.versions` MUST be present — REQUIRED". |
230+
| `docs/v3.7/crd.md` CKProject schema | `spec.hostname` is the canonical fully-qualified hostname; `domain` and `serving.subdomain` are derivation sources, optional in the CR but required (in some form) in the manifest. |
231+
232+
**Code-side fix brief**:
233+
234+
> `spec.hostname` is the authoritative project hostname per the CKProject CRD schema (`docs/v3.7/crd.md` §CKProject CRD). CK.Operator MUST read it directly and use it for:
235+
> - DNS / serving subdomain
236+
> - Filer path prefix (`/ck-data/{hostname}/...`)
237+
> - Namespace naming (derived: `ck-{serving.subdomain}` where subdomain is the hostname's leftmost label, unless `serving.subdomain` is explicitly set)
238+
> - All gateway HTTPRoute hostnames
239+
>
240+
> If `spec.hostname` is absent, the reconcile MUST fail with a clear error before any resources are created, not silently synthesise a fallback. Synthesising hides operator-config errors and produces hostnames that will conflict with whatever `.ckproject` later declares.
241+
>
242+
> Spot-check after the fix: `kubectl get ckp -A -o jsonpath='{range .items[*]}{.metadata.name}{":"}{.spec.hostname}{" → ns="}{.status.namespace}{"\n"}{end}'` should show one-to-one alignment between `.spec.hostname`'s leftmost label and the namespace prefix.
243+
244+
### #24.c — Dual kopf handlers for the same CR
245+
246+
**Reported**: two kopf handlers fire on the same `CKProject` CR event.
247+
248+
**Spec contract violated** (interpretation, since the spec doesn't enumerate handler counts directly):
249+
250+
| Citation | Statement |
251+
|---|---|
252+
| `docs/v3.7/operator.md:212-217` | "Dual Control Plane: kopf + NATS" — two **entry points** (kopf CRD watch and NATS message listener) trigger the **same `reconcile()` function**. The dual entry refers to *transports*, not duplicated handlers on the same transport. |
253+
| `docs/v3.7/operator.md:33` | `kopf operator watching CKProject CR` is described as a single watcher. |
254+
| `docs/v3.7/changelog.md:59` | "kopf + NATS dual control plane" — explicitly two entry points triggering one `reconcile()`. |
255+
256+
The spec does not contemplate two kopf handlers competing for the same CR event, and the dual-control-plane design assumes a single `reconcile()` is the convergence point. Two handlers means non-deterministic ordering, possible double-creation, and split log/proof streams.
257+
258+
**Code-side fix brief**:
259+
260+
> Per `docs/v3.7/operator.md:212-217` and `changelog.md:59`, the kopf+NATS dual control plane is two **entry points** converging on a single `reconcile()` function. There should be exactly one kopf handler per CR kind:
261+
> - `@kopf.on.create('ck.tech.games', 'v1', 'ckprojects')`
262+
> - `@kopf.on.update('ck.tech.games', 'v1', 'ckprojects')`
263+
> - `@kopf.on.delete('ck.tech.games', 'v1', 'ckprojects')`
264+
>
265+
> All three must call into the same `reconcile()` function (with the action variant — `project.deploy`, `project.update`, `project.teardown` — selected by handler kind). The NATS listener calls the same `reconcile()` for the same actions; there is one convergence point.
266+
>
267+
> Audit step: `grep -rnE '@kopf\.on\.|kopf\.handler' <operator-src>` should produce one handler per (kind, verb), not multiple. If there are two `@kopf.on.create('ckprojects')` decorators in the codebase, that is the bug.
268+
269+
---
270+
271+
## How to use this document
272+
273+
1. Treat each `### Recommended insertion into SPEC.CK.NEW` block (in #14, #15, #16) as paste-ready text for the external spec.
274+
2. Treat each `**Code-side fix brief**` block (in #24.a/b/c) as a paste-ready dev ticket description.
275+
3. The internal v3.7 drift noted in #15 (changelog 12-step breakdown without `deploy.security`) is a candidate for a small follow-up Pass on this site, separate from the external work.
276+
4. Once external reconciliations land in `SPEC.CK.NEW` and operator fixes land in `CK.Operator`, the corresponding tasks (#14, #15, #16, #24) can be closed.
277+
278+
This file is a working artefact, not a specification. Do not link it from any published CKP page.

0 commit comments

Comments
 (0)