Skip to content

Commit 17cb734

Browse files
hyperpolymathclaude
hyperpolymath
and
claude
committed
docs(architecture): rewrite ARCHITECTURE.md to reflect v1.1.0 stack
Replaces the stale V-lang/triple-API description with the actual current architecture: Idris2 ABI → Zig FFI (.so) → boj-invoke dlopen → Elixir REST (port 7700). Key corrections: - Three-layer stack → canonical four-layer stack with Elixir REST layer - Removes Zig HTTP adapter / V-lang adapter references - Documents the ADR-0006 five-symbol ABI and seven return codes - Documents both dispatch paths (boj-invoke for .so; JsWorkerPool for mod.js transitional) and the cartridge.json selector - Documents HTTP surface (5 routes + pubkey endpoint; gRPC/GraphQL/SSE marked as not yet live) - Documents ECDH credential forwarding format - Documents cartridge production process via launch-scaffolder; flags cartridge-minter as legacy (Node.js, banned) - Thread safety: attributes BEAM concurrency to Elixir layer; retains Zig mutex discipline as defence-in-depth description - Removes V-lang references from Unified-Zig-API section Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 9d9c629 commit 17cb734

1 file changed

Lines changed: 219 additions & 106 deletions

File tree

‎docs/ARCHITECTURE.md‎

Lines changed: 219 additions & 106 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,14 @@
11
# BoJ Server Architecture
22

3+
<!-- SPDX-License-Identifier: PMPL-1.0-or-later -->
4+
<!-- Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) <j.d.a.jewell@open.ac.uk> -->
5+
<!-- Last updated: 2026-04-25 — reflects v1.1.0 Elixir REST + Deno transitional stack -->
6+
37
## The Problem
48

5-
The developer server ecosystem is fragmenting. MCP servers, LSP servers, DAP servers, build servers — each tool has its own server, each AI needs different capabilities. Developers drown in configuration. AI agents hunt across dozens of endpoints.
9+
The developer server ecosystem is fragmenting. MCP servers, LSP servers, DAP servers,
10+
build servers — each tool has its own server, each AI needs different capabilities.
11+
Developers drown in configuration. AI agents hunt across dozens of endpoints.
612

713
**BoJ solves this.** AI goes to ONE place — the Teranga menu — and orders what it needs.
814

@@ -17,68 +23,235 @@ BoJ organises server capabilities as a 2D matrix:
1723
```
1824
MCP LSP DAP BSP NeSy Agent Fleet gRPC REST
1925
+------+------+------+------+------+------+------+------+------+
20-
Cloud | | | | | | | | | |
21-
Container | | | | | | | | | |
26+
Cloud | * | | | | | | | | |
27+
Container | * | | | | | | | | |
2228
Database | * | * | | | | | | | |
23-
K8s | | | | | | | | | |
24-
Git/VCS | | | | | | | | | |
29+
K8s | * | | | | | | | | |
30+
Git/VCS | * | | | | | | | | |
2531
Secrets | * | | | | | | | | |
26-
Queues | | | | | | | | | |
27-
IaC | | | | | | | | | |
32+
Queues | * | | | | | | | | |
33+
IaC | * | | | | | | | | |
2834
Observe | * | | | | | | | | |
29-
SSG | | | | | | | | | |
30-
Proof | | | | | | | | | |
31-
Fleet | * | | | | | | * | * | |
32-
NeSy | * | * | | | * | | | * | |
35+
SSG | * | | | | | | | | |
36+
Proof | * | | | | | | | | |
37+
Fleet | * | | | | * | * | * | | |
38+
NeSy | * | * | | | * | | | | |
3339
+------+------+------+------+------+------+------+------+------+
3440
35-
* = cartridge exists (may be Development or Ready)
41+
* = cartridge exists (Zig .so or mod.js transitional)
3642
```
3743

38-
The matrix is sparse — not every cell needs to be filled. The most common use case is MCP+LSP, but the architecture supports any combination.
44+
The matrix is sparse — not every cell needs to be filled. The most common use case
45+
is MCP, but the architecture supports any combination.
3946

40-
## Three-Layer Stack (per cartridge)
47+
## Canonical Four-Layer Stack
4148

42-
Every cartridge follows the same three-layer pattern:
49+
Every cartridge follows the same pattern from proof to execution:
4350

4451
| Layer | Language | Purpose |
4552
|-------|----------|---------|
46-
| **ABI** | Idris2 | Formal proofs, state machines, `%default total`, zero `believe_me` |
47-
| **FFI** | Zig | C-compatible native execution, zero runtime dependencies |
48-
| **Adapter** | Zig | Triple API (REST + gRPC + GraphQL) on dedicated ports (see note) |
49-
50-
> **Note (2026-04-10):** V-lang was banned estate-wide on 2026-04-10. The adapter
51-
> layer previously used V-lang; it has been replaced by Zig (`ffi/zig/` adapters,
52-
> swept in commit c4674f8). Historical V-lang API interfaces live in
53-
> `developer-ecosystem/v-ecosystem/v-api-interfaces/` for potential donation to the
54-
> V community — they are not HP infrastructure. The long-term target for the adapter
55-
> layer is the **unified-zig-api stack** (`developer-ecosystem/zig-api/`); see
56-
> [§ Unified-Zig-API Stack](#unified-zig-api-stack) and ADR-0002.
53+
| **ABI** | Idris2 | Formal proofs; `IsUnbreakable`; ADR-0006 5-symbol interface |
54+
| **FFI** | Zig | C-ABI `.so` shared library; 5 symbols + `cartridge_shim` |
55+
| **Dispatcher** | Zig (`boj-invoke`) | `dlopen` the `.so`; single CLI call; zero persistent process |
56+
| **REST server** | Elixir (Plug/Cowboy) | HTTP surface on port 7700; routes calls to Invoker or JsWorkerPool |
5757

5858
### Why these languages?
5959

60-
**Idris2** has dependent types that prove interface correctness at compile-time. The `IsUnbreakable` proof type mathematically guarantees that only `Ready` cartridges can be activated. This isn't aspirational — it's enforced by the type checker.
60+
**Idris2** has dependent types that prove interface correctness at compile-time. The
61+
`IsUnbreakable` proof type mathematically guarantees that only `Ready` cartridges can
62+
be activated. This isn't aspirational — it's enforced by the type checker.
63+
64+
**Zig** provides native C ABI compatibility without runtime overhead. It bridges the
65+
gap between Idris2's proofs and actual system calls. Cross-compilation is built-in,
66+
which matters for community nodes running on varied hardware.
67+
68+
**Elixir/OTP** provides the HTTP surface, supervision tree, ETS-backed catalog, and
69+
BEAM concurrency — replacing the former V-lang adapter layer (V-lang was banned
70+
estate-wide 2026-04-10; all `.v` files removed 2026-04-12, commit c4674f8).
71+
72+
## The ADR-0006 Cartridge ABI
73+
74+
Every compiled cartridge exposes exactly five C-ABI symbols:
75+
76+
```
77+
boj_cartridge_init() → i32 initialise module state
78+
boj_cartridge_deinit() → void release module state
79+
boj_cartridge_name() → [*c]u8 null-terminated name string
80+
boj_cartridge_version() → [*c]u8 semver string
81+
boj_cartridge_invoke(
82+
tool_name: [*c]const u8,
83+
json_args: [*c]const u8,
84+
out_buf: [*c]u8,
85+
in_out_len:[*c]usize,
86+
) → i32
87+
```
88+
89+
Return codes (frozen by ADR-0006; new failure modes use error JSON, not new integers):
90+
91+
| Code | Meaning |
92+
|------|---------|
93+
| `0` | success |
94+
| `-1` | unknown tool |
95+
| `-2` | bad args (null pointer) |
96+
| `-3` | buffer too small (retry with larger buffer) |
97+
| `-4` | runtime error |
98+
| `-5` | panic |
99+
| `-6` | auth denied |
100+
101+
The `cartridge_shim.zig` in every `ffi/` directory centralises null-guards,
102+
tool-name comparison, and the buffer-too-small retry pattern so domain `.zig` files
103+
stay short (tool table + `shim.writeResult`).
104+
105+
## Dispatch Path
106+
107+
### Zig FFI path (canonical — boj-invoke dlopen)
108+
109+
```
110+
POST /cartridge/<name>/invoke
111+
→ BojRest.Invoker
112+
→ boj-invoke probe|invoke <so_path> <tool> <json> (OS exec)
113+
→ dlopen(so_path) → boj_cartridge_invoke(...)
114+
→ JSON response back via stdout
115+
→ Elixir reads stdout → 200 OK
116+
```
117+
118+
The `boj-invoke` binary is a single-shot CLI. Each invocation opens the `.so`,
119+
calls the symbol, writes JSON to stdout, and exits. There is no persistent FFI
120+
process — the `.so` is loaded and unloaded per call. This is suitable for
121+
correctness and simplicity; hot-path caching is future work.
122+
123+
### Deno/JS path (transitional — pending .so completion)
124+
125+
```
126+
POST /cartridge/<name>/invoke
127+
→ BojRest.JsWorkerPool
128+
→ consistent-hash(mod_js_path) → JsWorker (persistent Deno process)
129+
→ send JSON over port → mod.js handleTool(tool, args)
130+
→ JSON response
131+
→ Elixir reads response → 200 OK
132+
```
133+
134+
The `mod.js` files serve as behavioural reference implementations for what each
135+
cartridge's Zig FFI symbols must produce. They are **transitional** — as each
136+
cartridge's `.so` is compiled and verified, the Elixir Router switches to the
137+
Zig FFI path automatically (presence of `"ffi"` key in `cartridge.json` controls
138+
which path is taken). Once all 112 cartridges have verified `.so` builds,
139+
the Deno pool is retired.
140+
141+
**Dispatch selector (BojRest.Router):**
142+
```elixir
143+
if Map.has_key?(cart, "ffi") do
144+
BojRest.Invoker.invoke(so_path, tool, args, creds)
145+
else
146+
BojRest.JsWorkerPool.invoke(mod_js_path, tool, args)
147+
end
148+
```
149+
150+
## The HTTP Surface (port 7700)
151+
152+
Five routes, all returning `application/json`:
153+
154+
| Method | Path | Description |
155+
|--------|------|-------------|
156+
| `GET` | `/health` | Server status, version, cartridges_loaded |
157+
| `GET` | `/menu` | Teranga menu — name/domain/tier/description per cartridge |
158+
| `GET` | `/cartridges` | Name list with count |
159+
| `GET` | `/cartridge/:name` | Full cartridge.json for a single cartridge |
160+
| `POST` | `/cartridge/:name/invoke` | Dispatch a tool call |
161+
| `GET` | `/.well-known/boj-node-pubkey` | Node's X25519 public key (base64url) |
162+
163+
gRPC (7701), GraphQL (7702), and SSE (7703) are **not yet live** — tracked as
164+
future work in ROADMAP.adoc.
165+
166+
## Credential Forwarding
167+
168+
The invoke endpoint accepts an optional `"credentials"` envelope in the request body.
169+
Two paths:
170+
171+
- **Loopback (`is_local = true`)**: plaintext string→string map accepted
172+
- **Remote**: must be ECDH-encrypted with ChaCha20-Poly1305 using the node's X25519 key
173+
174+
```json
175+
{
176+
"tool": "search",
177+
"arguments": {"query": "..."},
178+
"credentials": {
179+
"v": 1,
180+
"encrypted": true,
181+
"caller_pubkey": "<base64url X25519 ephemeral pub>",
182+
"nonce": "<base64url 12 bytes>",
183+
"ciphertext": "<base64url ciphertext+tag>"
184+
}
185+
}
186+
```
187+
188+
`BojRest.CredentialDecryptor.extract/2` handles both paths. Private key material
189+
never appears in error messages (verified by `aspect_test.exs` security tests).
190+
191+
## Cartridge Production Process
192+
193+
New cartridges are produced via the **launch-scaffolder** (`launch-scaffolder` repo):
194+
195+
```
196+
launch-scaffolder mint <name> <domain> <tier>
197+
→ generates cartridge.json + Idris2 ABI + Zig FFI scaffold
198+
199+
launch-scaffolder provision <name>
200+
→ compiles .so, runs boj-invoke probe, updates cartridge.json ffi section
201+
202+
launch-scaffolder config <name>
203+
→ configures auth method, credential requirements, loopback port
61204
62-
**Zig** provides native C ABI compatibility without runtime overhead. It bridges the gap between Idris2's proofs and actual system calls. Cross-compilation is built-in, which matters for community nodes running on varied hardware. Zig now covers both the FFI layer and the adapter (HTTP server) layer.
205+
launch-scaffolder realign <name>
206+
→ re-applies standards (ADR-0006 ABI, spdx header, schema version)
207+
```
208+
209+
All 112 cartridges have the scaffold. Status:
210+
- **1 complete** (boj-health): Zig source compiled, `.so` verified, in production
211+
- **111 in progress**: Zig source present and substantial (100–1,260 lines each),
212+
`.so` not yet compiled — transitional `mod.js` in use
213+
214+
> **Note:** `tools/cartridge-minter/` exists but uses Node.js (banned language).
215+
> The `launch-scaffolder` is the canonical tool; cartridge-minter is a legacy artefact.
63216
64217
## The Teranga Menu
65218

66-
The menu (`.machine_readable/servers/menu.a2ml`) is the public catalogue of available capabilities. It has three sections:
219+
The menu (`.machine_readable/servers/menu.a2ml`) is the public catalogue of available
220+
capabilities. Three sections:
67221

68222
- **Teranga (Core)**: Cartridges maintained by the project
69223
- **Shield**: Privacy and security cartridges (SDP, DoQ/DoH, oDNS)
70224
- **Ayo (Community)**: Community-contributed cartridges
71225

72-
AI agents act as the "Maitre D'" — presenting the menu to users, taking their order, and having the kitchen prepare it.
226+
AI agents act as the "Maitre D'" — presenting the menu to users, taking their order,
227+
and having the kitchen prepare it.
73228

74229
## The Order-Ticket Protocol
75230

76231
1. AI reads the Teranga menu (`menu.a2ml`)
77232
2. AI writes an order ticket (`order-ticket.scm`)
78-
3. BoJ validates the order against the catalogue (checks `IsUnbreakable`)
79-
4. BoJ mounts requested cartridges via Zig FFI
80-
5. Zig adapter exposes mounted cartridges as REST+gRPC+GraphQL
81-
6. AI receives confirmation with endpoints
233+
3. BoJ validates the order against the catalogue (`IsUnbreakable` check)
234+
4. BoJ routes invoke calls via `BojRest.Router`:
235+
- Cartridge has `"ffi"` key → `BojRest.Invoker` → `boj-invoke dlopen`
236+
- Cartridge has no `"ffi"` key → `BojRest.JsWorkerPool` → Deno `mod.js`
237+
5. Results returned as JSON; credentials forwarded encrypted end-to-end
238+
6. AI receives tool output
239+
240+
## Thread Safety
241+
242+
**BEAM layer (Elixir):** Concurrency is handled by OTP. Each invoke request runs in
243+
its own Elixir process. `BojRest.Catalog` uses a named ETS table (`:boj_catalog`)
244+
for lock-free concurrent reads. `BojRest.NodeKey` uses `:boj_node_key` ETS for the
245+
same reason. `BojRest.JsWorkerPool` serialises per-worker via `GenServer.call/3`.
246+
247+
**Zig FFI layer:** Each `.so`'s domain module uses `std.Thread.Mutex` to guard
248+
global state, following the two-tier convention:
249+
- `pub export fn boj_<domain>_*` — acquires mutex; called by `boj-invoke` / external C
250+
- `fn <domain>_*_impl` — mutex-free; called by other functions already holding the lock
251+
252+
Since `boj-invoke` is a single-shot process, re-entrant deadlocks cannot occur in
253+
the current dispatch model. The mutex discipline remains as defence-in-depth and for
254+
future hot-path call modes.
82255

83256
## Distributed Hosting (Umoja Network)
84257

@@ -88,7 +261,7 @@ BoJ servers are community-hosted, like Tor or IPFS:
88261
- **Hash attestation**: each node's binary hash must match the canonical build
89262
- **Tampered nodes**: excluded from the community network, but can still run locally
90263
- **Gossip protocol**: nodes discover each other via IPv6 gossip (Byzantine fault tolerant)
91-
- **Load-aware routing**: requests go to healthy nodes (under 80% capacity)
264+
- **Load-aware routing**: requests go to healthy nodes (under 80% capacity) — _not yet wired_
92265
- **PMPL provenance**: the license's cryptographic provenance requirements ARE the attestation
93266

94267
### Seed Nodes (Day 1)
@@ -112,85 +285,25 @@ BoJ servers are community-hosted, like Tor or IPFS:
112285
BoJ doesn't start from scratch:
113286

114287
- **proven-servers** (108 components, zero `believe_me`): MCP types, connectors, core primitives
115-
- **polystack** (13 components — being superseded by BoJ): capability domain mapping
288+
- **polystack** (13 components — superseded by BoJ): capability domain mapping
116289
- **stapeln**: container supply chain
117290

118-
## Thread Safety
119-
120-
All 9 Zig FFI modules (`database.zig`, `secrets.zig`, `observe.zig`, `fleet.zig`, `nesy.zig`, `federation.zig`, `lsp.zig`, `dap.zig`, `bsp.zig`) use `std.Thread.Mutex` to guarantee safe concurrent access from the Zig HTTP adapter worker threads.
121-
122-
**Mutex discipline:**
123-
124-
- Every `pub export fn` (C-ABI boundary) acquires the module's mutex before touching any global state and releases it on return via `defer`.
125-
- 55 global variables are protected across 120+ exported functions.
126-
- No Zig FFI function can be entered concurrently on two threads with conflicting access to the same global.
127-
128-
**Deadlock prevention:**
129-
130-
Some exported functions need to call other exported functions within the same module (e.g. `federation.zig` where a gossip handler invokes node-lookup logic). If both the caller and callee acquire the same mutex, the thread deadlocks on itself. The solution is a two-tier naming convention:
131-
132-
1. `pub export fn boj_federation_*` — acquires the mutex, called only from the Zig adapter / external C code.
133-
2. `fn federation_*_impl` — mutex-free internal implementation, called by other functions that already hold the lock.
134-
135-
Exported functions delegate to `_impl` helpers, and re-entrant internal call paths use `_impl` directly, so the mutex is acquired exactly once per external call.
136-
137-
**Concurrency model:**
138-
139-
The Zig HTTP adapter spawns worker threads that call into the Zig FFI concurrently. Because every C-ABI entry point serialises on the module mutex, workers never observe torn or partially-updated state. The cost is per-module serialisation, which is acceptable at current scale; future work may introduce per-resource fine-grained locks if profiling warrants it.
140-
141-
**panic-attack validation:**
142-
143-
The panic-attack security scanner validated the thread-safety model across all 9 modules. Results: 1 expected weak point (QUIC crypto — inherent to the protocol's 0-RTT replay window, mitigated at the application layer), 0 critical vulnerabilities.
144-
145-
## Transport Layer
146-
147-
BoJ supports multiple transport protocols:
148-
149-
- **stdio** (default): JSON-RPC 2.0 over stdin/stdout (Claude Code, Glama, etc.)
150-
- **REST** (port 7700): HTTP/1.1 JSON API
151-
- **gRPC** (port 7701): Binary protocol for high-performance clients
152-
- **GraphQL** (port 7702): Flexible querying for UI/integration layers
153-
- **SSE** (port 7703): Server-Sent Events for real-time updates
154-
155-
Each cartridge declares its supported protocols in its manifest. The MCP bridge routes calls to the appropriate transport.
156-
157-
### Auto-Reconnect
158-
159-
The transport layer includes an exponential backoff with jitter mechanism for improved resilience. This ensures that the system can recover from connection issues and continue to operate smoothly.
160-
161-
### Health Monitoring
162-
163-
The transport layer provides detailed health information, including cartridge connection status. This makes it easy to monitor the system's health and identify any issues.
164-
165-
## Unified-Zig-API Stack
291+
## Unified-Zig-API Stack Alignment
166292

167293
The estate standard for all Zig-edge service boundaries is the **unified-zig-api
168-
stack** documented in `developer-ecosystem/UNIFIED-ZIG-API-STACK.adoc`. It provides
169-
four vertically-stacked layers:
170-
171-
| Layer | Location | Description |
172-
|-------|----------|-------------|
173-
| Idris2 core (proven) | `verification-ecosystem/proven/` | 104 formally verified primitives, `%default total`, zero `believe_me` |
174-
| Idris2 ABI | `developer-ecosystem/zig-api/src/ZigApi/ABI/` | Dependent-type proofs of the C ABI surface |
175-
| Zig runtime | `developer-ecosystem/zig-api/ffi/zig/src/` | `uapi_*` symbol set; gnosis HTTP server + connector pool |
176-
| C adaptor | `developer-ecosystem/zig-api/generated/abi/zig_api.h` | Auto-generated; never edit by hand |
294+
stack** documented in `developer-ecosystem/UNIFIED-ZIG-API-STACK.adoc`.
177295

178-
**BoJ alignment status (2026-04-17):** BoJ does **not yet** consume `libzig_api`
296+
**BoJ alignment status (2026-04-25):** BoJ does **not yet** consume `libzig_api`
179297
in code. The cartridge ABI + FFI layers (Idris2 + Zig) are BoJ-local today.
180298
Full alignment — calling `uapi_gnosis_*` from the BoJ adapter and linking
181-
`libzig_api.so` — is tracked in ADR-0002 as future work. First estate consumers
182-
wired on 2026-04-17: lol-gateway, aerie, emergency-button/emergency-room.
183-
184-
The `uapi_gnosis_set_handler` pattern (single-port handler dispatch) is in flight
185-
as of this version; BoJ will adopt it once stabilised upstream.
186-
187-
**Open question:** When BoJ adopts `uapi_gnosis_*`, the per-port V-legacy adapter
188-
pattern (ports 7700/7701/7702) will consolidate to a single-port gnosis server.
189-
Port assignments are documented in `docs/API-CONTRACT.md` and are stable until
190-
a major version bump. Track progress in ADR-0002.
299+
`libzig_api.so` — is tracked in ADR-0002 as future work.
191300

192-
See `developer-ecosystem/UNIFIED-ZIG-API-STACK.adoc` for the canonical wiring guide.
301+
When BoJ adopts `uapi_gnosis_*`, the per-port adapter pattern (REST/7700) will
302+
consolidate to a single-port gnosis server. Port assignments are in `docs/API-CONTRACT.md`
303+
and are stable until a major version bump.
193304

194305
## License
195306

196-
PMPL-1.0-or-later. The license's provenance requirements (crypto signatures, emotional lineage) align directly with the hash attestation model — the legal framework and the technical framework say the same thing.
307+
PMPL-1.0-or-later. The license's provenance requirements (crypto signatures,
308+
emotional lineage) align directly with the hash attestation model — the legal
309+
framework and the technical framework say the same thing.

0 commit comments

Comments
 (0)