Skip to content

Commit dc58de1

Browse files
feat(abi): binary ABI + Zig↔Idris layout cross-check (#38)
* chore(maintenance): reconcile licence headers + honest state (corrective/adaptive pass) Corrective: - Re-stamp 24 .zig/.idr source SPDX headers MPL-2.0 -> AGPL-3.0-or-later to match LICENSE, META.a2ml, and steam_client.zig. Machine-readable .a2ml/.contractile files and the emitted-A2ML output string keep MPL-2.0 (deliberate carve-out). - Fill INTENT.contractile (was raw {{PLACEHOLDER}} template): purpose, anti-purpose, architectural-invariants, ask-before-touching, ecosystem. Adaptive (stale-state reconciliation): - Exported FFI symbol count 24/22 -> 27 (Steam +3, Nexus +2) in CLAUDE.md and ROADMAP.adoc. - Completion 100% -> honest 93% to match ROADMAP and the open v1.0.0 gates. - Strike stale critical-next-action: the Nexus FFI exports already exist (server_actions.zig:573 / :713); replace with the real open item (Ephapax parser gaps, 23/25 chain tests). - Refresh ROADMAP revdate and log the April Steam/Nexus/CryoFall additions. Test counts (111/112; integration 36/39) left unchanged pending a verified `zig build test` run -- zig is not installed in this environment. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LvsZgNxFbeqfRmrVFNhJ8G * feat(abi): implement binary ABI + Zig↔Idris layout cross-check Close the open HIGH proof item (Zig↔Idris layout cross-check) and make the previously-aspirational binary ABI real. Layout.idr's proofs show the layout *model* is internally consistent, but nothing checked those hand-written offsets against the C layout Zig actually emits, and the offset-reader FFI that GSA.ABI.Foreign binds was unimplemented (9 dead bindings; no extern structs anywhere). - abi_layout.zig: declare the 8 ABI structs as Zig `extern struct`s and assert at comptime (+ `zig build test`) that @offsetOf/@sizeOf/@Alignof of every field equal the proven Layout.idr constants. A divergence is a compile error (negative-tested). - scripts/gen_abi_expected.py: lift the constants from Layout.idr (single source of truth) into the generated abi_layout_expected.zig used by the check. - abi_serde.zig: implement the readers/helpers Foreign.idr binds — read_int/read_double/read_ptr/read_string, array_len/array_get_string, is_null/free, apply_config/close_handle — plus emitters and an exported gossamer_gsa_drift_struct producer. Round-trip tests read each field back at its proven Layout.idr offset, making the offsets a live runtime contract. - Foreign.idr: getDrift now decodes a DriftReport at offsetOf-derived offsets (was wrong hand-tuned 0/4/12/20/28) via prim__driftStruct + read_ptr/read_int/ read_double; adds prim__readPtr / prim__driftStruct bindings. Verified: zig build + test/integration/smoke (124 tests) green on Zig 0.15.2; idris2 0.7.0 typechecks the ABI. Exported FFI symbols 27 -> 38. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LvsZgNxFbeqfRmrVFNhJ8G * fix(abi): resolve Hypatia new-alert errors (banned Python, unchecked casts) Address the 3 new Hypatia alerts the PR introduced: - banned_language_file: rewrite the layout-table generator in Zig (scripts/gen_abi_expected.zig, run via `zig run`) and delete the Python version. The generator reproduces abi_layout_expected.zig byte-for-byte (only the @generated banner differs). - zig_ptr_cast / zig_align_cast (CWE-704): refactor abi_serde.zig to use zero @ptrCast/@aligncast, matching the rest of the FFI (which has none). Readers reinterpret memory via @ptrFromInt + @memcpy; emitters build a typed view of a c_allocator byte block with std.mem.bytesAsValue / bytesAsSlice. Behaviour unchanged: zig build + test/integration/smoke (124 tests) green, idris2 0.7.0 typechecks. Docs updated for the .zig generator. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LvsZgNxFbeqfRmrVFNhJ8G --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 07fda92 commit dc58de1

8 files changed

Lines changed: 905 additions & 23 deletions

File tree

.claude/CLAUDE.md

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ panic-attack assail .
4141
```
4242
Gossamer GUI (Ephapax .eph) -- src/core/, src/gui/panels/
4343
| IPC (gossamer:// protocol)
44-
Zig FFI (libgsa.so + gsa CLI) -- src/interface/ffi/src/ (9 modules)
44+
Zig FFI (libgsa.so + gsa CLI) -- src/interface/ffi/src/ (11 modules)
4545
| C ABI (13 result codes)
4646
Idris2 ABI (Types/Foreign/Layout) -- src/interface/abi/
4747
| REST (port 8090)
@@ -60,8 +60,13 @@ VeriSimDB (8-modality octads) -- container/verisimdb/
6060
| `server_actions.zig` | Start/stop/restart/logs via Podman/Docker/systemd |
6161
| `game_profiles.zig` | A2ML profile registry + parser |
6262
| `groove_client.zig` | .well-known Groove voice alerting |
63+
| `abi_layout.zig` | Canonical C-ABI `extern struct`s + comptime Zig↔Idris layout cross-check |
64+
| `abi_serde.zig` | Binary ABI emitters + offset readers (`read_int`/`read_ptr`/…) — makes `Layout.idr` offsets a live contract |
6365
| `cli.zig` | Standalone CLI executable (status, probe, profiles, version) |
6466

67+
`abi_layout_expected.zig` is generated from `Layout.idr` by
68+
`scripts/gen_abi_expected.zig` (`zig run`; regenerate when `Layout.idr` changes).
69+
6570
## Key Conventions
6671

6772
- **All exported FFI functions** are prefixed `gossamer_gsa_` and use `pub export fn ... callconv(.c)`
@@ -78,13 +83,15 @@ VeriSimDB (8-modality octads) -- container/verisimdb/
7883

7984
- **Completion**: 93% (15 phases built; v1.0.0 gates still open — see Remaining)
8085
- **Zig version**: 0.15.2 (see `.tool-versions`)
81-
- **Exported FFI symbols**: 27 (comptime linker hints in main.zig)
82-
- **Tests**: 111 Zig tests across 3 suites (unit: 67, integration: 39, smoke: 5). All passing.
86+
- **Exported FFI symbols**: 38 (comptime linker hints in main.zig)
87+
- **Tests**: 124 Zig tests across 3 suites (unit: 80, integration: 39, smoke: 5). All passing.
8388
- Security tests for command injection in server_actions
8489
- Config parser edge cases for all 8 formats
8590
- A2ML round-trip, diff, and secret redaction tests
8691
- Groove target registry overflow and buffer truncation tests
92+
- ABI layout cross-check + binary ABI round-trip (read at proven `Layout.idr` offsets)
8793
- **Idris2 ABI**: Alignment postulate replaced with constructive proof (`alignUpCeil` + `alignUpCeilIsMultiple`)
94+
- **Cross-language ABI**: `abi_layout.zig` asserts (at compile time + `zig build test`) that the 8 `extern struct`s match the proven `Layout.idr` offsets/sizes; `abi_serde.zig` implements the offset readers/emitters so those offsets are a live runtime contract (was the open HIGH proof item)
8895
- **VeriSimDB**: Main on 8090 (built, running), backup on 8091 (game saves)
8996
- **Container**: Containerfile wired with real Zig build, entrypoint.sh execs gsa
9097
- **Nix/Guix**: Both flake.nix and guix.scm have real build/install phases

PROOF-NEEDS.md

Lines changed: 32 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
11
# Proof Requirements
22

3-
## Current State (2026-06-21 — verified with Idris2 0.7.0)
3+
## Current State (2026-06-21 — verified with Idris2 0.7.0 + Zig 0.15.2)
44

55
The ABI package typechecks end-to-end (all modules `%default total`, no
6-
`postulate` / `believe_me` / `assert_total`):
6+
`postulate` / `believe_me` / `assert_total`), and the hand-written Idris layout
7+
constants are now cross-checked against the real C layout the Zig compiler emits
8+
(see *Cross-language ABI* below):
79

810
```bash
911
cd src/interface/abi && idris2 --typecheck gsa-abi.ipkg
@@ -54,13 +56,36 @@ Decidable `portInRange`, `nonEmptyId`, `configFieldCountPositive`,
5456
`allPortsValid`, the `Valid*` smart constructors, and the linear
5557
`ServerHandle` with its erased non-null witness.
5658

59+
### Cross-language ABI (`abi_layout.zig` / `abi_serde.zig`)
60+
The Idris proofs above show the *model* is internally consistent; the Zig side
61+
now checks that model against reality and makes it a live runtime contract.
62+
63+
- **Layout cross-check (was the open HIGH item).** Each of the 8 structs is
64+
declared as a Zig `extern struct` (`abi_layout.zig`). A comptime block asserts
65+
`@offsetOf` / `@sizeOf` / `@alignOf` of every field equals the proven Idris
66+
constant — lifted verbatim from `Layout.idr` into `abi_layout_expected.zig` by
67+
`scripts/gen_abi_expected.zig`. Any divergence is a **compile error**, so the
68+
library cannot build against a layout the proofs do not describe. A negative
69+
test (perturbing one offset) confirms the guard bites; the check also runs
70+
under `zig build test`.
71+
- **Live offset contract.** The previously-unimplemented FFI readers that
72+
`GSA.ABI.Foreign` binds are now real (`abi_serde.zig`): `read_int` /
73+
`read_double` / `read_ptr` decode fields at a byte offset; `read_string`,
74+
`array_len`, `array_get_string`, `is_null`, `free` handle strings/arrays;
75+
`apply_config` / `close_handle` round out the surface. Emitters
76+
(`serializeDriftReport` / `serializeFingerprint` / `serializeStringArray`)
77+
write the wire structs, and `gossamer_gsa_drift_struct` exports one. Round-trip
78+
tests read each field back at its **proven** `Layout.idr` offset, standing in
79+
for the Idris reader — so agreement is an end-to-end cross-language guarantee.
80+
- **Idris consumer.** `Foreign.getDrift` now decodes a `DriftReport` at offsets
81+
taken from `offsetOf … driftReportLayout` (not the previous, wrong hand-tuned
82+
`0/4/12/20/28`), via `prim__driftStruct` + `prim__readPtr`/`readInt`/`readDouble`.
83+
84+
> Regenerate the expected table whenever `Layout.idr` changes:
85+
> `zig run scripts/gen_abi_expected.zig` (then the Zig cross-check re-validates).
86+
5787
## What still needs proving
5888

59-
- **Zig ↔ Idris layout cross-check (HIGH).** `Layout.idr` proves the Idris
60-
model is internally consistent, but nothing yet checks the hand-written
61-
field offsets/sizes against Zig's `@sizeOf`/`@offsetOf`. A generated test
62-
emitting Zig's numbers and comparing them to the `Layout` constants would
63-
close the actual cross-language ABI guarantee.
6489
- **Server probe safety (MEDIUM).** Prove probes cause no side effects on
6590
targets. Needs a specification first.
6691
- **Configuration drift-detection completeness (LOW).** Needs a richer spec.

scripts/gen_abi_expected.zig

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
// SPDX-License-Identifier: AGPL-3.0-or-later
2+
// Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) <j.d.a.jewell@open.ac.uk>
3+
//
4+
// gen_abi_expected.zig — extract the canonical C-ABI layout numbers from the
5+
// machine-checked Idris2 model (src/interface/abi/Layout.idr) and emit the Zig
6+
// source file (src/interface/ffi/src/abi_layout_expected.zig) consumed by the
7+
// Zig-side cross-check (abi_layout.zig).
8+
//
9+
// Layout.idr is the single source of truth: it carries the proofs (NoOverlap,
10+
// AllFieldsAligned, SizeAligned, SizeCoversFields). This tool lifts the same
11+
// `MkFieldDesc name offset size align` / `MkStructLayout [...] total align`
12+
// constants into Zig so the compiler can assert the canonical extern structs
13+
// agree with the proven model.
14+
//
15+
// Run from the repository root:
16+
// zig run scripts/gen_abi_expected.zig
17+
// (or `python3 -m ziglang run scripts/gen_abi_expected.zig`). CI re-runs this and
18+
// `git diff --exit-code`s the result so the Zig table never drifts from Layout.idr.
19+
20+
const std = @import("std");
21+
22+
const LAYOUT_IDR = "src/interface/abi/Layout.idr";
23+
const OUT_ZIG = "src/interface/ffi/src/abi_layout_expected.zig";
24+
25+
// Idris layout variable (without the "Layout" suffix) -> Zig struct type name.
26+
const Struct = struct { idris: []const u8, zig: []const u8 };
27+
const structs = [_]Struct{
28+
.{ .idris = "serverHandle", .zig = "ServerHandle" },
29+
.{ .idris = "probeResult", .zig = "ProbeResult" },
30+
.{ .idris = "configField", .zig = "ConfigField" },
31+
.{ .idris = "a2mlConfig", .zig = "A2MLConfig" },
32+
.{ .idris = "gameProfile", .zig = "GameProfile" },
33+
.{ .idris = "serverOctad", .zig = "ServerOctad" },
34+
.{ .idris = "fingerprint", .zig = "Fingerprint" },
35+
.{ .idris = "driftReport", .zig = "DriftReport" },
36+
};
37+
38+
const HEADER =
39+
\\// SPDX-License-Identifier: AGPL-3.0-or-later
40+
\\// Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) <j.d.a.jewell@open.ac.uk>
41+
\\//
42+
\\// @generated by scripts/gen_abi_expected.zig from src/interface/abi/Layout.idr
43+
\\// DO NOT EDIT. Regenerate with: zig run scripts/gen_abi_expected.zig
44+
\\//
45+
\\// Canonical C-ABI layout numbers lifted from the machine-checked Idris2 model
46+
\\// (GSA.ABI.Layout). Consumed by abi_layout.zig to assert that the Zig extern
47+
\\// structs agree, byte-for-byte, with the proven Idris layout.
48+
\\
49+
\\pub const Field = struct {
50+
\\ name: []const u8,
51+
\\ offset: u32,
52+
\\ size: u32,
53+
\\ alignment: u32,
54+
\\};
55+
\\
56+
\\pub const StructExpect = struct {
57+
\\ name: []const u8,
58+
\\ total_size: u32,
59+
\\ struct_align: u32,
60+
\\ fields: []const Field,
61+
\\};
62+
\\
63+
\\
64+
;
65+
66+
fn nextUint(it: *std.mem.TokenIterator(u8, .any)) !u32 {
67+
return std.fmt.parseInt(u32, it.next() orelse return error.ParseFailed, 10);
68+
}
69+
70+
fn emitStruct(out: *std.array_list.Managed(u8), src: []const u8, s: Struct) !void {
71+
var nbuf: [128]u8 = undefined;
72+
const needle = try std.fmt.bufPrint(&nbuf, "{s}Layout = MkStructLayout", .{s.idris});
73+
const at = std.mem.indexOf(u8, src, needle) orelse return error.StructNotFound;
74+
const after = src[at + needle.len ..];
75+
const lb = std.mem.indexOfScalar(u8, after, '[') orelse return error.NoFieldList;
76+
const rel_rb = std.mem.indexOfScalar(u8, after[lb..], ']') orelse return error.NoFieldList;
77+
const body = after[lb + 1 .. lb + rel_rb];
78+
const tail = after[lb + rel_rb + 1 ..]; // "<total> <align>" follow the ']'
79+
80+
var tit = std.mem.tokenizeAny(u8, tail, " \r\n\t");
81+
const total = try nextUint(&tit);
82+
const salign = try nextUint(&tit);
83+
84+
const w = out.writer();
85+
try w.print(
86+
"pub const {s} = StructExpect{{\n .name = \"{s}\",\n .total_size = {d},\n .struct_align = {d},\n .fields = &.{{\n",
87+
.{ s.zig, s.zig, total, salign },
88+
);
89+
90+
var rest = body;
91+
while (std.mem.indexOf(u8, rest, "MkFieldDesc")) |fi| {
92+
rest = rest[fi + "MkFieldDesc".len ..];
93+
const q1 = std.mem.indexOfScalar(u8, rest, '"') orelse return error.BadField;
94+
const q2 = std.mem.indexOfScalarPos(u8, rest, q1 + 1, '"') orelse return error.BadField;
95+
const fname = rest[q1 + 1 .. q2];
96+
rest = rest[q2 + 1 ..];
97+
98+
var fit = std.mem.tokenizeAny(u8, rest, " \r\n\t");
99+
const foff = try nextUint(&fit);
100+
const fsize = try nextUint(&fit);
101+
const falign = try nextUint(&fit);
102+
103+
if (!std.mem.startsWith(u8, fname, "padding")) {
104+
try w.print(
105+
" .{{ .name = \"{s}\", .offset = {d}, .size = {d}, .alignment = {d} }},\n",
106+
.{ fname, foff, fsize, falign },
107+
);
108+
}
109+
}
110+
try w.writeAll(" },\n};\n\n");
111+
}
112+
113+
pub fn main() !void {
114+
var gpa = std.heap.GeneralPurposeAllocator(.{}){};
115+
defer _ = gpa.deinit();
116+
const alloc = gpa.allocator();
117+
118+
const src = try std.fs.cwd().readFileAlloc(alloc, LAYOUT_IDR, 4 * 1024 * 1024);
119+
defer alloc.free(src);
120+
121+
var out = std.array_list.Managed(u8).init(alloc);
122+
defer out.deinit();
123+
124+
try out.appendSlice(HEADER);
125+
for (structs) |s| try emitStruct(&out, src, s);
126+
127+
try out.appendSlice("pub const all = [_]StructExpect{ ");
128+
for (structs, 0..) |s, i| {
129+
if (i != 0) try out.appendSlice(", ");
130+
try out.appendSlice(s.zig);
131+
}
132+
try out.appendSlice(" };\n");
133+
134+
try std.fs.cwd().writeFile(.{ .sub_path = OUT_ZIG, .data = out.items });
135+
std.debug.print("wrote {s} ({d} structs)\n", .{ OUT_ZIG, structs.len });
136+
}

src/interface/abi/Foreign.idr

Lines changed: 40 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -192,6 +192,23 @@ export
192192
%foreign "C:gossamer_gsa_array_get_string, libgossamer_gsa"
193193
prim__arrayGetString : AnyPtr -> Int -> PrimIO String
194194

195+
||| Read a pointer field at a byte offset from a serialised struct. Combined
196+
||| with prim__readString this decodes char* fields at their proven offsets:
197+
||| readString (readPtr struct (offsetOf "serverId" layout)).
198+
|||
199+
||| C signature: void* gossamer_gsa_read_ptr(void* ptr, int32_t offset)
200+
export
201+
%foreign "C:gossamer_gsa_read_ptr, libgossamer_gsa"
202+
prim__readPtr : AnyPtr -> Int -> PrimIO AnyPtr
203+
204+
||| Emit a binary DriftReport wire struct (canonical Layout.idr layout) for a
205+
||| tracked server. The returned pointer must be released with prim__free.
206+
|||
207+
||| C signature: void* gossamer_gsa_drift_struct(const char* server_id)
208+
export
209+
%foreign "C:gossamer_gsa_drift_struct, libgossamer_gsa"
210+
prim__driftStruct : String -> PrimIO AnyPtr
211+
195212
--------------------------------------------------------------------------------
196213
-- Internal Helpers
197214
-- Utility functions used by the safe wrappers to convert between C
@@ -457,30 +474,40 @@ checkHealth = do
457474
Just status => pure (Right status)
458475
Nothing => pure (Left Error)
459476

460-
||| Get a drift report for a specific server.
461-
||| Compares the server's current VeriSimDB octad against its historical
462-
||| baseline across all modalities (config, semantic, temporal).
477+
||| Get a drift report for a specific server, decoded from the binary
478+
||| DriftReport wire struct emitted by gossamer_gsa_drift_struct.
479+
|||
480+
||| Every field is read at the offset *proven* in GSA.ABI.Layout
481+
||| (driftReportLayout): the char* field serverId via prim__readPtr, the scalars
482+
||| via prim__readInt / prim__readDouble. These are the same offsets the Zig
483+
||| layer is compile-time-checked against (abi_layout.zig), so this is the live
484+
||| cross-language ABI contract rather than hand-tuned constants.
485+
|||
486+
||| status reflects the tracked liveness; the configDrift / semanticDrift /
487+
||| temporalConsistency / overallScore metrics are 0.0 until VeriSimDB scoring
488+
||| is wired into the emitter.
463489
|||
464490
||| @param serverId The server identifier to check
465491
||| @return Left Result on failure, Right DriftReport on success
466492
export
467493
covering
468494
getDrift : String -> IO (Either Result DriftReport)
469495
getDrift serverId = do
470-
ptr <- primIO (prim__verisimdbDrift serverId)
496+
ptr <- primIO (prim__driftStruct serverId)
471497
case prim__isNull ptr /= 0 of
472-
True => pure (Left VeriSimDBUnavailable)
498+
True => pure (Left Error)
473499
False => do
474-
-- Deserialise the DriftReport from the returned pointer.
475-
-- Field offsets match the Zig struct layout (see Layout.idr).
476-
statusCode <- primIO (prim__readInt ptr 0)
477-
configDrift <- primIO (prim__readDouble ptr 4)
478-
semanticDrift <- primIO (prim__readDouble ptr 12)
479-
temporalCons <- primIO (prim__readDouble ptr 20)
480-
overallScore <- primIO (prim__readDouble ptr 28)
500+
let o = \name => the Int (cast (fromMaybe 0 (Layout.offsetOf name Layout.driftReportLayout)))
501+
sidPtr <- primIO (prim__readPtr ptr (o "serverId"))
502+
sid <- primIO (prim__readString sidPtr)
503+
statusCode <- primIO (prim__readInt ptr (o "status"))
504+
configDrift <- primIO (prim__readDouble ptr (o "configDrift"))
505+
semanticDrift <- primIO (prim__readDouble ptr (o "semanticDrift"))
506+
temporalCons <- primIO (prim__readDouble ptr (o "temporalConsistency"))
507+
overallScore <- primIO (prim__readDouble ptr (o "overallScore"))
481508
primIO (prim__free ptr)
482509
let status = fromMaybe Warning (healthStatusFromInt statusCode)
483-
pure (Right (MkDriftReport serverId status configDrift semanticDrift temporalCons overallScore))
510+
pure (Right (MkDriftReport sid status configDrift semanticDrift temporalCons overallScore))
484511

485512
--------------------------------------------------------------------------------
486513
-- Safe Wrappers: Profile Management

0 commit comments

Comments
 (0)