Skip to content

Commit 86319db

Browse files
verisimiser: make ABI/FFI/codegen provably-real + honest docs (#167)
## What this does Turns VeriSimiser's non-Rust claims from prose into machine-checked facts, and adds a CI gate (`provable.yml`) so they stay checked. This is the verisimiser slice of the "-iser family → provably-real + honest" effort (after chapeliser and typedqliser). Everything below was **verified locally** with a full toolchain (idris2 0.7.0, zig 0.14.0, cargo), not just asserted. ### Idris2 ABI — full, hole-free proofs `src/interface/abi/Verisimiser/ABI/{Types,Layout,Foreign}.idr` all `idris2 --check` clean (no holes, no warnings). - **Layout.idr** completed: each of the four C-struct layouts (`OctadRecord`, `ProvenanceEntry`, `DriftMeasurement`, `TemporalSnapshot`) carries machine-checked size/alignment (`CABICompliant`) and per-field-offset (`FieldsAligned`) witnesses. `alignUp` is redefined as `ceilDiv size align * align` so `alignUpCorrect` becomes unconditional. - Removed generic helpers that **cannot** be proven honestly — `checkCABI`, `offsetInBounds` (false for an arbitrary field), `verifyLayout`, `calcStructSize`, `PlatformLayout`/`verifyAllPlatforms` — each with a NOTE explaining why. The concrete, checkable `*Valid` instances remain. ### Zig FFI — builds + tests green `cd src/interface/ffi && zig build test` → **43/43 tests pass** (8 unit + 35 C-ABI integration through the compiled `libverisimiser.so`); `zig build` produces `.so` + `.a`. - Fixed `build.zig` for Zig 0.14 (dropped the `lib.version` panic + the missing `addInstallHeader`; added `linkLibC` since the impl uses `std.heap.c_allocator`; wired integration tests into `zig build test`). - `Handle`/`DbConnection`: `extern struct` → `struct` (they hold a `std.mem.Allocator`, which has no C layout; C only ever sees them behind a pointer). - Fixed a pointless-discard compile error and the integration test's per-declaration `?*opaque {}` (now one shared opaque `Handle` type). ### Codegen golden + CI - `examples/golden/`: frozen manifest + schema + committed generated overlay (deterministic — regenerates byte-identically). - `.github/workflows/provable.yml` (4 jobs, SHA-pinned actions, timeouts on all): 1. `idris2-proofs` — `idris2 --check` the ABI 2. `zig-ffi` — `zig build test` + `zig build` 3. `codegen-drift` — regenerate golden and diff 4. `sql-golden` — apply the generated overlay to a real SQLite DB; assert every sidecar table + enrichment view builds and is queryable (the views use `ROW_NUMBER() OVER (…)`, so this is a real validity check) The Rust CLI stays covered by the existing `rust-ci.yml`; `cargo test` is green locally too. ### Honesty docs - Rewrote `docs/developer/ABI-FFI-README.adoc` — it was an **unrendered RSR template** full of `{{project}}` placeholders, wrong paths, banned-language bindings, and fictional Idris commands (`%runElab verifyABI`, `idris2 --cg c-header`). Now describes the real, proven contract. - `README` / `ROADMAP` / `EXPLAINME` / `STATE.a2ml`: mark what is machine-checked vs aspirational and point to `provable.yml`. ROADMAP's two relevant Phase-1 items are annotated as *partial* (sidecar-isolation + layout proofs done; hash-chain/version/lineage proofs and live DB interception still TODO) rather than over-ticked. ## 🚩 Flag for owner (no action taken) `.claude/CLAUDE.md` declares this repo **PMPL-1.0-or-later**, but every actual file (Cargo.toml `license-file`, all SPDX headers, `LICENSE`) is **MPL-2.0**, and verisimiser is **not** one of the three PMPL carve-out repos (palimpsest-license / palimpsest-plasma / consent-aware-http). Per the estate license policy this sole-owner repo should be MPL-2.0, so the `CLAUDE.md` line looks like the drift. **I changed nothing license-related** — new files I authored carry MPL-2.0 to match the repo's actual classification. Surfacing only; your call. ## Verification summary | Gate | Result (local) | |---|---| | `idris2 --check` Types/Layout/Foreign | ✅ clean, 0 holes/warnings | | `zig build test` | ✅ 43/43 | | codegen determinism (regen + diff) | ✅ byte-identical | | generated SQL → SQLite apply | ✅ all tables + views build & query | | `cargo test` | ✅ pass | 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01Mbq6yKF9RhFai6EQ7WqKhQ --- _Generated by [Claude Code](https://claude.ai/code/session_01Mbq6yKF9RhFai6EQ7WqKhQ)_ Co-authored-by: Claude <noreply@anthropic.com>
1 parent f4ea008 commit 86319db

8 files changed

Lines changed: 531 additions & 59 deletions

File tree

.github/workflows/provable.yml

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
# SPDX-License-Identifier: MPL-2.0
2+
# Provable — machine-checks the claims VeriSimiser makes outside the Rust layer.
3+
#
4+
# The Rust CLI/codegen is covered by rust-ci.yml (fmt + clippy + test). This
5+
# workflow verifies the four things that are otherwise only asserted in prose:
6+
# 1. idris2-proofs — the Idris2 ABI proofs actually type-check (`idris2 --check`).
7+
# 2. zig-ffi — the Zig FFI reference impl builds and its tests pass.
8+
# 3. codegen-drift — the committed golden sample matches fresh codegen output.
9+
# 4. sql-golden — the generated overlay SQL actually applies to a real
10+
# SQLite database and every table/view builds and queries.
11+
#
12+
# Until every job here is green, the proofs / FFI / generated SQL are "written"
13+
# but NOT "verified" — ROADMAP.adoc and README.adoc are worded accordingly.
14+
name: Provable
15+
16+
on:
17+
push:
18+
branches: [main, master, "claude/**"]
19+
pull_request:
20+
21+
permissions:
22+
contents: read
23+
24+
jobs:
25+
idris2-proofs:
26+
name: Idris2 — machine-check ABI proofs
27+
runs-on: ubuntu-latest
28+
timeout-minutes: 20
29+
container: ghcr.io/stefan-hoeck/idris2-pack:latest
30+
steps:
31+
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
32+
- name: idris2 build the ABI package (Types, Layout, Proofs, Foreign)
33+
working-directory: src/interface/abi
34+
# Modules are namespaced Verisimiser.ABI.* under Verisimiser/ABI/.
35+
# Building the .ipkg type-checks every module in the package — including
36+
# Proofs.idr, which carries the machine-checked C-ABI / layout theorems —
37+
# in one invocation, which is also the canonical command in the .ipkg.
38+
run: |
39+
idris2 --version
40+
idris2 --build verisimiser-abi.ipkg
41+
42+
zig-ffi:
43+
name: Zig — build + test FFI reference impl
44+
runs-on: ubuntu-latest
45+
timeout-minutes: 15
46+
steps:
47+
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
48+
- uses: mlugg/setup-zig@53fc45b17fe98b52f92ee5ea08ff48a85a3e7eb7 # v1.2.2
49+
with:
50+
version: 0.14.0
51+
- name: zig build + test
52+
working-directory: src/interface/ffi
53+
# `zig build test` runs the 8 in-module unit tests AND the 35 C-ABI
54+
# integration tests (which link the compiled libverisimiser.so).
55+
run: |
56+
zig version
57+
zig build test
58+
zig build
59+
60+
codegen-drift:
61+
name: Codegen — golden sample is up to date
62+
runs-on: ubuntu-latest
63+
timeout-minutes: 15
64+
steps:
65+
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
66+
- name: Regenerate golden artifacts and diff against committed tree
67+
run: |
68+
# retry: fresh runners occasionally flake fetching crates from crates.io
69+
built=0
70+
for i in 1 2 3; do
71+
cargo build --quiet && { built=1; break; }
72+
echo "build attempt $i failed; retrying after backoff…"; sleep $((i * 5))
73+
done
74+
[ "$built" = 1 ] || { echo "cargo build failed after retries"; exit 1; }
75+
( cd examples/golden && ../../target/debug/verisimiser generate -m verisimiser.toml -o /tmp/golden )
76+
diff -ru examples/golden/generated /tmp/golden
77+
78+
sql-golden:
79+
name: SQLite — generated overlay applies and views build
80+
runs-on: ubuntu-latest
81+
timeout-minutes: 10
82+
steps:
83+
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
84+
- name: Ensure sqlite3 is available
85+
run: sudo apt-get update && sudo apt-get install -y sqlite3
86+
- name: Apply schema + generated overlay + interceptors into a fresh SQLite DB
87+
run: |
88+
sqlite3 --version
89+
DB="$(mktemp -d)/golden.db"
90+
sqlite3 "$DB" < examples/golden/schema.sql
91+
sqlite3 "$DB" < examples/golden/generated/sidecar_schema.sql
92+
sqlite3 "$DB" < examples/golden/generated/interceptors.sql
93+
94+
# Every enabled octad dimension must have produced its sidecar table.
95+
for t in verisimdb_metadata verisimdb_provenance_log \
96+
verisimdb_lineage_graph verisimdb_temporal_versions \
97+
verisimdb_access_policies; do
98+
sqlite3 "$DB" "SELECT name FROM sqlite_master WHERE type='table' AND name='$t';" \
99+
| grep -qx "$t" || { echo "missing sidecar table: $t"; exit 1; }
100+
done
101+
102+
# Every target table must have provenance + temporal enrichment views,
103+
# and each view must be queryable (catches invalid SQL in view bodies).
104+
for v in verisimdb_users_with_provenance verisimdb_users_with_temporal \
105+
verisimdb_posts_with_provenance verisimdb_posts_with_temporal; do
106+
sqlite3 "$DB" "SELECT name FROM sqlite_master WHERE type='view' AND name='$v';" \
107+
| grep -qx "$v" || { echo "missing enrichment view: $v"; exit 1; }
108+
sqlite3 "$DB" "SELECT count(*) FROM $v;" >/dev/null \
109+
|| { echo "view not queryable: $v"; exit 1; }
110+
done
111+
112+
echo "Generated overlay applied cleanly; all sidecar tables + views present and queryable."

examples/golden/README.adoc

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
// SPDX-License-Identifier: MPL-2.0
2+
= VeriSimiser Codegen Golden Sample
3+
:toc: macro
4+
:source-highlighter: rouge
5+
6+
This directory is the *frozen golden sample* that the `codegen-drift` and
7+
`sql-golden` jobs of `.github/workflows/provable.yml` check on every push and
8+
pull request. It exists to turn "the generator produces correct SQL" from a
9+
prose claim into a machine-checked fact.
10+
11+
toc::[]
12+
13+
== What's here
14+
15+
[cols="1,3"]
16+
|===
17+
| File | Role
18+
19+
| `verisimiser.toml`
20+
| Frozen input manifest (SQLite backend, 7 of 8 octad dimensions enabled).
21+
22+
| `schema.sql`
23+
| Frozen input schema — two tables (`users`, `posts`) with a typical author
24+
relationship and a spread of column types.
25+
26+
| `generated/sidecar_schema.sql`
27+
| Committed expected output: the octad sidecar overlay DDL.
28+
29+
| `generated/interceptors.sql`
30+
| Committed expected output: the per-table provenance/temporal enrichment
31+
views and lineage/access-control query templates.
32+
|===
33+
34+
== What the CI checks
35+
36+
`codegen-drift`:: regenerates `generated/` from `verisimiser.toml` + `schema.sql`
37+
into a temp directory and `diff`s it against the committed tree. The generator
38+
is deterministic, so any difference fails the job — this catches *unintended*
39+
changes to codegen output.
40+
41+
`sql-golden`:: applies `schema.sql`, then the two generated files, into a fresh
42+
SQLite database and asserts every sidecar table and enrichment view is created
43+
and queryable. This is the SQL analogue of "the generated code compiles and
44+
runs": it proves the emitted DDL is valid and applies cleanly against the
45+
target schema (the views use `ROW_NUMBER() OVER (…)`, so SQLite >= 3.25).
46+
47+
== Regenerating after an intentional codegen change
48+
49+
If you change the generator and the new output is correct, refresh the golden
50+
in the *same* PR as the code change:
51+
52+
[source,sh]
53+
----
54+
cargo build
55+
( cd examples/golden && ../../target/debug/verisimiser generate -m verisimiser.toml -o generated )
56+
git add examples/golden/generated
57+
----
58+
59+
Then review the diff carefully — it is the human-readable record of how your
60+
change affected every consumer's generated overlay.
Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
-- SPDX-License-Identifier: MPL-2.0
2+
-- VeriSimiser query interceptors (auto-generated)
3+
4+
-- ==========================================================
5+
-- Table: users
6+
-- ==========================================================
7+
8+
-- Provenance-enriched view for 'users'
9+
-- Joins each row with its latest provenance entry from the sidecar.
10+
CREATE VIEW IF NOT EXISTS verisimdb_users_with_provenance AS
11+
SELECT
12+
users.id,
13+
users.username,
14+
users.email,
15+
users.created_at,
16+
prov.operation AS _verisimdb_last_operation,
17+
prov.actor AS _verisimdb_last_actor,
18+
prov.timestamp AS _verisimdb_last_modified,
19+
prov.hash AS _verisimdb_provenance_hash
20+
FROM users
21+
LEFT JOIN (
22+
SELECT entity_id, operation, actor, timestamp, hash
23+
FROM (
24+
SELECT entity_id, operation, actor, timestamp, hash,
25+
ROW_NUMBER() OVER (PARTITION BY entity_id ORDER BY timestamp DESC) AS _rn
26+
FROM verisimdb_provenance_log
27+
WHERE table_name = 'users'
28+
) ranked
29+
WHERE _rn = 1
30+
) prov ON prov.entity_id = (CAST(users.id AS TEXT));
31+
32+
-- Temporal-enriched view for 'users'
33+
-- Joins each row with its current version metadata.
34+
CREATE VIEW IF NOT EXISTS verisimdb_users_with_temporal AS
35+
SELECT
36+
users.id,
37+
users.username,
38+
users.email,
39+
users.created_at,
40+
tv.version AS _verisimdb_version,
41+
tv.valid_from AS _verisimdb_valid_from,
42+
tv.operation AS _verisimdb_version_operation
43+
FROM users
44+
LEFT JOIN verisimdb_temporal_versions tv
45+
ON tv.entity_id = (CAST(users.id AS TEXT))
46+
AND tv.table_name = 'users'
47+
AND tv.valid_to IS NULL;
48+
49+
-- Lineage queries for 'users'
50+
51+
-- Upstream: what data was this entity derived from?
52+
-- SELECT * FROM verisimdb_lineage_graph
53+
-- WHERE target_entity = :entity_id AND target_table = 'users';
54+
55+
-- Downstream: what entities depend on this entity?
56+
-- SELECT * FROM verisimdb_lineage_graph
57+
-- WHERE source_entity = :entity_id AND source_table = 'users';
58+
59+
-- Access control filter for 'users'
60+
-- Apply this as a WHERE clause addition to enforce row-level security.
61+
--
62+
-- Example usage (parameterised):
63+
-- SELECT * FROM users
64+
-- WHERE ... AND EXISTS (
65+
-- SELECT 1 FROM verisimdb_access_policies
66+
-- WHERE target_table = 'users'
67+
-- AND principal = :current_principal
68+
-- AND access_level IN ('read', 'admin')
69+
-- AND active = 1
70+
-- AND (condition IS NULL OR :row_matches_condition)
71+
-- );
72+
73+
-- ==========================================================
74+
-- Table: posts
75+
-- ==========================================================
76+
77+
-- Provenance-enriched view for 'posts'
78+
-- Joins each row with its latest provenance entry from the sidecar.
79+
CREATE VIEW IF NOT EXISTS verisimdb_posts_with_provenance AS
80+
SELECT
81+
posts.id,
82+
posts.author_id,
83+
posts.title,
84+
posts.body,
85+
posts.published,
86+
posts.created_at,
87+
prov.operation AS _verisimdb_last_operation,
88+
prov.actor AS _verisimdb_last_actor,
89+
prov.timestamp AS _verisimdb_last_modified,
90+
prov.hash AS _verisimdb_provenance_hash
91+
FROM posts
92+
LEFT JOIN (
93+
SELECT entity_id, operation, actor, timestamp, hash
94+
FROM (
95+
SELECT entity_id, operation, actor, timestamp, hash,
96+
ROW_NUMBER() OVER (PARTITION BY entity_id ORDER BY timestamp DESC) AS _rn
97+
FROM verisimdb_provenance_log
98+
WHERE table_name = 'posts'
99+
) ranked
100+
WHERE _rn = 1
101+
) prov ON prov.entity_id = (CAST(posts.id AS TEXT));
102+
103+
-- Temporal-enriched view for 'posts'
104+
-- Joins each row with its current version metadata.
105+
CREATE VIEW IF NOT EXISTS verisimdb_posts_with_temporal AS
106+
SELECT
107+
posts.id,
108+
posts.author_id,
109+
posts.title,
110+
posts.body,
111+
posts.published,
112+
posts.created_at,
113+
tv.version AS _verisimdb_version,
114+
tv.valid_from AS _verisimdb_valid_from,
115+
tv.operation AS _verisimdb_version_operation
116+
FROM posts
117+
LEFT JOIN verisimdb_temporal_versions tv
118+
ON tv.entity_id = (CAST(posts.id AS TEXT))
119+
AND tv.table_name = 'posts'
120+
AND tv.valid_to IS NULL;
121+
122+
-- Lineage queries for 'posts'
123+
124+
-- Upstream: what data was this entity derived from?
125+
-- SELECT * FROM verisimdb_lineage_graph
126+
-- WHERE target_entity = :entity_id AND target_table = 'posts';
127+
128+
-- Downstream: what entities depend on this entity?
129+
-- SELECT * FROM verisimdb_lineage_graph
130+
-- WHERE source_entity = :entity_id AND source_table = 'posts';
131+
132+
-- Access control filter for 'posts'
133+
-- Apply this as a WHERE clause addition to enforce row-level security.
134+
--
135+
-- Example usage (parameterised):
136+
-- SELECT * FROM posts
137+
-- WHERE ... AND EXISTS (
138+
-- SELECT 1 FROM verisimdb_access_policies
139+
-- WHERE target_table = 'posts'
140+
-- AND principal = :current_principal
141+
-- AND access_level IN ('read', 'admin')
142+
-- AND active = 1
143+
-- AND (condition IS NULL OR :row_matches_condition)
144+
-- );
145+

0 commit comments

Comments
 (0)