Skip to content

Commit 0f86e4e

Browse files
committed
verisimiser: make ABI/FFI/codegen provably-real + honest docs
Machine-check the claims VeriSimiser makes outside the Rust layer, and add a CI gate (provable.yml) so they stay checked. Idris2 ABI (src/interface/abi/Verisimiser/ABI/): - Complete Layout.idr with full, hole-free proofs. Each of the four C-struct layouts (OctadRecord, ProvenanceEntry, DriftMeasurement, TemporalSnapshot) now carries machine-checked size/alignment (CABICompliant) and per-field offset (FieldsAligned) witnesses; alignUp is redefined so alignUpCorrect is unconditional. - Remove generic helpers that cannot be proven honestly (checkCABI, offsetInBounds, verifyLayout, calcStructSize, PlatformLayout) with NOTEs explaining why; keep the concrete, checkable instances. - Types/Layout/Foreign all `idris2 --check` clean (no holes, no warnings). Zig FFI (src/interface/ffi/): - Fix build.zig for Zig 0.14: drop the lib.version panic and the missing addInstallHeader; add linkLibC (the impl uses std.heap.c_allocator); wire the integration tests into `zig build test`. - Handle/DbConnection: extern struct -> struct (they hold a std.mem.Allocator, which has no C layout; C only ever sees them behind a pointer). - Fix a pointless-discard compile error, and the integration test's per-decl `?*opaque {}` (now one shared opaque Handle type). - `zig build test` is green: 43/43 (8 unit + 35 C-ABI integration through the compiled libverisimiser.so). Codegen golden + CI: - examples/golden/: frozen manifest + schema + committed generated overlay. - .github/workflows/provable.yml: idris2-proofs, zig-ffi, codegen-drift, and sql-golden (applies the generated overlay to a real SQLite DB and asserts every sidecar table + enrichment view builds and is queryable). Honesty docs: - Rewrite docs/developer/ABI-FFI-README.adoc (was an unrendered template full of fictional commands/paths) into the real, proven ABI/FFI contract. - README/ROADMAP/EXPLAINME/STATE: mark what is machine-checked vs aspirational and point to provable.yml. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mbq6yKF9RhFai6EQ7WqKhQ
1 parent caeb8fd commit 0f86e4e

18 files changed

Lines changed: 981 additions & 582 deletions

File tree

.github/workflows/provable.yml

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
# SPDX-License-Identifier: MPL-2.0
2+
# Provable — machine-checks the claims VeriSimiser makes outside the Rust layer.
3+
#
4+
# The Rust CLI/codegen is covered by rust-ci.yml (fmt + clippy + test). This
5+
# workflow verifies the four things that are otherwise only asserted in prose:
6+
# 1. idris2-proofs — the Idris2 ABI proofs actually type-check (`idris2 --check`).
7+
# 2. zig-ffi — the Zig FFI reference impl builds and its tests pass.
8+
# 3. codegen-drift — the committed golden sample matches fresh codegen output.
9+
# 4. sql-golden — the generated overlay SQL actually applies to a real
10+
# SQLite database and every table/view builds and queries.
11+
#
12+
# Until every job here is green, the proofs / FFI / generated SQL are "written"
13+
# but NOT "verified" — ROADMAP.adoc and README.adoc are worded accordingly.
14+
name: Provable
15+
16+
on:
17+
push:
18+
branches: [main, master, "claude/**"]
19+
pull_request:
20+
21+
permissions:
22+
contents: read
23+
24+
jobs:
25+
idris2-proofs:
26+
name: Idris2 — machine-check ABI proofs
27+
runs-on: ubuntu-latest
28+
timeout-minutes: 20
29+
container: ghcr.io/stefan-hoeck/idris2-pack:latest
30+
steps:
31+
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
32+
- name: idris2 --check the ABI modules
33+
working-directory: src/interface/abi
34+
# Modules are namespaced Verisimiser.ABI.*, so they live under
35+
# Verisimiser/ABI/ (idris2 requires the file path to match the module
36+
# name). Foreign imports Types + Layout, so checking it checks the lot;
37+
# we check each explicitly for clearer failure attribution.
38+
run: |
39+
idris2 --version
40+
idris2 --check Verisimiser/ABI/Types.idr
41+
idris2 --check Verisimiser/ABI/Layout.idr
42+
idris2 --check Verisimiser/ABI/Foreign.idr
43+
44+
zig-ffi:
45+
name: Zig — build + test FFI reference impl
46+
runs-on: ubuntu-latest
47+
timeout-minutes: 15
48+
steps:
49+
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
50+
- uses: mlugg/setup-zig@53fc45b17fe98b52f92ee5ea08ff48a85a3e7eb7 # v1.2.2
51+
with:
52+
version: 0.14.0
53+
- name: zig build + test
54+
working-directory: src/interface/ffi
55+
# `zig build test` runs the 8 in-module unit tests AND the 35 C-ABI
56+
# integration tests (which link the compiled libverisimiser.so).
57+
run: |
58+
zig version
59+
zig build test
60+
zig build
61+
62+
codegen-drift:
63+
name: Codegen — golden sample is up to date
64+
runs-on: ubuntu-latest
65+
timeout-minutes: 15
66+
steps:
67+
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
68+
- name: Regenerate golden artifacts and diff against committed tree
69+
run: |
70+
# retry: fresh runners occasionally flake fetching crates from crates.io
71+
built=0
72+
for i in 1 2 3; do
73+
cargo build --quiet && { built=1; break; }
74+
echo "build attempt $i failed; retrying after backoff…"; sleep $((i * 5))
75+
done
76+
[ "$built" = 1 ] || { echo "cargo build failed after retries"; exit 1; }
77+
( cd examples/golden && ../../target/debug/verisimiser generate -m verisimiser.toml -o /tmp/golden )
78+
diff -ru examples/golden/generated /tmp/golden
79+
80+
sql-golden:
81+
name: SQLite — generated overlay applies and views build
82+
runs-on: ubuntu-latest
83+
timeout-minutes: 10
84+
steps:
85+
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
86+
- name: Ensure sqlite3 is available
87+
run: sudo apt-get update && sudo apt-get install -y sqlite3
88+
- name: Apply schema + generated overlay + interceptors into a fresh SQLite DB
89+
run: |
90+
sqlite3 --version
91+
DB="$(mktemp -d)/golden.db"
92+
sqlite3 "$DB" < examples/golden/schema.sql
93+
sqlite3 "$DB" < examples/golden/generated/sidecar_schema.sql
94+
sqlite3 "$DB" < examples/golden/generated/interceptors.sql
95+
96+
# Every enabled octad dimension must have produced its sidecar table.
97+
for t in verisimdb_metadata verisimdb_provenance_log \
98+
verisimdb_lineage_graph verisimdb_temporal_versions \
99+
verisimdb_access_policies; do
100+
sqlite3 "$DB" "SELECT name FROM sqlite_master WHERE type='table' AND name='$t';" \
101+
| grep -qx "$t" || { echo "missing sidecar table: $t"; exit 1; }
102+
done
103+
104+
# Every target table must have provenance + temporal enrichment views,
105+
# and each view must be queryable (catches invalid SQL in view bodies).
106+
for v in verisimdb_users_with_provenance verisimdb_users_with_temporal \
107+
verisimdb_posts_with_provenance verisimdb_posts_with_temporal; do
108+
sqlite3 "$DB" "SELECT name FROM sqlite_master WHERE type='view' AND name='$v';" \
109+
| grep -qx "$v" || { echo "missing enrichment view: $v"; exit 1; }
110+
sqlite3 "$DB" "SELECT count(*) FROM $v;" >/dev/null \
111+
|| { echo "view not queryable: $v"; exit 1; }
112+
done
113+
114+
echo "Generated overlay applied cleanly; all sidecar tables + views present and queryable."

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,10 @@ flake.lock
6565
# Chapel
6666
*.chpl.tmp.*
6767

68+
# Zig (FFI reference impl build artifacts under src/interface/ffi/)
69+
zig-out/
70+
.zig-cache/
71+
6872
# Secrets
6973
.env
7074
.env.*

.machine_readable/6a2/STATE.a2ml

Lines changed: 15 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,9 @@
55
[metadata]
66
project = "verisimiser"
77
version = "0.1.0"
8-
last-updated = "2026-03-21"
8+
last-updated = "2026-06-27"
99
status = "active"
10-
session = "converted from scheme — 2026-04-11"
10+
session = "provable: ABI proofs + Zig FFI + codegen golden machine-checked — 2026-06-27"
1111

1212
[project-context]
1313
name = "Verisimiser"
@@ -32,9 +32,19 @@ issues = [
3232
actions = [
3333
"Begin Phase 2 — implement core domain logic for verisimiser",
3434
"Write property-based tests for manifest parsing",
35-
"Define Idris2 ABI proof obligations for Phase 3",
35+
"Lift hash-chain integrity / version ordering / lineage acyclicity into Idris proofs (currently runtime-only)",
3636
]
3737

38+
# What is machine-checked outside the Rust layer, gated by
39+
# .github/workflows/provable.yml. Verified locally 2026-06-27:
40+
# idris2 --check clean, `zig build test` 43/43, codegen golden deterministic,
41+
# generated overlay applies to SQLite.
42+
[provable-status]
43+
idris2-abi = "type-checks: octad cardinality, CABICompliant layout/alignment (x4 structs), non-null handle invariants, Tier-1 sidecar isolation"
44+
zig-ffi = "builds (shared + static) and passes 43 tests (8 unit + 35 C-ABI integration)"
45+
codegen-golden = "examples/golden regenerates deterministically and the SQL applies cleanly to SQLite"
46+
not-yet-proven = "hash-chain integrity, version ordering, lineage acyclicity (runtime/SQL-CHECK only); live DB interception (Phase 1+)"
47+
3848
[maintenance-status]
39-
last-run-utc = "2026-03-21T00:00:00Z"
40-
last-result = "unknown" # unknown | pass | warn | fail
49+
last-run-utc = "2026-06-27T00:00:00Z"
50+
last-result = "pass" # unknown | pass | warn | fail

EXPLAINME.adoc

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,34 @@ ____
1010
Jonathan D.A. Jewell <j.d.a.jewell@open.ac.uk>
1111
____
1212

13+
== The Receipts (machine-checked)
14+
15+
The non-Rust claims are verified on every push/PR by
16+
`.github/workflows/provable.yml` — these are checks, not prose:
17+
18+
[cols="1,3"]
19+
|===
20+
| Job | What it proves
21+
22+
| `idris2-proofs`
23+
| `idris2 --check` the ABI: octad cardinality, C-struct layout/alignment
24+
(`CABICompliant` ×4), non-null handle invariants, Tier-1 sidecar isolation.
25+
26+
| `zig-ffi`
27+
| `zig build test` — the Zig FFI builds and 43 tests pass (8 unit + 35 C-ABI
28+
integration through the compiled `libverisimiser.so`).
29+
30+
| `codegen-drift`
31+
| `examples/golden/` regenerates byte-identically (the generator is deterministic).
32+
33+
| `sql-golden`
34+
| the generated overlay SQL applies to a fresh SQLite DB and every sidecar
35+
table + enrichment view builds and is queryable.
36+
|===
37+
38+
The Rust CLI is covered separately by `rust-ci.yml` (fmt + clippy + test). See
39+
`docs/developer/ABI-FFI-README.adoc` for the full ABI/FFI contract.
40+
1341
== Technology Choices
1442

1543
[cols="1,2"]

README.adoc

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,23 @@ therefore split into two tiers:
2727
storage alongside your database. Honest about being "VeriSimDB with your
2828
database as one backend" rather than pretending to be invisible.
2929

30+
[NOTE]
31+
====
32+
*What is machine-checked today.* The parts of VeriSimiser that live outside the
33+
Rust layer are verified in CI by `.github/workflows/provable.yml`, so the
34+
claims below are not just prose:
35+
36+
* the Idris2 ABI proofs type-check (`idris2 --check`) — octad cardinality,
37+
C-struct layout/alignment (`CABICompliant`), non-null handle invariants, and
38+
Tier-1 sidecar isolation;
39+
* the Zig FFI builds and its 43 tests pass (`zig build test`); and
40+
* the SQL the CLI generates applies cleanly to a real SQLite database.
41+
42+
The database *interception* daemon itself (Phase 1+ in `ROADMAP.adoc`) is not
43+
yet built — capabilities that are still aspirational are marked as such here
44+
and in the roadmap.
45+
====
46+
3047
== Octad: Eight Concerns
3148

3249
Each entity in a VeriSimiser-augmented database is enriched along

ROADMAP.adoc

Lines changed: 22 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,14 @@
44
:toc:
55
:icons: font
66

7+
[NOTE]
8+
====
9+
Items are ticked `[x]` only when machine-checked or otherwise demonstrably
10+
done. The ABI proofs, FFI, and codegen are gated by
11+
`.github/workflows/provable.yml` (see `docs/developer/ABI-FFI-README.adoc`):
12+
an unticked box means the claim is not yet verified, not merely undocumented.
13+
====
14+
715
== Phase 0: Scaffold (COMPLETE)
816
* [x] RSR template with full CI/CD
917
* [x] CLI with subcommands (init, start, drift, provenance, history, status, octad, doctor, gc, validate, version)
@@ -13,6 +21,10 @@
1321
* [x] ABI module stubs (Idris2 + Zig FFI) and domain-tagged `compute_hash`
1422
* [x] Drift categories pinned (ADR-0003): input / distance / threshold per category
1523
* [x] README rewritten around concerns octad (V-L1-A2, #20)
24+
* [x] ABI proofs machine-checked + FFI built/tested in CI (`provable.yml`):
25+
octad cardinality, C-struct layout/alignment (`CABICompliant` ×4), non-null
26+
handle invariants, Tier-1 sidecar isolation; Zig FFI 43 tests; codegen golden
27+
+ SQLite-apply check
1628

1729
== Phase 1: PostgreSQL Tier 1 MVP — concerns layered onto your DB
1830
* [ ] PostgreSQL logical replication interception (Data / Metadata feed)
@@ -23,8 +35,16 @@
2335
* [ ] **Constraints** / drift detection — read-path observer; eight categories (ADR-0003)
2436
* [x] **Constraints** category 1 of 8: Temporal drift detector (V-L1-E2 / #49)
2537
* [ ] **Constraints** categories 2–8 of 8: Structural, Semantic, Statistical, Referential, Provenance, Spatial, Embedding
26-
* [ ] Idris2 ABI proofs: sidecar isolation, hash-chain integrity, version ordering, lineage acyclicity
27-
* [ ] Zig FFI bridge: database connection, sidecar operations, VCL-total queries
38+
* [ ] Idris2 ABI proofs (partial — gated by `provable.yml`):
39+
** [x] sidecar isolation (Tier-1 never writes the target table)
40+
** [x] C-struct layout + alignment (`CABICompliant` for all four ABI structs)
41+
** [ ] hash-chain integrity, version ordering, lineage acyclicity — today
42+
enforced at runtime in Rust + SQL `CHECK` constraints; not yet lifted into
43+
Idris proofs
44+
* [ ] Zig FFI bridge (partial — gated by `provable.yml`):
45+
** [x] C-ABI surface declared (`Foreign.idr`), implemented in Zig, 43 tests pass
46+
** [ ] live database connection, sidecar operations, VCL-total queries —
47+
TODO-stubbed in the Zig layer; the working implementation lives in the Rust CLI
2848
* [ ] End-to-end test: PostgreSQL → verisimiser concerns → VCL-total query
2949

3050
== Phase 2: Multi-Backend Support

0 commit comments

Comments
 (0)