Skip to content

Commit 7bd05b6

Browse files
hyperpolymathclaude
andcommitted
docs: resync status claims with the verified state of the repo
A ground-truth audit (6-way parallel survey, 2026-07-01) catalogued ~23 stale or contradictory claims across the status docs. This brings them in line with what is actually built, proven, and CI-enforced: - ROADMAP: '54% complete' + unchecked Validator/Serializer/Merkle/ Verify/Repair boxes were stale — all are implemented with their soundness proofs landed (each box now carries its honest residual); the 'libochrance.so must be built and linked' blocker was resolved by 703fa00 and is replaced by the real named boundaries. Critical Next Actions replaced with the actual open items (canonical signing serialization + KAT, Stage 4 proofs, placeholder integration tests, required-checks ruleset). - TOPOLOGY dashboard: 'Verify soundness 0%' vs reality (Stages 2.1-2.4 proven), 'Crypto linked 20%/NOT linked' vs reality (CI-gated end to end), Repair/binding rows updated; new rows for validator+policy and the CI proof gate; overall ~40% -> ~70% on the proof axis. - README status: dropped the stale 'stub fallbacks must be removed' paragraph; now states the CI-verified FFI path and names the residual boundaries (CollisionResistant, ABI blake3Hash, trust-root). - EXPLAINME: fixed the self-contradicting Merkle caveat (hashPairStub no longer exists) and replaced the four nonexistent test-file references with the real, CI-wired suites. - PROOFS.adoc: Stage 2.1 EXTENDED entry for the policy-enforcement theorems (PR #71) incl. the goal-rewrite proof technique note, and a CI-ENFORCED note (idris2.yml gates the whole ledger per PR). - ABI-FFI-README: fixed wrong test ipkg path. - ComprehensiveTest: stale 'FFI Crypto (stubbed)' banner corrected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent fbfc201 commit 7bd05b6

7 files changed

Lines changed: 100 additions & 42 deletions

File tree

ABI-FFI-README.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -218,7 +218,9 @@ signature rejection.
218218
### Idris2 Integration Tests
219219

220220
```bash
221-
idris2 --build tests/integration.ipkg
221+
idris2 --install ochrance.ipkg
222+
idris2 --build tests/integration/tests.ipkg
223+
./tests/integration/build/exec/integration-tests
222224
```
223225

224226
## Contributing

EXPLAINME.adoc

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ Three components orchestrate this: A2ML parser/validator (parses filesystem mani
2929

3030
**How verified**: The Merkle tree is defined as a size-indexed binary tree: `MerkleTree : (size : Nat) → Type`. At compile time, Idris2 verifies that all leaf nodes are at depth `log2(size)`. The tree operations (leaf insertion, hash computation, proof generation) take dependent proofs that structure is maintained. README (§Architecture) claims "Verified Merkle trees — Size-indexed binary trees with compile-time structure proofs." This is proven by the type signature itself: if you try to construct a tree of size 8 with only 7 leaves, the type checker rejects it because the dependent type `MerkleTree 8` requires exactly 8 leaves.
3131

32-
**Caveat**: The size-indexed *structure* proofs are independent of hash strength. Real hashing now exists — the Zig FFI implements BLAKE3/SHA-256/SHA3-256 (`ffi/zig/src/main.zig`, with known-answer-vector tests) and the IO Merkle path calls `hashPairBlake3` — but a pure XOR combiner (`hashPairStub`) remains for totality/offline testing, and `libochrance.so` must be built and linked for the cryptographic path to be live. Until the stub paths are removed and the FFI is wired into the verification flow, the structure proofs hold but cryptographic integrity is not yet claimed.
32+
**Caveat**: The size-indexed *structure* proofs are independent of hash strength. The cryptographic path is live: the Zig FFI implements BLAKE3/SHA-256/SHA3-256 (`ffi/zig/src/main.zig`, known-answer-vector tested), `build.zig` emits `libochrance.so`, and the IO Merkle path calls `hashPairBlake3` across `%foreign` — confirmed end to end by `tests/ffi/CryptoFFITest.idr` (the production root is the BLAKE3 fold, not the XOR spec root) and CI-gated in the `Idris2` and `Zig FFI` workflows. The pure XOR combiner (`xorCombiner`) is the totality-friendly *spec instance* of the combiner-generic theorems, not a fallback. The irreducible crypto assumption is `CollisionResistant`, isolated in `Filesystem.MerkleAssumption`.
3333

3434
== Dogfooded Across The Account
3535

@@ -60,12 +60,12 @@ Also integrates with ECHIDNA for neural proof synthesis — FFI calls to libechi
6060

6161
== Testing Critical Paths
6262

63-
* **Lexer totality**: `idris2 --check ochrance-core/A2ML/Lexer.idr` — Idris2 type checker verifies structurally recursive, no partial functions
64-
* **Parser totality**: `idris2 --check ochrance-core/A2ML/Parser.idr` — sized types ensure termination proof
65-
* **Validator**: `tests/validator_test.idr` — roundtrip A2ML → AST → A2ML preserves structure
66-
* **Merkle tree**: `tests/merkle_test.idr` — size-indexed tree construction and hash verification
67-
* **Repair linearity**: `tests/repair_test.idr` — linear type system prevents use-after-repair bugs (compile-time error if violated)
68-
* **ECHIDNA FFI**: `tests/echidna_ffi_test.idr` — call libechidna.so via Zig FFI, verify proof synthesis works
63+
* **Lexer/Parser totality**: `idris2 --build ochrance.ipkg` (opts `--total`) — every core module, including the lexer's structural recursion and the parser's sized-type termination, type-checks under the totality checker; CI-gated by the `Idris2` workflow
64+
* **Parser/roundtrip**: `tests/A2ML/ParserTests.idr` — lex/parse/serialize roundtrip and error handling (fail-capable, CI-gated)
65+
* **Properties**: `tests/property/PropertyTests.idr` — 47 property checks across hex roundtrip, Merkle, repair idempotence, and validation (incl. policy rejection cases)
66+
* **Integration**: `tests/integration/IntegrationTests.idr` — manifest generation → validation → verification → repair scenarios
67+
* **Crypto FFI runtime**: `tests/ffi/CryptoFFITest.idr` (via `tests/ffi/run_ffi_test.sh`) — real BLAKE3/SHA-256/SHA3-256 across `%foreign` into `libochrance.so`; proves the production Merkle root ≠ XOR spec root
68+
* **ECHIDNA FFI**: not yet testable — `Ochrance.FFI.Echidna` is a design stub (`Left "FFI not yet implemented"`); see ROADMAP Phase 2
6969

7070
== Known Limitations & TODOs
7171

README.adoc

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -22,11 +22,16 @@ v0.1.0, 2026-02-06
2222

2323
Formal verification is **in progress** — see link:docs/PROOFS.adoc[`docs/PROOFS.adoc`]
2424
for the authoritative, dependency-sorted plan and the currently-proven surface
25-
(combiner-generic Merkle soundness, A2ML parser totality, manifest round-trip,
26-
progressive assurance). *Cryptographic integrity is not yet claimed*: the real
27-
BLAKE3 / SHA-256 / SHA3-256 / Ed25519 lives in the Zig FFI, but the Idris-side
28-
stub fallbacks must be removed and `libochrance.so` linked into the verification
29-
flow first (issue #39).
25+
(combiner-generic Merkle soundness + binding, verify/validator soundness with
26+
policy enforcement, repair correctness, A2ML parser totality, manifest
27+
round-trip, progressive assurance). The real BLAKE3 / SHA-256 / SHA3-256 /
28+
Ed25519 lives in the Zig FFI (`libochrance.so`), is wired into the Idris
29+
production path via `%foreign`, and is CI-verified end to end: the `Idris2`
30+
workflow builds every proof module under `--total` and runs the Idris→Zig
31+
runtime test on each PR. Named residual boundaries, not hidden: the
32+
`CollisionResistant` hypothesis (isolated in `Filesystem.MerkleAssumption`),
33+
the ABI-layer `blake3Hash` placeholder (`src/abi/`, issue #39), and the
34+
attestation trust-root / key-management story.
3035

3136
== Architecture
3237

ROADMAP.adoc

Lines changed: 24 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -8,44 +8,51 @@ v0.1.0, 2026-02-07
88

99
== Current Status
1010

11-
**Phase 1: Ochránce Core** (54% complete - In Progress)
12-
13-
The A2ML parsing pipeline is fully functional with total lexer and covering parser. Type-safe framework with q/p/z error taxonomy is complete. Current focus: completing validator, serializer, and Merkle tree implementation.
11+
**Phase 1: Ochránce Core** — 10/11 components complete (updated 2026-07-02)
12+
13+
The proof program's Stages 1–3 are machine-checked and CI-enforced: the
14+
`Idris2` workflow builds every core module under `--total` on each PR, and
15+
the Idris→Zig FFI runtime test proves the production Merkle path computes
16+
real BLAKE3 across the `%foreign` boundary. The validator enforces policy
17+
(require_sig, max_age freshness) with soundness theorems. Ledger of record:
18+
`docs/PROOFS.adoc`. Remaining engineering: filling in the placeholder
19+
integration scenarios, canonical signing serialization, and the Stage 4
20+
proofs (Merkle completeness, progressive monotonicity, VerifiedSubsystem
21+
law).
1422

1523
== Development Phases
1624

1725
=== Phase 1: Ochránce Core (v0.1.0) - **In Progress**
1826

1927
Target: Complete core verification framework with full A2ML support
2028

21-
**Completion: 54%**
29+
**Completion: 10/11 components**
2230

2331
Components:
2432

2533
* [x] A2ML Lexer (total, tested)
2634
* [x] A2ML Parser (total, tested)
27-
* [ ] A2ML Validator (comprehensive signature verification)
28-
* [ ] A2ML Serializer (roundtrip: Manifest ↔ String)
35+
* [x] A2ML Validator (structural + policy enforcement proven sound; Ed25519 signature verification via FFI; residual: canonical signing serialization, positive-path signature KAT, trust-root story)
36+
* [x] A2ML Serializer (roundtrip proven for the reference token codec; production pipeline runtime-checked — the String-primitive wall, see PROOFS.adoc)
2937
* [x] Framework Interface (VerifiedSubsystem)
3038
* [x] Proof Witnesses (Lax/Checked/Attested modes)
3139
* [x] Error Taxonomy (q/p/z classification)
32-
* [ ] Merkle Tree Implementation (size-indexed; BLAKE3 FFI now implemented in Zig — build/link wiring still pending)
33-
* [ ] Filesystem Verification (VerifiedSubsystem instance)
34-
* [ ] Linear Type Repair (consume-old-state semantics)
35-
* [ ] Integration Tests (50+ scenarios)
40+
* [x] Merkle Tree Implementation (size-indexed; real BLAKE3 via libochrance.so, CI-gated end to end; residual assumption: CollisionResistant, isolated)
41+
* [x] Filesystem Verification (verify soundness proven — Stages 2.2/2.4; residual: proof-token constructor visibility)
42+
* [x] Linear Type Repair (repair correctness proven — Stages 3.1/3.2; impl models hashes only, no block I/O; full linearity deferred, see PROOFS.adoc)
43+
* [ ] Integration Tests (10 real scenarios green + fail-capable and CI-gated; 45 remaining placeholders to fill)
3644

3745
**Critical Next Actions:**
3846

39-
1. Complete A2ML Validator (signature verification via FFI)
40-
2. Implement A2ML Serializer (roundtrip verification)
41-
3. Wire the implemented Zig BLAKE3 FFI into Merkle and remove the XOR stub combiner
42-
4. Implement Filesystem.Verify module
43-
5. Implement Filesystem.Repair with linear types (Quantity 1)
47+
1. Canonical signing serialization (delimited, all-fields-bound) + positive-path Ed25519 known-answer test through validateManifestIO
48+
2. Stage 4 proofs: Merkle completeness (4.1), remaining SatisfiesMinimum monotonicity cases (4.3), VerifiedSubsystem law stated + proven for FSState
49+
3. Fill in the 45 placeholder integration scenarios
50+
4. Make real BLAKE3 reachable from a root-verify entry point (IO path via the proven IO↔pure bridge; VerifyRoot currently requires a pure Combiner)
51+
5. Require the Idris2 / Zig FFI checks in the repo ruleset (currently no required status checks — auto-merge lands before CI finishes)
4452

4553
**Blockers:**
4654

47-
* Idris2 0.8.0+ compiler required
48-
* `libochrance.so` (Zig crypto FFI) must be built and linked into the Idris2 verification flow; the primitives themselves (BLAKE3/SHA-256/SHA3-256/Ed25519) are now implemented and tested
55+
* None for the components above. Named boundaries: CollisionResistant hypothesis (isolated); ABI-layer blake3Hash placeholder + ECHIDNA FFI stub (issue #39, Phase 2 scope); attestation trust-root / key management (design decision needed)
4956

5057
=== Phase 2: ECHIDNA Integration (v0.2.0) - **Planned**
5158

TOPOLOGY.md

Lines changed: 28 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ SPDX-License-Identifier: CC-BY-SA-4.0
33
Copyright (c) Jonathan D.A. Jewell <j.d.a.jewell@open.ac.uk>
44
-->
55
<!-- TOPOLOGY.md — Project architecture map and completion dashboard -->
6-
<!-- Last updated: 2026-06-17 -->
6+
<!-- Last updated: 2026-07-02 -->
77

88
# Ochránce — Project Topology
99

@@ -65,25 +65,42 @@ COMPONENT STATUS NOTES (proof / verification
6565
─────────────────────────────────── ────────────── ────────────────────────────────────────────
6666
CORE VERIFICATION (IDRIS2)
6767
Merkle soundness + round-trip ███████░░░ 70% merkleCorrect(With) + buildGetLeaf proven
68-
Merkle completeness + binding ░░░░░░░░░░ 0% converse + CollisionResistant (Stage 1.4/4)
68+
Merkle completeness + binding █████░░░░░ 50% binding DONE (merkleBinding; CR isolated in
69+
MerkleAssumption); completeness converse open (4.1)
6970
A2ML parser + reference round-trip ███████░░░ 70% total; ref-codec proven; prod pipeline runtime-only
70-
Verify soundness ░░░░░░░░░░ 0% verify success ⇏ a proof yet (Stage 2)
71-
Linear-type Repair ██░░░░░░░░ 20% IMPL IS A STUB (no block I/O); unproven (Stage 3)
72-
VerifiedSubsystem law █████░░░░░ 50% interface + FSState instance; law unproven (Stage 3)
73-
Progressive assurance ███████░░░ 70% attestedSatisfiesLax proven; monotonicity (Stage 4)
71+
Verify soundness ████████░░ 80% Stages 2.1–2.4 proven (verifyRefsSound,
72+
merkleRootVerifyHashSound mod CR+DecodeInjective);
73+
residual: proof-token constructor visibility
74+
Validator + policy enforcement ████████░░ 80% validateManifestSound (4-tuple incl. policy);
75+
require_sig/max_age proven enforced; residual:
76+
canonical signing serialization + positive KAT
77+
Linear-type Repair ██████░░░░ 60% Stages 3.1/3.2 proven (mod GoodRefs+hashRefl);
78+
impl hash-map-only, no block I/O; linearity deferred
79+
VerifiedSubsystem law █████░░░░░ 50% interface + FSState instance; law not yet STATED
80+
in Interface.idr (repairThenVerify proven at
81+
pure-core level, not lifted)
82+
Progressive assurance ███████░░░ 70% attestedSatisfiesLax proven; monotonicity (Stage 4.3)
7483
7584
SYSTEM & EXTERNAL
76-
Zig crypto (BLAKE3/SHA/Ed25519) ███████░░░ 70% implemented + known-answer-vector tested in Zig
77-
Crypto linked into verify flow ██░░░░░░░░ 20% NOT linked; Idris-side stubs still live (#39)
85+
Zig crypto (BLAKE3/SHA/Ed25519) █████████░ 90% implemented + KAT-tested; Zig 0.15.2; 24 tests
86+
(unit + linked C-ABI integration) CI-gated
87+
Crypto linked into verify flow ███████░░░ 70% libochrance.so built+linked+CI-gated end to end
88+
(Idris2 workflow runs run_ffi_test.sh per PR);
89+
residual: root-verify entry needs pure Combiner
90+
(IO path open), ABI blake3Hash placeholder (#39)
7891
ECHIDNA integration █░░░░░░░░░ 10% design types only; FFI entirely stubbed
7992
8093
REPO INFRASTRUCTURE
8194
Justfile automation ██████████ 100% build / verify tasks
82-
.machine_readable/ ██████████ 100% STATE tracking active
83-
Idris2 .ipkg / totality gate ██████████ 100% 19 core modules; --total build green
95+
.machine_readable/ ██████████ 100% STATE tracking active; descriptiles (post-6a2)
96+
Idris2 .ipkg / totality gate ██████████ 100% 29 core modules; --total build green
97+
CI proof gate (Idris2 workflow) ██████████ 100% every PR: full --total build + 3 fail-capable
98+
suites + Idris→Zig FFI runtime test
8499
85100
────────────────────────────────────────────────────────────────────────────────────────────────
86-
OVERALL (proof axis): ████░░░░░░ ~40% Stage 1.1 done; 1.2–4 open. See docs/PROOFS.adoc.
101+
OVERALL (proof axis): ███████░░░ ~70% Stages 1–3 done (mod named hypotheses);
102+
Stage 4 open: Merkle completeness, monotonicity
103+
cases, VerifiedSubsystem law. See docs/PROOFS.adoc.
87104
```
88105

89106
## Key Dependencies

docs/PROOFS.adoc

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44

55
// SPDX-License-Identifier: CC-BY-SA-4.0
66
// SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell (hyperpolymath)
7+
// Copyright (c) Jonathan D.A. Jewell <j.d.a.jewell@open.ac.uk>
78

89
This is the authoritative, dependency-sorted plan for driving the formal
910
verification of the Ochránce estate to completion. It is the durable source of
@@ -213,6 +214,32 @@ only `map id y` (functor-identity *law*, not definitional) — so invert by casi
213214
on the head outcome *and* the tail fold, keeping each step definitional; (ii) `with`
214215
abstracts only a hypothesis's WHNF, so case on `validateRef ref` (exposed there),
215216
not the nested `isValidHexString`.
217+
+
218+
EXTENDED (2026-07-02): policy enforcement is now part of the proven surface.
219+
`validatePolicy` — previously dead code in the production path (a manifest with
220+
`require_sig = true` and no attestation *passed*) — is wired into
221+
`validateManifest`, and `validateManifestSound` is a 4-tuple: acceptance also
222+
forces `validatePolicy m = Right ()`. New theorems, all machine-checked,
223+
axiom-free: `requireSigNoAttestationRejected` (the gate is provably wired in);
224+
`staleManifestRejected` (end-to-end: `validatePolicyAt` rejects a manifest
225+
older than `max_age` — proved by rewriting the goal along the
226+
`parseTimestamp ts = Just issued` hypothesis, which unblocks both stuck case
227+
scrutinees at once; the `with`-abstraction route fails here because `with`
228+
cannot rewrite an already-fixed hypothesis); `checkFreshnessRejectsStale` /
229+
`checkFreshnessAcceptsFresh`; `hexAcceptSound` (strengthened check: exactly 64
230+
hex chars AND hex-only content — the old check accepted `'.'`, empty and
231+
any-length strings) + `hexWrongLengthRejected` + `hexEmptyRejected`; and
232+
known-answer proofs for the total ISO-8601-subset `parseTimestamp` (epoch = 0,
233+
modern date cross-checked, garbage and month-13 rejected). Freshness is
234+
clock-free in the pure layer (`validatePolicyAt` takes `now`;
235+
`validateManifestIO` fetches `System.time`) — IO↔pure-bridge style.
236+
+
237+
CI-ENFORCED (2026-07-01): the entire ledger is now gated per PR — the `Idris2`
238+
workflow builds every core module under `--total` (a broken proof is a red
239+
check) and runs the Idris→Zig FFI runtime test (`tests/ffi/run_ffi_test.sh`),
240+
so the production-Merkle-root-is-real-BLAKE3 fact is re-verified on every
241+
change. The three Idris test suites are fail-capable (exit 1) as of the same
242+
change.
216243

217244
. *[DONE — 2.2 (inversion); merkle-wiring deferred]* `verify` soundness. Lifted
218245
`verifyRefsHelper` / `parseBlockIdx` out of the instance `where` to top-level

ochrance-core/ComprehensiveTest.idr

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,5 +69,5 @@ main = do
6969
putStrLn " • A2ML (Types/Lexer/Validator): ✓"
7070
putStrLn " • Framework (Interface/Proofs): ✓"
7171
putStrLn " • Filesystem (Types/Merkle): ✓"
72-
putStrLn " • FFI Crypto (stubbed): "
72+
putStrLn " • FFI Crypto (real, via libochrance):"
7373
putStrLn ""

0 commit comments

Comments
 (0)