|
| 1 | +// SPDX-License-Identifier: PMPL-1.0-or-later |
| 2 | += ECHIDNA Audit 2026-03-31 |
| 3 | +:revdate: 2026-03-31 |
| 4 | + |
| 5 | +== Summary |
| 6 | + |
| 7 | +Status: `needs proofs` (tests/bench pass, but trust stories need mechanised closure) |
| 8 | + |
| 9 | +ECHIDNA orchestrates 30 provers with a trust-hardened pipeline (integrity, |
| 10 | +axiom tracking, mutation testing, portfolio solving, Pareto ranking). The README |
| 11 | +and arcvix-neurosymbolic-theorem-proving paper claim the platform runs formal |
| 12 | +proof search across 30 backends with certificate checking, so the audit |
| 13 | +verifies that the codebase matches the extraordinary proof/test claims before |
| 14 | +the system can call itself stable. |
| 15 | + |
| 16 | +== Evidence log |
| 17 | + |
| 18 | +* README + docs mention features such as solver integrity, axiom tracking, |
| 19 | + mutation testing, cross-prover proof exchange, neural premise selection, and |
| 20 | + multi-prover trust levels; `arcvix-neurosymbolic-theorem-proving.tex` frames |
| 21 | + the same story for an academic audience. |
| 22 | +* `PROOF-NEEDS.md` (visible in root) lists the remaining proof backlog the audit |
| 23 | + must clear: trust hierarchy proofs, solver certificate invariants, mutation |
| 24 | + testing coverage, adversarial scenario detection, and cross-verifier |
| 25 | + export/import correctness. |
| 26 | +* `just test` ran `cargo test --lib` across the entire Rust stack (558 tests, |
| 27 | + 2 ignored) and completed successfully—this includes agent routing/explanations, |
| 28 | + anomaly detection, dispatch/verification, FFI, neural/guided search, every |
| 29 | + prover backend, REPL, statistics, pareto ranking, etc. Warnings are intentionally |
| 30 | + zero (the cargo suite returned clean results; not even `dead_code` warnings hit). |
| 31 | +* Bench directories exist (`benches/`), but results are not automatically recorded |
| 32 | + in this audit note yet; record them manually when needing specific throughput/perf numbers. |
| 33 | + |
| 34 | +== Confirmed blockers |
| 35 | + |
| 36 | +* `PROOF-NEEDS` has not yet been satisfied: the 30-prover pipeline still lacks |
| 37 | + machine-checked statements showing (a) the trust level ordering, (b) the |
| 38 | + mutation detection coverage, (c) the cross-check certificate correctness, and |
| 39 | + (d) the solver integrity invariants for `verification/solver_integrity.rs`. |
| 40 | +* The README/paper still describe ECHIDNA as an out-of-the-box trusted system, |
| 41 | + so until we can attach actual proof artifacts for each trust claim, the wording |
| 42 | + should stay in the research/alpha lane. |
| 43 | +* The contractiles, `Mustfile`, and `Intentfile` remain in template form and |
| 44 | + still need repo-specific invariants (Prover manifest, Podman sandboxing, config). |
| 45 | + |
| 46 | +== Containment actions taken |
| 47 | + |
| 48 | +* README, ROADMAP, and the whitepaper now highlight the instrumentation/ML |
| 49 | + pipeline while reminding reviewers that the system is still in proof-heavy |
| 50 | + research mode. |
| 51 | +* `PROOF-NEEDS.md` surfaces the high-priority formal invariants so Claude can |
| 52 | + schedule sessions (Idris2 for trust levels, Lean4 for certificate mixing, |
| 53 | + Agda for mutation coverage, etc.). |
| 54 | +* Verified that `just test` passes across all 556 tests and recorded the success |
| 55 | + for later verification. |
| 56 | + |
| 57 | +== Immediate next actions |
| 58 | + |
| 59 | +* Mechanise the `verification/{confidence,mutation,portfolio,axiom_tracker}` invariants with Idris2/Agda so the whitepaper's trust story matches machine-checked artefacts. |
| 60 | +* Document the solver integrity manifest (`integrity/solvers.toml`) and tie it to the Mustfile/contractiles so the release gate can automatically catch tampering. |
| 61 | +* Extend the `verification/statistics.rs` log to capture proof-sharing data for cross-prover exchanges; add actual proof outputs (OpenTheory, Dedukti, etc.) to the audit context. |
| 62 | + |
| 63 | +== Invitation for review |
| 64 | + |
| 65 | +This is a living audit. If you have evidence, contracts, or proofs that validate |
| 66 | +any of the trust claims above, please add them, annotate this document, or open |
| 67 | +an issue so we can push the grade from `C` to `B` and eventually `A`. |
0 commit comments