|
| 1 | +# Echidna Production-Wiring — State of Things |
| 2 | + |
| 3 | +**Where we are now, in relation to the forward vision.** Complements |
| 4 | +`ECHIDNA-TODO.md` (actionable backlog) and the full continuation |
| 5 | +prompts at `verification-ecosystem/echidna/docs/handover/`. |
| 6 | + |
| 7 | +Last updated: 2026-04-20. |
| 8 | + |
| 9 | +--- |
| 10 | + |
| 11 | +## Vision (one paragraph) |
| 12 | + |
| 13 | +Take Echidna from "48 / 105 backends trait-wired with mock-only CI" to |
| 14 | +"production-level live subprocess CI across ~38 backends, Cap'n Proto |
| 15 | +IPC end-to-end, Chapel as first-class parallel execution layer." Three |
| 16 | +phases, deliberately sequenced **L3 → L1 → L2**: live-prover CI first |
| 17 | +because it surfaces real bugs mocks hide, Cap'n Proto next so Chapel |
| 18 | +can consume its schemas, Chapel last because its sub-waves are the |
| 19 | +largest piece. Guix is the authoritative package manager throughout |
| 20 | +with Nix as fallback. GitHub is the single source of truth; no other |
| 21 | +forges pushed directly. |
| 22 | + |
| 23 | +## Decisions locked |
| 24 | + |
| 25 | +| # | Decision | Rationale | |
| 26 | +|---|----------|-----------| |
| 27 | +| D1 | **Serialization = Cap'n Proto** | Chosen over Bebop3 for dependability + maturity (Cloudflare Workers use at scale), zero-copy reads, strong schema-evolution. Tradeoff: heavier codegen; shim Julia/Chapel via C-ABI (fits existing Idris2-ABI + Zig-FFI). | |
| 28 | +| D2 | **Chapel = first-class, maximal** | 420-LoC POC promoted to `src/chapel/` across 7 sub-waves — portfolio dispatch, speculative tactic search, corpus-parallel ops, mutation-testing parallelism, multi-locale distributed, numeric hot paths. | |
| 29 | +| D3 | **Guix primary, Nix fallback** | Per project CLAUDE.md. `guix.scm` / `manifests/live-provers.scm` authoritative; `flake.nix` mirrors. | |
| 30 | +| D4 | **Execution order = L3 → L1 → L2** | Live-prover CI first: highest-leverage gap, surfaces real bugs mocks hide. Cap'n Proto next, since Chapel consumes those schemas. | |
| 31 | +| D5 | **Live-prover CI cadence tiered** | T1 every PR, T2 nightly, T3 weekly, T4 quarterly allow-fail. | |
| 32 | +| D6 | **No JSON emit on hot path** | Per `feedback_no_json_emit_a2ml`. Cap'n Proto replaces HTTP-JSON Rust↔Julia. Tool config stays Nickel/A2ML. | |
| 33 | + |
| 34 | +## Current state per phase |
| 35 | + |
| 36 | +### L3 — Live-Prover CI — PARTIALLY SHIPPED |
| 37 | + |
| 38 | +Four waves. First two done, third and fourth scaffolded only. |
| 39 | + |
| 40 | +| Wave | Scope | Status | Commits | |
| 41 | +|------|-------|--------|---------| |
| 42 | +| Wave-1 | Tier-1 apt-installable (9 backends) every PR: Z3, CVC5, Vampire, EProver, SPASS, Alt-Ergo, GLPK, MiniZinc, Chuffed | **DONE** 2026-04-19 | `b022bf4` | |
| 43 | +| Wave-2 | Tier-2 build-from-source (10 backends) nightly: coq/agda/why3 (apt), idris2 (source bootstrap), lean4 (elan), isabelle (Isabelle2024 tarball), dafny (dotnet tool), fstar (release tarball), tlaps (installer). hol-light deferred to Wave-3. | **DONE 2026-04-19** locally; CI-unverified | `9a4aeeb`, `6717b12` | |
| 44 | +| Wave-3 | Tier-3 weekly, 9 backends (Tamarin, ProVerif, Imandra, SCIP, OR-Tools, HOL4, ACL2, Twelf, Metamath). Needs per-backend Containerfiles (Podman). | **SCAFFOLD ONLY** — handover hints in STATE.a2ml | — | |
| 45 | +| Wave-4 | Tier-4 quarterly, 19 backends. Retained as mock-only unless a maintainer volunteers. | **SCAFFOLD ONLY** | — | |
| 46 | + |
| 47 | +Local verification of Wave-1 + Wave-2: **18/18 live tests pass** (13 real versions returned, 5 auto-skipped for missing binaries: GLPK/SPASS/MiniZinc/TLAPS/Chuffed). |
| 48 | + |
| 49 | +**Dafny flagged as shallow** (165 LoC subprocess wrapper). Live version-check passes but the wiring is stub-ish — needs L3-phase deepening before mocks retire. |
| 50 | + |
| 51 | +### L1 — Cap'n Proto Protocol Swap — NOT STARTED |
| 52 | + |
| 53 | +Blockers: L3 Tier-1 green on main for ≥ 7 days. Current IPC: |
| 54 | +- **Rust ↔ Julia**: HTTP + JSON (`src/rust/gnn/client.rs:1-195` → `src/julia/api_server.jl:8090`) — violates `no_json_emit`. |
| 55 | +- **Rust ↔ Chapel**: Stub (Zig bridge self-links but not in dispatch path). |
| 56 | +- **Rust ↔ gRPC**: schema-only; `dispatch.rs` uses in-process traits. |
| 57 | + |
| 58 | +### L2 — Chapel Maximum Integration — PARTIALLY PREPARED |
| 59 | + |
| 60 | +- 420-LoC POC at `chapel_poc/parallel_proof_search.chpl` + Chapel `export` functions at `chapel_poc/chapel_ffi_exports.chpl`. |
| 61 | +- Zig FFI bridge at `src/zig_ffi/chapel_bridge.zig` — **now self-links** against bundled stubs (commit `53ab9b8`, 2026-04-19). `cargo build --features chapel` works standalone; 6/6 `proof_search` tests pass. |
| 62 | +- Nothing in dispatch path yet. No `src/chapel/` directory. All 7 sub-waves pending. |
| 63 | + |
| 64 | +## Wiring depth snapshot (from 2026-04-19 audit) |
| 65 | + |
| 66 | +- **Deep** (persistent process + structured protocol parsing): Z3, CVC5, Coq (SerAPI), Lean 4, Idris2. |
| 67 | +- **Medium** (subprocess + output parse, no streaming): Agda, Vampire. |
| 68 | +- **Stub-ish / thin**: Dafny (165 LoC) — flagged for hardening during L3. |
| 69 | + |
| 70 | +Also corrected 2026-04-19 (were mis-listed as "planned"): |
| 71 | +- **Tamarin** — fully wired (`provers/tamarin.rs`, 592 LoC, registered in `ProverFactory`, 4 unit tests). |
| 72 | +- **ProVerif** — fully wired (`provers/proverif.rs`, 799 LoC, registered). |
| 73 | +- **No TODO/FIXME in `src/rust/`** — 0 matches; standing property. |
| 74 | + |
| 75 | +## Corpus |
| 76 | + |
| 77 | +66,674 proofs across 16 prover systems (`COMPLETE_CORPUS_SUMMARY.md`). 179,933 tactics, 10,599 unique tactic signatures, 300 indexed premises. Untouched this session. |
| 78 | + |
| 79 | +## CI today |
| 80 | + |
| 81 | +- `.github/workflows/rust-ci.yml` — mock-only tests on every PR. Baseline smoke. |
| 82 | +- `.github/workflows/chapel-ci.yml` — compiles Chapel POC and the Zig FFI bridge. **Does not feed into dispatch** and **does not link real Chapel**; tests run against bundled stubs only. |
| 83 | +- `.github/workflows/live-provers.yml` — tiered workflow (T1 PR, T2 nightly, T3 weekly, T4 quarterly). T1 + T2 matrices filled with real provisioning; T3 + T4 placeholder jobs. |
| 84 | +- `.github/workflows/agda-meta-checker.yml` — formally-verified trust-pipeline properties. |
| 85 | + |
| 86 | +## Architectural invariants (from 0-AI-MANIFEST.a2ml + CLAUDE.md) |
| 87 | + |
| 88 | +- **Idris2 ABI** for formal proofs; zero `believe_me`. |
| 89 | +- **Zig FFI** for C-ABI bridges to polyglot consumers. |
| 90 | +- **Justfile** primary build system; not Make. |
| 91 | +- **Containerfile** + Podman; not Dockerfile / Docker. |
| 92 | +- **State files in `.machine_readable/6a2/` only**; never root. |
| 93 | +- **All interfaces under `src/interfaces/`**; never extract to separate repos. |
| 94 | +- **When adding provers: update all 3 layers** (Rust backend, Julia ML, Chapel). |
| 95 | +- **Original name: "Cognitive Hybrid"** not "Computational Heuristic". |
| 96 | +- **PMPL-1.0-or-later** throughout. |
| 97 | +- **No Python** — Julia for ML, Rust for systems, ReScript for apps. |
| 98 | + |
| 99 | +## Tech stack |
| 100 | + |
| 101 | +- **Primary**: Rust (48 / 105 prover backends, trust pipeline, CLI, REPL, API servers). |
| 102 | +- **Secondary**: Julia (ML inference, port 8090), ReScript + Deno (UI, 33 files, zero TypeScript). |
| 103 | +- **Optional**: Chapel (parallel proof dispatch). |
| 104 | +- **ABI**: Idris2 (7+ modules, zero `believe_me`). |
| 105 | +- **FFI**: Zig (4 shared libraries). |
| 106 | +- **Interfaces**: GraphQL (8081), gRPC (50051), REST (8000). |
| 107 | +- **Build**: Justfile primary, Cargo workspace. |
| 108 | +- **Container**: Podman + Containerfile. |
| 109 | + |
| 110 | +## 11-step trust pipeline (v1.5+) |
| 111 | + |
| 112 | +1. Solver binary integrity (SHAKE3-512 + BLAKE3). |
| 113 | +2. SMT portfolio solving / cross-checking. |
| 114 | +3. Proof certificate checking (Alethe, DRAT/LRAT, TSTP). |
| 115 | +4. Axiom usage tracking (4 danger levels). |
| 116 | +5. Solver sandboxing (Podman, bubblewrap). |
| 117 | +6. 5-level trust hierarchy for confidence scoring. |
| 118 | +7. Mutation testing for specifications. |
| 119 | +8. Prover dispatch pipeline. |
| 120 | +9. Cross-prover proof exchange (OpenTheory, Dedukti). |
| 121 | +10. Pareto frontier (multi-objective proof search). |
| 122 | +11. Bayesian timeout estimation. |
| 123 | + |
| 124 | +## v2.x roadmap (from CLAUDE.md) |
| 125 | + |
| 126 | +- **v2.1 (landed)**: GNN proof graph construction (7 node kinds, 8 edge kinds); 32-dim local term embeddings + GNN inference client; GNN-guided proof search (hybrid GNN + symbolic scoring); Julia `/gnn/rank` endpoint with cosine fallback; Idris2 formal proofs (7 GNN properties, 0 `believe_me`); 28 new tests. |
| 127 | +- **v2.2**: Train GNN/Transformer on larger corpus (Flux.jl); Chapel → Rust C FFI bridge **(the Zig layer is done; dispatch-path integration is the L2 work above)**; Tamarin/ProVerif bridge **(already landed — stale in the roadmap)**. |
| 128 | + |
| 129 | +## Handover artefacts (in-repo, canonical) |
| 130 | + |
| 131 | +| File | Role | |
| 132 | +|------|------| |
| 133 | +| `verification-ecosystem/echidna/docs/handover/PRODUCTION-WIRING-PLAN.md` | Master plan | |
| 134 | +| `verification-ecosystem/echidna/docs/handover/L1-CAPNPROTO-PROMPT.md` | L1 Cap'n Proto continuation prompt | |
| 135 | +| `verification-ecosystem/echidna/docs/handover/L2-CHAPEL-PROMPT.md` | L2 Chapel continuation prompt | |
| 136 | +| `verification-ecosystem/echidna/docs/handover/L3-LIVE-PROVER-CI-PROMPT.md` | L3 live-prover continuation prompt (marked Wave-2 DONE, pointing to Wave-3) | |
| 137 | +| `verification-ecosystem/echidna/.machine_readable/6a2/STATE.a2ml` | Session ledger incl. `[l3-status-after-wave-2]` + `[wave-3-handover-hints]` | |
| 138 | +| `verification-ecosystem/echidna/docs/handover/README.md` | Index + drift-handling policy | |
| 139 | + |
| 140 | +## Session log highlights (2026-04-19) |
| 141 | + |
| 142 | +- **Chapel FFI self-link fix** (`53ab9b8`) — `-Dstubs=true` default in `src/zig_ffi/build.zig`; `-fno-sanitize=undefined` flag; `use anyhow::Context;` in `proof_search.rs`. `cargo build --features chapel` now links standalone. |
| 143 | +- **Stale-gap corrections** — Tamarin + ProVerif marked fully wired; zero TODO/FIXME standing property; Chapel FFI "not yet wired" was actually a link-time gap, fixed. |
| 144 | +- **L3 Wave-2 installers** — idris2 source bootstrap against Chez Scheme; isabelle Isabelle2024 tarball; dafny `dotnet tool install`; fstar release tarball (binary `fstar.exe` even on Linux); tlaps self-extracting installer (`tlapm`). hol-light deferred to Wave-3. |
| 145 | +- **Tests extended** — `live_fstar_version` + `live_tlaps_version` added; `kind_label` gained FStar / TLAPS; `ProverConfig` literal gained missing `library_paths` field (pre-existing compile error fixed). |
| 146 | +- **Docs + `.gitignore`** — `QUICKSTART-DEV.adoc` chapel-feature build instructions; `chapel_poc/README.md` "Add FFI bindings to call from Rust" marked DONE; `.gitignore` adds `models/e*/`, `/models_e*/`, `src/zig_ffi/zig-out/`, `.zig-cache/`. |
| 147 | +- **Mirrored** — all 4 Desktop handover docs now in-repo at `docs/handover/` (commit `b6d437c`). |
0 commit comments