|
| 1 | +# Echidna L3 — Live-Prover CI — Continuation Prompt |
| 2 | + |
| 3 | +**Context**: Echidna (`/var/mnt/eclipse/repos/verification-ecosystem/echidna/`) has 48 trait-wired |
| 4 | +prover backends, but CI only tests them with `MockProver`. Mission: make every provisionable |
| 5 | +backend run against a canonical micro-goal on a predictable cadence. Guix primary, Nix fallback. |
| 6 | + |
| 7 | +**Master plan**: `~/Desktop/ECHIDNA-PRODUCTION-WIRING-PLAN.md` |
| 8 | +**Order**: L3 (this) → L1 (Cap'n Proto) → L2 (Chapel max). |
| 9 | + |
| 10 | +--- |
| 11 | + |
| 12 | +## Where L3 is when this prompt fires |
| 13 | + |
| 14 | +Wave-1 (Tier-1, every PR — 9 backends) DONE in kickoff session, commit `b022bf4`: |
| 15 | +- `manifests/live-provers.scm` — Guix manifest with Tier-1 + Tier-2 provers |
| 16 | +- `flake.nix` — expanded with matching `liveProverDeps` |
| 17 | +- `.github/workflows/live-provers.yml` — tiered workflow (Tier-1 PR, Tier-2 nightly, etc.) |
| 18 | +- `tests/live_prover_suite.rs` — Rust test harness with Tier-1 version-check tests |
| 19 | + |
| 20 | +Wave-2 (Tier-2, nightly — 10 backends) **DONE 2026-04-19** in commits `9a4aeeb` + `6717b12`: |
| 21 | +- Real provisioning commands for coq, agda, why3 (apt); idris2 (source bootstrap |
| 22 | + against Chez Scheme); lean4 (elan); isabelle (Isabelle2024 tarball); dafny |
| 23 | + (`dotnet tool install --global Dafny`); fstar (GitHub release tarball, binary |
| 24 | + `fstar.exe`); tlaps (self-extracting installer for `tlapm`). |
| 25 | +- `hol-light` **deferred** to Wave-3 container path — no prebuilt binary and opam |
| 26 | + build is ~20 min + camlp5. |
| 27 | +- `tests/live_prover_suite.rs` extended with `live_fstar_version` + `live_tlaps_version`. |
| 28 | +- Local verification (not CI yet): 18/18 tests pass, 13 backends return real versions, |
| 29 | + 5 auto-skip when binaries are absent. |
| 30 | + |
| 31 | +`tests/live_goals/` was listed in the handover-artefacts but not actually created. |
| 32 | +Wave-2 version-check tests do not need it; Wave-2 / Wave-3 goal-check extensions will. |
| 33 | + |
| 34 | +## What to finish in L3 (remaining) |
| 35 | + |
| 36 | +### Wave-2 verification in CI (not just local) |
| 37 | +The Wave-2 provisioning installers are not yet exercised by a real CI run — |
| 38 | +nightly needs to fire (cron `0 3 * * *`) to surface any CI-specific breakage |
| 39 | +(apt mirror changes, GitHub release URL format drift, dotnet sdk availability, |
| 40 | +etc.). Watch the next nightly after merge; fix any red matrix cells in-place. |
| 41 | + |
| 42 | +### Wave-3: Tier-3 backends (Weekly) |
| 43 | +Container/special-env path: Tamarin, ProVerif, Imandra, SCIP, OR-Tools, HOL4, ACL2, Twelf, Metamath. |
| 44 | +These likely need per-backend Containerfiles (Podman, not Docker — per project CLAUDE.md). |
| 45 | + |
| 46 | +### Wave-4: Tier-4 backends (Quarterly, allow-fail) |
| 47 | +Best-effort: Mizar, Nuprl, PVS, Minlog, Dedukti, Arend, KeY, Prism, UPPAAL, ViPER, NuSMV, Spin, |
| 48 | +TLC, CBMC, Seahorn, dReal, Boogie, Kissat, Alloy. Many will stay mock-only (document why). |
| 49 | + |
| 50 | +### Harden wiring depth for shallow backends |
| 51 | +Audit flagged **Dafny (165 LoC)** as stub-ish — upgrade to deep during L3 so the live test isn't |
| 52 | +measuring a broken subprocess wrapper. |
| 53 | + |
| 54 | +### VeriSimDB integration |
| 55 | +Per `feedback_verisimdb_policy`: the live-prover harness should emit a VeriSimDB record per run |
| 56 | +(prover, goal, outcome, duration, exit code). Schema TBD — coordinate with VeriSimDB repo. |
| 57 | + |
| 58 | +## Acceptance for L3 complete |
| 59 | + |
| 60 | +- Tier-1 green on every PR |
| 61 | +- Tier-2 green on nightly (or documented flaky backends with `continue-on-error: true`) |
| 62 | +- Tier-3 runs weekly with per-backend allow-fail |
| 63 | +- Tier-4 runs quarterly, results archived |
| 64 | +- Dafny upgraded from 165 LoC stub to deep subprocess wiring |
| 65 | +- `guix shell -m manifests/live-provers.scm -- just test-live` works locally |
| 66 | +- VeriSimDB records emitted (if schema ready) |
| 67 | +- Mock tests retained as fast pre-CI smoke |
| 68 | + |
| 69 | +## Hand to L1 when |
| 70 | + |
| 71 | +- All four waves land or are explicitly deferred with rationale in STATE.a2ml |
| 72 | +- Tier-1 has run green on main for ≥7 days |
| 73 | + |
| 74 | +--- |
| 75 | + |
| 76 | +## Rules active in this session |
| 77 | + |
| 78 | +- GitHub-only mirroring (per memory `feedback_github_only_mirroring`) |
| 79 | +- Commit ASAP, specific paths (`feedback_commit_asap`) |
| 80 | +- Pre-commit: `panic-attack assail` (`feedback_panic_attack_precommit`) |
| 81 | +- No `.unwrap() → .expect("TODO")` refactor (`feedback_unwrap_to_expect_antipattern`) |
| 82 | +- Session close marker: `SAFE TO CLOSE` literal at end of final message |
| 83 | + |
| 84 | +## Key files to read first |
| 85 | + |
| 86 | +1. `~/Desktop/ECHIDNA-PRODUCTION-WIRING-PLAN.md` |
| 87 | +2. `/var/mnt/eclipse/repos/verification-ecosystem/echidna/.machine_readable/6a2/STATE.a2ml` |
| 88 | +3. `/var/mnt/eclipse/repos/verification-ecosystem/echidna/manifests/live-provers.scm` |
| 89 | +4. `/var/mnt/eclipse/repos/verification-ecosystem/echidna/.github/workflows/live-provers.yml` |
| 90 | +5. `/var/mnt/eclipse/repos/verification-ecosystem/echidna/tests/live_prover_suite.rs` |
0 commit comments