Skip to content

Commit 3b110e7

Browse files
hyperpolymathclaude
andcommitted
docs(handover): mirror ~/Desktop/ECHIDNA-*.md into docs/handover/
Makes the production-wiring plan + L1/L2/L3 continuation prompts part of the repo so a fresh clone has the full handover context without needing Desktop access. - docs/handover/PRODUCTION-WIRING-PLAN.md (← ECHIDNA-PRODUCTION-WIRING-PLAN.md) - docs/handover/L1-CAPNPROTO-PROMPT.md - docs/handover/L2-CHAPEL-PROMPT.md - docs/handover/L3-LIVE-PROVER-CI-PROMPT.md - docs/handover/README.md — drift-handling policy (both copies updated in same commit; in-repo copy wins on conflict). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent ba7944e commit 3b110e7

5 files changed

Lines changed: 496 additions & 0 deletions

File tree

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
# Echidna L1 — Cap'n Proto Protocol Swap — Continuation Prompt
2+
3+
**Context**: Echidna currently uses HTTP + JSON for Rust↔Julia (GNN inference) via
4+
`src/rust/gnn/client.rs``src/julia/api_server.jl` port 8090. This violates the
5+
`feedback_no_json_emit_a2ml` memory rule and blocks a clean polyglot IPC story. Swap to
6+
**Cap'n Proto** (chosen over Bebop3 for dependability + maturity + zero-copy reads).
7+
8+
**Master plan**: `~/Desktop/ECHIDNA-PRODUCTION-WIRING-PLAN.md`
9+
**Prerequisite**: L3 Tier-1 green for ≥7 days on main.
10+
**Follows**: L2 (Chapel) consumes these schemas.
11+
12+
## Deliverables
13+
14+
### Schema
15+
- `schemas/echidna.capnp` — canonical wire schemas:
16+
- `ProofGoal` — a theorem/claim to prove
17+
- `ProofResult` — success/failure + metadata
18+
- `TacticSuggestion` — ML-suggested tactic
19+
- `GnnRankRequest` / `GnnRankResponse` — replaces current HTTP+JSON
20+
- `ProverInvocation` — dispatch message to a backend (used later by Chapel in L2)
21+
- `TrustedOutcome` — post-trust-pipeline result
22+
- `schemas/VERSIONING.md` — forward/backward compat rules
23+
24+
### Rust side (hot-path consumer)
25+
- `src/rust/ipc/mod.rs` — Cap'n Proto transport module
26+
- `src/rust/ipc/uds.rs` — Unix-domain socket transport (primary)
27+
- `src/rust/ipc/tcp.rs` — TCP transport (fallback for containerised cases)
28+
- Replace `src/rust/gnn/client.rs` HTTP calls with Cap'n Proto over UDS
29+
- Keep `serde_json` out of hot path; retain only for config/log emission (Nickel for config proper)
30+
31+
### Julia side
32+
- `src/julia/ipc.jl` — Cap'n Proto reader/writer
33+
- Use `CapnProto.jl` if mature enough; otherwise shim via C-ABI (our Zig FFI layer)
34+
- `src/julia/api_server.jl` — switch from HTTP to UDS listener
35+
36+
### Idris2 ABI proofs
37+
- `src/abi/CapnSchemas.idr` — formal proofs that schema round-trip preserves meaning
38+
- Zero `believe_me` (per estate rule); constructive proofs only
39+
- Coordinate with `src/abi/Types.idr` / `src/abi/Foreign.idr`
40+
41+
### Zig FFI bridge
42+
- `ffi/zig/capnp_bridge.zig` — C-ABI bridge for polyglot consumers (esp. Chapel in L2)
43+
44+
### ReScript bindings (UI path)
45+
- `bindings/rescript/echidna_capnp.res` — typed ReScript bindings for the UI layer
46+
- Keep the existing 3 API interfaces (GraphQL/gRPC/REST) as external surfaces; Cap'n Proto is the
47+
**internal** wire format
48+
49+
### Tooling
50+
- `just capnp-gen` — regenerates Rust/Julia/Idris2/Zig/ReScript bindings from `.capnp` schemas
51+
- CI check: `capnp compile` runs clean + generated code is committed
52+
53+
## Acceptance criteria
54+
55+
- Zero `serde_json::to_*` / `serde_json::from_*` calls on the Rust↔Julia hot path
56+
- `src/rust/gnn/client.rs` uses UDS+Cap'n Proto
57+
- Benchmarks: Cap'n Proto round-trip ≤ 50% of JSON latency for GNN rank request
58+
- Idris2 ABI compiles with zero `believe_me`
59+
- Round-trip property tests on all six schemas (Rust, Julia, Idris2)
60+
- `schemas/VERSIONING.md` explains migration story
61+
62+
## Open design questions to settle early
63+
64+
1. **UDS path convention**`/run/echidna/ipc.sock` vs `$XDG_RUNTIME_DIR/echidna/ipc.sock`?
65+
2. **Authentication** — initial handshake signed with existing BLAKE3/SHAKE3-512 integrity keys?
66+
3. **Streaming vs request-response** — Cap'n Proto RPC streams for GNN batch inference?
67+
4. **Multi-locale Chapel** — schemas need to survive locale-to-locale transit; design for that now
68+
so L2 doesn't re-spec.
69+
70+
## Hand to L2 when
71+
72+
- Cap'n Proto is the default Rust↔Julia path
73+
- `ProverInvocation` + `TrustedOutcome` schemas are stable (L2 Chapel will consume them)
74+
- Zig FFI bridge passes round-trip test
75+
76+
---
77+
78+
## Rules active
79+
80+
- `no_json_emit_a2ml` — the whole reason this phase exists
81+
- `feedback_verisimdb_policy` — all IPC traffic should be observable as VeriSimDB records
82+
- `feedback_code_only_grep_for_banned_patterns` — when auditing for `serde_json` removal, use
83+
code-only grep (`^[^-|]*serde_json`) to avoid comment/doc noise
84+
- `feedback_wire_everything` — no stubs; every schema has a live consumer on all sides
85+
86+
## Key files to read first
87+
88+
1. `~/Desktop/ECHIDNA-PRODUCTION-WIRING-PLAN.md`
89+
2. `/var/mnt/eclipse/repos/verification-ecosystem/echidna/src/rust/gnn/client.rs` (current JSON path)
90+
3. `/var/mnt/eclipse/repos/verification-ecosystem/echidna/src/julia/api_server.jl` (Julia server)
91+
4. `/var/mnt/eclipse/repos/verification-ecosystem/echidna/src/abi/*.idr` (existing Idris2 ABI)
92+
5. `/var/mnt/eclipse/repos/verification-ecosystem/echidna/src/zig_ffi/` (existing Zig FFI layer)

docs/handover/L2-CHAPEL-PROMPT.md

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
# Echidna L2 — Chapel Maximum Integration — Continuation Prompt
2+
3+
**Context**: Echidna has a 420-LoC Chapel POC (`chapel_poc/parallel_proof_search.chpl`) that implements
4+
30-prover parallel portfolio dispatch but is **not wired into `src/rust/dispatch.rs`**. User directive
5+
(2026-04-19): "expand Chapel to do EVERYTHING it can — no point investing in hard Chapel code and not
6+
using it to maximum value."
7+
8+
**Master plan**: `~/Desktop/ECHIDNA-PRODUCTION-WIRING-PLAN.md`
9+
**Prerequisites**: L3 Tier-1+Tier-2 green; L1 Cap'n Proto schemas stable.
10+
**Scope**: ~5–6 weeks, broken into 7 sub-waves.
11+
12+
---
13+
14+
## What Chapel is actually good at (the full opportunity)
15+
16+
1. **Task parallelism** (`coforall`, `cobegin`, `begin`) — parallel prover dispatch, speculative tactic branches
17+
2. **Data parallelism** (`forall`) — corpus-wide premise scoring, tactic mining
18+
3. **Distributed PGAS** (locales, on-clauses) — multi-node proof search with sharded corpus
19+
4. **Domains & distributions** — data-dependent workload partitioning
20+
5. **Atomics** — first-wins racing, lock-free coordination
21+
6. **Reductions** — parallel statistics, confidence aggregation
22+
7. **GPU support** — Chapel-on-NVIDIA for embedding batches
23+
8. **NUMA awareness** — locale-aware dispatch on multi-socket machines
24+
25+
## Sub-wave breakdown
26+
27+
### L2.1 — Portfolio dispatch (promote POC) ~1 week
28+
- `src/chapel/portfolio.chpl` — migrate + adapt from `chapel_poc/parallel_proof_search.chpl`
29+
- Input: Cap'n Proto `ProverInvocation` batch (from L1)
30+
- Output: Cap'n Proto `TrustedOutcome[]`
31+
- Race semantics: atomic first-wins, SIGKILL losers, cancellation within `timeout`
32+
- Wire into `src/rust/dispatch.rs` behind `--chapel` feature flag via Zig FFI
33+
- Default: feature-off; default-on flip after L2.7 benchmarks
34+
35+
### L2.2 — Speculative tactic search ~1 week
36+
- `src/chapel/tactic_search.chpl`
37+
- Parallel beam search (k parallel beams)
38+
- Parallel MCTS over tactic trees (UCB1 selection with `coforall` rollouts)
39+
- Speculative branch execution + cancellation on winner commit
40+
- Consumes `TacticSuggestion` stream from GNN (L1 schemas)
41+
42+
### L2.3 — Corpus-parallel ops ~1 week
43+
- `src/chapel/corpus.chpl`
44+
- `forall` over 66,674-proof corpus
45+
- Parallel proof replay (re-verify corpus)
46+
- Parallel premise scoring (vs GNN rankings)
47+
- Parallel tactic mining + inverted index build
48+
- Locality-aware: `forall ... with (ref corpusShard)` to avoid cross-locale traffic
49+
50+
### L2.4 — Mutation testing parallelism ~3 days
51+
- `src/chapel/mutation.chpl`
52+
- Fan out 1000s of mutants across all cores/locales
53+
- Integrate with existing Rust `verification/mutation.rs` (consume its mutant stream, run in Chapel)
54+
55+
### L2.5 — Multi-locale distributed ~1.5 weeks
56+
- PGAS-sharded corpus across locales (`Block` or `Cyclic` distribution)
57+
- Locale-aware prover dispatch: heavy backends on dedicated locales
58+
- GPU-locale offload: embedding batches on NVIDIA locale if present
59+
- Reproducibility: locale seed propagation
60+
- Single-locale covers 80% of value; multi-locale proven on dev hardware
61+
62+
### L2.6 — Numeric hot paths ~4 days
63+
- `src/chapel/numeric.chpl`
64+
- Parallel GNN embedding pre-processing (batch pack + pad)
65+
- Parallel Pareto frontier computation (`forall` + atomic minimum tracking)
66+
- Parallel confidence statistics / Bayesian timeout updates
67+
68+
### L2.7 — CI + bench ~3 days
69+
- `.github/workflows/chapel-live.yml` — runs Chapel-in-dispatch tests
70+
- Criterion benchmarks: Chapel portfolio vs Rust+Rayon baseline
71+
- Must show measurable speedup on 8+ core machines to flip default-on
72+
73+
## Acceptance criteria
74+
75+
- Chapel on hot path by default after benchmarks prove it
76+
- `src/chapel/` has 6+ modules wired via Zig FFI + Cap'n Proto
77+
- `chapel_poc/` archived (or deleted) with redirect note in its README
78+
- Multi-locale path proven on at least one dev-hardware config
79+
- Benchmarks show ≥1.5× speedup for portfolio solving on 8-core machines (or Chapel stays feature-flagged)
80+
- No Rust code duplicates what Chapel does best (avoid double-paths)
81+
82+
---
83+
84+
## Rules active
85+
86+
- `feedback_wire_everything` — no stubs; POC promoted fully, not re-implemented
87+
- `feedback_meander_resource_costs` — Chapel builds slow; cache Chapel artefacts in CI, clean after
88+
- `feedback_rust_means_rust_spark` — Rust side admits SPARK/Ada; Chapel doesn't change that
89+
- `feedback_verisimdb_policy` — Chapel-dispatched runs still emit VeriSimDB records (via Cap'n Proto
90+
back-channel to Rust)
91+
- `feedback_full_battery_before_claims` — before claiming speedup, full battery: tests + benches +
92+
panic-attack + causality + verifiable I/O
93+
94+
## Key files to read first
95+
96+
1. `~/Desktop/ECHIDNA-PRODUCTION-WIRING-PLAN.md`
97+
2. `/var/mnt/eclipse/repos/verification-ecosystem/echidna/chapel_poc/parallel_proof_search.chpl` (420-LoC POC to promote)
98+
3. `/var/mnt/eclipse/repos/verification-ecosystem/echidna/src/zig_ffi/chapel_bridge.zig` (existing Zig FFI stub)
99+
4. `/var/mnt/eclipse/repos/verification-ecosystem/echidna/src/rust/dispatch.rs` (integration target)
100+
5. `/var/mnt/eclipse/repos/verification-ecosystem/echidna/.github/workflows/chapel-ci.yml` (current Chapel CI for compile-only)
101+
6. `schemas/echidna.capnp` (must exist from L1 — if not, L1 isn't done)
Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# Echidna L3 — Live-Prover CI — Continuation Prompt
2+
3+
**Context**: Echidna (`/var/mnt/eclipse/repos/verification-ecosystem/echidna/`) has 48 trait-wired
4+
prover backends, but CI only tests them with `MockProver`. Mission: make every provisionable
5+
backend run against a canonical micro-goal on a predictable cadence. Guix primary, Nix fallback.
6+
7+
**Master plan**: `~/Desktop/ECHIDNA-PRODUCTION-WIRING-PLAN.md`
8+
**Order**: L3 (this) → L1 (Cap'n Proto) → L2 (Chapel max).
9+
10+
---
11+
12+
## Where L3 is when this prompt fires
13+
14+
Wave-1 (Tier-1, every PR — 9 backends) DONE in kickoff session, commit `b022bf4`:
15+
- `manifests/live-provers.scm` — Guix manifest with Tier-1 + Tier-2 provers
16+
- `flake.nix` — expanded with matching `liveProverDeps`
17+
- `.github/workflows/live-provers.yml` — tiered workflow (Tier-1 PR, Tier-2 nightly, etc.)
18+
- `tests/live_prover_suite.rs` — Rust test harness with Tier-1 version-check tests
19+
20+
Wave-2 (Tier-2, nightly — 10 backends) **DONE 2026-04-19** in commits `9a4aeeb` + `6717b12`:
21+
- Real provisioning commands for coq, agda, why3 (apt); idris2 (source bootstrap
22+
against Chez Scheme); lean4 (elan); isabelle (Isabelle2024 tarball); dafny
23+
(`dotnet tool install --global Dafny`); fstar (GitHub release tarball, binary
24+
`fstar.exe`); tlaps (self-extracting installer for `tlapm`).
25+
- `hol-light` **deferred** to Wave-3 container path — no prebuilt binary and opam
26+
build is ~20 min + camlp5.
27+
- `tests/live_prover_suite.rs` extended with `live_fstar_version` + `live_tlaps_version`.
28+
- Local verification (not CI yet): 18/18 tests pass, 13 backends return real versions,
29+
5 auto-skip when binaries are absent.
30+
31+
`tests/live_goals/` was listed in the handover-artefacts but not actually created.
32+
Wave-2 version-check tests do not need it; Wave-2 / Wave-3 goal-check extensions will.
33+
34+
## What to finish in L3 (remaining)
35+
36+
### Wave-2 verification in CI (not just local)
37+
The Wave-2 provisioning installers are not yet exercised by a real CI run —
38+
nightly needs to fire (cron `0 3 * * *`) to surface any CI-specific breakage
39+
(apt mirror changes, GitHub release URL format drift, dotnet sdk availability,
40+
etc.). Watch the next nightly after merge; fix any red matrix cells in-place.
41+
42+
### Wave-3: Tier-3 backends (Weekly)
43+
Container/special-env path: Tamarin, ProVerif, Imandra, SCIP, OR-Tools, HOL4, ACL2, Twelf, Metamath.
44+
These likely need per-backend Containerfiles (Podman, not Docker — per project CLAUDE.md).
45+
46+
### Wave-4: Tier-4 backends (Quarterly, allow-fail)
47+
Best-effort: Mizar, Nuprl, PVS, Minlog, Dedukti, Arend, KeY, Prism, UPPAAL, ViPER, NuSMV, Spin,
48+
TLC, CBMC, Seahorn, dReal, Boogie, Kissat, Alloy. Many will stay mock-only (document why).
49+
50+
### Harden wiring depth for shallow backends
51+
Audit flagged **Dafny (165 LoC)** as stub-ish — upgrade to deep during L3 so the live test isn't
52+
measuring a broken subprocess wrapper.
53+
54+
### VeriSimDB integration
55+
Per `feedback_verisimdb_policy`: the live-prover harness should emit a VeriSimDB record per run
56+
(prover, goal, outcome, duration, exit code). Schema TBD — coordinate with VeriSimDB repo.
57+
58+
## Acceptance for L3 complete
59+
60+
- Tier-1 green on every PR
61+
- Tier-2 green on nightly (or documented flaky backends with `continue-on-error: true`)
62+
- Tier-3 runs weekly with per-backend allow-fail
63+
- Tier-4 runs quarterly, results archived
64+
- Dafny upgraded from 165 LoC stub to deep subprocess wiring
65+
- `guix shell -m manifests/live-provers.scm -- just test-live` works locally
66+
- VeriSimDB records emitted (if schema ready)
67+
- Mock tests retained as fast pre-CI smoke
68+
69+
## Hand to L1 when
70+
71+
- All four waves land or are explicitly deferred with rationale in STATE.a2ml
72+
- Tier-1 has run green on main for ≥7 days
73+
74+
---
75+
76+
## Rules active in this session
77+
78+
- GitHub-only mirroring (per memory `feedback_github_only_mirroring`)
79+
- Commit ASAP, specific paths (`feedback_commit_asap`)
80+
- Pre-commit: `panic-attack assail` (`feedback_panic_attack_precommit`)
81+
- No `.unwrap() → .expect("TODO")` refactor (`feedback_unwrap_to_expect_antipattern`)
82+
- Session close marker: `SAFE TO CLOSE` literal at end of final message
83+
84+
## Key files to read first
85+
86+
1. `~/Desktop/ECHIDNA-PRODUCTION-WIRING-PLAN.md`
87+
2. `/var/mnt/eclipse/repos/verification-ecosystem/echidna/.machine_readable/6a2/STATE.a2ml`
88+
3. `/var/mnt/eclipse/repos/verification-ecosystem/echidna/manifests/live-provers.scm`
89+
4. `/var/mnt/eclipse/repos/verification-ecosystem/echidna/.github/workflows/live-provers.yml`
90+
5. `/var/mnt/eclipse/repos/verification-ecosystem/echidna/tests/live_prover_suite.rs`

0 commit comments

Comments
 (0)