|
| 1 | +// SPDX-License-Identifier: MPL-2.0 |
| 2 | +// SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell <j.d.a.jewell@open.ac.uk> |
| 3 | +:toc: left |
| 4 | +:toclevels: 2 |
| 5 | +:sectnums: |
| 6 | +:icons: font |
| 7 | + |
| 8 | += L3 → L1 gate path: precise checklist |
| 9 | + |
| 10 | +_Closes audit item *D18* from the 2026-06-03_ |
| 11 | +_"what stops echidna from running proof work across the estate"_ |
| 12 | +_punch list._ |
| 13 | + |
| 14 | +Status as of *2026-06-03*: **gate NOT satisfied**. L1 (Cap'n Proto) and |
| 15 | +L2 (Chapel full integration) are blocked until every box below is |
| 16 | +ticked. This document enumerates exactly what remains. |
| 17 | + |
| 18 | +== Where the criteria come from |
| 19 | + |
| 20 | +`docs/handover/L3-LIVE-PROVER-CI-PROMPT.md` "Hand to L1 when": |
| 21 | + |
| 22 | +* All four waves land or are explicitly deferred with rationale in |
| 23 | + `STATE.a2ml`. |
| 24 | +* Tier-1 has run green on main for ≥7 days. |
| 25 | + |
| 26 | +`docs/handover/TODO.md` adds two operational pre-requisites (P0, P1). |
| 27 | + |
| 28 | +These five criteria are unpacked below. |
| 29 | + |
| 30 | +== Criterion 1 — Wave-1 (Tier-1, every PR) |
| 31 | + |
| 32 | +Wave-1 is the every-PR Tier-1 matrix (9 backends): coq, agda, lean4, |
| 33 | +isabelle, z3, cvc5, eprover, alt-ergo, why3. |
| 34 | + |
| 35 | +[%header,cols="2,1,3"] |
| 36 | +|=== |
| 37 | +| Sub-criterion | Status | Notes |
| 38 | + |
| 39 | +| Wave-1 manifest exists |
| 40 | +| ✅ DONE |
| 41 | +| `manifests/live-provers.scm` + `.github/workflows/live-provers.yml` |
| 42 | + (b022bf4, 2026-04-18). |
| 43 | + |
| 44 | +| Wave-1 binaries reachable on CI |
| 45 | +| ✅ DONE |
| 46 | +| 9 backends present; verified locally 2026-04-19 (18/18 tests pass). |
| 47 | + |
| 48 | +| Wave-1 green on main for ≥7 days |
| 49 | +| ⚠ UNKNOWN |
| 50 | +| `STATE.a2ml [session-…l3-triage] wave-1-status = "UNKNOWN — run data |
| 51 | + inaccessible"`. Needs a `gh run list --workflow live-provers.yml |
| 52 | + --branch main --status success` count over the last 7 days. Cannot |
| 53 | + be ticked without that evidence. |
| 54 | +|=== |
| 55 | + |
| 56 | +**To close:** run the gh query, confirm ≥7 consecutive successful |
| 57 | +nightly invocations on main, and amend `STATE.a2ml` to record the |
| 58 | +window. |
| 59 | + |
| 60 | +== Criterion 2 — Wave-2 (Tier-2, nightly) |
| 61 | + |
| 62 | +Wave-2 is the nightly Tier-2 matrix (10 backends): idris2, isabelle, |
| 63 | +dafny, fstar, tlaps, vampire, gnatprove, glpk, spass, minizinc. (HOL |
| 64 | +Light deferred to Wave-3.) |
| 65 | + |
| 66 | +[%header,cols="2,1,3"] |
| 67 | +|=== |
| 68 | +| Sub-criterion | Status | Notes |
| 69 | + |
| 70 | +| Wave-2 installers landed (local) |
| 71 | +| ✅ DONE |
| 72 | +| Real provisioning commands shipped 2026-04-19 (9a4aeeb + 6717b12). |
| 73 | + 18/18 local tests pass; 13 real-version returns; 5 auto-skipped. |
| 74 | + |
| 75 | +| Wave-2 installers green in CI |
| 76 | +| ⚠ UNVERIFIED |
| 77 | +| TODO.md P0 row: "Wave-2 installers (idris2 / isabelle / dafny / |
| 78 | + fstar / tlaps) are local-pass but CI-unverified." Likely failure |
| 79 | + modes documented: Isabelle 500 MB download timeout, tlapm release |
| 80 | + URL drift, `fstar.exe` symlink resolution, apt mirror changes. |
| 81 | + |
| 82 | +| HOL Light explicitly deferred to Wave-3 |
| 83 | +| ✅ DONE |
| 84 | +| Deferred + rationale (no prebuilt binary, opam build ~20 min + |
| 85 | + camlp5). Recorded in STATE.a2ml. |
| 86 | +|=== |
| 87 | + |
| 88 | +**To close:** wait for the next `0 3 * * *` UTC nightly to fire and |
| 89 | +fix any red matrix cells in-place. Then run the 7-day green window |
| 90 | +check (Criterion 1, sub-row 3) against the Tier-2 matrix. |
| 91 | + |
| 92 | +== Criterion 3 — Wave-3 (Tier-3, weekly) |
| 93 | + |
| 94 | +Wave-3 is the weekly Tier-3 container matrix (9 backends): tamarin, |
| 95 | +proverif, imandra, scip, or-tools, hol4, acl2, twelf, metamath. |
| 96 | + |
| 97 | +[%header,cols="2,1,3"] |
| 98 | +|=== |
| 99 | +| Sub-criterion | Status | Notes |
| 100 | + |
| 101 | +| Wave-3 containers landed |
| 102 | +| ✅ DONE |
| 103 | +| PR #73 (2026-05-18) consolidated `.containerization/Containerfile.wave3` — |
| 104 | + one shared `rust-builder`, `--target` per prover. |
| 105 | + |
| 106 | +| All non-proprietary backends runtime-smoke-verified REAL |
| 107 | +| ✅ DONE |
| 108 | +| 8/8 confirmed by exercising the binary in the built image (not by |
| 109 | + trusting the build log). Five silent-stub failures fixed at-source |
| 110 | + during verification: scip dead-download, tamarin dead-download, |
| 111 | + metamath missing libc6-dev, or-tools missing build-number, twelf/acl2 |
| 112 | + baked build paths, proverif opam-solve. |
| 113 | + |
| 114 | +| Imandra documented as honest licence-gated stub |
| 115 | +| ✅ DONE |
| 116 | +| Real Rust adapter + Idris2 ABI proofs retained; `IMANDRA_TOKEN` |
| 117 | + secret path documented; KEEP decision recorded 2026-05-18. |
| 118 | + |
| 119 | +| Container CI workflow green |
| 120 | +| ⚠ UNVERIFIED |
| 121 | +| `container-ci.yml` weekly tier3-container matrix is allow-fail. No |
| 122 | + recent green-run count on main. |
| 123 | + |
| 124 | +| PR #74 (proverif opam) resolved |
| 125 | +| ✅ DONE |
| 126 | +| Closed by the 2026-05-18 source-tarball pivot. |
| 127 | +|=== |
| 128 | + |
| 129 | +**To close:** verify `container-ci.yml` weekly Tier-3 runs are not |
| 130 | +silently red-but-allow-fail; record green-run count in `STATE.a2ml`. |
| 131 | + |
| 132 | +== Criterion 4 — Wave-4 (Tier-4, quarterly) |
| 133 | + |
| 134 | +Wave-4 is the quarterly allow-fail Tier-4 list (19 backends): Mizar, |
| 135 | +Nuprl, PVS, Minlog, Dedukti, Arend, KeY, Prism, UPPAAL, ViPER, NuSMV, |
| 136 | +Spin, TLC, CBMC, Seahorn, dReal, Boogie, Kissat, Alloy. |
| 137 | + |
| 138 | +[%header,cols="2,1,3"] |
| 139 | +|=== |
| 140 | +| Sub-criterion | Status | Notes |
| 141 | + |
| 142 | +| Per-backend mock-only rationale in STATE.a2ml |
| 143 | +| ✅ DONE |
| 144 | +| `STATE.a2ml [wave-4-rationale]` carries one-line rationale per |
| 145 | + backend (1ca9862, 2026-04-26). |
| 146 | + |
| 147 | +| Scaffold placeholder in `live-provers.yml` |
| 148 | +| ✅ DONE |
| 149 | +| Tier-4 placeholder job announces the list. |
| 150 | + |
| 151 | +| At least one Wave-4 quarterly run executed |
| 152 | +| ⚠ UNVERIFIED |
| 153 | +| No evidence in STATE.a2ml that the placeholder has ever fired. |
| 154 | + Possibly cron-pinned but never visited. |
| 155 | +|=== |
| 156 | + |
| 157 | +**To close:** check that the quarterly cron has fired at least once |
| 158 | +since scaffolding, OR explicitly defer Wave-4 with a STATE.a2ml note |
| 159 | +"Wave-4 is scaffold-only and acceptable in the L3 hand-off — to be |
| 160 | +revisited when a maintainer contributes a Containerfile". |
| 161 | + |
| 162 | +Both readings satisfy the L3 prompt's phrasing |
| 163 | +("...all four waves land *or are explicitly deferred with rationale* |
| 164 | +in STATE.a2ml"). The choice is the owner's. |
| 165 | + |
| 166 | +== Criterion 5 — Tier-1 green on main for ≥7 days |
| 167 | + |
| 168 | +Same as Criterion 1, sub-row 3 — broken out here because the L3 prompt |
| 169 | +states it as a top-level acceptance criterion. |
| 170 | + |
| 171 | +[%header,cols="2,1,3"] |
| 172 | +|=== |
| 173 | +| Sub-criterion | Status | Notes |
| 174 | + |
| 175 | +| Run data accessible from a host with `gh` + auth |
| 176 | +| ❌ TODO |
| 177 | +| `STATE.a2ml [session-…l3-triage]` blocked on this. Local sessions |
| 178 | + have lacked `gh` + GITHUB_TOKEN. |
| 179 | + |
| 180 | +| Tier-1 nightly green count ≥7 over last 7 days on main |
| 181 | +| ❌ TODO |
| 182 | +| Requires the above. Suggested query: |
| 183 | ++ |
| 184 | +[source,shell] |
| 185 | +---- |
| 186 | +gh run list \ |
| 187 | + --workflow live-provers.yml \ |
| 188 | + --branch main \ |
| 189 | + --status success \ |
| 190 | + --limit 14 \ |
| 191 | + --json conclusion,startedAt |
| 192 | +---- |
| 193 | + |
| 194 | +| Record green window in STATE.a2ml |
| 195 | +| ❌ TODO |
| 196 | +| Single key, e.g. `l3-tier1-green-window = "2026-MM-DD .. 2026-MM-DD"`. |
| 197 | +|=== |
| 198 | + |
| 199 | +== Criterion 6 — Operational pre-requisites |
| 200 | + |
| 201 | +From `docs/handover/TODO.md`: |
| 202 | + |
| 203 | +[%header,cols="2,1,3"] |
| 204 | +|=== |
| 205 | +| Sub-criterion | Status | Notes |
| 206 | + |
| 207 | +| `guix shell -m manifests/live-provers.scm -- just test-live` |
| 208 | + end-to-end on a dev machine |
| 209 | +| ❌ TODO |
| 210 | +| Stated as L3 acceptance in the prompt. Has not been run end-to-end |
| 211 | + on a fresh dev host (Guix-extend was rejected; sealed container is |
| 212 | + the fallback for the not-in-Guix tail). |
| 213 | + |
| 214 | +| Dafny stub-shape upgrade (165 LoC → deep) |
| 215 | +| ⚠ UNKNOWN |
| 216 | +| Stated as L3 acceptance in the prompt. No commit found that closes |
| 217 | + this; needs an audit pass against `src/rust/provers/dafny.rs`. |
| 218 | + |
| 219 | +| VeriSimDB integration emits live-run records |
| 220 | +| ⚠ PARTIAL |
| 221 | +| Per `feedback_verisimdb_policy`, write paths exist in |
| 222 | + `dispatch.rs::emit_live_result` (2026-04-26). CI workflow gated on |
| 223 | + upstream `verisimdb#ghcr-publish.yml` — runbook YAML kept but not |
| 224 | + merged upstream. Tracked separately by audit item A5. |
| 225 | +|=== |
| 226 | + |
| 227 | +== Summary |
| 228 | + |
| 229 | +[%header,cols="1,4,1"] |
| 230 | +|=== |
| 231 | +| Criterion | Description | Status |
| 232 | + |
| 233 | +| 1 | Wave-1 (every-PR Tier-1) green ≥7 days | ⚠ UNKNOWN (data not collected) |
| 234 | +| 2 | Wave-2 (nightly Tier-2) green ≥1 cycle | ⚠ UNVERIFIED |
| 235 | +| 3 | Wave-3 (weekly Tier-3) green ≥1 cycle | ⚠ UNVERIFIED |
| 236 | +| 4 | Wave-4 (quarterly Tier-4) executed or explicitly deferred | ⚠ owner choice |
| 237 | +| 5 | Tier-1 green ≥7 days on main (data) | ❌ TODO |
| 238 | +| 6a | `guix shell` local end-to-end | ❌ TODO |
| 239 | +| 6b | Dafny stub upgrade | ⚠ UNKNOWN |
| 240 | +| 6c | VeriSimDB live-record emission | ⚠ PARTIAL (gated on verisimdb#ghcr) |
| 241 | +|=== |
| 242 | + |
| 243 | +**Minimum sufficient path to close the gate:** |
| 244 | + |
| 245 | +. Run the `gh run list` query for the last 14 nightlies of |
| 246 | + `live-provers.yml` on main; record the green window in |
| 247 | + `STATE.a2ml`. (Closes Criteria 1, 2, 5.) |
| 248 | +. Same query against `container-ci.yml` for Tier-3 weekly. |
| 249 | + (Closes Criterion 3 sub-row 4.) |
| 250 | +. Owner decision on Wave-4: execute one quarterly run OR formally |
| 251 | + defer with a STATE.a2ml note. (Closes Criterion 4 sub-row 3.) |
| 252 | +. One dev-host run of `guix shell -m manifests/live-provers.scm -- |
| 253 | + just test-live`. (Closes Criterion 6a.) |
| 254 | +. Audit `src/rust/provers/dafny.rs` against the "deep wiring" bar |
| 255 | + used for Coq / Lean4. If it falls short, file a follow-up; if it |
| 256 | + meets the bar, record that in STATE.a2ml. (Closes Criterion 6b.) |
| 257 | +. (Criterion 6c stays parked on `verisimdb#ghcr-publish.yml` — that |
| 258 | + is upstream-blocked and not on this gate's critical path.) |
| 259 | + |
| 260 | +Estimated effort: **one session** if the gh tokens are available. |
| 261 | +Without gh access, Criteria 1/2/3/5 cannot close, full stop. |
| 262 | + |
| 263 | +== Cross-references |
| 264 | + |
| 265 | +* `docs/handover/L3-LIVE-PROVER-CI-PROMPT.md` — original L3 mission. |
| 266 | +* `docs/handover/TODO.md` — P0 / P1 rows referenced above. |
| 267 | +* `docs/handover/L1-CAPNPROTO-PROMPT.md` — the prompt that fires |
| 268 | + once this gate closes. |
| 269 | +* `docs/handover/L2-CHAPEL-PROMPT.md` — blocked behind L1 + this gate. |
| 270 | +* C12 PR (echidnabot manifest) — per-repo opt-in shape for consumers. |
| 271 | +* C14 PR (cross-repo proof DAG design) — what L1's Cap'n Proto schemas |
| 272 | + must carry. |
| 273 | +* C15 PR (gitbot-fleet hypatia wiring) — the closed-loop signal the |
| 274 | + L3 gate depends on for VeriSimDB record quality. |
| 275 | +* B7 PR (HP backend correction) — confirms the HP ecosystem is not |
| 276 | + blocking L3. |
0 commit comments