Skip to content

Commit d9b4e7f

Browse files
hyperpolymathclaude
andcommitted
docs(handover): L3 gate path checklist (closes D18 estate blocker)
Closes the estate-onboarding blocker D18 surfaced in the "what stops echidna from running proof work across the estate" audit: L3 → L1 hand-off requires Tier-1 green ≥7 days + all 4 waves landed-or-deferred. As of today, the gate is NOT satisfied — but crucially, NO ONE HAD ENUMERATED EXACTLY WHY, so L1 (Cap'n Proto) and L2 (Chapel full integration) stayed parked on a vague signal. This doc enumerates the six L3 acceptance criteria across the L3 prompt and TODO.md, and walks each one through its sub-criteria with current status: Criterion 1 — Wave-1 (Tier-1 every-PR): 2/3 sub-rows ✅, 1 UNKNOWN (green-window data not collected) Criterion 2 — Wave-2 (Tier-2 nightly): 2/3 sub-rows ✅, 1 UNVERIFIED (CI cycle outcome not recorded) Criterion 3 — Wave-3 (Tier-3 weekly): 4/5 sub-rows ✅, 1 UNVERIFIED (container-ci.yml weekly green-count not recorded) Criterion 4 — Wave-4 (Tier-4 quarterly): 2/3 sub-rows ✅, 1 owner choice (one execution OR explicit defer; both satisfy the L3 prompt's "land or defer with rationale" phrasing) Criterion 5 — Tier-1 green ≥7 days on main: 0/3 sub-rows (needs gh + GITHUB_TOKEN; previous sessions lacked them — STATE.a2ml triage records the block) Criterion 6 — Operational: 6a guix shell end-to-end: ❌ TODO 6b Dafny stub upgrade audit: ⚠ UNKNOWN 6c VeriSimDB record emission: ⚠ PARTIAL (gated on verisimdb#ghcr-publish.yml upstream) Minimum sufficient path documented at the end (~one session if gh auth is available). Without gh access, criteria 1/2/3/5 cannot close. The doc cross-references the other four audit-pass PRs (C12, C14, C15, B7) so the L3 gate context stays linked to the surrounding work. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 0d27d43 commit d9b4e7f

1 file changed

Lines changed: 276 additions & 0 deletions

File tree

docs/handover/L3-GATE-PATH.adoc

Lines changed: 276 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,276 @@
1+
// SPDX-License-Identifier: MPL-2.0
2+
// SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell <j.d.a.jewell@open.ac.uk>
3+
:toc: left
4+
:toclevels: 2
5+
:sectnums:
6+
:icons: font
7+
8+
= L3 → L1 gate path: precise checklist
9+
10+
_Closes audit item *D18* from the 2026-06-03_
11+
_"what stops echidna from running proof work across the estate"_
12+
_punch list._
13+
14+
Status as of *2026-06-03*: **gate NOT satisfied**. L1 (Cap'n Proto) and
15+
L2 (Chapel full integration) are blocked until every box below is
16+
ticked. This document enumerates exactly what remains.
17+
18+
== Where the criteria come from
19+
20+
`docs/handover/L3-LIVE-PROVER-CI-PROMPT.md` "Hand to L1 when":
21+
22+
* All four waves land or are explicitly deferred with rationale in
23+
`STATE.a2ml`.
24+
* Tier-1 has run green on main for ≥7 days.
25+
26+
`docs/handover/TODO.md` adds two operational pre-requisites (P0, P1).
27+
28+
These five criteria are unpacked below.
29+
30+
== Criterion 1 — Wave-1 (Tier-1, every PR)
31+
32+
Wave-1 is the every-PR Tier-1 matrix (9 backends): coq, agda, lean4,
33+
isabelle, z3, cvc5, eprover, alt-ergo, why3.
34+
35+
[%header,cols="2,1,3"]
36+
|===
37+
| Sub-criterion | Status | Notes
38+
39+
| Wave-1 manifest exists
40+
| ✅ DONE
41+
| `manifests/live-provers.scm` + `.github/workflows/live-provers.yml`
42+
(b022bf4, 2026-04-18).
43+
44+
| Wave-1 binaries reachable on CI
45+
| ✅ DONE
46+
| 9 backends present; verified locally 2026-04-19 (18/18 tests pass).
47+
48+
| Wave-1 green on main for ≥7 days
49+
| ⚠ UNKNOWN
50+
| `STATE.a2ml [session-…l3-triage] wave-1-status = "UNKNOWN — run data
51+
inaccessible"`. Needs a `gh run list --workflow live-provers.yml
52+
--branch main --status success` count over the last 7 days. Cannot
53+
be ticked without that evidence.
54+
|===
55+
56+
**To close:** run the gh query, confirm ≥7 consecutive successful
57+
nightly invocations on main, and amend `STATE.a2ml` to record the
58+
window.
59+
60+
== Criterion 2 — Wave-2 (Tier-2, nightly)
61+
62+
Wave-2 is the nightly Tier-2 matrix (10 backends): idris2, isabelle,
63+
dafny, fstar, tlaps, vampire, gnatprove, glpk, spass, minizinc. (HOL
64+
Light deferred to Wave-3.)
65+
66+
[%header,cols="2,1,3"]
67+
|===
68+
| Sub-criterion | Status | Notes
69+
70+
| Wave-2 installers landed (local)
71+
| ✅ DONE
72+
| Real provisioning commands shipped 2026-04-19 (9a4aeeb + 6717b12).
73+
18/18 local tests pass; 13 real-version returns; 5 auto-skipped.
74+
75+
| Wave-2 installers green in CI
76+
| ⚠ UNVERIFIED
77+
| TODO.md P0 row: "Wave-2 installers (idris2 / isabelle / dafny /
78+
fstar / tlaps) are local-pass but CI-unverified." Likely failure
79+
modes documented: Isabelle 500 MB download timeout, tlapm release
80+
URL drift, `fstar.exe` symlink resolution, apt mirror changes.
81+
82+
| HOL Light explicitly deferred to Wave-3
83+
| ✅ DONE
84+
| Deferred + rationale (no prebuilt binary, opam build ~20 min +
85+
camlp5). Recorded in STATE.a2ml.
86+
|===
87+
88+
**To close:** wait for the next `0 3 * * *` UTC nightly to fire and
89+
fix any red matrix cells in-place. Then run the 7-day green window
90+
check (Criterion 1, sub-row 3) against the Tier-2 matrix.
91+
92+
== Criterion 3 — Wave-3 (Tier-3, weekly)
93+
94+
Wave-3 is the weekly Tier-3 container matrix (9 backends): tamarin,
95+
proverif, imandra, scip, or-tools, hol4, acl2, twelf, metamath.
96+
97+
[%header,cols="2,1,3"]
98+
|===
99+
| Sub-criterion | Status | Notes
100+
101+
| Wave-3 containers landed
102+
| ✅ DONE
103+
| PR #73 (2026-05-18) consolidated `.containerization/Containerfile.wave3` —
104+
one shared `rust-builder`, `--target` per prover.
105+
106+
| All non-proprietary backends runtime-smoke-verified REAL
107+
| ✅ DONE
108+
| 8/8 confirmed by exercising the binary in the built image (not by
109+
trusting the build log). Five silent-stub failures fixed at-source
110+
during verification: scip dead-download, tamarin dead-download,
111+
metamath missing libc6-dev, or-tools missing build-number, twelf/acl2
112+
baked build paths, proverif opam-solve.
113+
114+
| Imandra documented as honest licence-gated stub
115+
| ✅ DONE
116+
| Real Rust adapter + Idris2 ABI proofs retained; `IMANDRA_TOKEN`
117+
secret path documented; KEEP decision recorded 2026-05-18.
118+
119+
| Container CI workflow green
120+
| ⚠ UNVERIFIED
121+
| `container-ci.yml` weekly tier3-container matrix is allow-fail. No
122+
recent green-run count on main.
123+
124+
| PR #74 (proverif opam) resolved
125+
| ✅ DONE
126+
| Closed by the 2026-05-18 source-tarball pivot.
127+
|===
128+
129+
**To close:** verify `container-ci.yml` weekly Tier-3 runs are not
130+
silently red-but-allow-fail; record green-run count in `STATE.a2ml`.
131+
132+
== Criterion 4 — Wave-4 (Tier-4, quarterly)
133+
134+
Wave-4 is the quarterly allow-fail Tier-4 list (19 backends): Mizar,
135+
Nuprl, PVS, Minlog, Dedukti, Arend, KeY, Prism, UPPAAL, ViPER, NuSMV,
136+
Spin, TLC, CBMC, Seahorn, dReal, Boogie, Kissat, Alloy.
137+
138+
[%header,cols="2,1,3"]
139+
|===
140+
| Sub-criterion | Status | Notes
141+
142+
| Per-backend mock-only rationale in STATE.a2ml
143+
| ✅ DONE
144+
| `STATE.a2ml [wave-4-rationale]` carries one-line rationale per
145+
backend (1ca9862, 2026-04-26).
146+
147+
| Scaffold placeholder in `live-provers.yml`
148+
| ✅ DONE
149+
| Tier-4 placeholder job announces the list.
150+
151+
| At least one Wave-4 quarterly run executed
152+
| ⚠ UNVERIFIED
153+
| No evidence in STATE.a2ml that the placeholder has ever fired.
154+
Possibly cron-pinned but never visited.
155+
|===
156+
157+
**To close:** check that the quarterly cron has fired at least once
158+
since scaffolding, OR explicitly defer Wave-4 with a STATE.a2ml note
159+
"Wave-4 is scaffold-only and acceptable in the L3 hand-off — to be
160+
revisited when a maintainer contributes a Containerfile".
161+
162+
Both readings satisfy the L3 prompt's phrasing
163+
("...all four waves land *or are explicitly deferred with rationale*
164+
in STATE.a2ml"). The choice is the owner's.
165+
166+
== Criterion 5 — Tier-1 green on main for ≥7 days
167+
168+
Same as Criterion 1, sub-row 3 — broken out here because the L3 prompt
169+
states it as a top-level acceptance criterion.
170+
171+
[%header,cols="2,1,3"]
172+
|===
173+
| Sub-criterion | Status | Notes
174+
175+
| Run data accessible from a host with `gh` + auth
176+
| ❌ TODO
177+
| `STATE.a2ml [session-…l3-triage]` blocked on this. Local sessions
178+
have lacked `gh` + GITHUB_TOKEN.
179+
180+
| Tier-1 nightly green count ≥7 over last 7 days on main
181+
| ❌ TODO
182+
| Requires the above. Suggested query:
183+
+
184+
[source,shell]
185+
----
186+
gh run list \
187+
--workflow live-provers.yml \
188+
--branch main \
189+
--status success \
190+
--limit 14 \
191+
--json conclusion,startedAt
192+
----
193+
194+
| Record green window in STATE.a2ml
195+
| ❌ TODO
196+
| Single key, e.g. `l3-tier1-green-window = "2026-MM-DD .. 2026-MM-DD"`.
197+
|===
198+
199+
== Criterion 6 — Operational pre-requisites
200+
201+
From `docs/handover/TODO.md`:
202+
203+
[%header,cols="2,1,3"]
204+
|===
205+
| Sub-criterion | Status | Notes
206+
207+
| `guix shell -m manifests/live-provers.scm -- just test-live`
208+
end-to-end on a dev machine
209+
| ❌ TODO
210+
| Stated as L3 acceptance in the prompt. Has not been run end-to-end
211+
on a fresh dev host (Guix-extend was rejected; sealed container is
212+
the fallback for the not-in-Guix tail).
213+
214+
| Dafny stub-shape upgrade (165 LoC → deep)
215+
| ⚠ UNKNOWN
216+
| Stated as L3 acceptance in the prompt. No commit found that closes
217+
this; needs an audit pass against `src/rust/provers/dafny.rs`.
218+
219+
| VeriSimDB integration emits live-run records
220+
| ⚠ PARTIAL
221+
| Per `feedback_verisimdb_policy`, write paths exist in
222+
`dispatch.rs::emit_live_result` (2026-04-26). CI workflow gated on
223+
upstream `verisimdb#ghcr-publish.yml` — runbook YAML kept but not
224+
merged upstream. Tracked separately by audit item A5.
225+
|===
226+
227+
== Summary
228+
229+
[%header,cols="1,4,1"]
230+
|===
231+
| Criterion | Description | Status
232+
233+
| 1 | Wave-1 (every-PR Tier-1) green ≥7 days | ⚠ UNKNOWN (data not collected)
234+
| 2 | Wave-2 (nightly Tier-2) green ≥1 cycle | ⚠ UNVERIFIED
235+
| 3 | Wave-3 (weekly Tier-3) green ≥1 cycle | ⚠ UNVERIFIED
236+
| 4 | Wave-4 (quarterly Tier-4) executed or explicitly deferred | ⚠ owner choice
237+
| 5 | Tier-1 green ≥7 days on main (data) | ❌ TODO
238+
| 6a | `guix shell` local end-to-end | ❌ TODO
239+
| 6b | Dafny stub upgrade | ⚠ UNKNOWN
240+
| 6c | VeriSimDB live-record emission | ⚠ PARTIAL (gated on verisimdb#ghcr)
241+
|===
242+
243+
**Minimum sufficient path to close the gate:**
244+
245+
. Run the `gh run list` query for the last 14 nightlies of
246+
`live-provers.yml` on main; record the green window in
247+
`STATE.a2ml`. (Closes Criteria 1, 2, 5.)
248+
. Same query against `container-ci.yml` for Tier-3 weekly.
249+
(Closes Criterion 3 sub-row 4.)
250+
. Owner decision on Wave-4: execute one quarterly run OR formally
251+
defer with a STATE.a2ml note. (Closes Criterion 4 sub-row 3.)
252+
. One dev-host run of `guix shell -m manifests/live-provers.scm --
253+
just test-live`. (Closes Criterion 6a.)
254+
. Audit `src/rust/provers/dafny.rs` against the "deep wiring" bar
255+
used for Coq / Lean4. If it falls short, file a follow-up; if it
256+
meets the bar, record that in STATE.a2ml. (Closes Criterion 6b.)
257+
. (Criterion 6c stays parked on `verisimdb#ghcr-publish.yml` — that
258+
is upstream-blocked and not on this gate's critical path.)
259+
260+
Estimated effort: **one session** if the gh tokens are available.
261+
Without gh access, Criteria 1/2/3/5 cannot close, full stop.
262+
263+
== Cross-references
264+
265+
* `docs/handover/L3-LIVE-PROVER-CI-PROMPT.md` — original L3 mission.
266+
* `docs/handover/TODO.md` — P0 / P1 rows referenced above.
267+
* `docs/handover/L1-CAPNPROTO-PROMPT.md` — the prompt that fires
268+
once this gate closes.
269+
* `docs/handover/L2-CHAPEL-PROMPT.md` — blocked behind L1 + this gate.
270+
* C12 PR (echidnabot manifest) — per-repo opt-in shape for consumers.
271+
* C14 PR (cross-repo proof DAG design) — what L1's Cap'n Proto schemas
272+
must carry.
273+
* C15 PR (gitbot-fleet hypatia wiring) — the closed-loop signal the
274+
L3 gate depends on for VeriSimDB record quality.
275+
* B7 PR (HP backend correction) — confirms the HP ecosystem is not
276+
blocking L3.

0 commit comments

Comments
 (0)