Skip to content

Commit d966f2e

Browse files
ArghDA M6: Lean4 backend + honest cross-backend check verdicts (#38)
## What & why The **sixth backend** — Lean4 — plus a soundness-honesty fix it exposed. ## Lean4 backend `lean <file>` elaboration. The subtlety Lean forces us to be honest about: a green exit does **not** mean proven — `sorry` is only a *warning* and still exits 0. So the verdict is parsed from output, not the bare exit code: - exit non-zero → **Error** - exit 0 + `sorry` in output → **Admitted** (a hole rode along) - exit 0 clean → **Unknown** — elaborated, but arghda refuses to claim `Proven` without a `#print axioms` audit (it could still use `native_decide`/`sorryAx`). Exactly the plan's "honest Unknown when the audit is absent". Dotted modules reuse `graph::module_name_of`; `.lean` `module_to_path`; import parser for top-level `import Mod` (`open` is a namespace directive, **not** an edge — deliberately ignored); `Main.lean` root discovery. `LeanEscapeHatch` (sorry/admit/native_decide/unsafe), named `escape-hatch` to reuse the amber cap. Wired via `--backend lean4`. Ground-truthed against Lean 4.13.0. ## Honesty fix exposed by M6 (all backends) `check`'s verdict was collapsing to proven-eligible/rejected — which mislabelled a *clean, correct* Lean file as `rejected`, and an SMT `sat` as `rejected`. It now reports the real verdict word (**proven-eligible / refuted / unknown / admitted / postulated / rejected**), and the human line drops the assistant-centric ok/FAILED. ## Verified (ran, not inferred) - `cargo test` → **119 passing, 0 failed** (+5 lean unit, +4 lean-escape unit, +2 lean integration). - clippy -D warnings, fmt, SPDX clean. - **Dogfooded real lean:** clean proof → `unknown`, `sorry` → `admitted`; re-dogfooded `z3 sat` → `refuted`, `z3 unsat` → `proven-eligible`, `agda` → `proven-eligible`. `arghda doctor` → 6 backends runnable. ## Honest scope M6 = ~85%. Follow-on 15%: lake-env/`LEAN_PATH` for project-wide multi-file resolution, and the per-declaration `#print axioms` audit to promote `Unknown` → `Proven`. ## Milestone status Six backends across both interaction models. Next: M11 (studio JSON freeze + Groove manifest — the last open v0.1 milestone), M7 (Echidna seam). --- _Generated by [Claude Code](https://claude.ai/code/session_012MpYSh6Wy8YMBH2E3qVyT7)_ Co-authored-by: Claude <noreply@anthropic.com>
1 parent 25e38ba commit d966f2e

10 files changed

Lines changed: 485 additions & 24 deletions

File tree

.machine_readable/6a2/STATE.a2ml

Lines changed: 30 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ milestones = [
4646
{ name = "M3: Flying-Logic reasoning graph (src/reason: And|Or juncts, demote-only cycle-safe Verdict propagation, additive studio JSON)", completion = 100 },
4747
{ name = "M4: Idris2 adapter (core/ABI language: .ipkg/--check/totality; escape-hatches)", completion = 90 },
4848
{ name = "M5: SMT solver backends (Z3, CVC5): SMT-LIB2 -> sat/unsat/unknown -> Verdict", completion = 100 },
49-
{ name = "M6: Lean4 adapter (lake build + #print axioms audit)", completion = 0 },
49+
{ name = "M6: Lean4 adapter (lake build + #print axioms audit)", completion = 85 },
5050
{ name = "M7: Echidna dispatch seam (optional route to orchestrator :8090, same Outcome)", completion = 0 },
5151
{ name = "M8: Coq/Rocq adapter (--heavy provisioning; Section-aware postulate classifier)", completion = 0 },
5252
{ name = "M9: Isabelle adapter (--heavy; dogfood tropical-resource-typing .thy)", completion = 0 },
@@ -60,6 +60,28 @@ milestones = [
6060
# Remaining M0: `arghda doctor` Rust subcommand (interim: `just doctor` shells
6161
# the script's --verify-only table).
6262
#
63+
# M6 landed 2026-07-01 (85%): the Lean4 backend — sixth backend. src/prover/
64+
# lean.rs: `lean <file>` elaboration. The honesty subtlety Lean forces: a green
65+
# exit does NOT mean proven (sorry is only a warning; exit 0). So the verdict is
66+
# NOT exit-code-only like agda/idris2 — it PARSES output: exit≠0→Error; exit 0 +
67+
# "sorry" in output→Admitted; exit 0 clean→Unknown (elaborated but NOT audited —
68+
# arghda refuses to claim Proven without a #print axioms audit; matches the plan's
69+
# "honest Unknown when the audit is absent"). Dotted modules reuse module_name_of;
70+
# .lean module_to_path; import parser (top-level `import Mod`; `open` is a
71+
# namespace directive, NOT an edge — deliberately ignored); Main.lean roots.
72+
# src/lint/lean.rs: LeanEscapeHatch (sorry/admit/native_decide/unsafe) named
73+
# "escape-hatch". Wired via --backend lean4. Ground-truthed vs Lean 4.13.0.
74+
# HONESTY FIX exposed by M6 (applies to ALL backends): the `check` command's
75+
# verdict was collapsing to proven-eligible/rejected, mislabelling a clean Lean
76+
# file "rejected" and an SMT `sat` "rejected". Now reports the real verdict word
77+
# (proven-eligible/refuted/unknown/admitted/postulated/rejected); the human line
78+
# drops the assistant-centric ok/FAILED. 119 tests (86 lib + 33 integration;
79+
# +5 lean unit +4 lean-escape unit +2 lean integration). Dogfooded REAL lean:
80+
# clean proof→unknown, sorry→admitted; re-dogfooded z3 sat→refuted, unsat→
81+
# proven-eligible, agda→proven-eligible; doctor now 6 backends. Follow-ons (the
82+
# 15%): lake-env/LEAN_PATH for project-wide multi-file resolution; the per-decl
83+
# #print axioms audit to promote Unknown→Proven.
84+
#
6385
# M0 COMPLETED 2026-07-01: the `arghda doctor` subcommand lands, closing the
6486
# last M0 gap. Probes every known backend by running `<command> --version`;
6587
# reports runnable/detail honestly (runnable=false only when the binary truly
@@ -162,10 +184,10 @@ milestones = [
162184

163185
[critical-next-actions]
164186
actions = [
165-
"M0 + M1 + M2 + M3 + M4 + M5 DONE. Five backends across both interaction models (agda, agda-cubical, idris2 Assistant; z3, cvc5 Solver) + reasoning graph + doctor. Engine spine core complete. Next:",
166-
"Flying-Logic epic M6: Lean4 adapter (lake build + #print axioms audit → honest Unknown soundness when the audit is absent). Lean 4.13.0 provisioned. Sixth backend.",
167-
"Flying-Logic epic M11: studio JSON freeze (reason/0.1) + Groove /.well-known/groove manifest (the one open v0.1 milestone; `arghda doctor` output feeds it).",
187+
"M0-M6 DONE. SIX backends (agda, agda-cubical, idris2, lean4 Assistant; z3, cvc5 Solver) + reasoning graph + doctor. Engine spine essentially complete. Next:",
188+
"Flying-Logic epic M11: studio JSON freeze (reason/0.1) + Groove /.well-known/groove manifest (the one open v0.1 milestone; `arghda doctor` output feeds the manifest).",
168189
"Flying-Logic epic M7: Echidna dispatch seam (feature-gated client, same Outcome).",
190+
"M6 follow-on (15%): lake-env/LEAN_PATH multi-file resolution; per-decl #print axioms audit to promote Unknown→Proven.",
169191
"M4 follow-on: .ipkg-declared roots; totality-hole (?name) + per-def `partial` lint.",
170192
"M3 follow-on: feed real verdicts into `reason` from a workspace `proven/` state + wire staleness from the content-hash closure (proven.rs).",
171193
"M8-M10 heavy tail (Coq/Rocq, Isabelle, Mizar) — gated on --heavy provisioning.",
@@ -177,10 +199,10 @@ last-result = "pass" # unknown | pass | warn | fail
177199
open-warnings = 0
178200
open-failures = 0
179201
# `just check` (fmt-check + clippy -D warnings + build + test + SPDX) green
180-
# post-M0-doctor: 80 lib + 28 integration = 108 tests pass. `arghda doctor`
181-
# dogfooded: 5/5 backends runnable. All five backends dogfooded via real CLI:
182-
# agda + agda-cubical + idris2 (Assistant), z3 + cvc5 (Solver). Dogfooded
183-
# historically against echo-types + agda-unused.
202+
# post-M6: 86 lib + 33 integration = 119 tests pass. `arghda doctor` dogfooded:
203+
# 6/6 backends runnable. All six dogfooded via real CLI: agda + agda-cubical +
204+
# idris2 + lean4 (Assistant), z3 + cvc5 (Solver). Dogfooded historically against
205+
# echo-types + agda-unused.
184206

185207
[ecosystem]
186208
part-of = ["arghda"]

src/lib.rs

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,8 @@ pub use event::{Event, EventKind};
3131
pub use graph::{build as build_graph, ImportGraph};
3232
pub use lint::{default_rules, rules_with_config, run_lints, LintRule, RuleConfig};
3333
pub use prover::{
34-
default_backend, Agda, AgdaCubical, Backend, BackendKind, Idris2, Outcome, Probe, Smt, Verdict,
34+
default_backend, Agda, AgdaCubical, Backend, BackendKind, Idris2, Lean, Outcome, Probe, Smt,
35+
Verdict,
3536
};
3637
pub use reason::{build as build_reason, Junct, ReasonDocument, ReasonEdge, ReasonNode};
3738
pub use workspace::{StaleEntry, State, Workspace};

src/lint/lean.rs

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
// SPDX-License-Identifier: MPL-2.0
2+
// Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) <j.d.a.jewell@open.ac.uk>
3+
4+
//! Lean4 `escape-hatch` (warn) — surface soundness escape hatches.
5+
//!
6+
//! The Lean4 analogue of the Agda/Idris2 escape-hatch rules; shares the rule
7+
//! *name* `escape-hatch` so the reasoning graph caps any hit at amber.
8+
//!
9+
//! Flags `sorry` and `admit` (proof holes), `native_decide` (trusts the
10+
//! compiler's evaluation, outside the kernel) and `unsafe` (bypasses the
11+
//! termination/positivity checks). Lean `--check`/elaboration exits 0 even
12+
//! with `sorry` present (it is only a *warning*), so without this rule — and
13+
//! without the `#print axioms` audit — such gaps ride along invisibly. That
14+
//! is why the Lean backend's own verdict is at best `Unknown` on a green
15+
//! elaboration absent the audit.
16+
17+
use super::{LintContext, LintRule};
18+
use crate::diagnostic::{Diagnostic, LintReport, Severity};
19+
use anyhow::{Context, Result};
20+
use std::fs;
21+
use std::path::Path;
22+
23+
pub struct LeanEscapeHatch;
24+
25+
const ESCAPE_TOKENS: &[&str] = &["sorry", "admit", "native_decide", "unsafe"];
26+
27+
impl LintRule for LeanEscapeHatch {
28+
fn name(&self) -> &'static str {
29+
"escape-hatch"
30+
}
31+
32+
fn run(&self, file: &Path, _ctx: &LintContext<'_>, report: &mut LintReport) -> Result<()> {
33+
let contents =
34+
fs::read_to_string(file).with_context(|| format!("reading {}", file.display()))?;
35+
for (i, line) in contents.lines().enumerate() {
36+
let trimmed = line.trim_start();
37+
if trimmed.starts_with("--") {
38+
continue; // whole-line comment
39+
}
40+
let code = line.split(" --").next().unwrap_or(line);
41+
for tok in ESCAPE_TOKENS {
42+
if has_token(code, tok) {
43+
report.push(warn(
44+
self.name(),
45+
file,
46+
i + 1,
47+
format!("escape primitive `{tok}`"),
48+
));
49+
}
50+
}
51+
}
52+
Ok(())
53+
}
54+
}
55+
56+
fn warn(rule: &str, file: &Path, line: usize, message: String) -> Diagnostic {
57+
Diagnostic {
58+
rule: rule.to_string(),
59+
severity: Severity::Warn,
60+
file: file.to_path_buf(),
61+
message,
62+
line: Some(line),
63+
}
64+
}
65+
66+
/// `tok` appears in `s` as a delimited token (so `sorryAx_helper` or a
67+
/// namespaced `Foo.admit` do not spuriously match the bare keyword).
68+
fn has_token(s: &str, tok: &str) -> bool {
69+
s.split(|c: char| c.is_whitespace() || "(){};,.".contains(c))
70+
.any(|w| w == tok)
71+
}
72+
73+
#[cfg(test)]
74+
mod tests {
75+
use super::*;
76+
use crate::lint::run_lints;
77+
use std::path::PathBuf;
78+
79+
fn lint_str(body: &str) -> LintReport {
80+
let tmp = tempfile::NamedTempFile::new().unwrap();
81+
std::fs::write(tmp.path(), body).unwrap();
82+
let roots: [PathBuf; 0] = [];
83+
let ctx = LintContext {
84+
include_root: tmp.path().parent().unwrap(),
85+
entry_modules: &roots,
86+
};
87+
let rules: Vec<Box<dyn LintRule>> = vec![Box::new(LeanEscapeHatch)];
88+
run_lints(tmp.path(), &ctx, &rules).unwrap()
89+
}
90+
91+
#[test]
92+
fn sorry_is_warned_under_escape_hatch() {
93+
let r = lint_str("theorem t : 1 = 1 := by sorry\n");
94+
assert_eq!(r.warns().count(), 1);
95+
assert_eq!(r.diagnostics[0].rule, "escape-hatch");
96+
assert!(r.diagnostics[0].message.contains("sorry"));
97+
}
98+
99+
#[test]
100+
fn native_decide_and_unsafe_are_warned() {
101+
let r = lint_str("unsafe def f : Nat := 0\nexample : P := by native_decide\n");
102+
assert_eq!(r.warns().count(), 2);
103+
}
104+
105+
#[test]
106+
fn escape_token_in_comment_is_ignored() {
107+
let r = lint_str("def x : Nat := 0 -- TODO: no sorry here\n");
108+
assert!(r.diagnostics.is_empty());
109+
}
110+
111+
#[test]
112+
fn clean_lean_file_is_silent() {
113+
let r = lint_str("theorem t : 1 = 1 := rfl\n");
114+
assert!(r.diagnostics.is_empty());
115+
}
116+
}

src/lint/mod.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ use std::path::{Path, PathBuf};
77

88
pub mod escape_hatch;
99
pub mod idris2;
10+
pub mod lean;
1011
pub mod orphan_module;
1112
pub mod postulate;
1213
pub mod safe_pragma;

src/main.rs

Lines changed: 31 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ use anyhow::{Context, Result};
55
use arghda_core::lint::LintContext;
66
use arghda_core::{
77
build_dag, build_reason, event, run_lints, unused, watcher, Agda, AgdaCubical, Backend,
8-
BackendKind, Idris2, LintRule, RuleConfig, Smt, State, Workspace,
8+
BackendKind, Idris2, Lean, LintRule, RuleConfig, Smt, State, Verdict, Workspace,
99
};
1010
use clap::{Parser, Subcommand};
1111
use std::path::{Path, PathBuf};
@@ -16,7 +16,7 @@ use walkdir::WalkDir;
1616
type RuleSet = Vec<Box<dyn LintRule>>;
1717

1818
/// Every backend `--backend` accepts and `doctor` probes.
19-
const KNOWN_BACKENDS: &[&str] = &["agda", "agda-cubical", "idris2", "z3", "cvc5"];
19+
const KNOWN_BACKENDS: &[&str] = &["agda", "agda-cubical", "idris2", "lean4", "z3", "cvc5"];
2020

2121
/// Resolve a `--backend` name to a backend instance. Agda is the default and
2222
/// v0.1 reference; Idris2 is the estate ABI language.
@@ -25,19 +25,20 @@ fn backend_for(name: &str) -> Result<Box<dyn Backend>> {
2525
"agda" => Ok(Box::new(Agda)),
2626
"agda-cubical" => Ok(Box::new(AgdaCubical)),
2727
"idris2" => Ok(Box::new(Idris2)),
28+
"lean4" => Ok(Box::new(Lean)),
2829
"z3" => Ok(Box::new(Smt::z3())),
2930
"cvc5" => Ok(Box::new(Smt::cvc5())),
30-
other => {
31-
anyhow::bail!("unknown backend `{other}` (known: agda, agda-cubical, idris2, z3, cvc5)")
32-
}
31+
other => anyhow::bail!(
32+
"unknown backend `{other}` (known: agda, agda-cubical, idris2, lean4, z3, cvc5)"
33+
),
3334
}
3435
}
3536

3637
#[derive(Parser)]
3738
#[command(
3839
name = "arghda",
3940
version,
40-
about = "Proof-workspace manager for provers/solvers (Agda, Cubical, Idris2, Z3, CVC5)"
41+
about = "Proof-workspace manager for provers/solvers (Agda, Cubical, Idris2, Lean4, Z3, CVC5)"
4142
)]
4243
struct Cli {
4344
#[command(subcommand)]
@@ -64,7 +65,8 @@ enum Cmd {
6465
/// `<PATH>/.arghda/config.toml` if present.
6566
#[arg(long)]
6667
config: Option<PathBuf>,
67-
/// Backend: `agda` (default), `agda-cubical`, `idris2`, `z3`, `cvc5`.
68+
/// Backend: `agda` (default), `agda-cubical`, `idris2`, `lean4`,
69+
/// `z3`, `cvc5`.
6870
#[arg(long, default_value = "agda")]
6971
backend: String,
7072
/// Also run the external `agda-unused` analyser and re-emit its
@@ -83,7 +85,8 @@ enum Cmd {
8385
/// Include root (search path). Defaults to the file's directory.
8486
#[arg(long)]
8587
include_root: Option<PathBuf>,
86-
/// Backend: `agda` (default), `agda-cubical`, `idris2`, `z3`, `cvc5`.
88+
/// Backend: `agda` (default), `agda-cubical`, `idris2`, `lean4`,
89+
/// `z3`, `cvc5`.
8790
#[arg(long, default_value = "agda")]
8891
backend: String,
8992
/// Emit the report as JSON.
@@ -106,7 +109,8 @@ enum Cmd {
106109
/// `<PATH>/.arghda/config.toml` if present.
107110
#[arg(long)]
108111
config: Option<PathBuf>,
109-
/// Backend: `agda` (default), `agda-cubical`, `idris2`, `z3`, `cvc5`.
112+
/// Backend: `agda` (default), `agda-cubical`, `idris2`, `lean4`,
113+
/// `z3`, `cvc5`.
110114
#[arg(long, default_value = "agda")]
111115
backend: String,
112116
},
@@ -129,7 +133,8 @@ enum Cmd {
129133
/// `<PATH>/.arghda/config.toml` if present.
130134
#[arg(long)]
131135
config: Option<PathBuf>,
132-
/// Backend: `agda` (default), `agda-cubical`, `idris2`, `z3`, `cvc5`.
136+
/// Backend: `agda` (default), `agda-cubical`, `idris2`, `lean4`,
137+
/// `z3`, `cvc5`.
133138
#[arg(long, default_value = "agda")]
134139
backend: String,
135140
/// Run the backend on every node to populate REAL prover verdicts
@@ -405,12 +410,24 @@ fn check(file: &Path, include_root: Option<&Path>, backend_name: &str, json: boo
405410
run_lints(file, &ctx, &rules).with_context(|| format!("linting {}", file.display()))?;
406411
let outcome = backend.check_file(file, include_root)?;
407412

413+
// Honest verdict: a lint hard-block rejects; otherwise report the
414+
// backend's ACTUAL verdict word. Only `Proven` (with no hard block) is
415+
// "proven-eligible" — a clean-but-unaudited Lean file is `unknown`, an
416+
// SMT `sat` is `refuted`, never silently "rejected"/"ok".
408417
let verdict = if !outcome.available {
409418
"backend-unavailable"
410-
} else if outcome.ok && !report.has_hard_block() {
411-
"proven-eligible"
412-
} else {
419+
} else if report.has_hard_block() {
413420
"rejected"
421+
} else {
422+
match outcome.verdict {
423+
Verdict::Proven => "proven-eligible",
424+
Verdict::Refuted => "refuted",
425+
Verdict::Unknown => "unknown",
426+
Verdict::Admitted => "admitted",
427+
Verdict::Postulated => "postulated",
428+
Verdict::Error => "rejected",
429+
Verdict::Unavailable => "backend-unavailable",
430+
}
414431
};
415432

416433
if json {
@@ -426,13 +443,12 @@ fn check(file: &Path, include_root: Option<&Path>, backend_name: &str, json: boo
426443
println!("{}", file.display());
427444
if outcome.available {
428445
println!(
429-
" {}: exit {}, {}",
446+
" {}: exit {}",
430447
backend.name(),
431448
outcome
432449
.exit_code
433450
.map(|c| c.to_string())
434451
.unwrap_or_else(|| "?".into()),
435-
if outcome.ok { "ok" } else { "FAILED" }
436452
);
437453
} else {
438454
println!(

0 commit comments

Comments
 (0)