Skip to content

Commit 378e7e2

Browse files
committed
feat(correspondence): compiled Lean model as differential test oracle
Advances the Lean->Rust correspondence toward mechanization by making the compiled proof artifact the test oracle. - proofs/lean4/ModelOracle.lean + lean_exe model_oracle: compiles the exact proven model defs (mkdir/rmdir/createFile/deleteFile/fsUpdate from FilesystemModel + FileOperations) into a standalone executable that reports the node type (DIR/FILE/NONE) at probe paths. - impl/rust-cli/tests/model_oracle_correspondence.rs: generates precondition-respecting op sequences (shadow tracker ensures validity), applies them to the real Rust ShellState, drives the same sequence through the Lean oracle, and asserts model and impl agree at every touched path. 1218 probes agree across 200 sequences. Skips cleanly if the oracle is not built (no Lean toolchain required for ). - just build-model-oracle / test-correspondence-model; wired into lean-verification.yml CI. - docs/LEAN4_RUST_CORRESPONDENCE.md: documents the harness and, honestly, its limits — this is differential testing against the proof artifact, not a full refinement proof (still the v1.0 blocker). Records why the Coq extraction is not executable (Path -> option FSNode function domain + classical axioms in is_empty_dir_dec; needs FMaps.t FSNode migration) and the mechanization roadmap. Verified: lake build model_oracle OK; oracle smoke test matches model semantics (incl. non-recursive rmdir leaving orphan children); differential test passes with and without VSH_MODEL_ORACLE env (auto-detects .lake path). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KfgJznd6jzSeDYsSXGAXkU
1 parent 5632654 commit 378e7e2

7 files changed

Lines changed: 473 additions & 3 deletions

File tree

.github/workflows/lean-verification.yml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,18 @@ jobs:
106106
cargo test --test correspondence_tests
107107
cargo test --test property_correspondence_tests
108108
109+
- name: Build Lean model oracle (proof artifact as test oracle)
110+
run: |
111+
source $HOME/.elan/env
112+
cd proofs/lean4
113+
lake build model_oracle
114+
ls -lh .lake/build/bin/model_oracle
115+
116+
- name: Differential correspondence (proven Lean model vs Rust impl)
117+
run: |
118+
cd impl/rust-cli
119+
cargo test --test model_oracle_correspondence -- --nocapture
120+
109121
- name: Report Rust binary size
110122
run: |
111123
echo "Rust binary: $(stat -c%s impl/rust-cli/target/release/vsh) bytes"

CHANGELOG.adoc

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1919

2020
=== Added — 2026-07-17
2121

22+
==== Correspondence (mechanized differential oracle)
23+
- `proofs/lean4/ModelOracle.lean` + `lean_exe model_oracle` — compiles the
24+
**proven** Lean model definitions (`mkdir`/`rmdir`/`createFile`/`deleteFile`/
25+
`fsUpdate`) into a standalone oracle that reports the node type at probe paths.
26+
- `impl/rust-cli/tests/model_oracle_correspondence.rs` — differential test that
27+
generates precondition-respecting operation sequences, applies them to the real
28+
Rust `ShellState`, runs the same sequence through the compiled Lean model, and
29+
asserts they agree at every touched path. **1218 probes agree across 200
30+
sequences.** This makes the proof artifact itself the test oracle (stronger
31+
than hand-transcribed property tests); it remains differential testing, not a
32+
full refinement proof — the latter is still the v1.0 blocker. The test skips
33+
cleanly when the oracle is not built.
34+
- `just build-model-oracle` / `just test-correspondence-model`; wired into
35+
`lean-verification.yml` CI.
36+
- `docs/LEAN4_RUST_CORRESPONDENCE.md` documents the harness, why the Coq
37+
extraction is not executable (function-typed filesystem + classical axioms;
38+
needs `FMaps.t FSNode` migration), and the roadmap to full mechanization.
39+
2240
==== Launcher (E-Grade Launcher Standard)
2341
- `launch.sh` — a standard-conformant launcher implementing the Hyperpolymath
2442
E-Grade Launcher Standard mode surface: `--auto` (default), `--start`,

Justfile

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,18 @@ build-lean4:
4343
cd proofs/lean4 && lean FileOperations.lean
4444
@echo "✓ Lean 4 proofs checked"
4545

46+
# Build the executable model oracle (the compiled proven Lean model) used by the
47+
# differential correspondence test impl/rust-cli/tests/model_oracle_correspondence.rs
48+
build-model-oracle:
49+
@echo "Building Lean model oracle..."
50+
cd proofs/lean4 && lake build model_oracle
51+
@echo "✓ model oracle -> proofs/lean4/.lake/build/bin/model_oracle"
52+
53+
# Run the differential correspondence test (proven Lean model vs Rust impl).
54+
# Builds the oracle first so the test does not skip.
55+
test-correspondence-model: build-model-oracle
56+
cd impl/rust-cli && cargo test --test model_oracle_correspondence -- --nocapture
57+
4658
# Build Agda proofs
4759
build-agda:
4860
@echo "Building Agda proofs..."

docs/LEAN4_RUST_CORRESPONDENCE.md

Lines changed: 65 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -36,12 +36,74 @@ This document establishes the **correspondence** between:
3636
└──────────────────────────────┘
3737
```
3838

39-
**Verification Strategy**:
39+
**Verification Strategy** (weakest → strongest):
4040
- **Manual proofs**: Argue why Rust code matches Lean 4 spec
41-
- **Integration tests**: Validate behavior (28/28 passing)
42-
- **Echidna**: Property-based testing (planned)
41+
- **Integration tests**: Validate behavior
42+
- **Property tests**: Hand-transcribed reversibility laws checked generatively
43+
- **Mechanized differential oracle** *(new, 2026-07-17)*: the **compiled proven
44+
Lean model** is executed as the test oracle — see below.
4345
- **Code review**: Multiple reviewers check correspondence
4446

47+
## Mechanized differential correspondence (model-as-oracle)
48+
49+
`proofs/lean4/ModelOracle.lean` compiles the **exact** proven model definitions
50+
(`mkdir`, `rmdir`, `createFile`, `deleteFile`, `fsUpdate` from `FilesystemModel`
51+
+ `FileOperations`) into a standalone executable. Given an operation sequence and
52+
a set of probe paths, it reports the node type (`DIR`/`FILE`/`NONE`) the *model*
53+
computes at each path.
54+
55+
`impl/rust-cli/tests/model_oracle_correspondence.rs` then:
56+
57+
1. generates **precondition-respecting** operation sequences (the regime where the
58+
reversibility theorems hold — a shadow tracker guarantees every op is valid);
59+
2. applies each sequence to the real Rust `ShellState` (a sandboxed temp dir);
60+
3. runs the same sequence through the compiled Lean model oracle;
61+
4. asserts the model and the implementation agree at **every** touched path.
62+
63+
This is stronger than the property tests: the oracle is the **proof artifact
64+
itself**, not a hand-written re-statement of the laws. A divergence is a genuine
65+
model↔implementation mismatch. Current status: **1218 probes agree across 200
66+
sequences** (`just test-correspondence-model`).
67+
68+
```mermaid
69+
flowchart LR
70+
G[precondition-respecting<br/>op sequence] --> R[Rust ShellState<br/>real temp dir]
71+
G --> O[compiled Lean model<br/>ModelOracle]
72+
R --> C{agree at<br/>every probe?}
73+
O --> C
74+
C -->|yes| P[pass]
75+
C -->|no| D[correspondence<br/>divergence]
76+
```
77+
78+
### What this does and does not establish
79+
80+
- ✅ It executes the **proven** model (not a paraphrase) as the reference.
81+
- ✅ It is reproducible and CI-gatable (`just build-model-oracle`).
82+
- ⚠️ It is still **differential testing**, not a mechanized *refinement proof*.
83+
Full mechanization (a machine-checked proof that the Rust semantics refine the
84+
model) remains the v1.0 blocker.
85+
86+
### Why not extraction-based (Coq → OCaml)?
87+
88+
The Coq model extracts (`proofs/coq/extraction.v`) but is **not executable**: the
89+
filesystem is modelled as a total function `Path → option FSNode` over an infinite
90+
domain, and `is_empty_dir_dec` pulls in classical axioms
91+
(`constructive_indefinite_description`, `excluded_middle_informative`) that the
92+
extracted OCaml leaves unrealized. Making the *Coq* model executable requires
93+
migrating the representation to a finite map (`FMaps.t FSNode`) and re-proving the
94+
theorems over it — the same migration the proof-holes audit already recommends.
95+
The Lean model sidesteps this because its operations are ordinary computable
96+
`def`s that can be evaluated at any finite probe path, even though the state is
97+
also a function.
98+
99+
### Roadmap to full mechanization
100+
101+
1. **(done)** Compiled-model differential oracle (this section).
102+
2. Migrate the Coq filesystem model to `FMaps.t FSNode`; re-prove; extract an
103+
executable OCaml oracle and cross-check it against the Lean oracle.
104+
3. Model the Rust operational semantics (or a shared IR) and prove refinement —
105+
the machine-checked correspondence that closes the v1.0 gap.
106+
45107
---
46108

47109
## Filesystem Model Correspondence

0 commit comments

Comments
 (0)