You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# Phase progress heuristics derived from repo state
@@ -177,6 +181,7 @@ steps:
177
181
8: (3.0 if has_lean_specs else 1.0) if (has_rust and has_research) else 0.0, # aeneas: only for Rust codebases with research done
178
182
9: 12.0 if (has_lean_specs and not has_ci) else 2.0, # CI: critical when lean files exist but no CI; regular check otherwise
179
183
10: (8.0 if not has_report else 3.0) if has_proofs else (2.0 if has_lean_specs else 0.0), # report: important when proofs exist but no report; available once lean specs exist
184
+
11: (8.0 if not has_paper else 3.0) if has_proofs else 0.0, # paper: important when proofs exist but no paper; only available once proofs exist
'weights': {str(k): round(v, 2) for k, v in weights.items()},
@@ -581,7 +587,14 @@ This is a reflective task. The goal is not to prove more things, but to evaluate
581
587
- **Strength**: is the property tight (captures exactly the right behaviour) or weak (too easy to satisfy, even by incorrect implementations)?
582
588
3. For unproved / `sorry`-guarded theorems, assess whether they are worth proving or should be revised.
583
589
4. Identify the **highest-value gaps**: which properties, if proved, would give the most confidence in the codebase? Are there important invariants or safety properties that have not yet been attempted?
584
-
5. Write or update `formal-verification/CRITIQUE.md`:
590
+
5. **(Optional) Review the conference paper**: if `formal-verification/PAPER.md` exists, read it and assess it as a critical reviewer would:
591
+
- **Accuracy**: are all claims in the paper supported by the actual Lean proofs and FV artifacts? Are results overstated or understated?
592
+
- **Completeness**: does the paper cover all significant findings, including negative results and limitations?
593
+
- **Intellectual honesty**: are modelling approximations and their impact on proof validity clearly disclosed?
594
+
- **Clarity**: is the methodology well-explained? Would a reader unfamiliar with the codebase understand the contribution?
595
+
- **Missing content**: are there important results, findings, or limitations not yet reflected in the paper?
596
+
Include a `## Paper Review` section in CRITIQUE.md with specific, actionable feedback for improving the paper. Note any claims that need revision based on the current state of the proofs.
597
+
6. Write or update `formal-verification/CRITIQUE.md`:
585
598
- **Always** include a `## Last Updated` section at the top with the current UTC date/time and the HEAD commit SHA:
586
599
```
587
600
## Last Updated
@@ -593,8 +606,9 @@ This is a reflective task. The goal is not to prove more things, but to evaluate
593
606
- **Gaps and recommendations**: what should be proved next and why — prioritised by impact.
594
607
- **Concerns**: any theorems that look proved but may be vacuous due to model approximations (cross-reference CORRESPONDENCE.md).
595
608
- **Positive findings**: highlight any case where FV revealed or confirmed something non-obvious.
596
-
6. Create a PR with the updated CRITIQUE.md.
597
-
7. Update memory: record the critique findings, flag high-priority gaps for future runs.
609
+
- **Paper review** (if PAPER.md was reviewed in step 5): specific, actionable feedback on the conference paper — claims to revise, missing content, clarity issues.
610
+
7. Create a PR with the updated CRITIQUE.md.
611
+
8. Update memory: record the critique findings, flag high-priority gaps for future runs.
598
612
599
613
---
600
614
@@ -1076,6 +1090,170 @@ timeline
1076
1090
1077
1091
---
1078
1092
1093
+
### Task 11: Conference Paper
1094
+
1095
+
**Goal**: Write and maintain `formal-verification/PAPER.md` — an IEEE conference submission (11-page limit) on formal verification of the codebase, suitable for a conference on formal verification or software engineering. The paper covers the methodology, findings, proof architecture, modelling choices, and lessons learned.
1096
+
1097
+
This task produces a living document. Each run updates the paper to reflect the current state of the project rather than rewriting it from scratch.
1098
+
1099
+
> **Applicability gate**: This task only applies when meaningful proof work exists (proofs attempted or completed). If no Lean proofs exist yet, skip this task and substitute the most logically prior incomplete task.
1100
+
1101
+
1. Read all existing FV artifacts: Lean files, informal specs, CORRESPONDENCE.md, CRITIQUE.md, REPORT.md, TARGETS.md, RESEARCH.md, memory, open issues, and merged PRs. Read the existing PAPER.md if it exists.
1102
+
2.**Create or update**`formal-verification/PAPER.md` following standard IEEE conference paper structure. The document must fit within an **11-page limit** when rendered (use judgment to keep content appropriately scoped and concise).
1103
+
1104
+
#### Paper Structure
1105
+
1106
+
```markdown
1107
+
# <Title>: Formal Verification of <Repository/Component> with Lean 4
1108
+
1109
+
> 🔬 *Lean Squad — automated formal verification for `<owner>/<repo>`.*
1110
+
1111
+
## Abstract
1112
+
1113
+
{150–250 words. Summarise the contribution: what was verified, the approach taken
{Detail any bugs discovered through formal verification — these are the most
1189
+
compelling results. For each: the property expected to hold, the counterexample
1190
+
or proof failure, root cause, severity, and resolution. If no bugs were found,
1191
+
discuss what this means (implementation is correct w.r.t. verified properties,
1192
+
or the model was too abstract to catch bugs).}
1193
+
1194
+
### 4.3 Coverage Assessment
1195
+
1196
+
{Assess what fraction of the codebase's correctness-critical behaviour is covered
1197
+
by the current proofs. What remains unverified? What are the highest-value gaps?
1198
+
Cross-reference CRITIQUE.md.}
1199
+
1200
+
## 5. Discussion
1201
+
1202
+
### 5.1 Proof Utility
1203
+
1204
+
{Critically assess: are the proved properties meaningful? Would real bugs cause
1205
+
theorem failures? Or are the models too abstract to catch practical issues?
1206
+
Discuss the tension between model fidelity and proof tractability.}
1207
+
1208
+
### 5.2 Automation and Effort
1209
+
1210
+
{Discuss the level of automation achieved. How much was automatic (decide, simp)
1211
+
vs. manual proof engineering? What was the approximate effort per theorem?
1212
+
Discuss implications for scaling formal verification.}
1213
+
1214
+
### 5.3 Limitations
1215
+
1216
+
{Be explicit about limitations: model approximations, unverified assumptions,
1217
+
properties not yet attempted, toolchain limitations. This section strengthens
1218
+
the paper by demonstrating intellectual honesty.}
1219
+
1220
+
### 5.4 Lessons Learned
1221
+
1222
+
{Practical lessons from the verification effort: what worked, what didn't,
1223
+
what would you do differently, advice for others attempting similar work.}
1224
+
1225
+
## 6. Conclusion
1226
+
1227
+
{Summarise contributions, key findings, and future work directions.
1228
+
Restate headline numbers and most important results.}
1229
+
1230
+
## References
1231
+
1232
+
{Include references to: Lean 4, Mathlib, Aeneas/Charon (if used), related
1233
+
formal verification work, the target codebase documentation, any textbook
1234
+
algorithms or protocols that were verified against.}
1235
+
```
1236
+
1237
+
3.**Content quality requirements**:
1238
+
- Write in formal academic style appropriate for an IEEE conference submission.
1239
+
- All claims must be supported by evidence from the actual Lean proofs and FV artifacts.
1240
+
- Be intellectually honest: clearly distinguish between what is proved and what is assumed, between exact models and approximations.
1241
+
- Include concrete examples: show key theorem statements in Lean syntax, show counterexamples for bugs found, show representative proof fragments for interesting cases.
1242
+
- The paper should be self-contained: a reader unfamiliar with the repository should understand the contribution.
1243
+
- Respect the **11-page limit**: be concise. Prioritise depth on methodology, results, and discussion over exhaustive listing. Move detailed inventories to an appendix or reference REPORT.md.
1244
+
4.**Incremental updates**: read the existing PAPER.md (if any), update sections that have changed (new theorems, new findings, revised assessments), and maintain consistency throughout. Do not delete prior content unless it has become incorrect.
1245
+
5.**Always** include a `## Last Updated` section near the top with the current UTC date/time and the HEAD commit SHA:
1246
+
```
1247
+
## Last Updated
1248
+
- **Date**: YYYY-MM-DD HH:MM UTC
1249
+
- **Commit**: `<SHA>`
1250
+
```
1251
+
6. Cross-reference CORRESPONDENCE.md, CRITIQUE.md, and REPORT.md for modelling choices, proof utility assessments, and detailed inventories.
1252
+
7. Create a PR with the updated PAPER.md.
1253
+
8. Update memory: note that the paper exists and what state it covers.
1254
+
1255
+
---
1256
+
1079
1257
### Task Final: Update Lean Squad Status Issue *(ALWAYS DO THIS EVERY RUN)*
1080
1258
1081
1259
Maintain a single open issue titled `[Lean Squad] Formal Verification Status` as a continuously-updated dashboard for maintainers.
0 commit comments