Skip to content

Commit da4935c

Browse files
committed
Add goal printing flags (-upto, -lastgoals) and LLM agent guide
Add two new flags for the `easycrypt` CLI to support LLM coding agents: - `-upto <pos>`: compile up to a given position and print goals there - `-lastgoals`: print the last unproven goals Also add a dedicated `llm` command mode and an LLM agent guide (doc/llm/CLAUDE.md) documenting EasyCrypt tactics and workflow for use with AI coding assistants.
1 parent 5a50911 commit da4935c

8 files changed

Lines changed: 273 additions & 7 deletions

File tree

doc/llm/CLAUDE.md

Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
# EasyCrypt — LLM Agent Guide
2+
3+
EasyCrypt is a proof assistant for reasoning about the security of
4+
cryptographic constructions. It provides support for probabilistic
5+
computations, program logics (Hoare logic, probabilistic Hoare logic,
6+
probabilistic relational Hoare logic), and ambient mathematical
7+
reasoning.
8+
9+
## Using the `llm` command
10+
11+
The `llm` subcommand is designed for non-interactive, LLM-friendly
12+
batch compilation. It produces no progress bar and no `.eco` cache
13+
files.
14+
15+
```
16+
easycrypt llm [OPTIONS] FILE.ec
17+
```
18+
19+
### Options
20+
21+
- `-upto LINE` or `-upto LINE:COL` — Compile up to (but not
22+
including) the given location, then print the current goal state to
23+
stdout and exit with code 0. Use this to inspect the proof state at
24+
a specific point in a file.
25+
26+
- `-lastgoals` — On failure, print the goal state (as it was just
27+
before the failing command) to stdout, then print the error to
28+
stderr, and exit with code 1. Use this to understand what the
29+
failing tactic was supposed to prove.
30+
31+
Standard loader and prover options (`-I`, `-timeout`, `-p`, etc.) are
32+
also available.
33+
34+
### Output conventions
35+
36+
- **Goals** are printed to **stdout**.
37+
- **Errors** are printed to **stderr**.
38+
- **Exit code 0** means success (or `-upto` reached its target).
39+
- **Exit code 1** means a command failed.
40+
- If there is no active proof at the point where goals are requested,
41+
stdout will contain: `No active proof.`
42+
43+
### Workflow for writing and debugging proofs
44+
45+
1. Try to write a pen-and-paper proof first.
46+
47+
2. Write the `.ec` file with your proof attempt. For a large proof,
48+
write down skeleton and `admit` subgoals first, and then detail
49+
the proof.
50+
51+
3. Run `easycrypt llm -lastgoals FILE.ec` to check the full file.
52+
- If it succeeds (exit 0), you are done.
53+
- If it fails (exit 1), read the error from stderr and the goal
54+
state from stdout to understand what went wrong.
55+
56+
4. Use `-upto LINE` to inspect the proof state at a specific point
57+
without running the rest of the file. This is useful for
58+
incremental proof development.
59+
60+
5. Fix the proof and repeat from step 2. The ultimate proof should
61+
not contain `admit` or `admitted`.
62+
63+
## EasyCrypt language overview
64+
65+
### File structure
66+
67+
An EasyCrypt file typically begins with `require` and `import`
68+
statements, followed by type, operator, and module declarations, and
69+
then lemma statements with their proofs.
70+
71+
```
72+
require import AllCore List.
73+
74+
type key.
75+
op n : int.
76+
axiom gt0_n : 0 < n.
77+
78+
lemma foo : 0 < n + 1.
79+
proof. smt(gt0_n). qed.
80+
```
81+
82+
### Proofs
83+
84+
A proof is delimited by `proof.` and `qed.`. Inside, tactics are
85+
applied sequentially to transform the goal until it is discharged.
86+
87+
```
88+
lemma bar (x : int) : x + 0 = x.
89+
proof. by ring. qed.
90+
```
91+
92+
### Common tactics
93+
94+
<!-- TODO: expand this section with descriptions and examples -->
95+
96+
- `trivial` — solve trivial goals
97+
- `smt` / `smt(lemmas...)` — call SMT solvers, optionally with hints
98+
- `auto` — automatic reasoning
99+
- `split` — split conjunctions
100+
- `left` / `right` — choose a disjunct
101+
- `assumption` — close goal from a hypothesis
102+
- `apply H` — apply a hypothesis or lemma
103+
- `rewrite H` — rewrite using an equality
104+
- `have : P` — introduce an intermediate goal
105+
- `elim` — elimination / induction
106+
- `case` — case analysis
107+
- `congr` — congruence
108+
- `ring` / `field` — algebraic reasoning
109+
- `proc` — unfold a procedure (program logics)
110+
- `inline` — inline a procedure call
111+
- `sp` / `wp` — symbolic execution (forward / backward)
112+
- `if` — handle conditionals in programs
113+
- `while I` — handle while loops with invariant `I`
114+
- `rnd` — handle random sampling
115+
- `seq N : P` — split a program at statement `N` with mid-condition `P`
116+
- `conseq` — weaken/strengthen pre/postconditions
117+
- `byequiv` / `byphoare` — switch between program logics
118+
- `skip` — skip trivial program steps
119+
- `sim` — similarity (automatic relational reasoning)
120+
- `ecall` — external call
121+
122+
### Tactic combinators
123+
124+
- `by tac.` — apply `tac` and require all goals to be closed
125+
- `tac1; tac2` — sequence
126+
- `try tac` — try, ignore failure
127+
- `do tac` / `do N tac` — repeat
128+
- `[tac1 | tac2 | ...]` — apply different tactics to each subgoal
129+
- `tac => //.` — apply `tac`, then try `trivial` on generated subgoals
130+
- `move=> H` / `move=> /H` — introduction and views
131+
132+
### Key libraries
133+
134+
- `AllCore` — re-exports the core libraries (logic, integers, reals,
135+
lists, etc.)
136+
- `Distr` — probability distributions
137+
- `DBool`, `DInterval`, `DList` — specific distributions
138+
- `FSet`, `FMap` — finite sets and maps
139+
- `SmtMap` — maps with SMT support
140+
- `PROM` — programmable/lazy random oracles
141+
142+
### Guidelines
143+
144+
* Use SMT solver only in direct mode (smt() or /#) on simple goals (arithmetic goals, pure logical goals).
145+
146+
* Refrain from unfolding operator definitions unless necessary.
147+
If you need more properties on an operator, state this property in a dedicated lemma,
148+
but avoid unfolding definitions in higher level proofs.
149+

src/ec.ml

Lines changed: 52 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -415,6 +415,7 @@ let main () =
415415
(*---*) gccompact : int option;
416416
(*---*) docgen : bool;
417417
(*---*) outdirp : string option;
418+
(*---*) upto : (int * int option) option;
418419
mutable trace : trace1 list option;
419420
}
420421

@@ -493,6 +494,7 @@ let main () =
493494
; gccompact = None
494495
; docgen = false
495496
; outdirp = None
497+
; upto = None
496498
; trace = None }
497499

498500
end
@@ -528,10 +530,40 @@ let main () =
528530
; gccompact = cmpopts.cmpo_compact
529531
; docgen = false
530532
; outdirp = None
533+
; upto = None
531534
; trace = trace0 }
532535

533536
end
534537

538+
| `Llm llmopts -> begin
539+
let name = llmopts.llmo_input in
540+
541+
begin try
542+
let ext = Filename.extension name in
543+
ignore (EcLoader.getkind ext : EcLoader.kind)
544+
with EcLoader.BadExtension ext ->
545+
Format.eprintf "do not know what to do with %s@." ext;
546+
exit 1
547+
end;
548+
549+
let lastgoals = llmopts.llmo_lastgoals in
550+
let terminal =
551+
lazy (T.from_channel ~name ~progress:`Silent ~lastgoals (open_in name))
552+
in
553+
554+
{ prvopts = {llmopts.llmo_provers with prvo_iterate = true}
555+
; input = Some name
556+
; terminal = terminal
557+
; interactive = false
558+
; eco = true
559+
; gccompact = None
560+
; docgen = false
561+
; outdirp = None
562+
; upto = llmopts.llmo_upto
563+
; trace = None }
564+
565+
end
566+
535567
| `Runtest _ ->
536568
(* Eagerly executed *)
537569
assert false
@@ -572,6 +604,7 @@ let main () =
572604
; gccompact = None
573605
; docgen = true
574606
; outdirp = docopts.doco_outdirp
607+
; upto = None
575608
; trace = None }
576609
end
577610

@@ -585,7 +618,7 @@ let main () =
585618
| Some pwd -> EcCommands.addidir pwd);
586619

587620
(* Check if the .eco is up-to-date and exit if so *)
588-
(if not state.docgen then
621+
(if not state.docgen && state.upto = None then
589622
oiter
590623
(fun input -> if EcCommands.check_eco input then exit 0)
591624
state.input);
@@ -669,6 +702,16 @@ let main () =
669702
if T.interactive terminal then
670703
T.notice ~immediate:true `Warning copyright terminal;
671704

705+
(* Check if a location is past the -upto point *)
706+
let past_upto (loc : EcLocation.t) =
707+
match state.upto with
708+
| None -> false
709+
| Some (line, col) ->
710+
let (sl, sc) = loc.loc_start in
711+
sl > line || (sl = line && match col with
712+
| None -> true
713+
| Some c -> sc >= c) in
714+
672715
try
673716
if T.interactive terminal then Sys.catch_break true;
674717

@@ -737,6 +780,14 @@ let main () =
737780
List.iter
738781
(fun p ->
739782
let loc = p.EP.gl_action.EcLocation.pl_loc in
783+
784+
(* -upto: if this command starts past the target, print goals and exit *)
785+
if past_upto loc then begin
786+
T.finalize terminal;
787+
EcCommands.pp_current_goal_or_noproof ~all:true Format.std_formatter;
788+
exit 0
789+
end;
790+
740791
let timed = p.EP.gl_debug = Some `Timed in
741792
let break = p.EP.gl_debug = Some `Break in
742793
let ignore_fail = ref false in

src/ecCommands.ml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1024,6 +1024,13 @@ let pp_current_goal ?(all = false) stream =
10241024
end
10251025
end
10261026

1027+
(* -------------------------------------------------------------------- *)
1028+
let pp_current_goal_or_noproof ?(all = false) stream =
1029+
if Option.is_some (S.xgoal (current ())) then
1030+
pp_current_goal ~all stream
1031+
else
1032+
Format.fprintf stream "No active proof.@\n%!"
1033+
10271034
(* -------------------------------------------------------------------- *)
10281035
let pp_maybe_current_goal stream =
10291036
match (Pragma.get ()).pm_verbose with

src/ecCommands.mli

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,7 @@ val doc_comment : [`Global | `Item] * string -> unit
6060

6161
(* -------------------------------------------------------------------- *)
6262
val pp_current_goal : ?all:bool -> Format.formatter -> unit
63+
val pp_current_goal_or_noproof : ?all:bool -> Format.formatter -> unit
6364
val pp_maybe_current_goal : Format.formatter -> unit
6465
val pp_all_goals : unit -> string list
6566

src/ecOptions.ml

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ type command = [
1010
| `Runtest of run_option
1111
| `Why3Config
1212
| `DocGen of doc_option
13+
| `Llm of llm_option
1314
]
1415

1516
and options = {
@@ -47,6 +48,13 @@ and doc_option = {
4748
doco_outdirp : string option;
4849
}
4950

51+
and llm_option = {
52+
llmo_input : string;
53+
llmo_provers : prv_options;
54+
llmo_lastgoals : bool;
55+
llmo_upto : (int * int option) option;
56+
}
57+
5058
and prv_options = {
5159
prvo_maxjobs : int option;
5260
prvo_timeout : int option;
@@ -351,6 +359,12 @@ let specs = {
351359
`Spec ("trace" , `Flag , "Save all goals & messages in .eco");
352360
`Spec ("compact", `Int , "<internal>")]);
353361

362+
("llm", "LLM-friendly batch compilation", [
363+
`Group "loader";
364+
`Group "provers";
365+
`Spec ("lastgoals" , `Flag , "Print last unproved goals on failure");
366+
`Spec ("upto" , `String, "Compile up to LINE or LINE:COL and print goals")]);
367+
354368
("cli", "Run EasyCrypt top-level", [
355369
`Group "loader";
356370
`Group "provers";
@@ -533,6 +547,27 @@ let doc_options_of_values values input =
533547
{ doco_input = input;
534548
doco_outdirp = get_string "outdir" values; }
535549

550+
let parse_upto values =
551+
get_string "upto" values |> Option.map (fun s ->
552+
let invalid () =
553+
raise (Arg.Bad (Printf.sprintf
554+
"invalid -upto format: expected LINE or LINE:COL, got %S" s)) in
555+
match String.split_on_char ':' s with
556+
| [line] ->
557+
let line = try int_of_string line with Failure _ -> invalid () in
558+
(line, None)
559+
| [line; col] ->
560+
let line = try int_of_string line with Failure _ -> invalid () in
561+
let col = try int_of_string col with Failure _ -> invalid () in
562+
(line, Some col)
563+
| _ -> invalid ())
564+
565+
let llm_options_of_values ini values input =
566+
{ llmo_input = input;
567+
llmo_provers = prv_options_of_values ini values;
568+
llmo_lastgoals = get_flag "lastgoals" values;
569+
llmo_upto = parse_upto values; }
570+
536571
(* -------------------------------------------------------------------- *)
537572
let parse getini argv =
538573
let (command, values, anons) = parse specs argv in
@@ -604,6 +639,17 @@ let parse getini argv =
604639
raise (Arg.Bad "this command takes a single input file as argument")
605640
end
606641

642+
| "llm" -> begin
643+
match anons with
644+
| [input] ->
645+
let ini = getini (Some input) in
646+
let cmd = `Llm (llm_options_of_values ini values input) in
647+
(cmd, ini, true)
648+
649+
| _ ->
650+
raise (Arg.Bad "this command takes a single argument")
651+
end
652+
607653
| _ -> assert false
608654

609655
in {

src/ecOptions.mli

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ type command = [
66
| `Runtest of run_option
77
| `Why3Config
88
| `DocGen of doc_option
9+
| `Llm of llm_option
910
]
1011

1112
and options = {
@@ -43,6 +44,13 @@ and doc_option = {
4344
doco_outdirp : string option;
4445
}
4546

47+
and llm_option = {
48+
llmo_input : string;
49+
llmo_provers : prv_options;
50+
llmo_lastgoals : bool;
51+
llmo_upto : (int * int option) option;
52+
}
53+
4654
and prv_options = {
4755
prvo_maxjobs : int option;
4856
prvo_timeout : int option;

0 commit comments

Comments
 (0)