Skip to content

Commit cf30a76

Browse files
committed
Add LLM REPL improvements and built-in guide
New REPL features: - LOAD -nosmt: skip SMT calls during prefix compilation (uses Proofs:weak pragma), restores strict mode for interactive tactics - Error responses now include the current goal state, eliminating the need for a separate GOALS roundtrip after failures - Multi-line input support via <BEGIN>/<DONE> delimiters - HELP meta-command and -help flag: print the LLM agent guide (doc/llm/CLAUDE.md) from within the REPL or at startup The guide is installed as part of the distribution (doc site) so it stays version-matched with the binary. Agent frameworks can use `easycrypt llm -help` to populate their system prompt automatically. Documentation updates: - REVERT-to-LOAD-uuid workflow (instant restart vs recompile) - Pattern-based search syntax with examples - SMT usage guidance (pure arithmetic/logic only) - rewrite-in-H clarification
1 parent dd93ca9 commit cf30a76

5 files changed

Lines changed: 187 additions & 29 deletions

File tree

doc/llm/CLAUDE.md

Lines changed: 71 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,11 @@ easycrypt llm [OPTIONS]
1717
```
1818

1919
Standard loader and prover options (`-I`, `-timeout`, `-p`, etc.) are
20-
available.
20+
available. Use `-help` to print this guide and exit:
21+
22+
```
23+
easycrypt llm -help
24+
```
2125

2226
### Protocol
2327

@@ -52,14 +56,16 @@ These are protocol-level commands, not EasyCrypt syntax:
5256

5357
| Command | Description |
5458
|---------|-------------|
55-
| `LOAD "file.ec" [LINE[:COL]]` | Reset state, compile file up to the given line |
59+
| `LOAD "file.ec" [LINE[:COL]] [-nosmt]` | Reset state, compile file (optionally skip SMT) |
5660
| `UNDO` | Undo the last proof step |
5761
| `REVERT <uuid-or-name>` | Revert to a specific state (by uuid or checkpoint name) |
5862
| `GOALS` | Print the current goal (first subgoal only, with remaining count) |
5963
| `GOALS ALL` | Print all subgoals |
6064
| `CHECKPOINT <name>` | Save current uuid under a name for later `REVERT` |
6165
| `SEARCH <pattern>` | Search for lemmas matching a pattern |
6266
| `QUIET ON` / `QUIET OFF` | Suppress/enable automatic goal display after tactics |
67+
| `<BEGIN>` / `<DONE>` | Delimit multi-line EasyCrypt input |
68+
| `HELP` | Print this guide |
6369
| `QUIT` | Exit |
6470

6571
### EasyCrypt commands
@@ -75,6 +81,16 @@ search (%/).
7581
print mulzK.
7682
```
7783

84+
For multi-line statements, wrap with `<BEGIN>` and `<DONE>`:
85+
86+
```
87+
<BEGIN>
88+
lemma test :
89+
0 <= n =>
90+
0 < n + 1.
91+
<DONE>
92+
```
93+
7894
### Workflow
7995

8096
**1. Load a file up to the proof point:**
@@ -94,19 +110,37 @@ Current goal
94110
<END>
95111
```
96112

97-
**2. Try tactics interactively:**
113+
For large files, use `-nosmt` to skip SMT calls during prefix
114+
compilation (safe when the prefix was already verified):
98115

99116
```
100-
smt().
117+
LOAD "myfile.ec" 436 -nosmt
101118
```
102119

103-
If it fails, the state is unchanged — try another tactic immediately:
120+
**2. Try tactics, using REVERT to restart:**
121+
122+
The uuid returned by LOAD is a revertible state. Use `REVERT` to
123+
return to it after failed experiments — this is instant, unlike
124+
re-doing LOAD which recompiles the prefix.
104125

105126
```
106-
rewrite H1.
107-
smt(lemma1 lemma2).
127+
LOAD "myfile.ec" 42
128+
→ OK [uuid:15] [loaded:myfile.ec:42]
129+
130+
smt(). ← fails, state unchanged
131+
rewrite H1. smt(). ← succeeds (uuid:17)
132+
rewrite H2. ← wrong direction
133+
REVERT 17 ← back to after the successful smt()
108134
```
109135

136+
To restart the proof from scratch, revert to the LOAD uuid:
137+
138+
```
139+
REVERT 15 ← back to the state right after LOAD
140+
```
141+
142+
Always note the LOAD uuid so you can return to it.
143+
110144
**3. Use checkpoints for branching exploration:**
111145

112146
```
@@ -128,11 +162,23 @@ QUIET OFF
128162
GOALS
129163
```
130164

131-
**5. Search for lemmas:**
165+
**5. Search for lemmas using patterns:**
166+
167+
EasyCrypt `search` uses pattern syntax, not keywords. Use `_` as
168+
wildcard:
169+
170+
```
171+
search (fdom _). ← lemmas involving fdom
172+
search (_ %/ _). ← integer division lemmas
173+
search (card (_ `|` _)). ← card of union
174+
search (mu _ _) (_ <= _). ← mu lemmas with inequalities
175+
```
176+
177+
The SEARCH meta-command is a shorthand that adds `search`/`.`:
132178

133179
```
134-
SEARCH mulzK
135-
SEARCH dvdz
180+
SEARCH (fdom _)
181+
SEARCH (_ %/ _)
136182
```
137183

138184
## EasyCrypt proof strategy
@@ -176,6 +222,18 @@ SEARCH dvdz
176222
- For induction on naturals: `elim/natind: n` gives base (`n ≤ 0`)
177223
and step (`0 ≤ n → P n → P (n+1)`).
178224

225+
### SMT usage
226+
227+
`smt()` and `/#` are equivalent — both call external SMT solvers.
228+
229+
- Use `smt()` **only** on goals that are pure arithmetic or pure
230+
propositional logic. If the goal contains abstract operators,
231+
FMap terms, or `if-then-else`, reduce it first with `rewrite`,
232+
`case`, or `have` before calling `smt()`.
233+
- If `smt()` takes more than 1 second, the goal is too complex.
234+
Simplify with interactive tactics instead of increasing the
235+
timeout.
236+
179237
### Common pitfalls
180238

181239
- `rewrite (factS n) //` generates a side goal `0 <= n`. Use
@@ -185,6 +243,9 @@ SEARCH dvdz
185243
one.
186244
- When a tactic generates multiple subgoals, each subgoal must be
187245
closed in order. Use `GOALS ALL` to see them all.
246+
- `rewrite lemma in H` modifies hypothesis `H` in place (it does
247+
not consume it). If you need to preserve the original, copy it
248+
first: `have H' := H; rewrite lemma in H'`.
188249

189250
## EasyCrypt language overview
190251

dune

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,14 @@
1-
(dirs 3rdparty src etc theories examples assets scripts)
1+
(dirs 3rdparty src etc theories examples assets scripts doc)
22

33
(install
44
(section (site (easycrypt commands)))
55
(files (scripts/testing/runtest as runtest)))
66

77
(install
88
(section (site (easycrypt doc)))
9-
(files (assets/styles/styles.css as styles.css)))
9+
(files
10+
(assets/styles/styles.css as styles.css)
11+
(doc/llm/CLAUDE.md as llm-guide.md)))
1012

1113
(install
1214
(section (bin))

src/ec.ml

Lines changed: 106 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -408,7 +408,34 @@ let main () =
408408
(* LLM interactive mode *)
409409
(* -------------------------------------------------------------------- *)
410410

411-
let run_llm_repl (prvopts : prv_options) =
411+
let llm_guide_path () =
412+
let (module Sites) = EcRelocate.sites in
413+
match EcRelocate.sourceroot with
414+
| Some root ->
415+
Filename.concat (Filename.concat root "doc/llm") "CLAUDE.md"
416+
| None ->
417+
Filename.concat Sites.doc "llm-guide.md"
418+
in
419+
420+
let print_llm_guide () =
421+
let path = llm_guide_path () in
422+
try
423+
let ic = open_in path in
424+
begin try while true do
425+
print_char (input_char ic)
426+
done with End_of_file -> () end;
427+
close_in ic
428+
with Sys_error e ->
429+
Printf.eprintf "cannot read LLM guide: %s\n%!" e
430+
in
431+
432+
let run_llm_repl (llmopts : llm_option) =
433+
if llmopts.llmo_help then begin
434+
print_llm_guide ();
435+
exit 0
436+
end;
437+
438+
let prvopts = llmopts.llmo_provers in
412439
(* Initialize PRNG *)
413440
Random.self_init ();
414441

@@ -510,8 +537,15 @@ let main () =
510537
in
511538

512539
let reply_error msg =
513-
Printf.printf "ERROR [uuid:%d]\n%s\n<END>\n%!"
514-
(EcCommands.uuid ()) msg;
540+
let goals = goals_to_string () in
541+
Printf.printf "ERROR [uuid:%d]\n%s\n" (EcCommands.uuid ()) msg;
542+
if goals <> "" then begin
543+
print_string goals;
544+
let len = String.length goals in
545+
if len > 0 && goals.[len - 1] <> '\n' then
546+
print_char '\n'
547+
end;
548+
Printf.printf "<END>\n%!";
515549
Buffer.clear notices
516550
in
517551

@@ -592,16 +626,27 @@ let main () =
592626
| f :: rest -> (f, String.concat " " rest)
593627
in
594628

595-
(* Parse optional LINE[:COL] *)
596-
let upto =
597-
if rest = "" then None
629+
(* Parse optional LINE[:COL] and flags (-nosmt) *)
630+
let upto, nosmt =
631+
if rest = "" then (None, false)
598632
else
599-
match String.split_on_char ':' rest with
600-
| [line] ->
601-
Some (int_of_string line, None)
602-
| [line; col] ->
603-
Some (int_of_string line, Some (int_of_string col))
604-
| _ -> failwith "LOAD: invalid LINE[:COL] format"
633+
let words = String.split_on_char ' ' rest in
634+
let words = List.filter (fun s -> s <> "") words in
635+
let nosmt = List.mem "-nosmt" words in
636+
let words = List.filter (fun s -> s <> "-nosmt") words in
637+
let upto = match words with
638+
| [] -> None
639+
| [w] ->
640+
begin match String.split_on_char ':' w with
641+
| [line] ->
642+
Some (int_of_string line, None)
643+
| [line; col] ->
644+
Some (int_of_string line, Some (int_of_string col))
645+
| _ -> failwith "LOAD: invalid LINE[:COL] format"
646+
end
647+
| _ -> failwith "LOAD: unexpected arguments"
648+
in
649+
(upto, nosmt)
605650
in
606651

607652
(* Validate file extension *)
@@ -632,6 +677,9 @@ let main () =
632677

633678
let last_loc = ref None in
634679

680+
(* In -nosmt mode, admit all SMT calls during prefix loading *)
681+
if nosmt then EcCommands.pragma_check `WeakCheck;
682+
635683
begin try while true do
636684
let (src, prog) = EcIo.xparse reader in
637685
let src = String.strip src in
@@ -653,10 +701,17 @@ let main () =
653701
EcCommands.doc_comment doc
654702
done with
655703
| Exit | End_of_file -> ()
656-
| e -> EcIo.finalize reader; raise e
704+
| e ->
705+
EcIo.finalize reader;
706+
if nosmt then EcCommands.pragma_check `Check;
707+
raise e
657708
end;
658709

659710
EcIo.finalize reader;
711+
712+
(* Restore full SMT checking for interactive tactics *)
713+
if nosmt then EcCommands.pragma_check `Check;
714+
660715
let tag =
661716
match !last_loc with
662717
| None -> ""
@@ -685,14 +740,49 @@ let main () =
685740
(EcCommands.uuid ());
686741

687742
(* Main REPL loop *)
743+
let multi_buf = Buffer.create 256 in
744+
let in_multi = ref false in
745+
688746
begin try while true do
689747
let line = input_line stdin in
690748
let line = String.strip line in
691749

692-
if line = "" then
750+
(* Multi-line input: <BEGIN> starts, <DONE> flushes *)
751+
if line = "<BEGIN>" then begin
752+
Buffer.clear multi_buf;
753+
in_multi := true
754+
end
755+
else if line = "<DONE>" && !in_multi then begin
756+
let input = Buffer.contents multi_buf in
757+
Buffer.clear multi_buf;
758+
in_multi := false;
759+
if input <> "" then process_ec_input input
760+
end
761+
else if !in_multi then begin
762+
if Buffer.length multi_buf > 0 then
763+
Buffer.add_char multi_buf ' ';
764+
Buffer.add_string multi_buf line
765+
end
766+
767+
else if line = "" then
693768
()
694769
else if line = "QUIT" then
695770
exit 0
771+
else if line = "HELP" then begin
772+
Buffer.clear notices;
773+
let buf = Buffer.create 4096 in
774+
let path = llm_guide_path () in
775+
begin try
776+
let ic = open_in path in
777+
begin try while true do
778+
Buffer.add_char buf (input_char ic)
779+
done with End_of_file -> () end;
780+
close_in ic;
781+
reply_ok (Buffer.contents buf)
782+
with Sys_error e ->
783+
reply_error (Printf.sprintf "cannot read guide: %s" e)
784+
end
785+
end
696786
else if line = "UNDO" then begin
697787
Buffer.clear notices;
698788
let uuid = EcCommands.uuid () in
@@ -910,7 +1000,8 @@ let main () =
9101000

9111001
| `Llm llmopts ->
9121002
run_llm_repl
913-
{llmopts.llmo_provers with prvo_iterate = true}
1003+
{llmopts with llmo_provers =
1004+
{llmopts.llmo_provers with prvo_iterate = true}}
9141005

9151006
| `Runtest _ ->
9161007
(* Eagerly executed *)

src/ecOptions.ml

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,7 @@ and doc_option = {
5050

5151
and llm_option = {
5252
llmo_provers : prv_options;
53+
llmo_help : bool;
5354
}
5455

5556
and prv_options = {
@@ -358,7 +359,8 @@ let specs = {
358359

359360
("llm", "LLM-friendly interactive mode", [
360361
`Group "loader";
361-
`Group "provers"]);
362+
`Group "provers";
363+
`Spec ("help", `Flag, "Print the LLM agent guide and exit")]);
362364

363365
("cli", "Run EasyCrypt top-level", [
364366
`Group "loader";
@@ -543,7 +545,8 @@ let doc_options_of_values values input =
543545
doco_outdirp = get_string "outdir" values; }
544546

545547
let llm_options_of_values ini values =
546-
{ llmo_provers = prv_options_of_values ini values; }
548+
{ llmo_provers = prv_options_of_values ini values;
549+
llmo_help = get_flag "help" values; }
547550

548551
(* -------------------------------------------------------------------- *)
549552
let parse getini argv =

src/ecOptions.mli

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,7 @@ and doc_option = {
4646

4747
and llm_option = {
4848
llmo_provers : prv_options;
49+
llmo_help : bool;
4950
}
5051

5152
and prv_options = {

0 commit comments

Comments
 (0)