feat: restrict edits to the proof body (lock challenge-file imports/opens)#19
Draft
KrystianNowakowski wants to merge 11 commits into
Draft
feat: restrict edits to the proof body (lock challenge-file imports/opens)#19KrystianNowakowski wants to merge 11 commits into
KrystianNowakowski wants to merge 11 commits into
Conversation
Opt-in config flag (restrict_to_proof_body, default false) that blocks the agent from writing new imports or file-level opens into a challenge file, satisfying AI4Math competition Rules 1 and 2. Drops violating imports/opens mechanically and warns the proposer via build feedback. Independent of PR #15 (no messages.py changes); based on main. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add restrict_to_proof_body parameter to TemporaryProposal.__init__ (default False). When True, edit_imports and edit_opens are not called, leaving the file preamble unchanged and only allowing proof-body edits. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… code-still-applied) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… line Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The polish commit accidentally reverted the _proposer_node refactor, leaving build_proposer_system_prompt unused and the locked-file prompt fragment never appended. Restore the helper call (passing the restrict flag) and add a regression guard asserting the wiring exists — the existing helper-only tests passed despite the broken integration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 3c916b2. Configure here.
| self.base_folder, | ||
| state.item.location, | ||
| state.last_proposal, | ||
| restrict_to_proof_body=self.config.restrict_to_proof_body, |
There was a problem hiding this comment.
Timeout path omits preamble warning
Medium Severity
With restrict_to_proof_body enabled, dropped-import/open feedback is prepended only when Lean returns a normal compilation failure. A LeanBuildTimeout still emits plain BuildFailedFeedback text, so the proposer never sees the IMPORTS/OPENS IGNORED warning on that path.
Reviewed by Cursor Bugbot for commit 3c916b2. Configure here.
The dropped-preamble warning was only prepended to BuildFailedFeedback, so a proposal whose imports/opens were dropped but still built taught the proposer they worked. Compute the warning in the build-success branch, log it when non-empty, and pass it through SorriesGoalStateFeedback (new optional dropped_preamble_warning field) so it reaches the proposer on the iterative sorries path. The terminal clean-success case is left untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
KrystianNowakowski
marked this pull request as draft
June 8, 2026 12:21
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


What
Adds an opt-in
restrict_to_proof_bodyflag onProverConfig(defaultfalse). When enabled, the prover only edits the proof body of the target theorem — it never writes newimportor file-levelopenstatements into the file. This satisfies the AI4Math / Codabench TCS competition rules:Default-off, so ax-prover's general behavior (where adding imports is legitimate) is unchanged.
How
restrict_to_proof_body: bool = FalseonProverConfig.TemporaryProposalskipsedit_imports/edit_openswhen the flag is set — the proof body (edit_function) is still applied. Wired at both the builder node and the reviewer'sapply_permanentlypath, so the final committed file is clean too.error_output(nomessages.pychanges — keeps this independent of the lemma-stripping branch).PROOF_BODY_RESTRICTION_PROMPTfragment is appended via a new, testablebuild_proposer_system_prompt()helper when the flag is on (covers both iterative and single-shot prompts).Tests
253 unit tests pass; ruff clean. Includes a regression guard asserting
_proposer_nodeactually calls the prompt builder (helper-only tests can pass while the wiring is broken).Notes
restrict_to_proof_body: true) is applied to the experiment profile (opus48_local.yaml) on the experiment branch after merge — out of scope here.docs/superpowers/specs/2026-06-05-restrict-edits-to-proof-body-design.md🤖 Generated with Claude Code
Note
Medium Risk
Changes how proposals are applied to disk on build and final approval when the flag is on; default-off limits blast radius, but mis-enabled config could hide missing imports until compile failures.
Overview
Adds opt-in
restrict_to_proof_bodyonProverConfig(default false) so challenge-style runs only change the target theorem’s proof body, not new file-levelimport/openlines.When enabled,
TemporaryProposalskips applying proposed imports/opens (proof code still applies) at both the builder temp-build path and reviewerapply_permanently. The proposer gets a locked-file prompt viabuild_proposer_system_prompt()/PROOF_BODY_RESTRICTION_PROMPT, and_dropped_preamble_warningis prepended to failed-build feedback and passed intoSorriesGoalStateFeedbackon successful builds that still have sorries. Unit tests cover config defaults, temp-file behavior, prompt assembly, and wiring guards in_proposer_node/_builder_node. Design/plan docs are added underdocs/superpowers/.Reviewed by Cursor Bugbot for commit 01cc7c9. Bugbot is set up for automated code reviews on this repo. Configure here.