Skip to content

feat: restrict edits to the proof body (lock challenge-file imports/opens)#19

Draft
KrystianNowakowski wants to merge 11 commits into
mainfrom
restrict-edits-to-proof-body
Draft

feat: restrict edits to the proof body (lock challenge-file imports/opens)#19
KrystianNowakowski wants to merge 11 commits into
mainfrom
restrict-edits-to-proof-body

Conversation

@KrystianNowakowski

@KrystianNowakowski KrystianNowakowski commented Jun 5, 2026

Copy link
Copy Markdown
Collaborator

What

Adds an opt-in restrict_to_proof_body flag on ProverConfig (default false). When enabled, the prover only edits the proof body of the target theorem — it never writes new import or file-level open statements into the file. This satisfies the AI4Math / Codabench TCS competition rules:

  1. Only fill in the sorry placeholders … do not modify any other part of those files.
  2. Do not change the import statements in the challenge files.

Default-off, so ax-prover's general behavior (where adding imports is legitimate) is unchanged.

How

  • Config: restrict_to_proof_body: bool = False on ProverConfig.
  • Enforcement: TemporaryProposal skips edit_imports/edit_opens when the flag is set — the proof body (edit_function) is still applied. Wired at both the builder node and the reviewer's apply_permanently path, so the final committed file is clean too.
  • Feedback: when the proposer still emits imports/opens under a locked file, a warning is prepended to the build-failure error_output (no messages.py changes — keeps this independent of the lemma-stripping branch).
  • Prompt: a PROOF_BODY_RESTRICTION_PROMPT fragment is appended via a new, testable build_proposer_system_prompt() helper when the flag is on (covers both iterative and single-shot prompts).

Tests

253 unit tests pass; ruff clean. Includes a regression guard asserting _proposer_node actually calls the prompt builder (helper-only tests can pass while the wiring is broken).

Notes

  • Activation (restrict_to_proof_body: true) is applied to the experiment profile (opus48_local.yaml) on the experiment branch after merge — out of scope here.
  • Out of scope (per spec): auxiliary files (Rule 4), axiom checking (Rule 5).
  • Design spec: docs/superpowers/specs/2026-06-05-restrict-edits-to-proof-body-design.md

🤖 Generated with Claude Code


Note

Medium Risk
Changes how proposals are applied to disk on build and final approval when the flag is on; default-off limits blast radius, but mis-enabled config could hide missing imports until compile failures.

Overview
Adds opt-in restrict_to_proof_body on ProverConfig (default false) so challenge-style runs only change the target theorem’s proof body, not new file-level import / open lines.

When enabled, TemporaryProposal skips applying proposed imports/opens (proof code still applies) at both the builder temp-build path and reviewer apply_permanently. The proposer gets a locked-file prompt via build_proposer_system_prompt() / PROOF_BODY_RESTRICTION_PROMPT, and _dropped_preamble_warning is prepended to failed-build feedback and passed into SorriesGoalStateFeedback on successful builds that still have sorries. Unit tests cover config defaults, temp-file behavior, prompt assembly, and wiring guards in _proposer_node / _builder_node. Design/plan docs are added under docs/superpowers/.

Reviewed by Cursor Bugbot for commit 01cc7c9. Bugbot is set up for automated code reviews on this repo. Configure here.

KrystianNowakowski and others added 10 commits June 5, 2026 14:12
Opt-in config flag (restrict_to_proof_body, default false) that blocks
the agent from writing new imports or file-level opens into a challenge
file, satisfying AI4Math competition Rules 1 and 2. Drops violating
imports/opens mechanically and warns the proposer via build feedback.
Independent of PR #15 (no messages.py changes); based on main.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add restrict_to_proof_body parameter to TemporaryProposal.__init__ (default False).
When True, edit_imports and edit_opens are not called, leaving the file preamble
unchanged and only allowing proof-body edits.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… code-still-applied)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… line

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The polish commit accidentally reverted the _proposer_node refactor,
leaving build_proposer_system_prompt unused and the locked-file prompt
fragment never appended. Restore the helper call (passing the restrict
flag) and add a regression guard asserting the wiring exists — the
existing helper-only tests passed despite the broken integration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 3c916b2. Configure here.

self.base_folder,
state.item.location,
state.last_proposal,
restrict_to_proof_body=self.config.restrict_to_proof_body,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Timeout path omits preamble warning

Medium Severity

With restrict_to_proof_body enabled, dropped-import/open feedback is prepended only when Lean returns a normal compilation failure. A LeanBuildTimeout still emits plain BuildFailedFeedback text, so the proposer never sees the IMPORTS/OPENS IGNORED warning on that path.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 3c916b2. Configure here.

The dropped-preamble warning was only prepended to BuildFailedFeedback,
so a proposal whose imports/opens were dropped but still built taught the
proposer they worked. Compute the warning in the build-success branch,
log it when non-empty, and pass it through SorriesGoalStateFeedback (new
optional dropped_preamble_warning field) so it reaches the proposer on the
iterative sorries path. The terminal clean-success case is left untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@KrystianNowakowski
KrystianNowakowski marked this pull request as draft June 8, 2026 12:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant