Skip to content

Commit 1b4d17e

Browse files
committed
docs: add comprehensive guide for fixing plan step bugs
Added detailed FIX_PLAN_BUGS.md documenting: - Current implementation and flow - 5 specific bugs with acceptance criteria - Step-by-step implementation guide - Code structure and key methods - Testing strategy This document will help any agent understand and fix the bugs in the mandatory plan-before-execute feature. Related: #38, #37
1 parent 7fea63e commit 1b4d17e

1 file changed

Lines changed: 365 additions & 0 deletions

File tree

FIX_PLAN_BUGS.md

Lines changed: 365 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,365 @@
1+
# Fix Plan Step Bugs - Implementation Guide
2+
3+
## Overview
4+
5+
This branch fixes bugs in the mandatory plan-before-execute feature (merged in PR #37). The feature currently has several issues that prevent it from working correctly across all channels.
6+
7+
## Current Implementation (in src/agent/agent.rs)
8+
9+
The plan step is triggered in `turn()` method when:
10+
- First tool iteration (iteration == 0)
11+
- Has write operations detected
12+
13+
Current flow:
14+
1. User sends request: "build a coffee cat website"
15+
2. Agent detects write operations in tool calls
16+
3. Agent calls `generate_plan_with_llm()` to create plan
17+
4. Plan is printed via `println!()` and `tracing::info!()`
18+
5. Agent returns plan message immediately without executing tools
19+
6. On next user message, agent should check for approval
20+
21+
## Bugs to Fix
22+
23+
### Bug 1: Plan Not Displaying in Non-CLI Channels
24+
25+
**Problem:**
26+
When using Signal, WhatsApp, or other non-CLI channels, the plan is generated but not visible to the user. The agent shows "Processing..." then immediately executes tools without showing the plan.
27+
28+
**Current Code:**
29+
```rust
30+
println!("\n📋 Plan:\n{}", plan);
31+
println!("\nReply 'yes' to proceed or tell me what to change.");
32+
```
33+
34+
**Why it fails:**
35+
- `println!()` only outputs to stdout, not to Signal/WhatsApp channels
36+
- The plan is not returned as part of the response message
37+
38+
**Expected Fix:**
39+
- Plan must be returned as assistant message content
40+
- User should see the plan in their Signal/WhatsApp chat
41+
- Format: Clear, emoji-enhanced plan with approval prompt
42+
43+
**Acceptance Criteria:**
44+
- [ ] Plan visible in Signal channel
45+
- [ ] Plan visible in WhatsApp channel
46+
- [ ] Plan visible in Telegram channel
47+
- [ ] Plan visible in CLI mode
48+
49+
### Bug 2: User Confirmation Flow Not Implemented
50+
51+
**Problem:**
52+
After showing the plan, the agent doesn't properly wait for or process user confirmation. Currently it returns the plan but doesn't track approval state.
53+
54+
**Current Code:**
55+
```rust
56+
return Ok(format!("📋 Plan:\n{}\n\nReply 'yes' to proceed or tell me what to change.", plan));
57+
```
58+
59+
**Why it fails:**
60+
- No state tracking for pending plan approval
61+
- On next user message, agent doesn't check if previous was a plan awaiting approval
62+
- No logic to handle "yes", "ok", "build it", "stop", "cancel"
63+
64+
**Expected Fix:**
65+
Need a state machine approach:
66+
67+
1. **Add pending plan state to Agent struct:**
68+
```rust
69+
struct Agent {
70+
// ... existing fields
71+
pending_plan: Option<PendingPlan>, // New field
72+
}
73+
74+
struct PendingPlan {
75+
original_request: String,
76+
plan_text: String,
77+
timestamp: Instant,
78+
}
79+
```
80+
81+
2. **Modify turn() to check for pending plan:**
82+
```rust
83+
pub async fn turn(&mut self, user_message: &str) -> Result<String> {
84+
// Check if we have a pending plan awaiting approval
85+
if let Some(pending) = self.pending_plan.take() {
86+
match user_message.to_lowercase().as_str() {
87+
"yes" | "y" | "ok" | "sure" | "build it" | "go" => {
88+
// User approved - continue with execution
89+
// Store approval in history
90+
self.history.push(ConversationMessage::Chat(
91+
ChatMessage::assistant("Plan approved. Starting execution...".to_string())
92+
));
93+
// Continue with tool execution...
94+
}
95+
"no" | "n" | "cancel" | "stop" | "abort" => {
96+
// User rejected
97+
self.history.push(ConversationMessage::Chat(
98+
ChatMessage::assistant("Plan cancelled. No changes made.".to_string())
99+
));
100+
return Ok("Plan cancelled. You can request something else.".to_string());
101+
}
102+
_ => {
103+
// User wants to modify plan - treat as new request
104+
// Could regenerate plan with modifications
105+
let modified_prompt = format!("{original_request}\n\nUser wants to modify: {user_message}",
106+
original_request = pending.original_request
107+
);
108+
// Generate new plan...
109+
}
110+
}
111+
}
112+
113+
// Normal flow...
114+
}
115+
```
116+
117+
3. **Handle first-time plan generation:**
118+
```rust
119+
// On first write iteration
120+
if iteration == 0 && self.has_write_operations(&calls) && self.pending_plan.is_none() {
121+
let plan = self.generate_plan_with_llm(user_message).await?;
122+
self.pending_plan = Some(PendingPlan {
123+
original_request: user_message.to_string(),
124+
plan_text: plan.clone(),
125+
timestamp: Instant::now(),
126+
});
127+
return Ok(format_plan_message(&plan));
128+
}
129+
```
130+
131+
**Acceptance Criteria:**
132+
- [ ] User can approve with: "yes", "y", "ok", "sure", "build it", "go"
133+
- [ ] User can reject with: "no", "n", "cancel", "stop", "abort"
134+
- [ ] User can request modifications (triggers plan regeneration)
135+
- [ ] After approval, agent continues with tool execution
136+
- [ ] After rejection, agent clears state and waits for new request
137+
138+
### Bug 3: Plan Generation Error Handling
139+
140+
**Problem:**
141+
If `generate_plan_with_llm()` fails, the agent doesn't have a fallback. It should gracefully handle errors.
142+
143+
**Current Code:**
144+
```rust
145+
let plan = self.generate_plan_with_llm(user_message).await?;
146+
```
147+
148+
**Why it fails:**
149+
- If LLM call fails, the ? operator returns early with error
150+
- User sees error instead of fallback behavior
151+
152+
**Expected Fix:**
153+
```rust
154+
let plan = match self.generate_plan_with_llm(user_message).await {
155+
Ok(plan) => plan,
156+
Err(e) => {
157+
tracing::warn!("Failed to generate plan: {}, using fallback", e);
158+
// Fallback: Use tool calls to describe what we'll do
159+
self.generate_simple_plan_from_tools(&calls)
160+
}
161+
};
162+
```
163+
164+
Add `generate_simple_plan_from_tools()` method that creates a basic plan from the tool calls without calling LLM.
165+
166+
**Acceptance Criteria:**
167+
- [ ] If LLM fails, fallback to tool-based plan
168+
- [ ] User always sees a plan before execution
169+
- [ ] Error is logged but not shown to user
170+
171+
### Bug 4: Plan Not Added to Conversation History
172+
173+
**Problem:**
174+
The plan is printed but not stored in the conversation history properly.
175+
176+
**Current Code:**
177+
The plan is returned as string but not added to `self.history` before returning.
178+
179+
**Expected Fix:**
180+
```rust
181+
// Add plan to history before returning
182+
let plan_message = format!("📋 Plan:\n{}\n\nReply 'yes' to proceed or tell me what to change.", plan);
183+
self.history.push(ConversationMessage::Chat(
184+
ChatMessage::assistant(plan_message.clone())
185+
));
186+
self.pending_plan = Some(PendingPlan { /* ... */ });
187+
return Ok(plan_message);
188+
```
189+
190+
**Acceptance Criteria:**
191+
- [ ] Plan appears in conversation history
192+
- [ ] Plan visible when viewing conversation log
193+
- [ ] Plan included in context for subsequent messages
194+
195+
### Bug 5: Test Failures
196+
197+
**Problem:**
198+
Some agent tests may fail because the plan step changes the expected flow.
199+
200+
**Expected Fix:**
201+
Update tests in `src/agent/tests.rs`:
202+
203+
1. **Tests with write operations** should expect plan step:
204+
```rust
205+
#[tokio::test]
206+
async fn test_write_operation_shows_plan() {
207+
// Setup agent with write tools
208+
// Send request that triggers write
209+
// Expect plan message returned (not tools executed yet)
210+
// Send "yes"
211+
// Expect tools to execute
212+
}
213+
```
214+
215+
2. **Tests with read-only operations** should skip plan:
216+
```rust
217+
#[tokio::test]
218+
async fn test_read_only_skips_plan() {
219+
// Send request with only file_read
220+
// Expect immediate execution (no plan)
221+
}
222+
```
223+
224+
3. **Test user approval flow:**
225+
```rust
226+
#[tokio::test]
227+
async fn test_user_approval_yes() {
228+
// Send build request
229+
// Get plan
230+
// Send "yes"
231+
// Verify execution continues
232+
}
233+
234+
#[tokio::test]
235+
async fn test_user_rejection() {
236+
// Send build request
237+
// Get plan
238+
// Send "cancel"
239+
// Verify no execution, state cleared
240+
}
241+
```
242+
243+
**Acceptance Criteria:**
244+
- [ ] All existing tests pass
245+
- [ ] New tests for plan approval flow
246+
- [ ] New tests for rejection flow
247+
- [ ] New tests for modification flow
248+
249+
## Implementation Order
250+
251+
1. **Phase 1: Fix plan display** (Bug 1)
252+
- Modify return value to include plan as assistant message
253+
- Ensure plan visible in all channels
254+
255+
2. **Phase 2: Add pending plan state** (Bug 2, 4)
256+
- Add `pending_plan` field to Agent struct
257+
- Modify turn() to check for pending approval
258+
- Handle yes/no/modify responses
259+
260+
3. **Phase 3: Error handling** (Bug 3)
261+
- Add fallback plan generation
262+
- Add proper error logging
263+
264+
4. **Phase 4: Tests** (Bug 5)
265+
- Update existing tests
266+
- Add new tests for plan flow
267+
268+
## Testing Strategy
269+
270+
### Manual Testing Checklist:
271+
272+
- [ ] **CLI Mode:**
273+
- Request: "build a landing page"
274+
- Verify: Plan displayed with emoji
275+
- Reply: "yes"
276+
- Verify: Execution starts
277+
278+
- [ ] **Signal Channel:**
279+
- Request: "build a landing page"
280+
- Verify: Plan message received
281+
- Reply: "yes"
282+
- Verify: Execution starts
283+
284+
- [ ] **Rejection Flow:**
285+
- Request: "build a landing page"
286+
- Reply: "cancel"
287+
- Verify: "Plan cancelled" message
288+
- Verify: No sandbox created
289+
290+
- [ ] **Read-Only Flow:**
291+
- Request: "explain this code"
292+
- Verify: No plan, immediate response
293+
294+
### Automated Tests:
295+
296+
```bash
297+
# Run agent tests
298+
cargo test --lib agent::tests
299+
300+
# Run with output visible
301+
cargo test --lib agent::tests -- --nocapture
302+
303+
# Specific test
304+
cargo test --lib agent::tests::test_plan_approval
305+
```
306+
307+
## Code Structure
308+
309+
### Files to Modify:
310+
311+
1. **src/agent/agent.rs**
312+
- Add `pending_plan` field to Agent struct
313+
- Add `PendingPlan` struct
314+
- Modify `turn()` method
315+
- Add `generate_simple_plan_from_tools()` method
316+
- Add helper `format_plan_message()`
317+
318+
2. **src/agent/mod.rs**
319+
- Export new types if needed
320+
321+
3. **tests/** (if new tests needed)
322+
- Add plan flow tests
323+
324+
### Key Methods:
325+
326+
```rust
327+
impl Agent {
328+
/// Generate plan using LLM
329+
async fn generate_plan_with_llm(&mut self, user_message: &str) -> Result<String>
330+
331+
/// Generate simple plan from tool calls (fallback)
332+
fn generate_simple_plan_from_tools(&self, calls: &[ParsedToolCall]) -> String
333+
334+
/// Check if pending plan needs approval
335+
fn has_pending_plan(&self) -> bool
336+
337+
/// Handle user response to plan
338+
async fn handle_plan_response(&mut self, user_response: &str) -> Result<PlanAction>
339+
340+
/// Format plan for display
341+
fn format_plan_message(plan: &str) -> String
342+
}
343+
344+
enum PlanAction {
345+
Approve, // User said yes
346+
Reject, // User said no/cancel
347+
Modify(String), // User wants changes
348+
}
349+
```
350+
351+
## References
352+
353+
- Original Issue: #38
354+
- Original PR: #37
355+
- Related: AGENTS.md section 5.2
356+
357+
## Notes for Agent
358+
359+
When fixing these bugs:
360+
1. Keep changes minimal and focused
361+
2. Don't break existing functionality
362+
3. Add comprehensive tests
363+
4. Update AGENTS.md if behavior changes
364+
5. Run full test suite before committing
365+
6. Test manually with both CLI and Signal channels

0 commit comments

Comments
 (0)