Skip to content

Commit 1ca4a15

Browse files
CakeRepositoryCakeRepository
committed
Add summary of system prompt improvements
1 parent eea9965 commit 1ca4a15

1 file changed

Lines changed: 178 additions & 0 deletions

File tree

‎SYSTEM_PROMPT_IMPROVEMENTS.md‎

Lines changed: 178 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,178 @@
1+
# System Prompt Improvements Summary
2+
3+
## 🎯 Problem Solved
4+
5+
The agents were having difficulty knowing how to use the tools to control the computer effectively. The previous system prompts lacked:
6+
- Specific tool call formats and examples
7+
- Common mistake warnings
8+
- Step-by-step workflows
9+
- Emphasis on mandatory practices (like using window handles)
10+
11+
## ✅ Changes Made
12+
13+
### 1. Added "Reset to Default" Buttons
14+
15+
Added reset buttons to the UI for all three agent prompts:
16+
- **Actioner Tab**: "🔄 Reset to Default" button
17+
- **Planner Tab**: "🔄 Reset to Default" button
18+
- **Coordinator Tab**: "🔄 Reset to Default" button
19+
20+
Each button:
21+
- Shows confirmation dialog before resetting
22+
- Reminds user to click "Save" after reset
23+
- Loads default prompt from static methods
24+
25+
### 2. Completely Rewrote Actioner System Prompt
26+
27+
**New Features:**
28+
- 📋 **YOUR MISSION section** - Makes it clear the agent DOES things, not just advises
29+
- 🔧 **AVAILABLE TOOLS** - Complete tool reference with exact syntax
30+
- 📋 **MANDATORY WORKFLOW** - Step-by-step process: Observe → Plan → Execute → Verify
31+
- ✅ **EXAMPLES OF CORRECT USAGE** - 3 detailed examples:
32+
- Opening Notepad and typing
33+
- Clicking a button using bbox
34+
- Browser search automation
35+
- ❌ **COMMON MISTAKES** - Shows wrong vs right approaches
36+
- 🎓 **TOOL CALL FORMAT** - Exact format requirements
37+
- 💡 **PRO TIPS** - Best practices checklist
38+
39+
**Key Improvements:**
40+
- Emphasizes observation FIRST (CaptureWholeScreen before any action)
41+
- Makes window handles MANDATORY for all keyboard/mouse operations
42+
- Provides exact tool call syntax with examples
43+
- Uses emojis for visual organization
44+
- Shows step-by-step workflows for common tasks
45+
- Warns about common pitfalls
46+
47+
**Length:** ~300 lines (vs ~80 lines previously)
48+
49+
### 3. Improved Planner System Prompt
50+
51+
**New Features:**
52+
- Clear role explanation: "Output ONE step at a time"
53+
- Iterative approach emphasized
54+
- Good vs Bad step examples
55+
- Common patterns library for:
56+
- Opening applications
57+
- Browser navigation
58+
- UI interaction
59+
- Typing text
60+
- File operations
61+
- Critical reminders about window handles
62+
- Example task breakdown showing one-step-at-a-time approach
63+
64+
**Key Improvements:**
65+
- Emphasizes outputting single steps and waiting for results
66+
- Provides template patterns for common tasks
67+
- Shows complete example of breaking down a task
68+
- Makes it clear to adapt based on actual results
69+
70+
**Length:** ~180 lines (vs ~70 lines previously)
71+
72+
### 4. Enhanced Coordinator System Prompt
73+
74+
**New Features:**
75+
- Decision tree diagram
76+
- Clearer task routing guidelines
77+
- Communication style examples (good vs bad)
78+
- Tone guidance for user-friendly responses
79+
80+
**Key Improvements:**
81+
- Clearer distinction between simple/complex tasks
82+
- Better examples of routing decisions
83+
- Emphasis on user-friendly language vs technical jargon
84+
85+
**Length:** ~140 lines (vs ~60 lines previously)
86+
87+
### 5. Added Static Methods in ToolConfig
88+
89+
New methods to retrieve default prompts:
90+
```csharp
91+
ToolConfig.GetDefaultActionerPrompt()
92+
ToolConfig.GetDefaultPlannerPrompt()
93+
ToolConfig.GetDefaultCoordinatorPrompt()
94+
```
95+
96+
These are used both for:
97+
- Initial configuration (when creating new config)
98+
- Reset buttons (to restore defaults)
99+
100+
## 📁 Files Modified
101+
102+
1. **`FlowVision/lib/Classes/ToolConfig.cs`**
103+
- Changed property initializers to use static methods
104+
- Added 3 new static methods with comprehensive default prompts
105+
- ~400 lines added
106+
107+
2. **`FlowVision/ToolConfigForm.cs`**
108+
- Added 3 reset button event handlers
109+
- Each shows confirmation and success messages
110+
- ~60 lines added
111+
112+
3. **`FlowVision/ToolConfigForm.Designer.cs`**
113+
- Added 3 button declarations
114+
- Added buttons to group box controls
115+
- Added button property definitions
116+
- ~50 lines added
117+
118+
## 🎓 Key Improvements for Tool Usage
119+
120+
### Before:
121+
❌ Agents didn't know exact tool syntax
122+
❌ Missing examples of correct usage
123+
❌ No emphasis on observation-first approach
124+
❌ Unclear about window handle requirements
125+
❌ No common mistake warnings
126+
127+
### After:
128+
✅ Exact tool call format with examples
129+
✅ Step-by-step workflows for common tasks
130+
✅ MANDATORY observation before action
131+
✅ Clear emphasis on window handles
132+
✅ Common mistakes section with wrong vs right
133+
✅ Pro tips and best practices
134+
✅ Visual organization with emojis
135+
✅ Iterative approach explained clearly
136+
137+
## 💡 Expected Impact
138+
139+
Agents should now:
140+
1. **Always start with observation** (CaptureWholeScreen first)
141+
2. **Use correct tool syntax** (with all required parameters)
142+
3. **Include window handles** for all keyboard/mouse operations
143+
4. **Follow proper workflows** (observe → plan → execute → verify)
144+
5. **Avoid common mistakes** (outlined in prompts)
145+
6. **Work iteratively** (one step at a time, adapt based on results)
146+
147+
## 🔗 Pull Request
148+
149+
Branch: `improve-system-prompts`
150+
Create PR: https://github.com/flowdevs-io/Recursive-Control/pull/new/improve-system-prompts
151+
152+
## 🧪 Testing Recommendations
153+
154+
After merging:
155+
1. Reset all agent prompts to defaults using the new buttons
156+
2. Test common tasks:
157+
- Opening applications
158+
- Clicking UI elements
159+
- Typing text in windows
160+
- Browser automation
161+
3. Verify agents now:
162+
- Call CaptureWholeScreen first
163+
- Use correct tool syntax
164+
- Include window handles
165+
- Work step-by-step
166+
167+
## 📝 Notes
168+
169+
- All changes are backward compatible
170+
- Existing custom prompts are not affected until user clicks Reset
171+
- Default prompts are significantly more detailed and instructive
172+
- Build verified successful with MSBuild
173+
- No breaking changes to API or configuration structure
174+
175+
---
176+
177+
**Status**: ✅ Ready for review and merge
178+
**Branch**: improve-system-prompts

0 commit comments

Comments
 (0)