Skip to content

Commit 2c446ac

Browse files
Add droid-evolved plugin
Skills for continuous learning and improvement: - session-navigation: Search and navigate past Droid sessions - human-writing: Remove AI writing patterns, make text sound human - skill-creation: Create and improve Droid skills - visual-design: Image generation (nanobanana CLI), presentations (Slidev), frontend design - browser-navigation: Browser automation with agent-browser Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
1 parent 0b89da4 commit 2c446ac

11 files changed

Lines changed: 2347 additions & 0 deletions

File tree

README.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,18 @@ Security review, threat modeling, and vulnerability validation skills.
3535
- `commit-security-scan` - Scan commits/PRs for security vulnerabilities
3636
- `vulnerability-validation` - Validate and confirm security findings
3737

38+
### droid-evolved
39+
40+
Skills for continuous learning and improvement.
41+
42+
**Skills:**
43+
44+
- `session-navigation` - Search and navigate past Droid sessions
45+
- `human-writing` - Remove AI writing patterns, make text sound human
46+
- `skill-creation` - Create and improve Droid skills
47+
- `visual-design` - Image generation (nanobanana CLI), presentations (Slidev), frontend design
48+
- `browser-navigation` - Browser automation with agent-browser
49+
3850
## Plugin Structure
3951

4052
Each plugin follows the Factory plugin format:
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"name": "droid-evolved",
3+
"description": "Skills for continuous learning and improvement: session navigation, human writing, skill creation, visual design, and browser automation",
4+
"author": {
5+
"name": "Factory",
6+
"email": "support@factory.ai"
7+
}
8+
}
Lines changed: 361 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,361 @@
1+
---
2+
name: browser-navigation
3+
version: 1.0.0
4+
description: |
5+
Automate browser interactions for web testing, form filling, screenshots, and data extraction.
6+
Use when the user needs to:
7+
- Navigate websites and interact with web pages
8+
- Fill forms and click buttons
9+
- Take screenshots of web content
10+
- Test web applications
11+
- Extract information from web pages
12+
- Debug frontend issues in the browser
13+
- Monitor console logs and network requests
14+
- Record browser sessions as video
15+
This skill uses agent-browser for comprehensive browser automation.
16+
RECOMMENDATION: Disable Chrome DevTools or Playwright MCP when using this skill to save context.
17+
---
18+
19+
# Browser Automation with agent-browser
20+
21+
Comprehensive browser automation for testing, data extraction, and web interaction.
22+
23+
## Quick Start
24+
25+
```bash
26+
agent-browser open <url> # Navigate to page
27+
agent-browser snapshot -i # Get interactive elements with refs
28+
agent-browser click @e1 # Click element by ref
29+
agent-browser fill @e2 "text" # Fill input by ref
30+
agent-browser close # Close browser
31+
```
32+
33+
## Core Workflow
34+
35+
1. **Navigate**: `agent-browser open <url>`
36+
2. **Snapshot**: `agent-browser snapshot -i` (returns elements with refs like `@e1`, `@e2`)
37+
3. **Interact** using refs from the snapshot
38+
4. **Re-snapshot** after navigation or significant DOM changes
39+
40+
## Installation
41+
42+
```bash
43+
# Install globally
44+
npm install -g agent-browser
45+
46+
# Or use via npx
47+
npx agent-browser open https://example.com
48+
```
49+
50+
## Commands Reference
51+
52+
### Navigation
53+
54+
```bash
55+
agent-browser open <url> # Navigate to URL
56+
agent-browser back # Go back
57+
agent-browser forward # Go forward
58+
agent-browser reload # Reload page
59+
agent-browser close # Close browser
60+
```
61+
62+
### Page Analysis (Snapshot)
63+
64+
```bash
65+
agent-browser snapshot # Full accessibility tree
66+
agent-browser snapshot -i # Interactive elements only (RECOMMENDED)
67+
agent-browser snapshot -c # Compact output
68+
agent-browser snapshot -d 3 # Limit depth to 3
69+
agent-browser snapshot -s "#main" # Scope to CSS selector
70+
```
71+
72+
### Interactions (Use @refs from Snapshot)
73+
74+
```bash
75+
# Clicking
76+
agent-browser click @e1 # Click
77+
agent-browser dblclick @e1 # Double-click
78+
agent-browser hover @e1 # Hover
79+
80+
# Focus
81+
agent-browser focus @e1 # Focus element
82+
83+
# Text Input
84+
agent-browser fill @e2 "text" # Clear and type
85+
agent-browser type @e2 "text" # Type without clearing
86+
87+
# Keyboard
88+
agent-browser press Enter # Press key
89+
agent-browser press Control+a # Key combination
90+
agent-browser keydown Shift # Hold key down
91+
agent-browser keyup Shift # Release key
92+
93+
# Forms
94+
agent-browser check @e1 # Check checkbox
95+
agent-browser uncheck @e1 # Uncheck checkbox
96+
agent-browser select @e1 "value" # Select dropdown option
97+
98+
# Scrolling
99+
agent-browser scroll down 500 # Scroll page
100+
agent-browser scrollintoview @e1 # Scroll element into view
101+
102+
# Other
103+
agent-browser drag @e1 @e2 # Drag and drop
104+
agent-browser upload @e1 file.pdf # Upload files
105+
```
106+
107+
### Getting Information
108+
109+
```bash
110+
agent-browser get text @e1 # Get element text
111+
agent-browser get html @e1 # Get innerHTML
112+
agent-browser get value @e1 # Get input value
113+
agent-browser get attr @e1 href # Get attribute
114+
agent-browser get title # Get page title
115+
agent-browser get url # Get current URL
116+
agent-browser get count ".item" # Count matching elements
117+
agent-browser get box @e1 # Get bounding box
118+
```
119+
120+
### Checking State
121+
122+
```bash
123+
agent-browser is visible @e1 # Check if visible
124+
agent-browser is enabled @e1 # Check if enabled
125+
agent-browser is checked @e1 # Check if checked
126+
```
127+
128+
### Screenshots & PDF
129+
130+
```bash
131+
agent-browser screenshot # Screenshot to stdout
132+
agent-browser screenshot path.png # Save to file
133+
agent-browser screenshot --full # Full page
134+
agent-browser pdf output.pdf # Save as PDF
135+
```
136+
137+
### Video Recording
138+
139+
```bash
140+
agent-browser record start ./demo.webm # Start recording
141+
agent-browser click @e1 # Perform actions
142+
agent-browser record stop # Stop and save video
143+
agent-browser record restart ./take2.webm # Stop current + start new
144+
```
145+
146+
### Waiting
147+
148+
```bash
149+
agent-browser wait @e1 # Wait for element
150+
agent-browser wait 2000 # Wait milliseconds
151+
agent-browser wait --text "Success" # Wait for text
152+
agent-browser wait --url "**/dashboard" # Wait for URL pattern
153+
agent-browser wait --load networkidle # Wait for network idle
154+
agent-browser wait --fn "window.ready" # Wait for JS condition
155+
```
156+
157+
### Cookies & Storage
158+
159+
```bash
160+
agent-browser cookies # Get all cookies
161+
agent-browser cookies set name value # Set cookie
162+
agent-browser cookies clear # Clear cookies
163+
agent-browser storage local # Get all localStorage
164+
agent-browser storage local key # Get specific key
165+
agent-browser storage local set k v # Set value
166+
agent-browser storage local clear # Clear all
167+
```
168+
169+
### Network
170+
171+
```bash
172+
agent-browser network route <url> # Intercept requests
173+
agent-browser network route <url> --abort # Block requests
174+
agent-browser network route <url> --body '{}' # Mock response
175+
agent-browser network unroute [url] # Remove routes
176+
agent-browser network requests # View tracked requests
177+
agent-browser network requests --filter api # Filter requests
178+
```
179+
180+
### Browser Settings
181+
182+
```bash
183+
agent-browser set viewport 1920 1080 # Set viewport size
184+
agent-browser set device "iPhone 14" # Emulate device
185+
agent-browser set geo 37.7749 -122.4194 # Set geolocation
186+
agent-browser set offline on # Toggle offline mode
187+
agent-browser set headers '{"X-Key":"v"}' # Extra HTTP headers
188+
agent-browser set credentials user pass # HTTP basic auth
189+
agent-browser set media dark # Emulate color scheme
190+
```
191+
192+
### Tabs & Windows
193+
194+
```bash
195+
agent-browser tab # List tabs
196+
agent-browser tab new [url] # New tab
197+
agent-browser tab 2 # Switch to tab
198+
agent-browser tab close # Close tab
199+
agent-browser window new # New window
200+
```
201+
202+
### Frames
203+
204+
```bash
205+
agent-browser frame "#iframe" # Switch to iframe
206+
agent-browser frame main # Back to main frame
207+
```
208+
209+
### Dialogs
210+
211+
```bash
212+
agent-browser dialog accept [text] # Accept dialog
213+
agent-browser dialog dismiss # Dismiss dialog
214+
```
215+
216+
### JavaScript Execution
217+
218+
```bash
219+
agent-browser eval "document.title" # Run JavaScript
220+
```
221+
222+
## Example Workflows
223+
224+
### Form Submission
225+
226+
```bash
227+
agent-browser open https://example.com/form
228+
agent-browser snapshot -i
229+
# Output shows: textbox "Email" [ref=e1], textbox "Password" [ref=e2], button "Submit" [ref=e3]
230+
231+
agent-browser fill @e1 "user@example.com"
232+
agent-browser fill @e2 "password123"
233+
agent-browser click @e3
234+
agent-browser wait --load networkidle
235+
agent-browser snapshot -i # Check result
236+
```
237+
238+
### Authentication with Saved State
239+
240+
```bash
241+
# Login once
242+
agent-browser open https://app.example.com/login
243+
agent-browser snapshot -i
244+
agent-browser fill @e1 "username"
245+
agent-browser fill @e2 "password"
246+
agent-browser click @e3
247+
agent-browser wait --url "**/dashboard"
248+
agent-browser state save auth.json
249+
250+
# Later sessions: load saved state
251+
agent-browser state load auth.json
252+
agent-browser open https://app.example.com/dashboard
253+
```
254+
255+
### Scraping Data
256+
257+
```bash
258+
agent-browser open https://example.com/products
259+
agent-browser snapshot -i --json > page_structure.json
260+
261+
# Get specific data
262+
agent-browser get text @e5 # Product title
263+
agent-browser get attr @e6 href # Product link
264+
```
265+
266+
### Taking Full Page Screenshot
267+
268+
```bash
269+
agent-browser open https://example.com
270+
agent-browser wait --load networkidle
271+
agent-browser screenshot --full fullpage.png
272+
```
273+
274+
### Testing Login Flow
275+
276+
```bash
277+
# Navigate to login
278+
agent-browser open https://app.example.com/login
279+
280+
# Take initial snapshot
281+
agent-browser snapshot -i
282+
283+
# Fill credentials
284+
agent-browser fill @e1 "test@example.com"
285+
agent-browser fill @e2 "testpassword"
286+
287+
# Click login
288+
agent-browser click @e3
289+
290+
# Wait for redirect
291+
agent-browser wait --url "**/dashboard"
292+
293+
# Verify logged in
294+
agent-browser get text @e10 # Should show username
295+
```
296+
297+
## Debugging
298+
299+
```bash
300+
agent-browser open example.com --headed # Show browser window
301+
agent-browser console # View console messages
302+
agent-browser console --clear # Clear console
303+
agent-browser errors # View page errors
304+
agent-browser errors --clear # Clear errors
305+
agent-browser highlight @e1 # Highlight element
306+
agent-browser trace start # Start recording trace
307+
agent-browser trace stop trace.zip # Stop and save trace
308+
agent-browser --cdp 9222 snapshot # Connect via CDP
309+
```
310+
311+
## Sessions (Parallel Browsers)
312+
313+
```bash
314+
agent-browser --session test1 open site-a.com
315+
agent-browser --session test2 open site-b.com
316+
agent-browser session list
317+
```
318+
319+
## JSON Output
320+
321+
Add `--json` for machine-readable output:
322+
323+
```bash
324+
agent-browser snapshot -i --json
325+
agent-browser get text @e1 --json
326+
```
327+
328+
## Semantic Locators (Alternative to Refs)
329+
330+
```bash
331+
agent-browser find role button click --name "Submit"
332+
agent-browser find text "Sign In" click
333+
agent-browser find label "Email" fill "user@test.com"
334+
agent-browser find first ".item" click
335+
agent-browser find nth 2 "a" text
336+
```
337+
338+
## Best Practices
339+
340+
1. **Always snapshot first**: Get the current page state before interacting
341+
2. **Use interactive mode (-i)**: Shows only clickable/fillable elements
342+
3. **Wait appropriately**: Use `--load networkidle` after navigation
343+
4. **Re-snapshot after changes**: DOM updates invalidate refs
344+
5. **Save authentication state**: Avoid repeated logins
345+
6. **Use --headed for debugging**: See what the browser sees
346+
7. **Check console for errors**: `agent-browser console` reveals issues
347+
348+
## When to Use This Skill
349+
350+
- Testing web applications end-to-end
351+
- Automating repetitive web tasks
352+
- Scraping data from websites
353+
- Debugging frontend issues
354+
- Taking screenshots for documentation
355+
- Recording demo videos
356+
- Verifying UI functionality
357+
- Filling out forms programmatically
358+
359+
## Context Saving Tip
360+
361+
When using this skill, disable other browser-related MCPs (Chrome DevTools, Playwright) to reduce context usage. agent-browser provides comprehensive functionality for most browser automation needs.

0 commit comments

Comments
 (0)