Skip to content

Commit 6106db5

Browse files
thymikeethymikee
committed
docs: add agent batching guides across readme website and skills
1 parent cbefb4e commit 6106db5

7 files changed

Lines changed: 353 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,61 @@ agent-device click @e3
4545
agent-device close
4646
```
4747

48+
## Fast batching for agents (JSON steps)
49+
50+
Use `batch` to execute multiple commands in a single daemon request.
51+
52+
CLI examples:
53+
54+
```bash
55+
agent-device batch \
56+
--session sim \
57+
--platform ios \
58+
--udid 00008150-001849640CF8401C \
59+
--steps-file /tmp/batch-steps.json \
60+
--json
61+
```
62+
63+
```bash
64+
cat /tmp/batch-steps.json | agent-device batch \
65+
--session sim \
66+
--platform ios \
67+
--udid 00008150-001849640CF8401C \
68+
--steps-stdin \
69+
--json
70+
```
71+
72+
Small inline payloads are also supported:
73+
74+
```bash
75+
agent-device batch --steps '[{"command":"open","positionals":["settings"]},{"command":"wait","positionals":["100"]}]'
76+
```
77+
78+
Batch payload format:
79+
80+
```json
81+
[
82+
{ "command": "open", "positionals": ["settings"], "flags": {} },
83+
{ "command": "wait", "positionals": ["label=\"Privacy & Security\"", "3000"], "flags": {} },
84+
{ "command": "click", "positionals": ["label=\"Privacy & Security\""], "flags": {} },
85+
{ "command": "get", "positionals": ["text", "label=\"Tracking\""], "flags": {} }
86+
]
87+
```
88+
89+
Batch response includes:
90+
91+
- `total`, `executed`, `totalDurationMs`
92+
- per-step `results[]` with `durationMs`
93+
- failure context with failing `step` and `partialResults`
94+
95+
Agent usage guidelines:
96+
97+
- Keep each batch to one screen-local workflow.
98+
- Add sync guards (`wait`, `is exists`) after mutating steps (`open`, `click`, `fill`, `swipe`).
99+
- Treat refs/snapshot assumptions as stale after UI mutations.
100+
- Prefer `--steps-file` or `--steps-stdin` over inline JSON for reliability.
101+
- Keep batches moderate (about 5-20 steps) and stop on first error.
102+
48103
## CLI Usage
49104

50105
```bash
@@ -84,6 +139,7 @@ agent-device swipe 540 1500 540 500 120 --count 8 --pause-ms 30 --pattern ping-p
84139

85140
## Command Index
86141
- `boot`, `open`, `close`, `reinstall`, `home`, `back`, `app-switcher`
142+
- `batch`
87143
- `snapshot`, `find`, `get`
88144
- `click`, `focus`, `type`, `fill`, `press`, `long-press`, `swipe`, `scroll`, `scrollintoview`, `pinch`, `is`
89145
- `alert`, `wait`, `screenshot`
@@ -114,6 +170,11 @@ Flags:
114170
- `--pattern one-way|ping-pong` repeat pattern for `swipe`
115171
- `--verbose` for daemon and runner logs
116172
- `--json` for structured output
173+
- `--steps <json>` batch: JSON array of steps
174+
- `--steps-file <path>` batch: read step JSON from file
175+
- `--steps-stdin` batch: read step JSON from stdin
176+
- `--on-error stop` batch: stop when a step fails
177+
- `--max-steps <n>` batch: max allowed steps per request
117178

118179
Pinch:
119180
- `pinch` is supported on iOS simulators.

‎skills/agent-device/SKILL.md‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -155,6 +155,59 @@ agent-device replay -u ./session.ad # Update selector drift and rewrite .ad sc
155155
`--save-script` path is a file path; parent directories are created automatically.
156156
For ambiguous bare values, use `--save-script=workflow.ad` or `./workflow.ad`.
157157

158+
### Fast batching for agents (JSON steps)
159+
160+
Use `batch` when an agent already has a known short sequence and wants fewer orchestration round trips.
161+
162+
```bash
163+
agent-device batch \
164+
--session sim \
165+
--platform ios \
166+
--udid 00008150-001849640CF8401C \
167+
--steps-file /tmp/batch-steps.json \
168+
--json
169+
```
170+
171+
```bash
172+
cat /tmp/batch-steps.json | agent-device batch \
173+
--session sim \
174+
--platform ios \
175+
--udid 00008150-001849640CF8401C \
176+
--steps-stdin \
177+
--json
178+
```
179+
180+
Inline JSON works for small payloads:
181+
182+
```bash
183+
agent-device batch --steps '[{"command":"open","positionals":["settings"]},{"command":"wait","positionals":["100"]}]'
184+
```
185+
186+
Step format:
187+
188+
```json
189+
[
190+
{ "command": "open", "positionals": ["settings"], "flags": {} },
191+
{ "command": "wait", "positionals": ["label=\"Privacy & Security\"", "3000"], "flags": {} },
192+
{ "command": "click", "positionals": ["label=\"Privacy & Security\""], "flags": {} },
193+
{ "command": "get", "positionals": ["text", "label=\"Tracking\""], "flags": {} }
194+
]
195+
```
196+
197+
Batch best practices for agents:
198+
199+
- Batch one screen-local flow at a time.
200+
- Add sync guards (`wait`, `is exists`) after mutating steps (`open`, `click`, `fill`, `swipe`).
201+
- Treat prior refs/snapshot assumptions as stale after UI mutations.
202+
- Prefer `--steps-file` or `--steps-stdin` over inline JSON.
203+
- Keep batches moderate (about 5-20 steps).
204+
- Use failure context (`step`, `partialResults`) to replan from the failed step.
205+
206+
Stale accessibility tree note:
207+
208+
- Rapid mutations can outrun accessibility tree updates.
209+
- Mitigate with explicit waits and phase splitting (navigate, verify/extract, cleanup).
210+
158211
### Trace logs (XCTest)
159212

160213
```bash
@@ -208,3 +261,4 @@ agent-device apps --platform android --user-installed
208261
- [references/permissions.md](references/permissions.md)
209262
- [references/video-recording.md](references/video-recording.md)
210263
- [references/coordinate-system.md](references/coordinate-system.md)
264+
- [references/batching-for-agents.md](references/batching-for-agents.md)

‎skills/agent-device/references/batching-for-agents.md‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
# Batching for agents
2+
3+
## When to use batch
4+
5+
- The agent already knows a short sequence of commands.
6+
- Steps belong to one logical screen flow.
7+
- You want one result object with per-step timing and failure context.
8+
9+
## When not to use batch
10+
11+
- Flows are unrelated and should be retried independently.
12+
- The workflow is highly dynamic and requires replanning after each step.
13+
- You need human approvals between steps.
14+
15+
## CLI patterns
16+
17+
From file:
18+
19+
```bash
20+
agent-device batch --session sim --platform ios --steps-file /tmp/batch-steps.json --json
21+
```
22+
23+
From stdin:
24+
25+
```bash
26+
cat /tmp/batch-steps.json | agent-device batch --session sim --platform ios --steps-stdin --json
27+
```
28+
29+
Inline (small payloads only):
30+
31+
```bash
32+
agent-device batch --steps '[{"command":"open","positionals":["settings"]}]'
33+
```
34+
35+
## Step payload contract
36+
37+
```json
38+
[
39+
{ "command": "open", "positionals": ["settings"], "flags": {} },
40+
{ "command": "wait", "positionals": ["label=\"Privacy & Security\"", "3000"], "flags": {} },
41+
{ "command": "click", "positionals": ["label=\"Privacy & Security\""], "flags": {} },
42+
{ "command": "get", "positionals": ["text", "label=\"Tracking\""], "flags": {} }
43+
]
44+
```
45+
46+
Rules:
47+
48+
- `positionals` optional, defaults to `[]`.
49+
- `flags` optional, defaults to `{}`.
50+
- nested `batch` and `replay` are rejected.
51+
- stop-on-first-error is the supported mode (`--on-error stop`).
52+
53+
## Response handling
54+
55+
Success includes:
56+
57+
- `total`, `executed`, `totalDurationMs`
58+
- `results[]` entries with `step`, `command`, `durationMs`, and optional `data`
59+
60+
Failure includes:
61+
62+
- `details.step`
63+
- `details.command`
64+
- `details.executed`
65+
- `details.partialResults`
66+
67+
Use these fields to replan from the first failing step.
68+
69+
## Common error categories and agent actions
70+
71+
- `INVALID_ARGS`: payload/step shape issue; fix payload and retry.
72+
- `SESSION_NOT_FOUND`: open or select the correct session, then retry.
73+
- `UNSUPPORTED_OPERATION`: switch command/target to supported operation.
74+
- `AMBIGUOUS_MATCH`: refine selector/locator, then retry failed step.
75+
- `COMMAND_FAILED`: add sync guard (`wait`, `is exists`) and retry from failed step.
76+
77+
## Reliability guardrails
78+
79+
- Add sync guards after mutating steps.
80+
- Assume snapshot/ref drift after navigation.
81+
- Keep batch size moderate (about 5-20 steps).
82+
- Split long workflows into phases:
83+
1. navigate
84+
2. verify/extract
85+
3. cleanup

‎website/docs/docs/_meta.json‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,11 @@
1919
"type": "file",
2020
"label": "Commands"
2121
},
22+
{
23+
"name": "batching",
24+
"type": "file",
25+
"label": "Batching for Agents"
26+
},
2227
{
2328
"name": "selectors",
2429
"type": "file",

‎website/docs/docs/batching.md‎

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
---
2+
title: Batching for Agents
3+
---
4+
5+
# Batching for Agents
6+
7+
Use `batch` to run multiple commands in a single daemon request.
8+
9+
This is useful for agent workflows that already know the next sequence of actions and want to reduce orchestration overhead.
10+
11+
## CLI examples
12+
13+
From a file:
14+
15+
```bash
16+
agent-device batch \
17+
--session sim \
18+
--platform ios \
19+
--udid 00008150-001849640CF8401C \
20+
--steps-file /tmp/batch-steps.json \
21+
--json
22+
```
23+
24+
From stdin:
25+
26+
```bash
27+
cat /tmp/batch-steps.json | agent-device batch \
28+
--session sim \
29+
--platform ios \
30+
--udid 00008150-001849640CF8401C \
31+
--steps-stdin \
32+
--json
33+
```
34+
35+
Inline for small payloads:
36+
37+
```bash
38+
agent-device batch --steps '[{"command":"open","positionals":["settings"]},{"command":"wait","positionals":["100"]}]'
39+
```
40+
41+
## Step payload format
42+
43+
`batch` accepts a JSON array of steps:
44+
45+
```json
46+
[
47+
{ "command": "open", "positionals": ["settings"], "flags": {} },
48+
{ "command": "wait", "positionals": ["label=\"Privacy & Security\"", "3000"], "flags": {} },
49+
{ "command": "click", "positionals": ["label=\"Privacy & Security\""], "flags": {} },
50+
{ "command": "get", "positionals": ["text", "label=\"Tracking\""], "flags": {} }
51+
]
52+
```
53+
54+
Notes:
55+
56+
- `positionals` is optional (defaults to `[]`).
57+
- `flags` is optional (defaults to `{}`).
58+
- nested `batch` and `replay` steps are rejected.
59+
- `--on-error stop` is the supported behavior.
60+
61+
## Response shape
62+
63+
Success:
64+
65+
```json
66+
{
67+
"success": true,
68+
"data": {
69+
"total": 4,
70+
"executed": 4,
71+
"totalDurationMs": 1810,
72+
"results": [
73+
{ "step": 1, "command": "open", "ok": true, "durationMs": 1020 },
74+
{ "step": 2, "command": "wait", "ok": true, "durationMs": 320 },
75+
{ "step": 3, "command": "click", "ok": true, "durationMs": 260 },
76+
{ "step": 4, "command": "get", "ok": true, "durationMs": 210, "data": { "text": "..." } }
77+
]
78+
}
79+
}
80+
```
81+
82+
Failure:
83+
84+
```json
85+
{
86+
"success": false,
87+
"error": {
88+
"code": "COMMAND_FAILED",
89+
"message": "Batch failed at step 3 (click): ...",
90+
"details": {
91+
"step": 3,
92+
"command": "click",
93+
"positionals": ["label=\"Privacy & Security\""],
94+
"executed": 2,
95+
"total": 4,
96+
"partialResults": [
97+
{ "step": 1, "command": "open", "ok": true },
98+
{ "step": 2, "command": "wait", "ok": true }
99+
]
100+
}
101+
}
102+
}
103+
```
104+
105+
## Agent best practices
106+
107+
- Batch only one related screen flow at a time.
108+
- After mutating steps (`open`, `click`, `fill`, `swipe`), add a sync guard (`wait`, `is exists`) before critical reads.
109+
- Treat prior refs/snapshots as stale after UI changes.
110+
- Prefer `--steps-file` or `--steps-stdin` over inline JSON.
111+
- Keep batches moderate (about 5-20 steps).
112+
- Replan from the failing step using `details.step` and `details.partialResults`.
113+
114+
## Stale accessibility tree risk
115+
116+
Rapid UI changes can outpace accessibility tree updates. Mitigate by inserting explicit waits and splitting long workflows into phases:
117+
118+
1. navigate
119+
2. verify/extract
120+
3. cleanup

‎website/docs/docs/commands.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,21 @@ agent-device replay -u ./session.ad # Update selector drift and rewrite .ad sc
8888

8989
See [Replay & E2E (Experimental)](/docs/replay-e2e) for recording and CI workflow details.
9090

91+
## Batch
92+
93+
```bash
94+
agent-device batch --steps-file /tmp/batch-steps.json --json
95+
agent-device batch --steps-stdin --json
96+
agent-device batch --steps '[{"command":"open","positionals":["settings"]}]'
97+
```
98+
99+
- `batch` runs a JSON array of steps in a single daemon request.
100+
- each step has `command`, optional `positionals`, and optional `flags`.
101+
- stop-on-first-error is the supported behavior (`--on-error stop`).
102+
- use `--max-steps <n>` to tighten per-request safety limits.
103+
104+
See [Batching for Agents](/docs/batching) for payload format, response shape, and agent guidelines.
105+
91106
## App reinstall (fresh state)
92107

93108
```bash

0 commit comments

Comments
 (0)