You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Device automation CLI for AI agents. Mobile, TV, and desktop apps.
15
+
Device automation for AI agents.
16
16
17
-
`agent-device` lets coding agents run real apps, inspect UI state, interact with visible elements, and collect debugging evidence through one CLI.
17
+
`agent-device` lets coding agents open real apps, inspect the current UI, interact with visible elements, and collect debugging evidence through one CLI. Use it when an agent needs to verify what actually happens on a device, not just reason about code.
18
18
19
-
It is built around token-efficient accessibility snapshots, not pixel-first screenshots. Agents read compact UI trees, locate elements through refs like `@e3`, perform touch and text actions, and capture screenshots, video, logs, network, CPU/memory/perf, crash-related logs, and React profiles only when evidence is needed.
19
+
If you know Vercel's [agent-browser](https://github.com/vercel-labs/agent-browser), `agent-device` is the same idea for mobile, TV, and desktop apps.
20
20
21
-
Agents can ingest the current docs from [llms-full.txt](https://incubator.callstack.com/agent-device/llms-full.txt). The installed CLI help remains authoritative for exact command syntax.
22
-
23
-
## Agentic QA And Development
24
-
25
-
-**Quality Assurance**: dogfood flows, validate PR builds, check accessibility coverage, and turn stable explorations into `.ad` e2e tests.
26
-
-**Development**: build from specs, inspect real runtime behavior, and iterate until the UI matches the work.
27
-
28
-
`agent-device` closes the agentic development loop: agents can write code, run the real app, verify the UI end-to-end, collect screenshots/videos/logs/perf evidence, and feed bugs, crashes, or performance findings back into the next fix iteration before a human reviews the PR.
21
+

29
22
30
-

23
+
## Capabilities
31
24
32
-
If you know Vercel's [agent-browser](https://github.com/vercel-labs/agent-browser), this is the same idea for apps and devices.
25
+
-**Inspect** real app UI through compact accessibility snapshots, interactive refs like `@e3`, selectors, and React Native component trees.
26
+
-**Interact** by opening apps, tapping, typing, scrolling, performing gestures, waiting, asserting state, handling alerts, and closing sessions.
27
+
-**Capture evidence** with screenshots, videos, logs, traces, network traffic, performance samples, crash context, and React profiles.
28
+
-**Replay workflows** by recording `.ad` scripts for local runs, CI, and repeatable e2e checks.
29
+
-**Run across platforms** on iOS, Android, tvOS, Android TV, macOS, Linux, simulators, emulators, desktops, and physical devices.
33
30
34
-
Use it for AI mobile testing, AI QA for React Native and Expo apps, iOS Simulator automation, Android Emulator automation, tvOS/Android TV checks, and desktop app verification from coding agents. Humans install and configure `agent-device`; agents run the workflows.
31
+
## Why Teams Use It
35
32
36
-

33
+
- Verify PR builds on real apps before human review.
34
+
- Dogfood mobile flows with durable evidence.
35
+
- Give agents a device-side feedback loop while they write code.
36
+
- Convert exploratory QA into repeatable checks.
37
+
- Collect enough context for bugs, crashes, or performance regressions to feed the next fix.
37
38
38
-
Demo: Codex uses `agent-device` to inspect iOS Contacts through accessibility snapshots, interact with visible UI, and create a contact from a simple prompt.
39
+

39
40
40
41
## Quick Start
41
42
42
-
Install the CLI first:
43
+
Install the CLI:
43
44
44
45
```bash
45
46
npm install -g agent-device@latest
46
47
agent-device --version
47
48
agent-device help workflow
48
49
```
49
50
50
-
The CLI help is the source of truth for agents and is shipped with the installed version. Skills are optional but recommended when your agent runtime supports them: they auto-route device, React DevTools, and dogfood tasks to the right `agent-device help <topic>` page and verify the CLI is new enough before acting.
51
-
52
-
If you install skills separately, keep the CLI on `agent-device >= 0.14.0`. Older CLIs do not include the workflow help topics that the router skills expect.
53
-
54
-
### AI Agent Entry Points
55
-
56
-
-**Agent + terminal**: in Cursor, Codex, Claude Code, Windsurf, and similar clients, run `agent-device` in the integrated terminal. Start planning with `agent-device help workflow`; CLI help is authoritative.
57
-
-**Skills or rules**: install the skill with `npx skills add callstackincubator/agent-device`, use the bundled [agent-device skill](skills/agent-device/SKILL.md), or mirror it as a thin project rule, so the agent checks the installed version and reads `agent-device help workflow` before acting. Use `agent-device help react-native` for React Native apps, overlays, Metro/Fast Refresh blockers, and routing to React DevTools or debugging evidence.
58
-
-**MCP router**: use `agent-device mcp` when an MCP-aware client needs to discover the CLI package, install command, version check, and first help command. MCP is discovery-only; device automation still runs through terminal CLI commands.
59
-
60
-
For client-specific setup, see [AI Agent Setup](https://incubator.callstack.com/agent-device/docs/agent-setup). For agent-readable docs, use [llms-full.txt](https://incubator.callstack.com/agent-device/llms-full.txt).
61
-
62
-
### MCP Router
63
-
64
-
`agent-device` ships an official stdio MCP router for discovery-oriented clients. It exposes only a `status` tool that returns structured CLI handoff guidance: npm package name, installed version, CLI command name, install command, verify command, starting help command, and an explicit note that automation happens through the CLI.
65
-
66
-
MCP clients must not use this server as a device automation surface or generic shell runner. If the CLI is missing, agents should ask a human before installing or updating packages, then verify with `agent-device --version` and start with `agent-device help workflow`.
67
-
68
-
Paste one of these into clients that accept `mcpServers`, such as Cursor project `.cursor/mcp.json` or user-level MCP settings.
The installed CLI help is the source of truth for agents. Start with `agent-device help workflow`, then follow the topic-specific help when a task needs dogfooding, debugging, replay, or React Native profiling.
99
52
100
-
</details>
53
+
Prerequisites depend on the target platform: Node.js 22+, Xcode for iOS/tvOS/macOS targets, Android SDK + ADB for Android, and macOS Accessibility permission for desktop automation. See [Installation](https://incubator.callstack.com/agent-device/docs/installation) for platform setup.
101
54
102
-
Registry metadata uses MCP name `io.github.callstackincubator/agent-device`, npm package `agent-device`, stdio transport, `mcpName` package verification, `server.json`, and `smithery.yaml`.
55
+
Try the basic loop:
103
56
104
57
```bash
105
-
npm install -g agent-device@latest
106
-
agent-device --version
107
-
agent-device help
108
-
```
109
-
110
-
`agent-device` performs a lightweight background upgrade check for interactive CLI runs and, when a newer package is available, suggests a global reinstall command. Updating the package also refreshes the bundled `skills/` shipped with the CLI.
111
-
112
-
Prerequisites: Node.js 22+, Xcode for iOS/tvOS/macOS targets, Android SDK + ADB for Android, and macOS Accessibility permission for desktop automation. See [Installation](https://incubator.callstack.com/agent-device/docs/installation).
113
-
114
-
Try the loop.
115
-
116
-
```bash
117
-
# Find the app.
58
+
# Find an app.
118
59
agent-device apps --platform ios
119
60
agent-device apps --platform android
120
61
@@ -127,82 +68,31 @@ agent-device snapshot -i
127
68
# @e2 [button] "Sign In"
128
69
# @e3 [text-field] "Email"
129
70
130
-
# Act, capture a screenshot, and close.
131
-
agent-device fill @e3 "test"
71
+
# Act, capture evidence, and close.
72
+
agent-device fill @e3 "test@example.com"
132
73
agent-device screenshot ./artifacts/settings.png
133
74
agent-device close
134
75
```
135
76
136
-
Snapshots assign refs like `@e1`, `@e2`, and `@e3` to current-screen elements. Refs from the default snapshot are immediately actionable; for hidden content, scroll and re-snapshot.
137
-
138
-
### First 5 Minutes: Expo Test App
139
-
140
-
Use the bundled Expo fixture when you want a concrete first agent run with setup checks, screenshots, replay, and performance evidence. This path requires a repo checkout because `examples/test-app` and the `pnpm test-app:*` scripts are not included in the published npm package.
Snapshots assign refs like `@e1`, `@e2`, and `@e3` to elements on the current screen. Refs from the latest snapshot are immediately actionable; after scrolling or changing screens, take a fresh snapshot.
148
78
149
-
```bash
150
-
pnpm test-app:install
151
-
cd examples/test-app
152
-
npx expo-doctor@latest
153
-
cd ../..
154
-
pnpm test-app:ios
155
-
# or: pnpm test-app:android
156
-
```
79
+
## Next Steps
157
80
158
-
Then give your agent this prompt:
159
-
160
-
```text
161
-
Use agent-device to dogfood the bundled Expo app and produce an evidence-backed report.
162
-
163
-
Setup:
164
-
- Read `agent-device help workflow`, `agent-device help dogfood`, `agent-device help debugging`, and `agent-device help react-devtools` before planning commands.
165
-
- Confirm the test app setup commands were run: `pnpm test-app:install`, `cd examples/test-app && npx expo-doctor@latest`, then `pnpm test-app:ios` or `pnpm test-app:android`.
166
-
- If Metro prints an Expo URL, prefer opening the shell with that URL. On iOS use `agent-device open "Expo Go" <url> --platform ios`; on Android use the visible Expo/dev-client target or URL. Confirm the app UI with `snapshot -i`.
167
-
168
-
Run:
169
-
- Create `./dogfood-output/screenshots`, `./dogfood-output/videos`, `./dogfood-output/traces`, `./dogfood-output/perf`, and `./dogfood-output/replays`.
170
-
- Open a named session `expo-qa` and save a replay script to `./dogfood-output/replays/expo-test.ad`.
171
-
- Use command shapes like `agent-device --session expo-qa open "Expo Go" <url> --platform ios --save-script ./dogfood-output/replays/expo-test.ad`, `agent-device --session expo-qa screenshot ./dogfood-output/screenshots/home.png`, `agent-device --session expo-qa perf --json > ./dogfood-output/perf/baseline.json`, and `agent-device --session expo-qa record start ./dogfood-output/videos/checkout.mp4`.
172
-
- Capture a baseline `snapshot -i`, screenshot, and `perf --json` sample.
173
-
- Exercise Home, Catalog, product detail, Checkout, and Settings. Re-snapshot after each mutation and use refs/selectors from fresh snapshots.
174
-
- Capture at least one overlay-ref screenshot, one normal screenshot, one short video recording for a meaningful flow, logs marks around any issue, and trace output if a runtime symptom needs diagnostics.
175
-
- Run focused performance checks: compare `perf --json` before and after a navigation or form flow; if React DevTools connects, use one bounded first-pass profile survey (`slow --limit 5`, `rerenders --limit 5`, and `timeline --limit 20` only when timing matters), then drill into a specific `@c` ref with `profile report`. If it cannot connect, include the status and continue.
176
-
- Close the session so the `.ad` replay is written.
177
-
178
-
Report:
179
-
- Write `./dogfood-output/report.md`.
180
-
- Link every screenshot, video, trace, log path, replay file, and performance artifact you used.
181
-
- Include setup results, platform/device, Expo doctor outcome, coverage, severity counts, findings with repro commands, and a short performance section summarizing startup/CPU/memory/frame-health or React profile findings.
182
-
- If no issues are found, report covered flows and residual risk instead of claiming the app is bug-free.
183
-
```
81
+
-**Set up your agent**: run the CLI from Cursor, Codex, Claude Code, Windsurf, or another agent terminal. For skills, rules, MCP discovery, and client-specific setup, see [AI Agent Setup](https://incubator.callstack.com/agent-device/docs/agent-setup).
82
+
-**Try the sample app**: clone the repo and run the bundled Expo fixture when you want a guided first dogfood run with screenshots, replay, and performance evidence. See [Quick Start](https://incubator.callstack.com/agent-device/docs/quick-start).
83
+
-**Go deeper**: use [Commands](https://incubator.callstack.com/agent-device/docs/commands), [Replay & E2E](https://incubator.callstack.com/agent-device/docs/replay-e2e), and [Debugging & Profiling](https://incubator.callstack.com/agent-device/docs/debugging-profiling) for production workflows.
184
84
185
85
## Where To Run agent-device
186
86
187
87
| Path | Best for | Start with |
188
88
| --- | --- | --- |
189
89
| Local | Exploration, debugging, and development loops on simulators, emulators, physical devices, macOS apps, and Linux desktop targets. | Follow the Quick Start. |
190
-
| CI/CD | Automated PR and merge validation with replay scripts and captured artifacts. |Start with the [EAS workflow template](https://github.com/callstackincubator/eas-agent-device/blob/main/.eas/workflows/agent-qa-mobile.yml). GitHub Actions template coming soon. |
90
+
| CI/CD | Automated PR and merge validation with replay scripts and captured artifacts. |Try the [EAS workflow template](https://github.com/callstackincubator/eas-agent-device/blob/main/.eas/workflows/agent-qa-mobile.yml). GitHub Actions template coming soon. |
191
91
| Cloud / remote execution | Linux runners, managed devices, and remote execution. | Use [Agent Device Cloud](https://agent-device.dev/cloud), see [Commands](https://incubator.callstack.com/agent-device/docs/commands) for remote profiles, or [contact Callstack](mailto:hello@callstack.com) for team-scale QA. |
192
92
193
-
## Capabilities
194
-
195
-
-**Platforms**: iOS, Android, tvOS, Android TV, macOS, and Linux. Real devices and simulators are supported.
196
-
-**Agent-native UI model**: token-efficient accessibility snapshots, current-screen refs for exploration, selectors for durable replay, and skill-tested workflow guidance.
197
-
-**Capture and debug**: screenshots, video, logs, network traffic, CPU/memory/performance data, crash-related logs, accessibility snapshots, and React render profiles.
198
-
-**Produce**: replayable `.ad` scripts (recorded replay files that run locally or in CI), e2e test runs, snapshot and screenshot diffs, and debugging artifacts.
199
-
-**React Native and Expo**: component tree inspection, props/state/hooks, and render profiling.
200
-
-**MCP boundary**: discovery and help over MCP; app/device control through the CLI for explicit, auditable commands.
201
-
-**License**: MIT. Free to use.
202
-
203
93
## How It Works
204
94
205
-
`agent-device` runs session-aware commands through platform backends: XCTest for iOS and tvOS, ADB plus the Android snapshot helper for Android, a local helper for macOS desktop automation, and AT-SPI for Linux desktop targets. See [Introduction](https://incubator.callstack.com/agent-device/docs/introduction) and [Commands](https://incubator.callstack.com/agent-device/docs/commands) for platform details.
95
+
`agent-device` runs session-aware commands through platform backends: XCTest for iOS and tvOS, ADB plus the Android snapshot helper for Android, a local helper for macOS desktop automation, and AT-SPI for Linux desktop targets.
206
96
207
97
Node consumers can use the typed client and public subpaths for bridge integrations. `agent-device/android-adb` exposes the Android ADB provider contract, logcat/clipboard/keyboard/app helpers, and port reverse management.
208
98
@@ -212,26 +102,13 @@ Used by teams and developers at Callstack, Expensify, Shopify, Kindred, Total Wi
agent-device is open source and MIT licensed. Visit [agent-device.dev](https://agent-device.dev/), try the [EAS workflow template](https://github.com/callstackincubator/eas-agent-device/blob/main/.eas/workflows/agent-qa-mobile.yml), read the [incubator docs](https://incubator.callstack.com/agent-device/), or contact us at hello@callstack.com.
114
+
`agent-device` is open source and MIT licensed. Visit [agent-device.dev](https://agent-device.dev/), try the [EAS workflow template](https://github.com/callstackincubator/eas-agent-device/blob/main/.eas/workflows/agent-qa-mobile.yml), read the [incubator docs](https://incubator.callstack.com/agent-device/), or contact us at hello@callstack.com.
0 commit comments