You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+9-1Lines changed: 9 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@
8
8
9
9
# agent-device
10
10
11
-
`agent-device` is a CLI for UI automation on iOS, tvOS, macOS, Android, and AndroidTV. It is designed for agent-driven workflows: inspect the UI, act on it deterministically, and keep that work session-aware and replayable.
11
+
`agent-device` is a CLI for UI automation and app observability on iOS, tvOS, macOS, Android, and AndroidTV. It is built for agent-driven workflows: inspect the UI, interact deterministically, collect logs/network/perf evidence when behavior breaks, and keep the whole flow session-aware and replayable.
12
12
13
13
If you know Vercel's [agent-browser](https://github.com/vercel-labs/agent-browser), this project applies the same broad idea to mobile apps and devices.
14
14
@@ -19,18 +19,26 @@ If you know Vercel's [agent-browser](https://github.com/vercel-labs/agent-browse
19
19
- Give agents a practical way to understand mobile UI state through structured snapshots.
20
20
- Keep automation flows token-efficient enough for real agent loops.
21
21
- Make common interactions reliable enough for repeated automation runs.
22
+
- Make debugging evidence easy to collect through logs, network inspection, and performance snapshots.
22
23
- Keep automation grounded in sessions, selectors, and replayable flows instead of one-off scripts.
23
24
24
25
## Core Ideas
25
26
26
27
- Sessions: open a target once, interact within that session, then close it cleanly.
27
28
- Snapshots: inspect the current accessibility tree in a compact form and get current-screen refs for exploration.
28
29
- Refs vs selectors: use refs for discovery, use selectors for durable replay and assertions.
30
+
- Observability: collect session logs, inspect recent HTTP traffic with `network dump`, and sample CPU/memory with `perf`.
29
31
- Tests: run deterministic `.ad` scripts as a light e2e test suite.
30
32
- Replay scripts: save `.ad` flows with `--save-script`, replay one script with `replay`, or run a folder/glob as a serial suite with `test`.
31
33
`test` supports metadata-aware retries up to 3 additional attempts, per-test timeouts, flaky pass reporting, and runner-managed artifacts under `.agent-device/test-artifacts` by default. Each attempt writes `replay.ad` and `result.txt`; failed attempts also keep copied logs and artifacts when available.
32
34
- Human docs vs agent skills: docs explain the system for people; skills provide compact operating guidance for agents.
33
35
36
+
## Complementary Tooling
37
+
38
+
Use `agent-device` for on-device UI automation, screenshots/recordings, app logs, network inspection, and performance snapshots.
39
+
40
+
When the task needs the React component tree, props, state, hooks, or render profiling, pair it with the complementary [`agent-react-devtools`](https://github.com/callstackincubator/agent-react-devtools) project. The two tools solve different layers of the same debugging workflow.
Copy file name to clipboardExpand all lines: skills/agent-device/SKILL.md
+2-1Lines changed: 2 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
---
2
2
name: agent-device
3
-
description: Automates interactions for Apple-platform apps (iOS, tvOS, macOS) and Android devices. Use when navigating apps, taking snapshots/screenshots, tapping, typing, scrolling, or extracting UI info across mobile, TV, and desktop targets.
3
+
description: Automates interactions for Apple-platform apps (iOS, tvOS, macOS) and Android devices. Use when navigating apps, taking snapshots/screenshots, tapping, typing, scrolling, extracting UI info, or collecting logs, network inspection, and perf snapshots across mobile, TV, and desktop targets.
4
4
---
5
5
6
6
# agent-device
@@ -71,3 +71,4 @@ Use this skill as a router with mandatory defaults. Read this file first. For no
71
71
- Need desktop surfaces, menu bar behavior, or macOS-specific interaction rules: [references/macos-desktop.md](references/macos-desktop.md)
72
72
- Need remote HTTP transport, `connect --remote-config`, or tenant leases on a remote macOS host: [references/remote-tenancy.md](references/remote-tenancy.md)
73
73
This includes remote React Native runs where `agent-device` now prepares Metro locally and manages the local Metro companion tunnel automatically.
74
+
- Need the React component tree, props, state, hooks, or render profiling: pair `agent-device` with the complementary [`agent-react-devtools`](https://github.com/callstackincubator/agent-react-devtools) project when available.
Copy file name to clipboardExpand all lines: skills/agent-device/references/debugging.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,6 +4,8 @@
4
4
5
5
Open this file when the task turns into failure triage, logs, network inspection, permission prompts, setup trouble, or unstable session behavior.
6
6
7
+
If the debugging task needs the React component tree, props, state, hooks, or render profiling, pair `agent-device` with the complementary [`agent-react-devtools`](https://github.com/callstackincubator/agent-react-devtools) project instead of trying to infer those internals from the accessibility tree or app logs alone.
Copy file name to clipboardExpand all lines: skills/agent-device/references/exploration.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -20,6 +20,7 @@ Open this file when the app or screen is already running and you need to discove
20
20
- User asks what is visible on screen: `snapshot`
21
21
- User asks for exact text from a known target: `get text`
22
22
- User asks you to tap, type, or choose an element: `snapshot -i`, then act
23
+
- User asks for the React component tree, props/state/hooks, or render profiling: pair `agent-device` with the complementary [`agent-react-devtools`](https://github.com/callstackincubator/agent-react-devtools) project
23
24
- React Native dev or debug build shows warning/error UI: capture enough evidence to identify it, dismiss it if it is not the requested behavior, then continue the flow and report it in the summary
24
25
- The on-screen keyboard is blocking the next step: `keyboard dismiss`; on iOS do this only while an app session is active, and use `keyboard status|get` only on Android
25
26
- UI does not expose the answer: say so plainly; do not browse or force the app into a new state unless asked
Copy file name to clipboardExpand all lines: skills/agent-device/references/verification.md
+3-2Lines changed: 3 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -129,5 +129,6 @@ agent-device perf --json
129
129
-`startup` is command round-trip timing around `open`.
130
130
- It is not true first-frame or first-interactive telemetry.
131
131
- Android app sessions also expose `memory` (`dumpsys meminfo`) and `cpu` (`dumpsys cpuinfo`) snapshots when the session has an app package context.
132
-
- Apple app sessions on macOS and iOS simulators also expose `memory` and `cpu` process snapshots when the session has an app bundle ID.
133
-
-`fps` is still unavailable, and physical iOS devices still leave `memory` and `cpu` unavailable in this release.
132
+
- Apple app sessions on macOS, iOS simulators, and physical iOS devices also expose `memory` and `cpu` process snapshots when the session has an app bundle ID.
133
+
- On physical iOS devices, sampling uses a short `xcrun xctrace` Activity Monitor capture, so keep the device unlocked, connected, and the app active in the foreground while sampling.
Copy file name to clipboardExpand all lines: skills/dogfood/SKILL.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -166,6 +166,7 @@ agent-device --session {SESSION} close
166
166
- Re-snapshot after any mutation (navigation, modal, list update, form submit).
167
167
- Use `fill` for clear-then-type semantics; use `type` for incremental typing behavior checks.
168
168
- Keep logs optional and targeted: enable/read app logs only when useful for diagnosis.
169
+
- If the issue appears rooted in React internals rather than device/app runtime behavior, pair `agent-device` with the complementary [`agent-react-devtools`](https://github.com/callstackincubator/agent-react-devtools) project for component-tree or render-profiling inspection.
169
170
- Never read source code of the app under test; findings must come from observed runtime behavior.
170
171
- Write each issue immediately to avoid losing evidence.
171
172
- Never delete screenshots/videos/report artifacts during a session.
Use `agent-device` when the task moves past UI automation and you need runtime evidence from the app or device layer.
8
+
9
+
## What `agent-device` covers well
10
+
11
+
- Session app logs for targeted debugging windows
12
+
- Network inspection from recent HTTP(s) entries in app logs via `network dump`
13
+
- Performance snapshots with `perf` / `metrics`
14
+
- Screenshots, recordings, and replayable repro flows
15
+
16
+
## What to use instead
17
+
18
+
If the task needs the React component tree, props, state, hooks, or render profiling, pair `agent-device` with the complementary [`agent-react-devtools`](https://github.com/callstackincubator/agent-react-devtools) project.
19
+
20
+
`agent-device` is centered on the device and app runtime layer. `agent-react-devtools` is the better fit for React internals.
21
+
22
+
## Fast path
23
+
24
+
```bash
25
+
agent-device open MyApp --platform ios
26
+
agent-device logs clear --restart
27
+
agent-device network dump 25 --include headers
28
+
agent-device perf --json
29
+
agent-device logs path
30
+
```
31
+
32
+
Use this flow when you need a clean repro window with logs, recent network activity, and a quick perf sample from the active app session.
33
+
34
+
## Core commands
35
+
36
+
### Logs
37
+
38
+
```bash
39
+
agent-device logs start
40
+
agent-device logs stop
41
+
agent-device logs clear --restart
42
+
agent-device logs path
43
+
agent-device logs doctor
44
+
agent-device logs mark "before submit"
45
+
```
46
+
47
+
- Logging is off by default; enable it only for focused debugging windows.
48
+
- Prefer `logs clear --restart` for clean repro loops.
49
+
- Use `logs path` and then grep the file instead of loading whole logs into agent context.
50
+
51
+
### Network inspection
52
+
53
+
```bash
54
+
agent-device network dump 25
55
+
agent-device network dump 25 --include headers
56
+
agent-device network dump 25 --include all
57
+
```
58
+
59
+
-`network dump` parses recent HTTP(s) entries from the session app log.
60
+
-`network log` is an alias for `network dump`.
61
+
- Parsed results depend on what the app emits into the platform log backend.
62
+
63
+
### Performance snapshots
64
+
65
+
```bash
66
+
agent-device perf --json
67
+
agent-device metrics --json
68
+
```
69
+
70
+
-`perf` returns session-scoped startup and, where supported, CPU and memory samples.
71
+
- Startup is measured around the `open` command; it is not first-frame instrumentation.
72
+
- CPU and memory availability depends on platform and whether the active session is bound to an app/package.
73
+
74
+
## Where to go deeper
75
+
76
+
- Full command reference: [Commands](/docs/commands)
Copy file name to clipboardExpand all lines: website/docs/docs/introduction.md
+14-4Lines changed: 14 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,22 +9,26 @@ title: Introduction
9
9
- Accessibility snapshots for UI understanding
10
10
- Deterministic interactions (tap, type, scroll)
11
11
- Session-aware workflows and replay
12
+
- Session logs and network inspection for debugging broken flows
13
+
- Performance snapshots with `perf`/`metrics`, including CPU and memory data where supported
12
14
13
-
If you know `agent-browser`, this is the mobile-native counterpart for iOS/Android UI automation.
15
+
If you know `agent-browser`, this is the mobile-native counterpart for iOS/Android UI automation and app-level observability.
14
16
For exploratory QA and bug-hunting workflows, see `skills/dogfood/SKILL.md` in this repository.
17
+
For React component trees, props/state/hooks, and render profiling, pair it with the complementary [`agent-react-devtools`](https://github.com/callstackincubator/agent-react-devtools) project.
15
18
16
19
## What it’s good at
17
20
18
-
-Capturing structured UI state for LLMs
19
-
-Driving common UI actions with refs or semantic selectors
20
-
- Replaying flows for regression checks
21
+
-Exploring and driving app flows on real devices and simulators
22
+
-Collecting debugging evidence through logs, network traffic, screenshots, recordings, and performance snapshots
23
+
- Replaying successful flows as lightweight regression checks
- Apple simulators and macOS desktop app sessions: `alert`, `pinch`.
31
+
- Session diagnostics: `logs` and `network dump` are available for debugging active app sessions, with network inspection based on recent HTTP(s) entries captured in the session app log.
28
32
- Session performance metrics: `perf`/`metrics` is available on iOS, macOS, and Android. Startup timing comes from `open` command round-trip duration. Android app sessions and Apple app sessions on macOS, iOS simulators, or connected iOS devices also expose CPU and memory snapshots when an app identifier is available in the session.
29
33
- iOS `record` supports simulators and physical devices.
30
34
- Simulators use native `simctl io ... recordVideo`.
@@ -43,6 +47,12 @@ For exploratory QA and bug-hunting workflows, see `skills/dogfood/SKILL.md` in t
43
47
3. iOS uses XCTest runner for snapshots and input on simulators and physical devices.
44
48
4. Android uses ADB-based tooling.
45
49
50
+
## Complementary React tooling
51
+
52
+
`agent-device` is intentionally centered on the device/app layer: UI automation, screenshots/recordings, app logs, network inspection, and performance sampling.
53
+
54
+
When a debugging workflow needs React internals such as the component tree, props, state, hooks, or render profiling, use the complementary `agent-react-devtools` project alongside `agent-device` rather than as a replacement.
0 commit comments