Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ The project is in early development and considered experimental. Pull requests a

## Features
- Platforms: iOS (simulator + limited device support) and Android (emulator + device).
- Core commands: `open`, `back`, `home`, `app-switcher`, `press`, `long-press`, `focus`, `type`, `fill`, `scroll`, `scrollintoview`, `wait`, `alert`, `screenshot`, `close`.
- Core commands: `open`, `back`, `home`, `app-switcher`, `press`, `long-press`, `focus`, `type`, `fill`, `scroll`, `scrollintoview`, `wait`, `alert`, `screenshot`, `close`, `reinstall`.
- Inspection commands: `snapshot` (accessibility tree).
- Device tooling: `adb` (Android), `simctl`/`devicectl` (iOS via Xcode).
- Minimal dependencies; TypeScript executed directly on Node 22+ (no build step).
Expand Down Expand Up @@ -75,7 +75,7 @@ Coordinates:
- X increases to the right, Y increases downward.

## Command Index
- `boot`, `open`, `close`, `home`, `back`, `app-switcher`
- `boot`, `open`, `close`, `reinstall`, `home`, `back`, `app-switcher`
- `snapshot`, `find`, `get`
- `click`, `focus`, `type`, `fill`, `press`, `long-press`, `scroll`, `scrollintoview`, `is`
- `alert`, `wait`, `screenshot`
Expand Down Expand Up @@ -123,6 +123,13 @@ Sessions:
- Session scripts are written to `~/.agent-device/sessions/<session>-<timestamp>.ad` when recording is enabled with `--save-script`.
- Deterministic replay is `.ad`-based; use `replay --update` (`-u`) to update selector drift and rewrite the replay file in place.

Navigation helpers:
- `boot --platform ios|android` ensures the target is ready without launching an app.
- Use `boot` mainly when starting a new session and `open` fails because no booted simulator/emulator is available.
- `open [app]` already boots/activates the selected target when needed.
- `reinstall <app> <path>` uninstalls and installs the app binary in one command (Android + iOS simulator in v1).
- `reinstall` accepts package/bundle id style app names and supports `~` in paths.

Find (semantic):
- `find <text> <action> [value]` finds by any text (label/value/identifier) using a scoped snapshot.
- `find text|label|value|role|id <value> <action> [value]` for specific locators.
Expand Down Expand Up @@ -188,7 +195,7 @@ Boot diagnostics:
- Boot failures include normalized reason codes in `error.details.reason` (JSON mode) and verbose logs.
- Reason codes: `IOS_BOOT_TIMEOUT`, `IOS_RUNNER_CONNECT_TIMEOUT`, `ANDROID_BOOT_TIMEOUT`, `ADB_TRANSPORT_UNAVAILABLE`, `CI_RESOURCE_STARVATION_SUSPECTED`, `BOOT_COMMAND_FAILED`, `UNKNOWN`.
- Android boot waits fail fast for permission/tooling issues and do not always collapse into timeout errors.
- Use `agent-device boot --platform ios|android` for explicit CI preflight readiness checks.
- Use `agent-device boot --platform ios|android` when starting a new session only if `open` cannot find/connect to an available target.
- Set `AGENT_DEVICE_RETRY_LOGS=1` to print structured retry telemetry (attempt, phase, delay, elapsed/remaining deadline, reason).

## App resolution
Expand Down
9 changes: 8 additions & 1 deletion skills/agent-device/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ npx -y agent-device

## Core workflow

1. Open app or just boot device: `open [app]`
1. Open app: `open [app]` (`open` handles target selection + boot/activation in the normal flow)
2. Snapshot: `snapshot` to get refs from accessibility tree
3. Interact using refs (`click @ref`, `fill @ref "text"`)
4. Re-snapshot after navigation/UI changes
Expand All @@ -38,12 +38,19 @@ npx -y agent-device
### Navigation

```bash
agent-device boot # Ensure target is booted/ready without opening app
agent-device boot --platform ios # Boot iOS simulator
agent-device boot --platform android # Boot Android emulator/device target
agent-device open [app] # Boot device/simulator; optionally launch app
agent-device open [app] --activity com.example/.MainActivity # Android: open specific activity
agent-device close [app] # Close app or just end session
agent-device reinstall <app> <path> # Uninstall + install app in one command
agent-device session list # List active sessions
```

`boot` requires either an active session or an explicit selector (`--platform`, `--device`, `--udid`, or `--serial`).
`boot` is a fallback, not a regular step: use it when starting a new session only if `open` cannot find/connect to an available target.

### Snapshot (page analysis)

```bash
Expand Down
1 change: 1 addition & 0 deletions src/core/__tests__/capabilities.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ test('iOS simulator + Android commands reject iOS devices', () => {
'home',
'long-press',
'open',
'reinstall',
'press',
'record',
'screenshot',
Expand Down
1 change: 1 addition & 0 deletions src/core/capabilities.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ const COMMAND_CAPABILITY_MATRIX: Record<string, CommandCapability> = {
home: { ios: { simulator: true }, android: { emulator: true, device: true, unknown: true } },
'long-press': { ios: { simulator: true }, android: { emulator: true, device: true, unknown: true } },
open: { ios: { simulator: true }, android: { emulator: true, device: true, unknown: true } },
reinstall: { ios: { simulator: true }, android: { emulator: true, device: true, unknown: true } },
press: { ios: { simulator: true }, android: { emulator: true, device: true, unknown: true } },
record: { ios: { simulator: true }, android: { emulator: true, device: true, unknown: true } },
screenshot: { ios: { simulator: true }, android: { emulator: true, device: true, unknown: true } },
Expand Down
219 changes: 219 additions & 0 deletions src/daemon/handlers/__tests__/session-reinstall.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,219 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { handleSessionCommands } from '../session.ts';
import { SessionStore } from '../../session-store.ts';
import type { DaemonRequest, DaemonResponse, SessionState } from '../../types.ts';

function makeStore(): SessionStore {
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-session-reinstall-'));
return new SessionStore(path.join(tempRoot, 'sessions'));
}

function makeSession(name: string, device: SessionState['device']): SessionState {
return {
name,
device,
createdAt: Date.now(),
actions: [],
};
}

const invoke = async (_req: DaemonRequest): Promise<DaemonResponse> => {
return { ok: false, error: { code: 'INVALID_ARGS', message: 'invoke should not be called in reinstall tests' } };
};

test('reinstall requires active session or explicit device selector', async () => {
const sessionStore = makeStore();
const response = await handleSessionCommands({
req: {
token: 't',
session: 'default',
command: 'reinstall',
positionals: ['com.example.app', '/tmp/app.apk'],
flags: {},
},
sessionName: 'default',
logPath: '/tmp/daemon.log',
sessionStore,
invoke,
});
assert.ok(response);
assert.equal(response.ok, false);
if (!response.ok) {
assert.equal(response.error.code, 'INVALID_ARGS');
assert.match(response.error.message, /active session or an explicit device selector/i);
}
});

test('reinstall validates required args before device operations', async () => {
const sessionStore = makeStore();
sessionStore.set(
'default',
makeSession('default', {
platform: 'ios',
id: 'sim-1',
name: 'iPhone',
kind: 'simulator',
booted: true,
}),
);
const response = await handleSessionCommands({
req: {
token: 't',
session: 'default',
command: 'reinstall',
positionals: ['com.example.app'],
flags: {},
},
sessionName: 'default',
logPath: '/tmp/daemon.log',
sessionStore,
invoke,
});
assert.ok(response);
assert.equal(response.ok, false);
if (!response.ok) {
assert.equal(response.error.code, 'INVALID_ARGS');
assert.match(response.error.message, /reinstall <app> <path-to-app-binary>/i);
}
});

test('reinstall reports unsupported operation on iOS physical devices', async () => {
const sessionStore = makeStore();
sessionStore.set(
'default',
makeSession('default', {
platform: 'ios',
id: 'device-1',
name: 'iPhone Device',
kind: 'device',
booted: true,
}),
);
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-reinstall-binary-'));
const appPath = path.join(tempRoot, 'Sample.app');
fs.writeFileSync(appPath, 'placeholder');

const response = await handleSessionCommands({
req: {
token: 't',
session: 'default',
command: 'reinstall',
positionals: ['com.example.app', appPath],
flags: {},
},
sessionName: 'default',
logPath: '/tmp/daemon.log',
sessionStore,
invoke,
});
assert.ok(response);
assert.equal(response.ok, false);
if (!response.ok) {
assert.equal(response.error.code, 'UNSUPPORTED_OPERATION');
assert.match(response.error.message, /reinstall is not supported/i);
}
});

test('reinstall succeeds on active iOS simulator session and records action', async () => {
const sessionStore = makeStore();
const session = makeSession('default', {
platform: 'ios',
id: 'sim-1',
name: 'iPhone',
kind: 'simulator',
booted: true,
});
sessionStore.set('default', session);
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-reinstall-success-ios-'));
const appPath = path.join(tempRoot, 'Sample.app');
fs.writeFileSync(appPath, 'placeholder');

const response = await handleSessionCommands({
req: {
token: 't',
session: 'default',
command: 'reinstall',
positionals: ['com.example.app', appPath],
flags: {},
},
sessionName: 'default',
logPath: '/tmp/daemon.log',
sessionStore,
invoke,
reinstallOps: {
ios: async (_device, app, pathToBinary) => {
assert.equal(app, 'com.example.app');
assert.equal(pathToBinary, appPath);
return { bundleId: 'com.example.app' };
},
android: async () => {
throw new Error('unexpected android reinstall');
},
},
});

assert.ok(response);
assert.equal(response.ok, true);
if (response.ok) {
assert.equal(response.data?.platform, 'ios');
assert.equal(response.data?.appId, 'com.example.app');
assert.equal(response.data?.bundleId, 'com.example.app');
assert.equal(response.data?.appPath, appPath);
}
assert.equal(session.actions.length, 1);
assert.equal(session.actions[0]?.command, 'reinstall');
});

test('reinstall succeeds on active Android session with normalized appId', async () => {
const sessionStore = makeStore();
sessionStore.set(
'default',
makeSession('default', {
platform: 'android',
id: 'emulator-5554',
name: 'Pixel',
kind: 'emulator',
booted: true,
}),
);
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-reinstall-success-android-'));
const appPath = path.join(tempRoot, 'Sample.apk');
fs.writeFileSync(appPath, 'placeholder');

const response = await handleSessionCommands({
req: {
token: 't',
session: 'default',
command: 'reinstall',
positionals: ['com.example.app', appPath],
flags: {},
},
sessionName: 'default',
logPath: '/tmp/daemon.log',
sessionStore,
invoke,
reinstallOps: {
ios: async () => {
throw new Error('unexpected ios reinstall');
},
android: async (_device, app, pathToBinary) => {
assert.equal(app, 'com.example.app');
assert.equal(pathToBinary, appPath);
return { package: 'com.example.app' };
},
},
});

assert.ok(response);
assert.equal(response.ok, true);
if (response.ok) {
assert.equal(response.data?.platform, 'android');
assert.equal(response.data?.appId, 'com.example.app');
assert.equal(response.data?.package, 'com.example.app');
assert.equal(response.data?.appPath, appPath);
}
});
Loading
Loading