Skip to content

Commit b35ea4a

Browse files
supportsupport
authored andcommitted
fix(ios): preserve session on AX snapshot failure
1 parent fa4e2d5 commit b35ea4a

12 files changed

Lines changed: 623 additions & 33 deletions

File tree

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
# ADR 0003: iOS AX Snapshot Failure Handling
2+
3+
## Status
4+
5+
Accepted
6+
7+
## Context
8+
9+
iOS XCTest can fail hierarchy capture with `kAXErrorIllegalArgument` when an accessibility tree is
10+
too deep to serialize. Appium's XCUITest guidance documents the practical depth limit: callers may
11+
raise `snapshotMaxDepth` only up to `62`, and elements at depth `63` or greater cannot be returned by
12+
XCTest. React Native screens are a common source of this shape.
13+
14+
Before this ADR, Agent Device let that XCTest failure escape as a slow runner command. The daemon
15+
could wait for the command deadline, invalidate or kill the runner session as a transport failure,
16+
and then later commands reported `SESSION_NOT_FOUND`. The app tree may still need flattening, but a
17+
snapshot limitation should not break screenshot, logs, app lifecycle, or direct selector commands in
18+
the same runner session.
19+
20+
Maestro handles this class of failure in its iOS view hierarchy route by using a depth cap of `60`,
21+
detecting `kAXErrorIllegalArgument`, and retrying from a child/window subtree when the app root
22+
cannot be serialized.
23+
24+
## Decision
25+
26+
Agent Device treats iOS AX snapshot serialization failure as a typed snapshot failure, not as a
27+
runner transport failure.
28+
29+
The runner snapshot path now:
30+
31+
- caps traversal depth at `60`, with lower user-provided `--depth` values still honored
32+
- catches Swift errors and Objective-C exceptions from `XCUIElement.snapshot()`
33+
- classifies `kAXErrorIllegalArgument` as `IOS_AX_SNAPSHOT_FAILED`
34+
- retries app-root failures from `windows.firstMatch`, first child, and first `.other` subtree
35+
- returns a partial snapshot with a warning when fallback succeeds
36+
- returns `IOS_AX_SNAPSHOT_FAILED` with an app-side flattening hint when fallback fails
37+
38+
Daemon and CLI output preserve runner warnings and runner error hints. Because the error code is not
39+
`COMMAND_FAILED`, runner-session retry and invalidation policy does not treat this typed failure as a
40+
dead transport.
41+
42+
Direct iOS selector interaction remains the first path for simple selector clicks, and `find id
43+
<value> click` now probes the runner `querySelector` path before taking a full snapshot. If the
44+
direct probe misses or has a transport fallback condition, the normal snapshot-based find path still
45+
executes.
46+
47+
## Alternatives Considered
48+
49+
- Flatten every problematic app screen: still useful when the screen must be fully inspectable, but
50+
it moves a tooling failure mode into each app codebase and does not protect other sessions.
51+
- Copy WebDriverAgent/Appium source generation: too broad for Agent Device. The immediate need is
52+
typed fast failure, partial recovery, and session preservation.
53+
- Copy Maestro's hierarchy implementation wholesale: Maestro builds a different AX model and has
54+
its own swizzled max-depth path. Agent Device keeps its existing snapshot model and adopts only the
55+
small recovery behavior that fits the runner protocol.
56+
- Always return an empty snapshot on AX failure: simple, but ambiguous. Users need to know this is an
57+
iOS AX serialization limit and that app-side flattening may be required.
58+
59+
## Consequences
60+
61+
Partial fallback snapshots are explicitly marked `truncated` and include a warning. Selectors may be
62+
less accurate against partial trees, so callers should treat screenshot as visual truth and flatten
63+
the app-side accessibility tree when full inspectability is required.
64+
65+
`IOS_AX_SNAPSHOT_FAILED` should remain a snapshot-domain error. Do not add it to generic retryable
66+
runner transport errors, and do not invalidate the runner session for it.
67+
68+
Future improvements can add a dedicated regression fixture for a minimal React Native tree that
69+
reproduces the XCTest depth failure. Until then, TypeScript tests guard warning propagation, typed
70+
error preservation, and direct `find id ... click` routing.

ios-runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -646,12 +646,25 @@ extension RunnerTests {
646646
scope: command.scope,
647647
raw: command.raw ?? false
648648
)
649-
if options.raw {
649+
do {
650+
let payload: DataPayload
651+
if options.raw {
652+
payload = try snapshotRaw(app: activeApp, options: options)
653+
} else {
654+
payload = try snapshotFast(app: activeApp, options: options)
655+
}
650656
needsPostSnapshotInteractionDelay = true
651-
return Response(ok: true, data: snapshotRaw(app: activeApp, options: options))
657+
return Response(ok: true, data: payload)
658+
} catch let failure as SnapshotCaptureFailure {
659+
return Response(
660+
ok: false,
661+
error: ErrorPayload(
662+
code: failure.code,
663+
message: failure.message,
664+
hint: failure.hint
665+
)
666+
)
652667
}
653-
needsPostSnapshotInteractionDelay = true
654-
return Response(ok: true, data: snapshotFast(app: activeApp, options: options))
655668
case .screenshot:
656669
let screenshot: XCUIScreenshot
657670
#if os(macOS)

ios-runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Models.swift

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,7 @@ struct DataPayload: Codable {
9494
let items: [String]?
9595
let nodes: [SnapshotNode]?
9696
let truncated: Bool?
97+
let warnings: [String]?
9798
let gestureStartUptimeMs: Double?
9899
let gestureEndUptimeMs: Double?
99100
let x: Double?
@@ -115,6 +116,7 @@ struct DataPayload: Codable {
115116
items: [String]? = nil,
116117
nodes: [SnapshotNode]? = nil,
117118
truncated: Bool? = nil,
119+
warnings: [String]? = nil,
118120
gestureStartUptimeMs: Double? = nil,
119121
gestureEndUptimeMs: Double? = nil,
120122
x: Double? = nil,
@@ -135,6 +137,7 @@ struct DataPayload: Codable {
135137
self.items = items
136138
self.nodes = nodes
137139
self.truncated = truncated
140+
self.warnings = warnings
138141
self.gestureStartUptimeMs = gestureStartUptimeMs
139142
self.gestureEndUptimeMs = gestureEndUptimeMs
140143
self.x = x
@@ -154,10 +157,12 @@ struct DataPayload: Codable {
154157
struct ErrorPayload: Codable {
155158
let code: String?
156159
let message: String
160+
let hint: String?
157161

158-
init(code: String? = nil, message: String) {
162+
init(code: String? = nil, message: String, hint: String? = nil) {
159163
self.code = code
160164
self.message = message
165+
self.hint = hint
161166
}
162167
}
163168

0 commit comments

Comments
 (0)