iOS: HealthKit cross-process sheet fix + O(N) hierarchy walk#3313
Open
Leland-Takamine wants to merge 6 commits into
Open
iOS: HealthKit cross-process sheet fix + O(N) hierarchy walk#3313Leland-Takamine wants to merge 6 commits into
Leland-Takamine wants to merge 6 commits into
Conversation
When a host app snapshot merges a remote window (e.g. HealthKit authorization), descendant frames stay in window-local coords while taps use screen coords. Traverse the XCTest snapshot, detect windowContextID boundaries whose subtree contains remote elements, and apply visibleFrame − frame origin as an inherited offset. AXElement accepts an optional frame override from the walker. Update the failing HealthKit e2e flow to assert dismissal via "Turn On All" rather than ambiguous "Health Access" text. Rebuild checked-in Simulator driver zips. Co-authored-by: Cursor <cursoragent@cursor.com>
The cross-process boundary fix called snapshot.dictionaryRepresentation at every recursion level. dictionaryRepresentation eagerly serializes the entire subtree per call, so the walk was O(N*D) (worst case O(N^2)) on deep trees. Benchmarked against a HealthKit screen on iOS 26.1 sim: p50 /viewHierarchy latency was 2673ms vs 1750ms on main (+52.7%). Walk the snapshot tree directly and read each node's attributes from the snapshot's own properties (label, frame, identifier, etc.) plus KVC for the two non-public ones (windowContextID, displayID). The cross-process heuristic is unchanged. Restores per-call latency to main parity (p50 1717ms).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
iOS cross-process system sheets (HealthKit, share sheet, photo picker, etc.) are stitched into the host app's snapshot, but the foreign process reports frames in its own window's local coordinates rather than screen coordinates. Maestro's coordinate-based tap was landing in the wrong place — most visibly, the HealthKit authorization sheet was untappable.
What changed
Cross-process frame correction in the iOS XCTest runner.
ViewHierarchyHandlernow walks theXCUIElementSnapshottree directly and, at each cross-process window boundary, accumulates a coordinate-system offset that's inherited through the subtree. Descendant frames are rewritten to screen coordinates before being shipped to the host.Boundary detection requires three signals to align before the correction is applied:
windowContextIDtransition between parent and descendant (both non-zero)isRemote = 1visibleFrameis finiteRequiring the remote signal in addition to the windowContextID transition guards against benign in-process boundaries (e.g.
UITextEffectsWindow) where a non-zerovisibleFramedelta is ordinary clipping rather than a coordinate-system mismatch.The offset itself is
visibleFrame.origin − frame.originat the boundary node, accumulated into descendants.Attributes are read from the snapshot's own properties (
label,frame,identifier, sizes, etc.) rather than viasnapshot.dictionaryRepresentation, because that call eagerly serializes the entire subtree and was making the walk O(N·D) when applied per-node. The two private attributes (windowContextID,displayID) go through KVC, matching the pattern already used forisRemote/visibleFrame.Demo app additions
Health Accessbutton on the demo app home screen exercises the HealthKit authorization sheet via aMethodChannelto the iOS side, which callsHKHealthStore.requestAuthorization(toShare:read:)against the standard quantity types.e2e/demo_app/.maestro/issues/fail_health_access.yamlreproduces the bug: launches the app, opens the HealthKit sheet, taps "Turn On All", asserts the toggles flipped, taps "Allow". Taggedfailing-when-implementedoriginally; passes once the fix is applied.Perf
Direct
/viewHierarchybenchmark against a real-world application with a very large view hierarchy (~1.3 MB JSON), iOS 26.1 sim, 50 samples (p50 / p90 / p99 ms):Parity with main while correctly handling cross-process sheets.