fix(ios): unify multi-touch gestures on two-finger synthesis + hinted unsupported errors#645
Conversation
Size Report
Startup median (7 runs, lower is better):
Top changed chunks:
|
… unsupported errors Make RunnerSynthesizedGesture the single iOS multi-touch engine and drop the older incompatible models: - rotateGesture now drives the two-finger XCTest synthesis path (dx=dy=0, scale=1, degrees), mirroring the pinch migration in #634. The native XCUIElement.rotate(withVelocity:) injected a single synthetic rotation that React Native's rotation recognizer did not read reliably; synthesis fixes it. velocity is ignored on iOS (kept in the wire contract for compatibility; rotation direction comes from the sign of degrees). - pinch is now synthesis on iOS and a clear UNSUPPORTED_OPERATION on tvOS/macOS. The macOS coordinate double-tap+drag heuristic (performCoordinatePinch) is removed: synthesis is iOS-only, so macOS multi-touch is reported honestly as unsupported rather than approximated. - RunnerInteractionOutcome.unsupported now carries an actionable hint, mapped to ErrorPayload.hint (#639). Every unsupported gesture/tvOS path returns a concise message plus a next-step hint (existing messages kept verbatim). Net: on iOS, pinch/rotate/transform all flow through one synthesis primitive. swipe/scroll/pan/ fling remain single-finger drags (correct, unchanged). Coverage: examples/test-app/replays/gesture-lab.ad exercises pinch + rotate against the gesture lab and asserts "pinch changed yes" / "rotate changed yes".
…s-only Follow-through for removing the macOS coordinate pinch path (the runner now returns UNSUPPORTED_OPERATION for macOS pinch): reject it at admission instead of round-tripping. - capabilities.ts: pinch now matches rotate-gesture/transform-gesture (Android + iOS simulator only); macOS dropped. Removes the now-unused isMacOsOrMobileAppleSimulator helper. - capabilities.test.ts: pinch expected unsupported on macOS and tvOS. - website/docs/docs/commands.md: pinch listed for Android + iOS simulators only (removed from the macOS app-session list); documents that iOS rotate ignores the optional velocity arg (synthesis uses a fixed duration; direction comes from the sign of degrees). Addresses PR #645 review HIGH #2 and MEDIUM #3.
thymikee
force-pushed
the
fix/ios-rotate-gesture-synthesis
branch
from
11bbacc to
7db7726
Compare
|
Review addressed + on-device verification ✅HIGH #1 — core behavior now verified. Ran That run executes HIGH #2 — macOS pinch follow-through (new commit
MEDIUM #3 — velocity. Documented in MEDIUM #4 (optional) — hint test. Deferred. As you noted in #6, MEDIUM #5 — fixed Notes: branch rebased onto current |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7db77267cd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // path (RunnerSynthesizedGesture), which is iOS-only. macOS has no multi-touch synthesis, so | ||
| // it is excluded and fails fast at admission rather than round-tripping to an unsupported | ||
| // runner. Matches rotate-gesture / transform-gesture. | ||
| supports: (device) => device.platform === 'android' || isIosMobileSimulator(device), |
There was a problem hiding this comment.
For macOS app sessions, ensureGenericCommandReady in src/daemon/request-generic-dispatch.ts checks this capability before dispatch, so gesture pinch now fails at admission with the generic pinch is not supported on this device payload and never reaches the new runner branch that returns the macOS-specific message and hint. That undercuts the hinted unsupported behavior added for removing macOS pinch; either admission needs to return the same specific hint or the command needs to be allowed through to the hinted failure path.
Useful? React with 👍 / 👎.
…t admission Removing macOS pinch from the capability matrix makes macOS pinch (and the already-excluded rotate-gesture/transform-gesture on macOS/tvOS/physical iOS) fail fast in ensureGenericCommandReady before reaching the runner. That left the runner's macOS-specific hint unreachable on the daemon path, so callers only saw the generic "<cmd> is not supported on this device". Add an optional unsupportedHint to the capability matrix and surface it at admission, so the synthesis-only gestures fail fast (no runner round-trip) AND return an actionable hint pointing to where they work (Android + iOS simulator). Applied to pinch / rotate-gesture / transform-gesture. Addresses PR #645 review (P2: route macOS pinch to the hinted failure).
|
P2 (route macOS pinch to the hinted failure) — fixed in Rather than re-open the runner round-trip, the hint now lives at admission:
So macOS pinch now fails fast at admission (no runner round-trip) and returns an actionable hint — e.g. |
1 participant
What
Make
RunnerSynthesizedGesture(the two-finger XCTest event synthesis) the single iOS multi-touch engine, and drop the older incompatible models.XCUIElement.rotate)rotateGesture→ synthesis (dx=dy=0, scale=1, degrees), mirroring the pinch migration in fix(ios): drive gesture pinch via two-finger synthesis (#629) #634. The nativeXCUIElement.rotate(withVelocity:)injected a single synthetic rotation RN's rotation recognizer didn't read reliably — same class of bug Porting to Dart and benchmarking a bit #629/fix(ios): drive gesture pinch via two-finger synthesis (#629) #634 fixed for pinch.velocityis now ignored on iOS (kept in the wire contract; direction is carried by the sign ofdegrees).performCoordinatePinchdeleted). Synthesis is iOS-only, so macOS multi-touch is now reported honestly asUNSUPPORTED_OPERATIONwith a hint, instead of a double-tap+drag approximation.RunnerInteractionOutcome.unsupportednow carries(message, hint)→ErrorPayload.hint(fix: preserve iOS AX snapshot failures #639). Every unsupported gesture/tvOS path returns a concise message + an actionable hint. Existing messages are kept verbatim (no matcher breakage); hints are additive.swipe/scroll/pan/flingare unchanged — the daemon already maps them to single-fingerdrag, which is the correct primitive.Why
Aligns with the design goal: every command returns a reliable outcome and a clear, hinted error.
rotateGesturewas the last iOS multi-touch path still on a model RN can't read; macOS pinch was a heuristic pretending to work.Verification (blocking before un-drafting)
gesture-lab.adon a simulator:gesture rotate 35 …→ "rotate changed yes" (the native path's failure mode)gesture pinch 1.25 …→ "pinch changed yes" (no regression)UNSUPPORTED_OPERATIONwith the new hintbuild-for-testinggreen