fix: handle iOS keyboard Done dismiss controls

Author: thymikee

AGENTS.md

@@ -103,6 +103,7 @@ Command-only flags (like `find --first`) that don't flow to the platform layer o
 ## Hard Rules
 - Use `runCmd`/`runCmdSync` from `src/utils/exec.ts` for process execution.
 - Use daemon session flow for interactions (`open` before interactions, `close` after).
+- Use `keyboard dismiss` for iOS keyboard dismissal; it may tap safe native controls such as `Done` but must not fall back to system back navigation.
 - Do not remove shared snapshot/session model behavior without full migration.
 - Command/device support must come from `src/core/capabilities.ts`.
 - Apple-family target changes must keep `src/utils/device.ts`, `src/core/capabilities.ts`, `src/core/dispatch-resolve.ts`, `src/platforms/ios/devices.ts`, and `src/platforms/ios/runner-xctestrun.ts` in sync.

ios-runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift

@@ -285,20 +285,44 @@ extension RunnerTests {
   }
 
   private func tapKeyboardDismissControl(app: XCUIApplication) -> Bool {
-    for label in ["Hide keyboard", "Dismiss keyboard"] {
+    let keyboardFrame = app.keyboards.firstMatch.frame
+    for label in ["Hide keyboard", "Dismiss keyboard", "Done"] {
       let candidates = [
         app.keyboards.buttons[label],
         app.keyboards.keys[label],
-        app.toolbars.buttons[label],
+        app.keyboards.toolbars.buttons[label],
       ]
       if let hittable = candidates.first(where: { $0.exists && $0.isHittable }) {
         hittable.tap()
         return true
       }
+
+      let toolbarButtonPredicate = NSPredicate(
+        format: "label == %@ OR identifier == %@",
+        label,
+        label
+      )
+      let toolbarButtons = app.toolbars.buttons
+        .matching(toolbarButtonPredicate)
+        .allElementsBoundByIndex
+      if let hittable = toolbarButtons.first(where: {
+        $0.exists && $0.isHittable && isKeyboardAccessoryControl($0, keyboardFrame: keyboardFrame)
+      }) {
+        hittable.tap()
+        return true
+      }
     }
     return false
   }
 
+  private func isKeyboardAccessoryControl(_ element: XCUIElement, keyboardFrame: CGRect) -> Bool {
+    let frame = element.frame
+    guard !frame.isEmpty && !keyboardFrame.isEmpty else {
+      return false
+    }
+    return frame.intersects(keyboardFrame) || abs(frame.maxY - keyboardFrame.minY) <= 80
+  }
+
   private func moveCaretToEnd(element: XCUIElement) {
     let frame = element.frame
     guard !frame.isEmpty else {

skills/agent-device/SKILL.md

@@ -31,4 +31,4 @@ agent-device help dogfood
 
 Default loop: `open -> snapshot/-i -> get/is/find or press/fill/scroll/wait -> verify -> close`.
 
-Keep refs current, prefer selectors/refs over coordinates, use `fill` to replace text, and use `back` for app-owned navigation. Let `help workflow` provide the exact command shapes.
+Keep refs current, prefer selectors/refs over coordinates, use `fill` to replace text, and use `back` for app-owned navigation. On iOS, use `keyboard dismiss` before manually pressing visible keyboard controls such as `Done`; fall back only if the command reports unsupported. Let `help workflow` provide the exact command shapes.

src/utils/command-schema.ts

@@ -283,7 +283,7 @@ Text entry:
     agent-device fill 'id="field-email"' "qa@example.com"
     agent-device press 'id="product-note"'
     agent-device type "Handle with care" --delay-ms 80
-  Debounced field with no result selector: agent-device wait 1000. Keyboard read-only: keyboard status/get. Blocked control: keyboard dismiss.
+  Debounced field with no result selector: agent-device wait 1000. Keyboard read-only: keyboard status/get. Blocked control: keyboard dismiss. On iOS, prefer keyboard dismiss before manually pressing visible Done; the runner can use safe native keyboard controls and still reports unsupported layouts explicitly.
   Search-as-you-type fields on iOS can drop characters when driven too fast; use --delay-ms on fill/type before trying clipboard paste.
   iOS Allow Paste prompt cannot be exercised under XCUITest. To test paste-driven app behavior, prefill first with agent-device clipboard write "some text"; test the system prompt manually.
   Android non-ASCII can fail on some system images. Try fill/type normally; agent-device uses safer fallbacks. If the shell reports unsupported non-ASCII input, configure a trusted ADB keyboard IME outside the command plan and restore the previous IME afterward.

test/skillgym/README.md

@@ -16,7 +16,7 @@ The included suite focuses on the first two layers so it stays stable and CI-saf
 
 - `../../examples/test-app/`: minimal Expo SDK 55 fixture app for broad UI coverage
 - `skillgym.config.ts`: starter config that runs Codex and Claude Haiku against this repo
-- `suites/agent-device-smoke-suite.ts`: 66-case suite for skill routing, fixture-aware planning, and skill-guidance regressions
+- `suites/agent-device-smoke-suite.ts`: 68-case suite for skill routing, fixture-aware planning, and skill-guidance regressions
 
 ## Current coverage
 

test/skillgym/suites/agent-device-smoke-suite.ts

@@ -329,6 +329,19 @@ const FIXTURE_SMOKE_CASES: TestCase[] = [
     outputs: [/field-name/i, /keyboard dismiss/i],
     forbiddenOutputs: [commandPattern('back')],
   }),
+  makeCase({
+    id: 'form-keyboard-dismiss-ios-done-control',
+    contract: [
+      'Platform: iOS',
+      'App name: Agent Device Tester',
+      'Current screen: Checkout form tab',
+      'testID=field-name',
+      'The focused field shows an iOS keyboard toolbar with a visible Done control',
+    ],
+    task: 'Plan the commands to focus the Full name field and dismiss the iOS keyboard without manually pressing Done.',
+    outputs: [/field-name/i, /keyboard dismiss/i],
+    forbiddenOutputs: [commandPattern('back'), /press\s+.*Done/i, /click\s+.*Done/i],
+  }),
   makeCase({
     id: 'form-reset',
     contract: [

website/docs/docs/commands.md

@@ -534,7 +534,7 @@ agent-device keyboard dismiss
 ```
 
 - `keyboard status` (or `keyboard get`) returns keyboard visibility and best-effort input type classification on Android.
-- `keyboard dismiss` attempts a non-navigation keyboard dismissal on Android and a native dismiss gesture/control on iOS, then confirms the keyboard is hidden.
+- `keyboard dismiss` attempts a non-navigation keyboard dismissal on Android and a native dismiss gesture/control on iOS, including common safe controls such as a keyboard toolbar `Done` button, then confirms the keyboard is hidden.
 - If the keyboard remains visible after the platform-native dismiss path, the command returns an explicit `UNSUPPORTED_OPERATION` error instead of falling back to back navigation.
 - Works with active sessions and explicit selectors (`--platform`, `--device`, `--udid`, `--serial`).
 - `keyboard status|get` is supported on Android emulator/device.

website/docs/docs/introduction.md

@@ -36,7 +36,7 @@ For agent-oriented operating guidance, start with `agent-device help` or `agent-
   - Physical-device recording defaults to 15 FPS and supports `--fps` caps.
   - `record start --quality <5-10>` scales recording resolution from 50% through native resolution; omitting it keeps native/current resolution.
 - Android supports the same core interaction set, plus `rotate`, `push` notification simulation, `clipboard read/write`, and `keyboard status|get|dismiss`.
-- iOS supports `keyboard dismiss` through the XCTest runner when the on-screen keyboard is visible.
+- iOS supports `keyboard dismiss` through the XCTest runner when the on-screen keyboard is visible, including common native controls such as keyboard toolbar `Done`.
 - App-event triggers are available on iOS and Android through app-defined deep-link hooks (`trigger-app-event`), using active session context or explicit device selectors.
 
 ## Architecture (high level)