fix(actions): keyboard-occlusion guard Phase 1 — inject hideKeyboard in Maestro generator (#356)#367
Conversation
…deKeyboard injection) Generation-time hideKeyboard injection in generateMaestro() to fix flaky action replays where bottom-pinned taps land on the soft keyboard. Phase 1 covers the L3/Maestro replay path (cdp_run_action -> maestro-runner); the L2/in-runner live-tap guard is deferred to Phase 2. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…tro generator) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
dist/ is tracked and loaded by the runtime MCP bridge; the src change in prior commits was inert in installs until this rebuilt artifact ships. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 47bf54fa0f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (sel) { | ||
| if (keyboardLikelyUp) { | ||
| lines.push('# rn-dev-agent: keyboard-occlusion guard (#356)'); | ||
| lines.push('- hideKeyboard'); |
There was a problem hiding this comment.
When a generated action includes this new - hideKeyboard step, cdp_run_action can no longer use its iOS 26.x CDP/JS fallback for the same fill→submit flows: runCdpReplay() normalizes the saved YAML through normalizeSteps, which only special-cases the string waitForAnimationToEnd and otherwise throws UnsupportedStepError for string commands such as hideKeyboard (scripts/cdp-bridge/src/domain/cdp-flow-replay.ts:38-46). In the known transport-blind path where Maestro/WDA reports SELECTOR_NOT_FOUND or UNKNOWN but the selector is present in CDP, actions generated here will now return UNSUPPORTED_STEP instead of replaying; please either map hideKeyboard in the CDP replay subset or avoid emitting it for flows that may need that fallback.
Useful? React with 👍 / 👎.
✅ Device verification complete (iOS + Android) — with one runtime findingRan the injected guard sequence on real devices using the wizard screen (
Finding (filed as B223 → will open a follow-up issue)The generator fix in this PR is correct — it emits the right YAML, verified on iOS and on Android via official Maestro. But the plugin's default L3 replay engine, the third-party maestro-runner (DeviceLab.dev 1.0.9), silently no-ops
Net: iOS is fully fixed via the default path. Android is fixed via official Maestro, but not via the default maestro-runner until that runner bug is addressed. Recommended follow-up (separate PR): in Proof artifacts: |
…223) maestro-runner v1.0.9 silently no-ops hideKeyboard on Android (reports pass in ~5ms, mInputShown stays true), defeating the keyboard-occlusion guard. chooseMaestroDispatch now prefers the official Maestro CLI for Android flows that contain hideKeyboard (verified to honor it on-device), and surfaces a degradedReason warning when the CLI is unavailable. iOS is unaffected (maestro-runner honors hideKeyboard there). Device-verified: official Maestro dismisses the Android keyboard and the bottom-pinned tap lands. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…+ changeset Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Update: Android
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 525ac0fbe7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return commands.some( | ||
| (c) => | ||
| c === 'hideKeyboard' || | ||
| (typeof c === 'object' && c !== null && 'hideKeyboard' in (c as Record<string, unknown>)), | ||
| ); |
There was a problem hiding this comment.
When a validated Android flow hides the keyboard inside an inline or conditional runFlow, for example { runFlow: { when, commands: ['hideKeyboard', ...] } }, this shallow some returns false because it only checks the root command object. parseAndValidateFlow() preserves those nested commands for inline/conditional runFlow cases (see scripts/cdp-bridge/src/domain/maestro-validator.ts:387-399), so the flow is routed to maestro-runner and Android hideKeyboard is no-oped; please recurse into runFlow.commands when computing flowHasHideKeyboard.
Useful? React with 👍 / 👎.
| // Android (`mInputShown=false` after). iOS maestro-runner honors hideKeyboard, | ||
| // so this only applies to Android. | ||
| const needsOfficialForKeyboard = | ||
| inputs.platform === 'android' && inputs.flowHasHideKeyboard === true; |
There was a problem hiding this comment.
When an Android flow containing the newly emitted top-level - hideKeyboard is run through other supported flow runners, those paths still call chooseMaestroDispatch({ platform }) without the new flag (scripts/cdp-bridge/src/tools/maestro-test-all.ts:77 and scripts/cdp-bridge/src/maestro-invoke.ts:60), so this strict check stays false and the dispatcher selects maestro-runner—the Android path the new comments say no-ops hideKeyboard. Please have every flow-running path pass the parsed flag, or move detection into dispatch, otherwise Android regression suites and inline Maestro flows do not get the keyboard-occlusion fix.
Useful? React with 👍 / 👎.
…223) The batch replay surface (maestro_test_all) chose dispatch once before its per-flow loop, so Android replays of saved actions containing hideKeyboard still hit maestro-runner's no-op. Now re-routes per flow to the official Maestro CLI when a flow uses hideKeyboard on Android, surfacing a degraded warning if the CLI is unavailable. Mirrors the maestro_run routing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Final whole-branch review (opus) flagged that maestro_test_all (batch replay of saved actions) chose its runner once before the per-flow loop — so Android batch replays of hideKeyboard flows would still hit maestro-runner's no-op. Fixed in c95f0d3: it now re-routes per flow to the official Maestro CLI when a flow uses hideKeyboard on Android (mirroring maestro_run), with the same degraded-warning fallback. Full suite 2429/2429; lint+format clean; dist committed. Both action-replay surfaces (maestro_run and maestro_test_all) are now covered on Android. |
1 participant
Closes #356 (Phase 1).
Problem
When a learned-action replay taps a bottom-pinned control (submit/continue) while the soft keyboard is up, the tap lands on the keyboard instead of the button, so the next screen is never reached. The #300 session flagged this as the single biggest source of flaky replays, on both iOS and Android. The reporter's proven manual workaround was
- hideKeyboardbefore each such tap.Why this is Phase 1 (and what it is NOT)
The bug spans two independent device-control engines that do not share a tap path:
cdp_run_action→maestro_run)runNative()device_press/device_batchrunNative()→ in-tree runnerA guard in our native runner is a no-op for action replays (they shell out to the external maestro-runner). So Phase 1 fixes the L3/Maestro replay path — the headline symptom. The L2/in-runner frame-precise guard for live taps is deferred to Phase 2 (its design decisions are recorded in the spec).
Approach
A pure, order-only transform inside
generateMaestro()— the single chokepoint every generated/saved Maestro flow passes through. It tracks one boolean (keyboardLikelyUp): atypeevent raises it; before atap/long_pressit emits a- hideKeyboardstep (with an audit comment) when up, then lowers it;navigateresets it;submitdoes not.Safe by construction:
hideKeyboardis already allowlisted (maestro-validator) and is a no-op when no keyboard is showing, so over-injection costs only a few hundred ms.Emitted shape
Scope
generateMaestro(coverscdp_record_test_generate+save-as-action).Testing
dist/omission, now fixed).⏳ Remaining pre-merge gate
On-device verification on iOS + Android (per the issue): record a fill→bottom-pinned-submit flow, replay, confirm the submit reliably reaches the next screen where it was previously flaky. Not yet run (environment setup) — will complete before merge.
Docs
Spec + TDD plan committed under
docs/superpowers/{specs,plans}/2026-06-20-356-*.🤖 Generated with Claude Code