Skip to content

Commit 683a132

Browse files
Lykhoydaclaude
andauthored
Story 04: shared two-tier settle engine + runner capability flags (#385) (#442)
* docs(plan): Story 04 settle-engine TDD implementation plan (#385) Amendments applied from the multi-LLM plan review (2026-07-04; Claude-Opus-only — Gemini tier-locked, Antigravity timed out, Codex quota-paused — plus codex-pair per-edit review): - M1: device_fill corrective retypes opt out of settle (nativeSettle read-back is their stability check; prevents up-to-5-settles-per-fill amplification) - M2: device_fill pins @ref→coords once up front (--at-x/--at-y builder option) so the pre-tap's settle ref-map refresh cannot retarget the fill - S1: device_batch per-step settle budget capped at 2500ms - S2: end-to-end wiring test gets _setActiveSessionForTest (no xcrun shelling) - S3: Android settle probes use thin typed helpers with a pinned host port (no per-poll startAndroidRunner ensure; endpoint assumption checked) - C1-C5 + iOS activation-preamble rationale documented as accepted trade-offs - codex-pair: capability sync-test regexes target the actual health payload; settleTimeoutMs documented as budget knob (not opt-out); Number.isFinite pin guard; literal NUL/SOH delimiters replaced with escapes; primary fill threads settleOpts; Task 7 commit stages agent-device-wrapper.ts Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(rn-fast-runner): isScreenStatic probe + SCREEN_STATIC capability (#385) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(rn-android-runner): isWindowUpdating probe + WINDOW_UPDATE capability (#385) Also hardens the Task-1 iOS surface per codex-pair review: lifecycle classification moved to a free function (pure-logic testable without the XCUITest harness), guard-let in the Swift surface pin, clamped timeoutMs (0..2000ms) on the Android window gate, block-comment-aware source-sync regexes. Both native targets compile-verified. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(settle): snapshot-node normalization + SHA-256 hashing (#385) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(settle): JSON-encode hash tuples (delimiter-collision proof) + bucket-boundary tests (#385) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(settle): capability-switched two-tier waitForSettle engine (#385) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(settle): runner-client probe verbs, capability caches, production probes (#385) Capability caches are runner-lifetime-scoped (cleared on state teardown and live-port probe failures, never by foreign-port probes); Android settle probes are thin typed helpers that skip the per-dispatch ensure path and pin the forwarded host port captured at probe-builder construction — a mid-settle state change degrades to null rather than posting to the wrong runner. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(settle): wire waitForSettle into runNative after every mutating verb (#385) attachMeta generalizes attachMetaNote with timings_ms deep-merge; settleAfterMutation is advisory by contract (never fails the action); per-call settleTimeoutMs on device_press/device_fill (zod 500..30000, engine re-clamps to SETTLE_MAX_BUDGET_MS as defense-in-depth); the iOS health probe now routes through fetchImpl so the fetch test seam covers it (found when the e2e wiring test's real-network probe classified the fake runner stale and the reap SIGTERM'd the test process). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(settle): device_fill settle-aware delay, coord pinning, retype settle opt-out (#385) M2: refCenter resolved ONCE before the pre-tap; press + fill + retypes all dispatch with --at-x/--at-y pins so the settle's ref-map refresh can never retarget this fill (both arg builders gain the pin option, Number.isFinite guarded). M1: corrective retypes pass settle:{enabled:false} — their stability check is the existing nativeSettle CDP read-back. The B122 Pressable-resolution fallback keeps the same settle-aware delay. Fixed FOCUS_DELAY_MS survives only as the no-settle fallback. Also hardens the e2e wiring test with a disposable child pid (a reap regression can no longer SIGTERM the test process). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(settle): device_batch settles between steps; per-step settle:false hatch (#385) Batch steps use a 2500ms settle budget (a 10-step walk at the 6000ms default could take ~60s on an animating screen); the inter-step delayMs zod default(300) becomes optional so the settle-aware 0ms default can engage — explicit delayMs still always wins. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style: oxfmt normalization of story-04 settle files (#385) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(settle): changeset + story status for Story 04 (#385) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(settle): changeset bumps rn-dev-agent-cdp too; story text matches as-implemented behavior (#385) Per codex-pair post-hoc review: bridge-touching changesets bump both packages (convention: #422-424 batch); the story's fill/budget/impact claims now state the settle-less 150ms fallback, the Story-05 deferral of cross-tool budget carryover, and the native-verbs-only scope. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(settle): pin B122 Pressable-fallback fill coords; carry settle telemetry through verified-native path (#385) Post-hoc Codex holistic review (IMPORTANT): the Pressable-resolution fallback tapped the inner input with settle enabled, letting the settle's ref-map refresh renumber positional refs before the fill re-resolved the same @en — the exact retargeting hazard the primary path pins against. Now pinned identically (tap by coords, fill with --at-x/--at-y + settleOpts). Advisories: verified-native fill returns carry the primary fill's meta.settle/timings_ms.settle instead of dropping them; settle-hash documents the value/focused fallback-tier limitation; story acceptance text now matches the documented settle-less fallback. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(settle): Pressable-fallback re-wrap preserves envelope meta; criterion names the Android clipboard sleep exception (#385) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(settle): acceptance-criterion clipboard-sleep exception cites tracking issue #443 (#385) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent 945d3cd commit 683a132

31 files changed

Lines changed: 3600 additions & 88 deletions
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
'rn-dev-agent-cdp': minor
3+
'rn-dev-agent-plugin': minor
4+
---
5+
6+
Story 04 (#385): shared two-tier settle engine. Every mutating device_* verb now waits for the UI to actually stabilize instead of relying on fixed sleeps: Android gates on a new `isWindowUpdating` runner probe (capability `WINDOW_UPDATE`) then falls back to snapshot-hash equality polling; iOS polls a new on-runner `isScreenStatic` SHA-256 screenshot compare (capability `SCREEN_STATIC`, Maestro's 3s screen-settle budget) with the same snapshot-hash fallback. Results surface `meta.settle: {method, settled}` + `meta.timings_ms.settle`. `device_fill` drops its fixed 150ms focus delay when settle ran and pins its target coordinates once up front (`--at-x/--at-y`) so the settle's ref-map refresh can never retarget the fill mid-call; its corrective retypes skip settle (their stability check is the CDP read-back). `device_batch` settles between steps by default at a batch-scoped 2500ms budget (per-step `settle: false` escape hatch) and its blanket 300ms inter-step delay defaults to 0 while settle is on. Legacy runner artifacts (no new capabilities) transparently degrade to snapshot polling — no rebuild required, the new verbs are deliberately NOT in the required-command gate. Opt out globally with `RN_SETTLE=0` or per batch step with `settle: false`; tune the per-call budget with `settleTimeoutMs` (a budget knob, not a disable switch). A perpetually-animating screen settles via hierarchy stability or returns `method: 'timeout'` at budget — bounded, never hanging.

docs/stories/04-settle-engine.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
# Story 04 — Shared two-tier settle engine + capability flags
22

3-
**Status:** Proposed (2026-07-02)
3+
**Status:** Implemented (2026-07-04, #385)
44
**Epic:** [Maestro adoption](README.md)
5-
**Impact:** Replaces fixed sleeps (`FOCUS_DELAY_MS=150` etc.) with a principled "UI is stable" invariant applied after every mutating action, on both platforms, from one implementation
5+
**Impact:** Replaces fixed sleeps (`FOCUS_DELAY_MS=150` etc.) with a principled "UI is stable" invariant applied after every mutating native `device_*` verb, on both platforms, from one implementation (`cdp_interact` JS-path settling is a deferred follow-up — see Open questions)
66
**Effort:** M
77
**Depends on:** Story 02 (capability negotiation via `/health`); pairs with Story 03 (bypass makes *us* responsible for settling)
88

@@ -41,9 +41,9 @@ async function waitForSettle(opts: {platform, appId, budgetMs?, initialSnapshotH
4141
### Wiring
4242

4343
- `runNative()` already knows which verbs mutate (`SNAPSHOT_MUTATING_VERBS`, `agent-device-wrapper.ts:265-277`it uses them to dirty the snapshot cache). After a mutating verb succeeds, call `waitForSettle` and attach `meta.timings_ms.settle` + `meta.settle.method`.
44-
- `device_fill`: replace the fixed `FOCUS_DELAY_MS=150` tapfill delay with `waitForSettle`; keep read-back verification unchanged (it verifies *content*, settle verifies *stability*complementary).
45-
- `device_batch`: settle between steps by default; per-step `settle: false` escape hatch.
46-
- Timeout budget arithmetic: adopt Maestro's `adjustedToLatestInteraction` trick (`Orchestra.kt:1646-1649`) — a slow settle eats into the *next* lookup's budget rather than stacking timeouts.
44+
- `device_fill`: skip the fixed `FOCUS_DELAY_MS=150` tapfill delay whenever the pre-tap's envelope carries `meta.settle` (the 150ms constant survives ONLY as the settle-less fallback — `RN_SETTLE=0` or a legacy-path failure); keep read-back verification unchanged (it verifies *content*, settle verifies *stability* — complementary). As implemented (#385), the fill also pins its target coords once up front (`--at-x/--at-y`) so the settle's ref-map refresh cannot retarget it mid-call, and corrective retypes skip settle (their stability check is the read-back).
45+
- `device_batch`: settle between steps by default (batch-scoped 2500ms budget); per-step `settle: false` escape hatch.
46+
- Timeout budget arithmetic: adopt Maestro's `adjustedToLatestInteraction` trick (`Orchestra.kt:1646-1649`) — a slow settle eats into the *next* lookup's budget rather than stacking timeouts. **Deferred to Story 05** (#385 shares one budget across tiers *within* a settle; cross-tool carryover needs the interaction clock Story 05's re-resolution loop introduces).
4747
- Env `RN_SETTLE=0` global opt-out; per-call `settleTimeoutMs` override.
4848

4949
## Implementation steps
@@ -55,7 +55,7 @@ async function waitForSettle(opts: {platform, appId, budgetMs?, initialSnapshotH
5555

5656
## Acceptance criteria
5757

58-
- After `device_press` on a button that opens a new screen: the tool returns only once the target screen is static; `meta.settle.method` recorded; no fixed sleeps remain in the press/fill paths (grep-enforced).
58+
- After `device_press` on a button that opens a new screen: the tool returns only once the target screen is static; `meta.settle.method` recorded; no fixed sleeps execute on the settle-enabled press/fill paths. Two documented exceptions: the 150ms focus constant survives as the settle-less fallback (`RN_SETTLE=0` or legacy-path failure), and the Android clipboard-workaround path keeps its 300ms sleep (upstream of the dispatch choke pointtracked in #443 item 4).
5959
- On a static screen, settle overhead150 ms on Android (window-gate short-circuit) and~250 ms on iOS (one screen-static probe).
6060
- On the Reanimated fixture (Story 03): settle returns `method: 'timeout'` at budget rather than hanginga perpetually-animating screen must degrade gracefully, matching Maestro's bounded loops.
6161
- `device_batch` TaskWizard walk passes with zero step-transition races across 10 consecutive runs (currently flaky without manual waits).

0 commit comments

Comments
 (0)