Skip to content

Commit 98d3fb7

Browse files
Lykhoydaclaude
andauthored
feat(resolver): Phase 1 — RNTL-style discovery resolver (byRole/byText/byPlaceholder + selector bundle) (#362)
* docs(spec): RNTL-style selector resolver + durable Maestro projector (design) Two-phase design: an RNTL-style discovery ladder (byRole/byText/byPlaceholder) on the live fiber tree emitting a selector bundle, projected to Maestro YAML via a fail-closed, CONTAINS-aware, TS-owned-gated fallback ladder, with bundle-aware self-heal. Grounded by a Codex adversarial debate (approach selection) + a 6-agent hardening pass across rn-dev-agent / RNTL / maestro-runner. Captures the verified CONTAINS-not-EXACT native-match verdict and two Section-1 corrections: the maestro-runner selector gate is advisory-only (so the hard gate is re-homed into the TS projector), and the RNTL helpers must be ported against a fiber->host adapter rather than copied verbatim. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(plan): RNTL selector resolver Phase 1 (discovery ladder) — TDD plan 9 bite-sized TDD tasks: Task 0 vm test harness, then __match / __hostKind / __role / __accessibleName / __hidden ports, fail-closed truncation, resolveLadder + interact() routing, and anchor capture — implementing the precision half of the selector-resolver design spec. Each task is failing-test (vm + buildFiber) -> ES5 port inside the INJECTED_HELPERS IIFE -> passing test -> commit, with source-drift guards. Drafted via an 8-agent grounding + drafting workflow against real RNTL v14 and injected-helpers source; task-number cross-refs and the bounds-null caveat reconciled in self-review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(selector): shared injected-helpers vm test harness (Phase 1 Task 0) * feat(resolver): port RNTL matches() + normalizer to __RN_AGENT.__match Single-matcher form ({value,exact?} | {regexSource,regexFlags?}); trim+collapse normalizer that does NOT lowercase (case-insensitivity lives in the non-exact compare). Bumps HELPERS_VERSION 26->27. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(cdp): add __RN_AGENT.__hostKind live-fiber host classifier Port RNTL host-component-names (isHostText/TextInput/Image/Switch/ ScrollView/Modal) into a single hostKind(fiber) that maps a host name (string fiber.type or fiber.type.displayName/name) to one of text|textinput|image|switch|scrollview|modal|null. Name lists widened to native view names (RCTSinglelineTextInputView, RCTImageView, RCTModalHostView, ...) since live fibers expose the platform view name. Placed at IIFE top level (not inside getTree); returns null for Views, user components, text nodes, and null types. HELPERS_VERSION already at 27 from Task 1. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(selector): make #321 HELPERS_VERSION guards value-agnostic (>=26) Phase 1 bumps HELPERS_VERSION past 26; the two #321 guards pinned it to exactly 26 and broke on the first bump. Assert the >=26 baseline instead so feature branches bump freely without false regressions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(injected-helpers): port RNTL getRole to __RN_AGENT.__role Add normalizeRole + __role (explicit role → accessibilityRole image→img → host Text → none), reusing __hostKind from Task 2. Does not reuse the digest inferRole (which defaults Pressable to button); divergence pinned by test. Bump HELPERS_VERSION 26→27. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(cdp-bridge): port computeAccessibleName to __accessibleName Port RNTL computeAccessibleName + computeAriaLabel + getAriaLabelledByIds + joinAccessibleNameParts to ES5 on __RN_AGENT. labelledBy nativeID refs are resolved to the referenced node's plain TEXT CONTENT (port of RNTL getTextContent via __refTextContent), NOT by recursively computing its accessible name — this matches RNTL's computeAriaLabel and makes a malformed labelledBy cycle (A->B->A) safe (no stack overflow). The normal child-name recursion in computeAccessibleName is retained. Inline host-text parts join with '' (so 'Sign'+'In' -> 'SignIn'), otherwise ' '. TextInput placeholder is the name only at root. HELPERS_VERSION unchanged (27). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(a11y): port isHiddenFromAccessibility to __RN_AGENT.__hidden Climbs fiber.return, flattens memoizedProps.style arrays manually (no StyleSheet.flatten in-page), and treats aria-hidden / accessibilityElementsHidden / importantForAccessibility=no-hide-descendants / display:none / aria-modal host siblings as hidden. opacity:0 is not hidden. HELPERS_VERSION stays at 27 (already bumped on this branch). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(injected): fail-closed truncation in interact() findFiber Replace the silent `if (findCount > 8000) return;` cap with a rootsSeeded-scaled node budget (Math.min(40000, 8000*roots)) plus a 3s wall-clock guard, mirroring the salient-digest budget. On trip, set a findTruncated flag and short-circuit interact() to return {error:"Resolution truncated", truncated:true, scanned:<n>, hint:...} BEFORE any tier[0] pick, "Component not found" branch, or onPress fire — so a partial scan can never trigger a false action. HELPERS_VERSION stays at 27 (single branch bump). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(resolver): resolveLadder + interact() ladder routing (Task 7) Add __RN_AGENT.resolveLadder(specJson) composing __match/__role/ __accessibleName/__hidden/hostKind into byRole/byText/byPlaceholder predicates. Collect-all across renderers: 0 -> Component not found, >1 -> Ambiguous component match (count + descriptors), 1 -> bundle. Hidden excluded unless includeHidden; bundle.bounds is null in Phase 1. interact() routes role/name/text/placeholder specs (no testID/ accessibilityLabel) through the resolver and presses the found fiber or its nearest onPress ancestor, via the fiber-returning twin __resolveLadderFiber. Legacy testID/accessibilityLabel paths and Task 6 fail-closed truncation untouched. HELPERS_VERSION stays 27 (no bump). Also widen the find-active-renderer-migration B145 guard window (3000 -> 8000 chars): a pre-existing brittle source-position assertion that Task 6 already pushed out of range; the guarded findFiber-in- forEachRootFiber behavior is intact, only its byte offset moved. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(plan): execution-time refinements (bounds-null note; cross-ref fixes; HELPERS_VERSION single-branch-bump policy) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(cdp-bridge): capture ancestor anchors into resolveLadder bundle Add __collectAnchors — a bounded (depth 8) fiber.return ancestor walk mirroring the setFieldValue ancestor walk — recording nearest-first {testID, text, relation: childOf, depth, provenance} entries for any ancestor with a testID/nativeID or explicit accessibility label (__ariaLabel only, no recursive child traversal so bare host Text nodes are skipped). provenance is authored-testID for testID/nativeID ancestors, else text. Populate bundle.anchors in resolveLadder. Expose __collectAnchors on the __RN_AGENT public surface. Full suite: 2437/2437 pass, 0 fail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(build): rebuild committed dist with Task 8 anchors (stale-mirror fix) Task 8 changed src/injected-helpers.ts but did not rebuild the tracked dist/injected-helpers.js. dist is the shipped artifact (imported by cdp/setup.ts; no build hook), so the committed resolver silently omitted bundle.anchors; CI masked it by rebuilding before tests. Regenerated dist so the committed artifact matches src. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(test): apply oxfmt to Phase 1 selector-resolver test files Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(resolver): matchDeepestOnly — collapse composite+host fiber pairs (live-device bug) Live-simulator testing revealed every RN text/input element matched TWICE in resolveLadder — once as the composite fiber (Text/TextInput) and once as its host child (RCTText/RCTSinglelineTextInputView), both passing hostKind — so byText/byPlaceholder always fail-closed as Ambiguous on a real device. The vm tests missed it because buildFiber made one node per element. Add __deepestOnly (RNTL matchDeepestOnly parity): drop any match that is an ancestor of another match, keeping the deepest; distinct siblings stay ambiguous. Applied in both resolveLadder and __resolveLadderFiber. Bump HELPERS_VERSION 27->28. Verified live on the running test app: byText "Go to Dashboard" and byPlaceholder "Add a task..." now resolve uniquely with correct anchors. Full suite 2495/2495. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(changeset): rn-dev-agent-cdp minor — Phase 1 discovery resolver Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(changeset): also bump rn-dev-agent-plugin (marketplace propagation) The 'Require changeset' gate (per the #361/#363 post-mortem) requires shippable src changes to bump the plugin manifest too, else the change reaches main but not marketplace installs via /plugin update. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(resolver): address Codex review — tool surface, fail-closed truncation, byText content, non-press guard PR #362 review (chatgpt-codex-connector) — 4 P2 findings, all verified + fixed: - #1 cdp_interact now accepts the ladder selectors (role / text / placeholder / exact / includeHidden): relax the host guard, forward the fields, add them to the zod schema. The resolver is reachable via the supported tool, not only via raw eval. - #2 resolveLadder / __resolveLadderFiber had their own silent 8000-fiber cap; a duplicate past the cap could leave matched.length===1 and press the wrong element. Now fail-closed (truncated:true) with a rootsSeeded-scaled budget + wall-clock guard, matching the legacy findFiber path. - #3 byText matched __accessibleName (accessibilityLabel precedence) instead of the visible text content — now uses __refTextContent (getTextContent port); accessible names stay for byRole/name; bundle.text is text content too. - #4 the ladder interact() branch pressed regardless of the requested action — now fails closed for any action other than press. Verified live on the running test app; full suite 2502/2502. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(resolver): Codex review round 2 — Android TextInput host + ladder testID matcher PR #362 (chatgpt-codex-connector) — 2 P2 findings: - #5 __hostKind now recognizes Android's `AndroidTextInput` host name; without it byPlaceholder/byText returned null for every TextInput in Android sessions. - #6 resolveLadder / __resolveLadderFiber now match a `testID` spec — the spec already accepted testID but had no matcher, so resolveLadder({testID:'x'}) returned Component not found for mounted ids. Bump HELPERS_VERSION 28->29 (the injected surface changed across the review fixes; a same-version re-inject is skipped by the __v freshness guard). Verified live (testID + byText resolve on the running app) + full suite 2507/2507. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(resolver): Codex review round 3 — limit matchDeepestOnly to same-element duplicates #7: __deepestOnly blanket-dropped every ancestor match, so two DISTINCT nested components both matching a selector (e.g. an outer card button + an inner button both named "Settings") collapsed to the inner one and got silently pressed instead of failing closed as Ambiguous. Now drop a match's nearest matching ancestor only when they are the SAME element: (1) composite+host pair (the composite wrapper of a host match), or (2) the same testID/nativeID (one element whose id propagates across nested fibers, e.g. a tab button). Distinct nested matches stay Ambiguous. The same-testID arm also keeps resolveLadder({testID}) at found:1 (a tab's id spans ~7 fibers on-device). Bump HELPERS_VERSION 29->30. Verified live (testID / byText / byPlaceholder resolve; nested stays ambiguous); full suite 2509/2509. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(resolver): Codex review round 4 — byRole RNTL parity (accessible opt-out + role normalization) #8: byRole now respects the explicit accessible={false} opt-out (RNTL isAccessibilityElement) — an opted-out duplicate (e.g. an offscreen Pressable kept mounted) no longer causes false ambiguity or gets pressed. #9: the REQUESTED role is now normalized (normalizeRole(spec.role)) like the element side, so byRole({role:'image'}) matches an element whose accessibilityRole 'image' normalizes to 'img' — callers pass the RN prop value, not 'img'. Bump HELPERS_VERSION 30->31. Full suite 2513/2513; byRole verified live (buttons still resolve, no accessible-exclusion regression). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(resolver): Codex review round 5 — gate byRole on the full RNTL accessibility-element predicate The earlier accessible={false} exclusion still let a plain <View accessibilityRole="button"> with accessible undefined match byRole, even though RNTL / screen readers would not expose it. byRole now requires the full isAccessibilityElement predicate (__isA11yElement): Text / TextInput / Switch and Image+alt are accessibility elements by default; everything else must opt in with accessible={true}. Applied in both isCandidate and __resolveLadderFiber. Verified live: real RN buttons expose accessible:true in their fiber memoizedProps (40/48 role=button fibers — host RCTView + inner Pressable layers), so byRole({role:'button'}) still resolves them (count unchanged) — only plain-View role props are now excluded. Updated synthetic byRole tests to model real a11y elements (accessible:true). Bump HELPERS_VERSION 31->32. Full suite 2518/2518. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 7879dfc commit 98d3fb7

29 files changed

Lines changed: 5672 additions & 19 deletions
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
"rn-dev-agent-cdp": minor
3+
"rn-dev-agent-plugin": minor
4+
---
5+
6+
Add an RNTL-style discovery resolver to the injected helpers. `resolveLadder` finds elements by `byRole(+name)` / `byText` / `byPlaceholder` — ported from React Native Testing Library (matcher + normalizer, accessible-name, role, hidden, host-kind) — with fail-closed truncation and fail-closed multiplicity (never silently picks the wrong element), hidden-element exclusion by default, and a selector bundle (`testID` / `text` / `accessibleName` / `role` / `placeholder` / `anchors`). `interact()` routes `role`/`name`/`text`/`placeholder` selectors through the ladder. Includes RNTL `matchDeepestOnly` so a composite+host fiber pair (e.g. `Text`+`RCTText`) resolves to a single on-device element instead of fail-closing as ambiguous.

docs/superpowers/plans/2026-06-19-rnt-selector-resolver-phase1.md

Lines changed: 2480 additions & 0 deletions
Large diffs are not rendered by default.

docs/superpowers/specs/2026-06-19-rnt-selector-resolver-projector-design.md

Lines changed: 198 additions & 0 deletions
Large diffs are not rendered by default.

scripts/cdp-bridge/dist/index.js

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -666,14 +666,33 @@ trackedTool('cdp_interact', 'Interact with React components by testID (preferred
666666
.string()
667667
.optional()
668668
.describe('accessibilityLabel prop (used if testID not provided). Tiered match: exact → normalized (trim+lowercase) → substring. Returns Ambiguous error if >1 component matches.'),
669-
text: z.string().optional().describe('Required for typeText: the text to enter'),
669+
text: z
670+
.string()
671+
.optional()
672+
.describe('For typeText: the text to enter. For the discovery ladder (no testID/accessibilityLabel, action:"press"): byText — match a host Text by its visible text content.'),
673+
role: z
674+
.string()
675+
.optional()
676+
.describe('Discovery ladder (press-only): match by accessibility role (e.g. button, tab, link). Combine with `name` for the accessible name. Needs an explicit accessibilityRole — Pressables without one resolve as role "none".'),
677+
placeholder: z
678+
.string()
679+
.optional()
680+
.describe('Discovery ladder (press-only): match a TextInput by its placeholder text.'),
681+
exact: z
682+
.boolean()
683+
.optional()
684+
.describe('Discovery ladder: require an exact (full-string) match for text/name/placeholder instead of case-insensitive substring.'),
685+
includeHidden: z
686+
.boolean()
687+
.optional()
688+
.describe('Discovery ladder: include accessibility-hidden elements (excluded by default).'),
670689
scrollX: z.number().optional().describe('For scroll: horizontal offset in pixels (default 0)'),
671690
scrollY: z.number().optional().describe('For scroll: vertical offset in pixels (default 300)'),
672691
animated: z.boolean().default(true).describe('For scroll: whether to animate'),
673692
name: z
674693
.string()
675694
.optional()
676-
.describe('Required for setFieldValue: the React Hook Form field name (same string you passed to useController({name}) or <Controller name="...">).'),
695+
.describe('For setFieldValue: the React Hook Form field name (same string you passed to useController({name}) or <Controller name="...">). For the discovery ladder with `role`: the accessible name to match (e.g. role:"button" + name:"Save").'),
677696
value: z
678697
.union([z.string(), z.number(), z.boolean()])
679698
.optional()

scripts/cdp-bridge/dist/injected-helpers.js

Lines changed: 729 additions & 2 deletions
Large diffs are not rendered by default.

scripts/cdp-bridge/dist/tools/interact.js

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
11
import { okResult, failResult, withConnection } from '../utils.js';
22
export function createInteractHandler(getClient) {
33
return withConnection(getClient, async (args, client) => {
4-
if (!args.testID && !args.accessibilityLabel) {
5-
return failResult('Either testID or accessibilityLabel is required');
4+
const hasLadderSelector = Boolean(args.role || args.text || args.placeholder);
5+
if (!args.testID && !args.accessibilityLabel && !hasLadderSelector) {
6+
return failResult('A selector is required: testID / accessibilityLabel, or a discovery-ladder selector (role / text / placeholder).');
67
}
78
if (args.action === 'typeText' && args.text === undefined) {
89
return failResult('text parameter is required for typeText action');
@@ -35,6 +36,14 @@ export function createInteractHandler(getClient) {
3536
opts.shouldValidate = args.shouldValidate;
3637
if (args.shouldDirty !== undefined)
3738
opts.shouldDirty = args.shouldDirty;
39+
if (args.role !== undefined)
40+
opts.role = args.role;
41+
if (args.placeholder !== undefined)
42+
opts.placeholder = args.placeholder;
43+
if (args.exact !== undefined)
44+
opts.exact = args.exact;
45+
if (args.includeHidden !== undefined)
46+
opts.includeHidden = args.includeHidden;
3847
const result = await client.evaluate(`__RN_AGENT.interact(${JSON.stringify(opts)})`);
3948
if (result.error) {
4049
return failResult(`Interact error: ${result.error}`);

scripts/cdp-bridge/src/index.ts

Lines changed: 27 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -945,15 +945,40 @@ trackedTool(
945945
.describe(
946946
'accessibilityLabel prop (used if testID not provided). Tiered match: exact → normalized (trim+lowercase) → substring. Returns Ambiguous error if >1 component matches.',
947947
),
948-
text: z.string().optional().describe('Required for typeText: the text to enter'),
948+
text: z
949+
.string()
950+
.optional()
951+
.describe(
952+
'For typeText: the text to enter. For the discovery ladder (no testID/accessibilityLabel, action:"press"): byText — match a host Text by its visible text content.',
953+
),
954+
role: z
955+
.string()
956+
.optional()
957+
.describe(
958+
'Discovery ladder (press-only): match by accessibility role (e.g. button, tab, link). Combine with `name` for the accessible name. Needs an explicit accessibilityRole — Pressables without one resolve as role "none".',
959+
),
960+
placeholder: z
961+
.string()
962+
.optional()
963+
.describe('Discovery ladder (press-only): match a TextInput by its placeholder text.'),
964+
exact: z
965+
.boolean()
966+
.optional()
967+
.describe(
968+
'Discovery ladder: require an exact (full-string) match for text/name/placeholder instead of case-insensitive substring.',
969+
),
970+
includeHidden: z
971+
.boolean()
972+
.optional()
973+
.describe('Discovery ladder: include accessibility-hidden elements (excluded by default).'),
949974
scrollX: z.number().optional().describe('For scroll: horizontal offset in pixels (default 0)'),
950975
scrollY: z.number().optional().describe('For scroll: vertical offset in pixels (default 300)'),
951976
animated: z.boolean().default(true).describe('For scroll: whether to animate'),
952977
name: z
953978
.string()
954979
.optional()
955980
.describe(
956-
'Required for setFieldValue: the React Hook Form field name (same string you passed to useController({name}) or <Controller name="...">).',
981+
'For setFieldValue: the React Hook Form field name (same string you passed to useController({name}) or <Controller name="...">). For the discovery ladder with `role`: the accessible name to match (e.g. role:"button" + name:"Save").',
957982
),
958983
value: z
959984
.union([z.string(), z.number(), z.boolean()])

0 commit comments

Comments
 (0)