feat(story-06): Phase B — nightly device smoke (#387)#480
Conversation
…387) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Amendments applied from the multi-LLM plan review (Codex + Claude verified; Antigravity failed tool-side): .mjs->.ts driver (check-typescript-only gate), scrollintoview one-swipe cap worked around, Android textAllCaps + show_ime_with_hard_keyboard, iOS keyboard preflight, swiftc fixture hard gate, plus minor detect/emulator/assert cleanups. Spec DerivedData path typo fixed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… main) (#387) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Also fixes the plan's seeded-bug step to commit+push the mutation before dispatching (codex-pair finding). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Without appBundleId the runner's executeOnMain clears its target and activates the RnFastRunner HOST app — every device_scroll/device_swipe foreground-stole the host and dragged on a blank screen (ok:true, zero movement). Device-proven on the Phase B contract fixture; caught by the golden-set smoke before reaching CI. Covers all three dispatch sites (device_swipe coords, device_swipe direction, device_scroll) with handler-level unit tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
TypeScript (Node type stripping — a .mjs would fail the check-typescript-only gate), reusing supervisor-harness.js. RN_RUNNER_BUILD=local pins main-HEAD runner provenance. Full set green locally on iPhone 17 / iOS 26.5 (87s): open, snapshot refs, indexed exact find, press + counter + settle/timings, bounded scroll loop to row 80, scrollintoview, screenshot magic-bytes, native-path fill, keyboard-guard refusal contract (#370), negative find, close. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…vice smoke (#387) Two Android defects surfaced by the golden-set smoke: - updateRefMap's screen-rect heuristic took the FIRST (0,0)-anchored wide node; with interactive-windows snapshots (#370) the status bar precedes the app window, so direction scroll/swipe computed gestures inside the status bar. Now picks the LARGEST full-bleed rect (unit-tested). - rn-android-runner could not re-foreground a target on API 30+ — its manifest lacked a package-visibility <queries>, so getLaunchIntentForPackage returned null ("No launch intent for package ..."). Added a MAIN/LAUNCHER queries entry. Fixture: fitsSystemWindows=true (targetSdk 35 edge-to-edge put top controls under the status bar). Driver: driver-owned fixture launch + attachOnly open (race-free), platform-aware scroll amplitude, honest failure on RUNNER_TIMEOUT, wall-clock launch deadline. Verified on-device through open/snapshot/find/press/counter-increment/ scroll-advance; the scroll-to-row-80 loop is gated locally only by a degraded nested-ARM emulator's >10s drag latency (runner drag is ~4s direct). CI's dedicated Android lane is the authoritative gate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
nightly-device-smoke.yml: schedule (03:00 UTC) + dispatch; parallel iOS (macos-15, DerivedData cache keyed on Xcode version + runner-source hash) and Android (ubuntu KVM, android-emulator-runner api-34 google_apis) smoke lanes driving the golden set; an artifact-integrity lane verifying the released runner zips against runner-manifest.json (sha256 + bytes + traversal-safe listing); a report job opening/closing a nightly-smoke-red tracking issue only on 2 consecutive red scheduled runs. Trusted-context only — no issue/PR input in any run: step. Driver hardening (codex-pair): bounded timeouts on the install probes, honest failure on RUNNER_TIMEOUT (no masking retry), wall-clock launch deadline, and an explicit snapshot-ok assert inside the scroll loop so a failing snapshot can't hide behind 'row 80 never appeared'. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Story doc: Phase B section marked implemented with as-built deviations (main-HEAD provenance, native fixtures) and the pre-Phase-A Problem framing marked historical. Workflow: scope issues:write to the report job, machine-readable zip-listing for the traversal check, any-non-success counts as a red night (existing prior run only). Changeset added. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 50c9e61d8d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| function assertFixtureInstalled() { | ||
| try { | ||
| if (PLATFORM === 'ios') { | ||
| execFileSync('xcrun', ['simctl', 'get_app_container', 'booted', APP_ID], { |
There was a problem hiding this comment.
When the nightly workflow has already selected and installed the fixture on SMOKE_UDID, this smoke driver still addresses iOS as booted (and later opens the session without a deviceId). In any runner/developer environment with another booted simulator, simctl ... booted becomes ambiguous or points at the wrong device, so the smoke can fail before opening despite the intended simulator being ready. Thread the selected UDID (for example from SMOKE_UDID) through these simctl calls and the device_snapshot open arguments.
Useful? React with 👍 / 👎.
| else | ||
| gh issue create --repo "$GITHUB_REPOSITORY" \ | ||
| --title "Nightly device smoke: 2 consecutive red nights" \ | ||
| --label nightly-smoke-red \ |
There was a problem hiding this comment.
On the first 2-red scheduled run in a repository where nightly-smoke-red has not already been created, this path reaches gh issue create --label nightly-smoke-red; GitHub CLI label creation is a separate gh label create command (see https://cli.github.com/manual/gh_label_create), so adding a missing label by name fails and no tracking issue is opened. Make the report step create/update the label first (or avoid requiring the label on creation) so the alerting path is self-contained.
Useful? React with 👍 / 👎.
1 participant
Story 06 Phase B — nightly device smoke
Drives the golden
device_*command set through the real bridge (dist/supervisor.jsover MCP stdio) against tiny native contract fixtures on a booted simulator/emulator, nightly and non-gating. Closes the last gap in the three-layer coverage contract: TS logic (ci.yml) + packaged artifact (#432) + executed native device behavior (this).Spec:
docs/superpowers/specs/2026-07-05-387-phase-b-device-smoke-design.md· Plan:docs/superpowers/plans/2026-07-05-387-phase-b-device-smoke.md(multi-LLM reviewed).What's here
.github/workflows/nightly-device-smoke.yml— parallel iOS (macos-15, DerivedData cache) + Android (ubuntu KVM,android-emulator-runnerapi-34google_apis) smoke lanes; a release-artifact-integrity lane (sha256 + bytes + traversal-safe listing + contents vsrunner-manifest.json); a report job opening/closing anightly-smoke-redtracking issue only on 2 consecutive red scheduled runs.test-fixtures/{ios,android}-fixture/— SwiftUI (swiftc, no Xcode project) + Kotlinandroid.widgetcontract apps: counter button, text field, 100-row list, keyboard-occlusion bottom bar.scripts/cdp-bridge/test/smoke/device-smoke.ts+ rootsmoke:{ios,android}— the golden-set driver (TypeScript, run via Node type-stripping so it clears thecheck-typescript-onlygate).Runner provenance (deliberate deviation from the plan)
Smoke lanes cache-build the runner from the tested commit (
RN_RUNNER_BUILD=local) so a red night is a real regression, never release lag. The released artifacts get their own integrity lane — the two signals stay separate.Two runner bugs the smoke caught before CI
fastSwipeomitted the targetappBundleId→ the runner activated its ownRnFastRunnerhost and dragged on a blank screen (everydevice_scroll/device_swipereturnedok:truewith zero movement, foreground-stealing the app under test). Fixed at all three dispatch sites, unit-tested.<queries>→ with interactive-windows snapshots (Phase 2: in-runner keyboard-occlusion guard for live device_* taps (#356 follow-on) #370) the status bar preceded the app window (direction gestures computed inside the status bar), andgetLaunchIntentForPackagereturned null on API 30+ (couldn't re-foreground). Both fixed, unit-tested.Verification
Scheduled runs begin after merge; the one-week-green + seeded-bug acceptance is post-merge observation (not a merge gate, per the spec). Phase C (LLM-behavior evals) remains open in #387.
🤖 Generated with Claude Code