Skip to content

Commit 35842ac

Browse files
committed
fix(ci): pass --maestro and match the fixture app's real UI in layer-3 flows
Run 4 (29500262301) reached the differential itself — build, install, simctl check and the pinned Maestro 2.5.1 verification all passed — and surfaced two real bugs, both mine: 1. The runner invoked `agent-device test <flow>` without --maestro, so every scenario failed with "test does not support this file type". The repo's own scripts/run-test-app-maestro-suite.mjs passes it; the flag is what routes a .yaml through the Maestro compat engine. 2. settle-after-tap and percent-swipe assumed home-open-form is on screen at launch. It is not: real Maestro reported "Element not found: home-open-form", and the app's own helper flow scrolls it into view first. settle-after-tap now scrolls before tapping, mirroring that helper; percent-swipe no longer navigates at all and swipes the scrollable home screen, so it tests the conversion and nothing else. The remaining two flows already reported maestro=pass, so only the agent-device invocation was wrong for those. Note the settle invariant correctly reported "no-data: no completed tapOn steps" and FAILED rather than passing — a detector that cannot run is a failure, as intended.
1 parent c37aa5f commit 35842ac

3 files changed

Lines changed: 14 additions & 6 deletions

File tree

scripts/maestro-conformance/differential/flows/percent-swipe.yaml

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,11 @@
1-
# Layer-3 device flow (bug class 1, runtime half). Percentage endpoints must
2-
# resolve to the same pixels on both engines (integer division, not rounding).
1+
# Layer-3 device flow. Percentage swipe endpoints must drive the same gesture on
2+
# both engines. Deliberately stays on the scrollable home screen: no navigation,
3+
# no off-screen controls, so the scenario tests the swipe and nothing else.
34
appId: com.callstack.agentdevicelab
45
---
56
- launchApp:
67
clearState: true
7-
- tapOn:
8-
id: home-open-form
9-
- assertVisible: Checkout form
8+
- assertVisible: Agent Device Tester
109
- swipe:
1110
start: 50%, 75%
1211
end: 50%, 35%
Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,16 @@
1-
# Layer-3 device flow (bug class 4). Real tap on a real element of the lab app,
1+
# Layer-3 device flow (bug class 4). A real tap on a real control of the lab app,
22
# so the post-tap settle loop actually runs and the timing invariant is meaningful.
3+
# home-open-form sits below the fold, so scroll it into view first — the app's own
4+
# helper flow (examples/test-app/maestro/helpers/open-checkout-form.yaml) does
5+
# exactly this; tapping it directly fails with "element not found".
36
appId: com.callstack.agentdevicelab
47
---
58
- launchApp:
69
clearState: true
710
- assertVisible: Agent Device Tester
11+
- scrollUntilVisible:
12+
element:
13+
id: home-open-form
814
- tapOn:
915
id: home-open-form
1016
- assertVisible: Checkout form

scripts/maestro-conformance/differential/run.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -119,9 +119,12 @@ function runScenario(scenario: DifferentialScenario, options: RunnerOptions): Sc
119119
const platformArgs = options.platform ? ['--platform', options.platform] : [];
120120

121121
const maestro = runEngine('maestro', options.maestroBin, ['test', flowPath, ...platformArgs]);
122+
// `--maestro` is required: without it `test` rejects a .yaml flow outright
123+
// ("test does not support this file type"). Matches scripts/run-test-app-maestro-suite.mjs.
122124
const agentDevice = runEngine('agent-device', `node ${options.agentDeviceCli}`, [
123125
'test',
124126
flowPath,
127+
'--maestro',
125128
...platformArgs,
126129
]);
127130

0 commit comments

Comments
 (0)