Commit 413da76
committed
fix(intro-screen-native): ignore the trailing drag a settled scroll emits
The Android IntroScreen flow failed at its last assertion: after re-entering
the page, NEXT moved slide 2 -> 3 and the attribute correctly became 3, then
immediately flipped to 1, so "Active slide: 3" was never visible. The failure
video shows the value on screen going 2 -> 3 -> 1 with no input in between.
Cause: once the programmatic scroll to slide 3 settles, Android's list emits a
SECOND onScrollBeginDrag + onMomentumScrollEnd pair, and that momentum event
carries the offset the list moved AWAY from (0). The existing pendingScrollTarget
guard had already been cleared by the first, legitimate momentum end, and
wasUserDragging was true because the settling list — not a finger — began the
drag. So the event was treated as a swipe back to slide 1 and rewrote the
attribute. This is the same class of bug as the earlier phantom-momentum fix,
one step later in the sequence.
Fix: remember the slide a programmatic scroll settled on, and ignore a momentum
end that moves more than one page from activeIndex while that memory is set.
The list is pagingEnabled, so a genuine drag moves exactly one page; a
multi-page jump right after a scroll settled can only be that scroll's tail.
One-page swipes, including a swipe straight after a programmatic scroll, still
report normally (covered by existing tests).
Regression test added; it fails with setValue receiving [3, 1] and a spurious
onSlideChange(0, 2) when the guard is removed.
Revert "fix(e2e): scroll the alphabet-index target into view before tapping it"
This reverts commit 31e9fd78c2fb70fb0e88b33b2d9bf8a983c4db51.
fix(e2e): scroll the alphabet-index target into view before tapping it
FloatingActionButton.yaml failed on iOS with
Tap on "F"... COMPLETED
Assert that "Floating action button" is visible... FAILED
then passed on retry with the same build. The flow already had the 10s
extendedWaitUntil added in 20d3d52, and that wait is what timed out — so
this is not the render race that commit fixed; nothing was going to change in
second 11 and a longer timeout would not have helped.
extendedWaitUntil: visible requires the element to be IN THE VIEWPORT, not
merely present in the hierarchy. Tapping the index scrolls to the letter's
section, and under F the entries are Feedback then Floating action button; if
the scroll settles with Feedback at the bottom edge, the target is rendered
just below the fold and `visible` stays false forever. The retry's scroll
landed a few pixels differently and it passed.
Use scrollUntilVisible instead: it succeeds immediately when the element is
already in view and scrolls to it when it is not, so it covers both the
original render race and the off-screen case. centerElement keeps the
subsequent tap away from the screen edge.
Applied to the 55 flows whose index letter hosts more than one widget entry
(B, C, F, I, L, P, R, S) — the case where the target can be pushed below the
fold. The 16 flows on a sole-entry letter keep the cheaper wait.
Verified: all 82 flows still parse, all 71 index taps remain guarded (55
scrollUntilVisible + 16 extendedWaitUntil, 0 unguarded), and the pinned 2.6.1
binary accepts every field used — confirmed it rejects an unknown property, so
that parse is a real schema check.
ci(e2e): archive maestro's debug output for failed flows
Maestro writes a debug bundle per run — failure screenshot, maestro.log
(which records the element Bounds each selector matched) and
commands-<flow>.json — to ~/.maestro/tests/<timestamp>/. That is outside the
workspace, so none of it was ever uploaded, and it is exactly what
distinguishes "element never rendered" from "element rendered off-screen
below the fold" when an assertVisible/extendedWaitUntil fails. Without it
those failures can only be diagnosed by inference.
Redirect it into the workspace with --debug-output (+ --flatten-debug-output,
which its own help text recommends for CI) one dir per flow, and archive it.
Kept only for FAILED flows, mirroring the existing video policy, so a green
shard uploads nothing — each bundle is ~300KB.
Also gitignore maestro/debug and maestro/videos: both are produced into the
workspace by local runs too. The existing *.log rule covered maestro.log but
not the bundle's screenshots or commands JSON.
Verified --debug-output and --flatten-debug-output against the pinned 2.6.1
binary, and drove run_tests with a stub to confirm a failing flow's bundle is
kept while a passing flow's is discarded.
fix(maestro): recover from iOS driver/accessibility faults instead of hanging
A shard could die with, and never recover from:
Exception in thread "main" UnknownFailure(errorResponse=Request for
viewHierarchy failed, code: 500, body: {"errorMessage":"Error getting
element frame kAXErrorInvalidUIElement","code":"internal"})
Traced through the pinned 2.6.1 jars, this is unrecoverable by design.
XCTestDriverClient.handleExceptions maps a 500 to a typed error only for
four known message strings and otherwise throws UnknownFailure;
XCTestIOSDevice.execute catches only AppCrash/OperationTimeout/Unreachable,
and IOSDriver.runDeviceCall catches only the matching IOSDeviceErrors.
UnknownFailure matches none, so it escapes to main — hence "Exception in
thread main" rather than a failed step, and hence a flow-level `retry:`
block cannot help: the CLI is already dead, no command failed. kAXError*
means the accessibility element reference is stale, so every later query
against that driver fails identically.
Recovery therefore has to live in the shell. Add run_maestro(), which:
- runs `maestro test` under a wall-clock watchdog (MAESTRO_FLOW_TIMEOUT,
480s) so a JVM that died in main but is held open by non-daemon threads
can't eat the 30-minute job budget, and
- classifies a FAILED run as infrastructure (exit 90) when the output
carries a driver/accessibility signature, so callers can tell "the
device broke" from "the widget is wrong".
run_tests then resets the device and gives the flow one clean attempt with
a freshly reinstalled driver, instead of recording a widget failure and
marching every remaining flow through the same wedged driver. All three
call sites (flows, smoke check, retries) now go through run_maestro, and
the duplicated device-reset branches collapse into reset_device().
Only non-zero exits are reclassified, so a flow that passed after Maestro's
own internal retry logged such an error still counts as a pass.
fix(maestro): actually apply the iOS driver startup timeout
MAESTRO_DRIVER_STARTUP_TIMEOUT was passed to maestro as `--env`, but
LocalXCTestInstaller.getStartupTimeout() reads it with System.getenv() —
`--env` only fills Maestro's per-flow variable map, which that function
never consults. So every value we passed was silently ignored and the
Kotlin runCatching kept falling back to its built-in 120000 default.
Export it instead (and raise it to 240000), and drop the three no-op
`--env MAESTRO_DRIVER_STARTUP_TIMEOUT` arguments.
Also stop reinstalling the XCUITest driver once per flow. Maestro's
TestCommand.reinstallDriver defaults to true, so each `maestro test`
process runs restartXCTestRunner (uninstall + install + start) — and that
install is the window that throws IOSDriverTimeoutException on a 3-vCPU
macOS runner. The smoke check still installs the driver fresh at the start
of every shard and retries still reinstall via restart_simulator, so only
the first-pass flows reuse it. MAESTRO_REUSE_DRIVER=false restores the
old behaviour.1 parent 238a920 commit 413da76
5 files changed
Lines changed: 275 additions & 16 deletions
File tree
- .github/actions/archive-test-results
- maestro/helpers
- packages/pluggableWidgets/intro-screen-native/src
- __tests__
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
67 | 67 | | |
68 | 68 | | |
69 | 69 | | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
70 | 85 | | |
71 | 86 | | |
72 | 87 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
23 | 29 | | |
24 | 30 | | |
25 | 31 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
14 | 21 | | |
15 | 22 | | |
16 | 23 | | |
| |||
78 | 85 | | |
79 | 86 | | |
80 | 87 | | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
81 | 232 | | |
82 | 233 | | |
83 | 234 | | |
| |||
184 | 335 | | |
185 | 336 | | |
186 | 337 | | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
187 | 358 | | |
188 | 359 | | |
189 | 360 | | |
| |||
194 | 365 | | |
195 | 366 | | |
196 | 367 | | |
197 | | - | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
198 | 371 | | |
199 | 372 | | |
| 373 | + | |
200 | 374 | | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
201 | 394 | | |
202 | 395 | | |
203 | 396 | | |
| |||
228 | 421 | | |
229 | 422 | | |
230 | 423 | | |
231 | | - | |
| 424 | + | |
232 | 425 | | |
233 | 426 | | |
| 427 | + | |
234 | 428 | | |
235 | 429 | | |
236 | 430 | | |
237 | 431 | | |
238 | 432 | | |
239 | 433 | | |
240 | | - | |
241 | | - | |
242 | | - | |
243 | | - | |
244 | | - | |
245 | | - | |
246 | | - | |
| 434 | + | |
247 | 435 | | |
248 | 436 | | |
249 | 437 | | |
| |||
260 | 448 | | |
261 | 449 | | |
262 | 450 | | |
263 | | - | |
264 | | - | |
265 | | - | |
266 | | - | |
267 | | - | |
| 451 | + | |
268 | 452 | | |
269 | 453 | | |
270 | 454 | | |
271 | | - | |
| 455 | + | |
272 | 456 | | |
273 | 457 | | |
| 458 | + | |
274 | 459 | | |
275 | 460 | | |
276 | 461 | | |
| |||
Lines changed: 20 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
80 | 80 | | |
81 | 81 | | |
82 | 82 | | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
83 | 87 | | |
84 | 88 | | |
85 | 89 | | |
| |||
91 | 95 | | |
92 | 96 | | |
93 | 97 | | |
| 98 | + | |
94 | 99 | | |
95 | 100 | | |
96 | 101 | | |
| |||
355 | 360 | | |
356 | 361 | | |
357 | 362 | | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
358 | 367 | | |
359 | 368 | | |
360 | 369 | | |
| |||
363 | 372 | | |
364 | 373 | | |
365 | 374 | | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
366 | 386 | | |
367 | 387 | | |
368 | 388 | | |
| |||
0 commit comments