Consolidate unified-input Maestro tests into one file#8628
Conversation
a8c7aa3 to
394607d
Compare
Replaces the six unified_input_with_favorites_* files (one per omnibarPosition x inputWithAiToggle combo) with a single file that runs the six combos sequentially. Verification of the seeded favorites is extracted to a shared flow so future journeys (Duck.ai chat to Search) can reuse the same launch shape. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add shared/open_new_chat.yaml covering chat entry, text input, clear, newline, and submit-opens-Duck.ai. Wire it into iteration 1/6. Align appId in assert_seeded_favorites_visible.yaml with the parent test. Drop per-iteration maxRetries wrappers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The test-seeder TD landed on per-parameter mutations (addFavorites: "3") rather than canned scenario keys. Update the consolidated test and its shared verify flow to match: replace testScenario: "favorites_3" with addFavorites: "3" and assert against the generic Favorite N titles produced by AddFavoritesSeederPlugin. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
394607d to
f443a04
Compare
Adds the five top-level test files for the From-Search/New-Tab batch. Each runs three iterations sweeping omnibarPosition (top/bottom/split) with nativeInputToggle and inputWithAiToggle pinned true. Inner flows land in subsequent commits. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The merged AddFavoritesSeederPlugin (PR 8418 on develop) takes a semicolon-separated URL list rather than a count, and uses the host as each favorite's title. Switch the existing unified-input tests to "example.com;duckduckgo.com;privacytest.com" and assert against those host names instead of "Favorite 1/2/3". Add an extendedWaitUntil at the head of the shared assertion to ride out the seeding latency. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Verifies the unified input shows autocomplete suggestions when the user types in Search mode, submits the query, and dismisses cleanly into the SERP. Adds addFavorites to all five new top-level tests as a workaround for the seeder/routing race that prevents Skip Onboarding from landing when no mutation seeders are queued — this matches what PR 8628's existing unified-input tests already do. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds: - shared/flow_ask_duckai_suggestion.yaml — types a query, taps the Ask Duck.ai autocomplete row, and asserts the Duck.ai chat opens with duckAIHeader + the "Ask anything privately…" input placeholder. - shared/launch_and_skip_onboarding.yaml — a robust local replacement for ../../shared/skip_all_onboarding.yaml. The upstream version falls back to ../../shared/pre_onboarding.yaml whose asserted welcome copy has drifted from the current onboarding build; that fallback throws and aborts our flows. This one taps Skip Onboarding directly and waits for inputModeWidget with a 30s budget. All seven unified-input tests (the two existing PR 8628 ones plus the five new ones) now route through the local skip flow. Adds an extendedWaitUntil for inputModeWidget at the head of flow_search_suggestions.yaml so it doesn't race the skip. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Verifies the model and reasoning-effort pickers in Duck.ai mode: - model picker opens, shows alternatives, swapping to GPT-4o mini updates the chip text from "GPT-5" to "4o-mini" - switching back to GPT-5 mini re-exposes the reasoning picker (4o-mini does not support reasoning so the pill is hidden) - reasoning picker opens with "Fast" and "Reasoning" options, picking "Reasoning" dismisses the menu and the selection persists on re-open Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Verifies the Web Search and Create Image tool flows in Duck.ai mode. Tools are picked from the options menu (sliders icon) which lists "Web Search" and "Create Image". Selecting one places an active pill in the bottom row with content-desc "Remove <tool>"; tapping the pill removes it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Verifies that Web Search applies to the first prompt and clears for the follow-up: - selects Web Search from the options menu, asserts the active pill - types a prompt with Web Search active and submits - after the response, asserts the unified input re-opens with Web Search cleared so the tool is not sticky across prompts - types a follow-up and confirms the pill remains absent Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Skip Onboarding button bounds straddle the status bar height on the Pixel 9 — tapping the button by id resolves to its centre, which the system status bar overlay intercepts before the tap reaches the app. Switch to a point tap at the bottom of the button (50%, 9%) so the tap lands inside the button rect but below the system bar. Without this, the skip step is intermittent: the button visibly flashes on tap but the welcome card does not dismiss. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
joshliebe
left a comment
There was a problem hiding this comment.
Check the comments from Cursor, otherwise LGTM
Match the rest of the unifiedInputTest suite — every other top-level test in this directory wraps each iteration in `retry: maxRetries: 3`, but these two were relying on Maestro's lower default retry count and were the only outliers. Bring them in line so CI flakiness is bounded the same way across the whole suite. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The non-debug build is what gets exercised in CI and on Robin/Maestro Cloud, so target com.duckduckgo.mobile.android directly. Strips the .debug suffix from every top-level test and shared flow under .maestro/unified_input_screen/. No resource-id selectors used the .debug prefix, so the in-flow taps and asserts are unaffected. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit d6b01c7. Configure here.
| text: 'Agree and Continue' | ||
| - tapOn: | ||
| optional: true | ||
| text: "Agree and Continue" |
There was a problem hiding this comment.
Missing wait for T&C dialog causes test flakiness
Medium Severity
The T&C handling in open_new_chat.yaml uses assertVisible: optional: true + tapOn: optional: true without any extendedWaitUntil, unlike the other new flows in this PR (flow_ask_duckai_suggestion.yaml, flow_duckai_web_search_prompt.yaml) which correctly use runFlow: ../../shared/accept_duckai_agreement.yaml. That shared flow waits up to 10 seconds for the dialog to appear. Since every iteration launches with clearState: true, the T&C dialog will always appear on first Duck.ai use, but the inline optional check will race the WebView load — if it completes before the dialog renders, the dialog goes undismissed and blocks subsequent steps like tapOn: "Ask anything privately…".
Reviewed by Cursor Bugbot for commit d6b01c7. Configure here.


Task/Issue URL: https://app.asana.com/1/137249556945/project/488551667048375/task/1214927873722256?focus=true
Testing scenarios: https://app.asana.com/1/137249556945/project/488551667048375/task/1215056189493585?focus=true
Description
Lands the full unified-input Maestro suite under
.maestro/unified_input_screen/. Replaces the six per-combinationunified_input_with_favorites_*.yamlfiles from the seeders work with a single consolidated test, and adds six new top-level tests covering the From-Search / New-Tab scenarios from the Native Input Widget E2E backlog plus a Duck.ai chat → Search navigation test.Every top-level test pins
nativeInputToggle: "true", usesaddFavorites: "example.com;duckduckgo.com;privacytest.com", and wraps each iteration inretry: maxRetries: 3. The Duck.ai-side tests pininputWithAiToggle: "true"and sweepomnibarPosition∈ {top, bottom, split} (three iterations each).unified_input_with_seeded_favoriteskeeps the AI on/off sweep (six iterations) because the assertion is AI-independent.Layout under
.maestro/unified_input_screen/:Scenarios covered
unified_input_with_seeded_favoritesunified_input_open_chat_move_to_searchunified_input_search_suggestionsunified_input_ask_duckai_suggestionunified_input_duckai_model_reasoningunified_input_duckai_tool_pillsunified_input_duckai_web_search_promptInfrastructure notes
shared/launch_and_skip_onboarding.yaml— robust local replacement for../../shared/skip_all_onboarding.yaml. The upstream skip falls back to../../shared/pre_onboarding.yamlwhose asserted welcome copy has drifted from the new onboarding build, and the Skip Onboarding button bounds straddle the status bar height so tap-by-id is intercepted by the system bar. This helper waits for the button, taps by point at the bottom of its bounds, and waits for the unified input to settle.addFavoritesvalues are URL-list strings (semicolon-separated hosts) — matches the landedAddFavoritesSeederPluginsemantics from Add Maestro test seeding for NativeInputWidget E2E tests #8418.appId: com.duckduckgo.mobile.android(not.debug) to match the binary used in CI / Robin / Maestro Cloud.CI wiring
.github/workflows/e2e-nightly-non-blockers-suite.ymladds aUnified Input Fieldstep that runs the suite on Maestro Cloud, scoped via theunifiedInputTesttag, and reports failures to Asana.Steps to test this PR
./gradlew installPlayDebugmaestro test .maestro/unified_input_screen --include-tags unifiedInputTestUI changes