feat(ci): run time-to-first-query e2e on minimum supported platforms COMPASS-9087#8221
Open
ivandevp wants to merge 3 commits into
Open
feat(ci): run time-to-first-query e2e on minimum supported platforms COMPASS-9087#8221ivandevp wants to merge 3 commits into
ivandevp wants to merge 3 commits into
Conversation
…OMPASS-9087 Add packaged-app e2e smoke variants that run only the time-to-first-query test on the minimum supported OS versions (Ubuntu 20.04, RHEL 8, macOS 13 arm64). Packages are already built on these oldest OSes, but the packaged-app suite previously only ran on newer OSes, leaving the minimum supported versions untested at launch time. - Thread an optional packaged_app_test_args filter through the test-packaged-app function (mirrors ATLAS_CLOUD_TEST_ARGS) - Add non-sharded test-packaged-app-min-version task - Add min-version build variants (run-on-pr, arm64 for macOS)
|
Assigned |
…PASS-9087 The packaged Electron app needs a GUI host to start a webdriver session; the non-GUI arm64 packaging distro timed out on POST /session. The only macOS 13 GUI distro is Intel + dev-tier (no arm64 GUI host at macOS 13), so run this variant on macos-13-amd64-gui-dev, off-PR and non-gating. Linux min-version variants remain arm-agnostic and gating.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds packaged-app e2e smoke variants that run only the time-to-first-query test against the packaged app on the minimum supported OS versions: Ubuntu 20.04, RHEL 8, and macOS 13 (arm64).
Today we build/package on the oldest supported OSes (
ubuntu2004,rhel80,macos-13) but the packaged-app e2e suite only runs on newer OSes (Ubuntu 22.04, RHEL 9.3, macOS 14/15). That left a gap: we never actually launched the packaged binary on the minimum OS versions we claim to support. This closes that gap with a lightweight, per-PR smoke.Changes:
functions.yml: thread an optionalpackaged_app_test_argsfilter through thetest-packaged-appfunction (mirrors the existingATLAS_CLOUD_TEST_ARGSpattern). Backward-compatible — empty default runs the full suite, so existing tasks are unaffected.buildvariants-and-tasks.in.yml: newtest-packaged-app-min-versiontask (single, non-sharded,--test-filter="time-to-first-query") + three newrun-on-prbuild variants depending on the existingpackage-*artifacts.buildvariants-and-tasks.yml: regenerated vianpm run update-evergreen-config(passesevergreen validate).Checklist
Motivation and Context
We support (and build on) Ubuntu 20.04, RHEL 8, and macOS 13, but the packaged-app e2e suite only ran on newer OS versions — so the minimum supported platforms were never exercised end-to-end at launch time. This adds a cheap per-PR smoke (just time-to-first-query) on those min-version distros to catch launch/runtime regressions there. Resolves COMPASS-9087.
Open Questions
Dependents
Types of changes