ci: Implement 4-way hash sharding for integration tests#133
Merged
Conversation
|
Note Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported. |
cgwalters
force-pushed
the
test-matrix
branch
2 times, most recently
from
eeb3409 to
45d65d9
Compare
cgwalters
force-pushed
the
test-matrix
branch
from
45d65d9 to
d3d4af1
Compare
Collaborator
Author
|
OK this definitely worked! Cool beans. However, one of the shards took a lot longer than the other, and now it's even more obvious that we have too many tests which are doing |
cgwalters
force-pushed
the
test-matrix
branch
from
d3d4af1 to
9fecf07
Compare
Collaborator
Author
|
Awesome! Got our wall clock down to approx 15m for build and 15m for testing, which is way way way better. |
Collaborator
Author
|
Hmm though what's a bit annoying here is wiring this up to automerge and required contexts...now we can't gate on a single |
Our integration tests are getting slower, and it's a bit hard to avoid because we really do want to do some medium-expensive thing like disk image generation across multiple base images. Split integration tests across 4 parallel GHA runners using nextest's really cool hash-based sharding. Assisted-by: Claude Code (Sonnet 4.5) Signed-off-by: Colin Walters <walters@verbum.org>
The bootc-ubuntu-setup action was recently added and provides consolidated setup for podman, just, libvirt, KVM permissions, and Rust caching. Both the build and integration-tests jobs were duplicating this setup code. Replace the duplicated setup steps with the shared action. The only additional dependency needed is go-md2man in the build job. Assisted-by: Claude Code (Sonnet 4.5) Signed-off-by: Colin Walters <walters@verbum.org>
The integration tests were creating separate disk images for tests that could share the same VM, causing unnecessary duplication of expensive `to-disk` operations. Changes: - Merged `test_base_disks_list_shows_timestamp` into `test_base_disk_creation_and_reuse` - the timestamp test no longer needs its own VM, it can verify the list output after the reuse test creates VMs - Created `test_libvirt_comprehensive_workflow` that consolidates multiple separate tests (`test_libvirt_run_list_json_ssh_metadata`, `test_libvirt_run_with_instancetype`, and `test_libvirt_run_label_functionality`) into a single test that creates one VM and verifies: - Instance type configuration - Label metadata and filtering - JSON output with SSH metadata - VM lifecycle This reduces the number of expensive disk image creations in CI, improving test performance while maintaining the same test coverage. Assisted-by: Claude Code (Sonnet 4.5) Signed-off-by: Colin Walters <walters@verbum.org>
cgwalters
force-pushed
the
test-matrix
branch
from
9fecf07 to
81ef571
Compare
Collaborator
Author
|
OK I reworked the CI gating to now require |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Our integration tests are getting slower, and it's a bit hard to avoid because we really do want to do some medium-expensive thing like disk image generation across multiple base images.
Split integration tests across 4 parallel GHA runners using nextest's really cool hash-based sharding.
Assisted-by: Claude Code (Sonnet 4.5)