.github,go/utils/cigotests: Replace our multi go-test workflows with one dynamic matrix workflow and a test runner.#11021
Open
reltuk wants to merge 7 commits into
Open
.github,go/utils/cigotests: Replace our multi go-test workflows with one dynamic matrix workflow and a test runner.#11021reltuk wants to merge 7 commits into
reltuk wants to merge 7 commits into
Conversation
A small Go program that reads an embedded config (config.yaml) and
drives the test matrix for the new ci-go-tests workflow. Three
subcommands:
list -> JSON {os, shard} cells
plan --shard X --os Y --event Z -> describe-only (no exec)
run --shard X --os Y --event Z -> exec go test
Each shard declares packages (with optional ./... + exclude),
timeout, env, and race_on conditions over (os, event). The list
command emits one matrix cell per (os, shard) the workflow should
run, with shards able to narrow OS coverage via runs_on. Adding,
splitting, or retuning a shard is a single-file edit — the
workflow re-discovers cells on every run.
This commit is the runner only; the workflow that invokes it is
in the next commit.
Consolidates ci-go-tests.yaml, ci-go-tests-adaptive.yaml,
ci-go-race-tests.yaml, and ci-binlog-tests.yaml into a single
ci-go-tests.yaml driven by the cigotests runner.
The workflow has two jobs:
enumerate-shards runs cigotests list, emits a JSON array of
{os, shard} cells via job outputs.
test strategy.matrix.include is fromJson of the
above. Each cell runs cigotests run for its
(shard, os, github.event_name) triple.
What changed from the old workflows:
- The per-package bash loop is gone. Each shard hands its full
package list to a single 'go test pkg1 pkg2 ...' invocation,
which lets go's test runner schedule packages concurrently.
- The OS-conditional '-race' baked into ci-go-tests.yaml is now
a per-shard race_on predicate in cigotests/config.yaml.
- ci-go-tests-adaptive.yaml is dropped entirely. After the
adaptive-encoding default flip in #11011 it was running the
same matrix as ci-go-tests.yaml.
- The push-only enginetest+race coverage from
ci-go-race-tests.yaml is now expressed as race_on:
{os: [ubuntu-22.04], events: [push, workflow_dispatch]} on
the enginetest shard. Concurrentmap, which had its own race
job, is just one of many packages in the 'rest' shard which
already gets -race on ubuntu.
- ci-binlog-tests.yaml folds in as a 'binlogreplication' shard
pinned to ubuntu (matching the workflow it replaces).
Adding/splitting/retuning a shard is now a one-file edit to
config.yaml; no workflow change is needed.
reltuk
force-pushed
the
aaron/unified-go-tests-workflow
branch
from
8e74269 to
a8b1190
Compare
Contributor
|
@reltuk DOLT
|
Contributor
Contributor
Contributor
|
@coffeegoddd DOLT
|
Contributor
Contributor
Contributor
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The test runner is configured to create shards and then run them when it gets asked to. See the commits for details.