Currently, BenchmarkDotNet CI tests takes about 1.5h - 3h. (Especially slow on macos(x64) image)
And it cause following issues.
- CI tests turnaround time is too long if there are failed tests.
- It takes GitHub Actions compute resources
(e.g. Running CI on macos 3h takes about $11, it's not charged on OSS project though)
So, It's needed to reduce CI execution times on PR.
Expected workaround
Add setting to run full tests after Ready for Review.
When draft PR is created, Run only limited set of tests.
- Skip some tests that are take long times (It's not works inProcessToolchain, and require build/publish operations)
- Skip Wasm/NativeAOT/R2R related tests and installation of
v8/wasm-tool workload
- Skip CI on
macos(x64) image
Currently, BenchmarkDotNet CI tests takes about 1.5h - 3h. (Especially slow on
macos(x64)image)And it cause following issues.
(e.g. Running CI on
macos3h takes about $11, it's not charged on OSS project though)So, It's needed to reduce CI execution times on PR.
Expected workaround
Add setting to run full tests after
Ready for Review.When draft PR is created, Run only limited set of tests.
v8/wasm-toolworkloadmacos(x64)image