Skip to content

test: parallelize suite with ctest#1903

Closed
vulcandth wants to merge 1 commit into
gbdev:masterfrom
vulcandth:ctest-parallel-tests
Closed

test: parallelize suite with ctest#1903
vulcandth wants to merge 1 commit into
gbdev:masterfrom
vulcandth:ctest-parallel-tests

Conversation

@vulcandth
Copy link
Copy Markdown
Contributor

Summary

  • register the RGBDS test suite as individual CTest cases
  • run per-test bash drivers in isolated temp directories so tests can execute safely in parallel
  • add labels, smoke-test fixtures, and downstream external tests to support filtering and ordering
  • document the new CTest workflow in CONTRIBUTING while keeping the legacy runner available

Metrics

Benchmarks on this machine:

Suite Old runner New CTest runner Speedup
Internal only 1m36s 1m14s 1.30x
External only 2m30s 2m04s 1.21x
Full suite 5m05s 2m37s 1.95x

Additional validation:

  • ctest -j$(nproc) -L internal: 728/728 passed
  • ctest -j$(nproc) -L external: 11/11 passed
  • ctest -j$(nproc): 735/735 passed

Notes

  • external tests now run against the fork/root correctly without overriding downstream RGBDS ?= variables
  • asm/section-unsigned-overflow gets a longer timeout because it is substantially slower than the rest of the suite
  • transient out*.png files are excluded from CTest discovery

@Rangi42 Rangi42 added tests This affects the test suite builds This affects the build process or release artifacts labels Mar 15, 2026
@Rangi42 Rangi42 added this to the 1.0.2 milestone Mar 15, 2026
@ISSOtm
Copy link
Copy Markdown
Member

ISSOtm commented Apr 10, 2026

Thank you for the idea! There are plenty of benefits to this, as I've outlined in #1927, but this specific implementation duplicates all of our test suite's logic into a separate set of runners without replacing the old ones. Which I'm rejecting on maintainability grounds :P

@ISSOtm ISSOtm closed this Apr 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

builds This affects the build process or release artifacts tests This affects the test suite

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants