hip-tests: expose disabled tests with HIP_TESTS_RUN_DISABLED cmake op…#8525
hip-tests: expose disabled tests with HIP_TESTS_RUN_DISABLED cmake op…#8525sunhmy-amd wants to merge 1 commit into
Conversation
…tion parse_config.py previously emitted [.] for disabled tests, hiding them from --list-tests and bare binary runs. Now emits [disabled][arch] so disabled tests are visible and runnable. CMake default (OFF): ctest registers [disabled] tests as DISABLED TRUE so CI sweeps skip them. Enabled tests use TEST_SPEC ~[disabled]. With -DHIP_TESTS_RUN_DISABLED=ON: all tests registered as enabled so plain ctest runs disabled tests too. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
❌ PR Check — Action Required
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
|
🚫 Please fix the failed policies before requesting reviews. The following policy checks failed:
The |
There was a problem hiding this comment.
Pull request overview
This PR improves auditability and controllability of HIP Catch2 tests that are disabled via YAML config by making them visible in --list-tests and by adding a CMake option to control whether CTest runs them.
Changes:
- Update YAML→Catch2 tag generation so disabled tests are tagged with
[disabled][<arch>]instead of being hidden via[.]. - Add
HIP_TESTS_RUN_DISABLEDCMake option (defaultOFF) to optionally register disabled tests as enabled in CTest. - Adjust
catch_discover_tests()registration so that, by default, enabled tests exclude[disabled]and disabled tests are registered separately asDISABLED TRUE.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| projects/hip-tests/catch/config/parse_config.py | Emits [disabled][arch] tags for YAML-disabled tests (instead of hidden [.]), and ensures [arch] tagging in both paths. |
| projects/hip-tests/catch/config/CMakeLists.txt | Introduces HIP_TESTS_RUN_DISABLED option and status message when enabled. |
| projects/hip-tests/catch/cmake/hip-tests.cmake | Splits Catch2 test discovery into enabled vs disabled registrations unless HIP_TESTS_RUN_DISABLED=ON. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Motivation
Previously, tests disabled via the YAML config were compiled with Catch2's
[.]hidden tag,making them completely invisible to
--list-testsand impossible to run without modifyingsource. There was no way to audit which tests were disabled or run them selectively for
investigation.
Technical Details
Two changes:
parse_config.py: Disabled tests now emit[disabled][arch]instead of[.]. This makesthem visible in
./binary --list-tests, runnable via./binary [disabled], and excludablevia
./binary ~[disabled].cmake/hip-tests.cmake+config/CMakeLists.txt: Adds a cmake optionHIP_TESTS_RUN_DISABLED(default OFF).
TEST_SPEC ~[disabled]and disabled testsseparately with
DISABLED TRUE, so plainctestskips them.ctestruns disabled tests too. Useful forlocal investigation or CI sweeps targeting known-failing tests.
JIRA ID
ROCM-27979
Test Plan
Built
ExecutionControlTest(containsUnit_hipFuncGetAttributes_Positive_Basic, disabled ongfx1201 due to EXSWHTEC-242) on gfx1201 host with ROCm 7.14.
--list-testsbefore PR (develop): disabled test not shown at all.--list-testsafter PR: test visible with[disabled]tag.ctest -Nwith default OFF: test shows as(Disabled),ctest -RgivesNot Run (Disabled).ctest -Nwith-DHIP_TESTS_RUN_DISABLED=ON: test shows as normal enabled test,ctest -Ractually executes it (fails with known defect, as expected).
Test Result
All four scenarios behave as designed. No regression on enabled tests.
Submission Checklist