Skip to content

hip-tests: expose disabled tests with HIP_TESTS_RUN_DISABLED cmake op…#8525

Open
sunhmy-amd wants to merge 1 commit into
developfrom
users/mingsun/run-disabled-tests
Open

hip-tests: expose disabled tests with HIP_TESTS_RUN_DISABLED cmake op…#8525
sunhmy-amd wants to merge 1 commit into
developfrom
users/mingsun/run-disabled-tests

Conversation

@sunhmy-amd

Copy link
Copy Markdown
Contributor

Motivation

Previously, tests disabled via the YAML config were compiled with Catch2's [.] hidden tag,
making them completely invisible to --list-tests and impossible to run without modifying
source. There was no way to audit which tests were disabled or run them selectively for
investigation.

Technical Details

Two changes:

parse_config.py: Disabled tests now emit [disabled][arch] instead of [.]. This makes
them visible in ./binary --list-tests, runnable via ./binary [disabled], and excludable
via ./binary ~[disabled].

cmake/hip-tests.cmake + config/CMakeLists.txt: Adds a cmake option HIP_TESTS_RUN_DISABLED
(default OFF).

  • OFF (default): ctest registers enabled tests with TEST_SPEC ~[disabled] and disabled tests
    separately with DISABLED TRUE, so plain ctest skips them.
  • ON: ctest registers all tests as enabled so plain ctest runs disabled tests too. Useful for
    local investigation or CI sweeps targeting known-failing tests.

JIRA ID

ROCM-27979

Test Plan

Built ExecutionControlTest (contains Unit_hipFuncGetAttributes_Positive_Basic, disabled on
gfx1201 due to EXSWHTEC-242) on gfx1201 host with ROCm 7.14.

  1. Binary --list-tests before PR (develop): disabled test not shown at all.
  2. Binary --list-tests after PR: test visible with [disabled] tag.
  3. ctest -N with default OFF: test shows as (Disabled), ctest -R gives Not Run (Disabled).
  4. ctest -N with -DHIP_TESTS_RUN_DISABLED=ON: test shows as normal enabled test, ctest -R
    actually executes it (fails with known defect, as expected).

Test Result

All four scenarios behave as designed. No regression on enabled tests.

Submission Checklist

…tion

parse_config.py previously emitted [.] for disabled tests, hiding them
from --list-tests and bare binary runs. Now emits [disabled][arch] so
disabled tests are visible and runnable.

CMake default (OFF): ctest registers [disabled] tests as DISABLED TRUE
so CI sweeps skip them. Enabled tests use TEST_SPEC ~[disabled].
With -DHIP_TESTS_RUN_DISABLED=ON: all tests registered as enabled so
plain ctest runs disabled tests too.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 14, 2026 03:31
@sunhmy-amd
sunhmy-amd requested review from a team as code owners July 14, 2026 03:31
@therock-pr-bot

Copy link
Copy Markdown

❌ PR Check — Action Required

Check Status Details
🌿 Branch Name ✅ Pass
📝 PR Title/Description ❌ Fail Error: Title does not follow Conventional Commits style.
Expected: start with a valid type (feat, fix, docs, …).
Desired format: type(optional-scope): short description
───
Error: PR description must reference a JIRA ID, ISSUE ID, or a GitHub closing keyword.
Expected: include a JIRA ID / ISSUE ID line (separator : or -, or omitted; value may be a JIRA key, a number with/without #, or a link), OR a closing keyword + issue reference. Accepted examples:
JIRA ID : TESTAUTO-6039
JIRA ID - #330
JIRA ID #330
ISSUE ID : TESTUTO-3334
ISSUE ID #3334
ISSUE ID - TESTAUTO-3433
ISSUE ID : https://github.com/<org_name>/<repo_name>/issues/1234
Closes #10
Fixes octo-org/octo-repo#100
Resolves: #123
#123
https://github.com/<org_name>/<repo_name>/issues/123
Current: no valid JIRA/ISSUE/closing-keyword reference found
Forbidden Files ✅ Pass
🧪 Unit Test ❌ Fail Error: Source/code files changed without an accompanying unit test.
Expected: add at least one test file named like test_<name>.py / test_<name>.cpp (or <name>_test.*).
Current: code file(s) changed: projects/hip-tests/catch/config/parse_config.py; no test file found
🔎 pre-commit ⏳ Pending ⏳ Still running…
🚫 Draft PR 🔜 To Be Enabled
🚩 Feature Flag 🔜 To Be Enabled
📊 Code Coverage 🔜 To Be Enabled

⚠️ 2 policy check(s) failed. Please address the issues above before this PR can be Reviewed.

🚫 Please fix the failed policies

  • ❌ PR Title/Description
  • ❌ Unit Test

The Not ready to Review label was added to this PR. Once all policies pass, the label is removed automatically.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

@therock-pr-bot

Copy link
Copy Markdown

🚫 Please fix the failed policies before requesting reviews.

The following policy checks failed:

  • ❌ PR Title/Description
  • ❌ Unit Test

The Not ready to Review label has been added to this PR.
Once all policies pass, the label will be removed automatically.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves auditability and controllability of HIP Catch2 tests that are disabled via YAML config by making them visible in --list-tests and by adding a CMake option to control whether CTest runs them.

Changes:

  • Update YAML→Catch2 tag generation so disabled tests are tagged with [disabled][<arch>] instead of being hidden via [.].
  • Add HIP_TESTS_RUN_DISABLED CMake option (default OFF) to optionally register disabled tests as enabled in CTest.
  • Adjust catch_discover_tests() registration so that, by default, enabled tests exclude [disabled] and disabled tests are registered separately as DISABLED TRUE.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
projects/hip-tests/catch/config/parse_config.py Emits [disabled][arch] tags for YAML-disabled tests (instead of hidden [.]), and ensures [arch] tagging in both paths.
projects/hip-tests/catch/config/CMakeLists.txt Introduces HIP_TESTS_RUN_DISABLED option and status message when enabled.
projects/hip-tests/catch/cmake/hip-tests.cmake Splits Catch2 test discovery into enabled vs disabled registrations unless HIP_TESTS_RUN_DISABLED=ON.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants