[hipsolver]Add test filter standardization for hipsolver. by dileepr1 · Pull Request #7546 · ROCm/rocm-libraries

dileepr1 · 2026-05-18T16:17:30Z

Motivation

Implements test filter standardization for hipSOLVER so that ctest -L {quick|standard|comprehensive|full} works both in the build tree and from the installed/packaged tree that TheRock CI consumes.

Technical Details

hipSOLVER-side wiring:

projects/hipsolver/clients/gtest/test_categories.yaml (new) is added which defines the test categories that can be used by theRock CI. Tests can be added/removed/excluded as required under these categories.
projects/hipsolver/CMakeLists.txt — sets ROCM_LIBRARIES_ROOT so shared/ctest/TestCategories.cmake is reachable from the project.
projects/hipsolver/clients/gtest/CMakeLists.txt — includes the shared module, calls apply_test_category_labels() on hipsolver-test, and installs the generated install-time CTestTestfile.cmake to bin/hipsolver/ (under COMPONENT tests) so TheRock's relocatable test artifact carries the labels. Guarded with if(EXISTS shared/ctest/TestCategories.cmake) so standalone / sparse checkouts keep building.

This work together with TheRock side PR ROCm/TheRock#5490.

Test Plan

Build tree:

ctest -N
ctest --print-labels        # quick, standard, comprehensive, full
ctest -L quick

Install tree (matches what TheRock CI sees):

cd <install_prefix>/bin/hipsolver
ctest -N
ctest --print-labels
ctest -L quick

CI: covered by both Linux (gfx94X-dcgpu, gfx950-dcgpu) and Windows (gfx1151) hipsolver test shards in this PR's therock-ci-* runs.

Test Result

Test run: https://github.com/ROCm/rocm-libraries/actions/runs/26532647528/job/78175500172?pr=7546

Submission Checklist

Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

Mirror the rocSOLVER test categorization pattern from PR #6943. - Add projects/hipsolver/clients/gtest/test_categories.yaml with quick/standard/comprehensive/full categories. The shared exclude list mirrors test/therock/test_hipsolver.py. - Set ROCM_LIBRARIES_ROOT in projects/hipsolver/CMakeLists.txt for shared/ctest infrastructure. - Wire apply_test_category_labels() in clients/gtest/CMakeLists.txt with install-time CTestTestfile.cmake for TheRock relocatable tests. No weekly_* tier is included; hipSOLVER has no weekly_* INSTANTIATE_TEST_SUITE_P entries. Co-authored-by: Cursor <cursoragent@cursor.com>

Switch therock-test-component.yml and therock-test-packages.yml to check out the paired TheRock branch users/dravindr/tr_hipsolver (ROCm/TheRock@1f02638b) instead of the pinned 2026-05-13 commit, so the test job uses test_runner.py and the artifact glob that bundles bin/hipsolver/CTestTestfile.cmake. Same pattern as PR #7215. WIP: revert to a pinned ROCm/TheRock SHA before merge. Co-authored-by: Cursor <cursoragent@cursor.com>

codecov-commenter · 2026-05-18T17:11:15Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

❌ Your project status has failed because the head coverage (69.18%) is below the target coverage (80.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #7546      +/-   ##
===========================================
+ Coverage    61.78%   66.35%   +4.57%     
===========================================
  Files         2091     1606     -485     
  Lines       359586   267985   -91601     
  Branches     54392    37395   -16997     
===========================================
- Hits        222148   177817   -44331     
+ Misses      118662    75227   -43435     
+ Partials     18776    14941    -3835

Flag	Coverage Δ		*Carryforward flag
TensileLite	`?`
hipBLAS	`90.65% <ø> (+<0.01%)`	⬆️	Carriedforward from 3af4355
hipBLASLt	`41.19% <ø> (-0.04%)`	⬇️	Carriedforward from 3af4355
hipCUB	`82.21% <ø> (ø)`		Carriedforward from 3af4355
hipDNN	`85.46% <ø> (-1.15%)`	⬇️	Carriedforward from 3af4355
hipFFT	`50.47% <ø> (+0.27%)`	⬆️	Carriedforward from 3af4355
hipRAND	`?`
hipSOLVER	`69.18% <ø> (-0.06%)`	⬇️
hipSPARSE	`85.37% <ø> (-1.17%)`	⬇️	Carriedforward from 3af4355
rocBLAS	`48.11% <ø> (+0.01%)`	⬆️	Carriedforward from 3af4355
rocFFT	`49.80% <ø> (+1.36%)`	⬆️	Carriedforward from 3af4355
rocRAND	`57.02% <ø> (-0.01%)`	⬇️	Carriedforward from 3af4355
rocSOLVER	`?`
rocSPARSE	`72.47% <ø> (-0.20%)`	⬇️	Carriedforward from 3af4355

*This pull request uses carry forward flags. Click here to find out more.
see 666 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

The build stage was pinned to TheRock SHA 4e469b56 (2026-05-13), which predates the artifact-blas.toml update that adds bin/hipsolver/** to the hipSOLVER test include list. As a result, the produced artifact tarball omitted bin/hipsolver/CTestTestfile.cmake, and test_runner.py failed with "Test directory does not exist: build/bin/hipsolver". Align the build ref with the test workflows (already on users/dravindr/tr_hipsolver) so build and test use the same TheRock state. Co-authored-by: Cursor <cursoragent@cursor.com>

…hipsolver The Windows build stage was pinned to TheRock SHA e44b42bb (2026-05-20), which predates the artifact-blas.toml update that adds bin/hipsolver/** to the hipSOLVER test include list. As a result, the produced Windows artifact tarball omitted bin/hipsolver/CTestTestfile.cmake, and test_runner.py failed with "Test directory does not exist: build\bin\hipsolver". Align the Windows build ref with therock-ci-linux.yml, therock-test-component.yml, and therock-test-packages.yml (already on users/dravindr/tr_hipsolver) so build and test use the same TheRock state on both platforms. Co-authored-by: Cursor <cursoragent@cursor.com>

geomin12

going to let hipsolver team review

also, please add a better PR description and it seems this is still testing, so a draft PR may be good too

Restore all four TheRock-side workflow refs to the develop-tracked pin e44b42bb (2026-05-20): - .github/workflows/therock-ci-linux.yml - .github/workflows/therock-ci-windows.yml - .github/workflows/therock-test-component.yml - .github/workflows/therock-test-packages.yml Reverts the workflow ref changes from 71e281c, 17acb08 and daaa6a4, which temporarily pointed all four files at the paired TheRock branch users/dravindr/tr_hipsolver. At the develop-pinned TheRock SHA, fetch_test_configurations.py still runs the legacy test_hipsolver.py for hipsolver (not test_runner.py), and math-libs/BLAS/artifact-blas.toml does not include bin/hipsolver/**, so the new install-tree CTestTestfile.cmake and apply_test_category_labels() metadata added by this PR remain dormant until ROCm/TheRock#5490 merges and these refs are bumped to a main-tracked SHA that contains it. Until then the hipsolver Linux/Windows test shards behave exactly as on develop. Co-authored-by: Cursor <cursoragent@cursor.com>

Copilot

Pull request overview

This PR standardizes hipSOLVER’s CTest labeling so that ctest -L {quick|standard|comprehensive|full} works consistently both from the build tree and from an installed/packaged test tree (as consumed by TheRock CI), by introducing a YAML-driven category configuration and wiring it into the hipSOLVER gtest client build/install.

Changes:

Add a test_categories.yaml defining quick/standard/comprehensive/full suites and exclusions for hipSOLVER gtests.
Wire in the shared shared/ctest/TestCategories.cmake infrastructure (via ROCM_LIBRARIES_ROOT) and apply YAML-derived labels to hipsolver-test.
Generate and install an install-time CTestTestfile.cmake under bin/hipsolver/ to keep labels available in relocatable test artifacts.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
projects/hipsolver/CMakeLists.txt	Defines `ROCM_LIBRARIES_ROOT` default so shared CTest category infrastructure can be found in the monorepo.
projects/hipsolver/clients/gtest/test_categories.yaml	Adds YAML configuration for standardized test categories/labels and excludes.
projects/hipsolver/clients/gtest/CMakeLists.txt	Includes shared category module when present, generates labeled CTest tests + install-time `CTestTestfile.cmake`, and installs it for TheRock.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

jmachado-amd

The PR looks good.

## Motivation Add test filter standardisation for hipsolver. This works with the companion PR ROCm/rocm-libraries#7546 ## Technical Details - fetch_test_configurations.py: hipsolver test_script -> test_runner.py so the generic runner drives ctest from THEROCK_BIN_DIR/hipsolver using the quick/standard/comprehensive/full category labels. - artifact-blas.toml: include bin/hipsolver/** in the hipSOLVER test component so the install-tree CTestTestfile.cmake is bundled (same pattern as bin/rocblas/**). ## Test Plan Confirm that the CI tests for hipsolver works good once ROCm/rocm-libraries#7546 is merged. ## Test Result https://github.com/ROCm/TheRock/actions/runs/27141532171/job/80365435590?pr=5490 https://github.com/ROCm/TheRock/actions/runs/27141532171/job/80365749440?pr=5490 ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests. Co-authored-by: Cursor <cursoragent@cursor.com>

dileepr1 and others added 2 commits May 18, 2026 10:57

dileepr1 requested review from a team as code owners May 18, 2026 16:17

github-actions Bot added project: hipsolver github actions labels May 18, 2026

assistant-librarian Bot added the organization: ROCm label May 18, 2026

dileepr1 and others added 4 commits May 20, 2026 14:36

Merge branch 'develop' into users/dravindr/tf_hipsolver

3be10e7

Merge branch 'develop' into users/dravindr/tf_hipsolver

dd6619b

geomin12 reviewed May 28, 2026

View reviewed changes

Comment thread .github/workflows/therock-ci-linux.yml Outdated

Comment thread .github/workflows/therock-ci-windows.yml Outdated

Comment thread .github/workflows/therock-test-component.yml Outdated

Comment thread .github/workflows/therock-test-packages.yml Outdated

dileepr1 mentioned this pull request May 28, 2026

Add test filter standardisation for hipsolver ROCm/TheRock#5490

Merged

1 task

jzuniga-amd reviewed Jun 1, 2026

View reviewed changes

Comment thread projects/hipsolver/clients/gtest/test_categories.yaml Outdated

jzuniga-amd reviewed Jun 1, 2026

View reviewed changes

Comment thread projects/hipsolver/clients/gtest/test_categories.yaml

dileepr1 requested a review from nunnikri June 1, 2026 16:30

EdDAzevedo requested a review from Copilot June 1, 2026 16:39

Copilot started reviewing on behalf of EdDAzevedo June 1, 2026 16:39 View session

Copilot AI reviewed Jun 1, 2026

View reviewed changes

Comment thread projects/hipsolver/clients/gtest/test_categories.yaml

Comment thread projects/hipsolver/clients/gtest/CMakeLists.txt

dileepr1 and others added 3 commits June 1, 2026 16:04

Updating test_categories.yaml as per review comments

080ad6b

Apply suggestions from code review

3af4355

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Merge branch 'develop' into users/dravindr/tf_hipsolver

5494595

jmachado-amd approved these changes Jun 3, 2026

View reviewed changes

dileepr1 merged commit efdb880 into develop Jun 4, 2026
34 of 38 checks passed

dileepr1 deleted the users/dravindr/tf_hipsolver branch June 4, 2026 13:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[hipsolver]Add test filter standardization for hipsolver.#7546

[hipsolver]Add test filter standardization for hipsolver.#7546
dileepr1 merged 10 commits into
developfrom
users/dravindr/tf_hipsolver

dileepr1 commented May 18, 2026 •

edited

Loading

Uh oh!

codecov-commenter commented May 18, 2026 •

edited

Loading

Uh oh!

geomin12 left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

jmachado-amd left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

Conversation

dileepr1 commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Technical Details

Test Plan

Test Result

Submission Checklist

Uh oh!

codecov-commenter commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

geomin12 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

jmachado-amd left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

dileepr1 commented May 18, 2026 •

edited

Loading

codecov-commenter commented May 18, 2026 •

edited

Loading