Skip to content

cmake-workflow: Always check test results if they run#34

Open
Nerixyz wants to merge 2 commits into
alandefreitas:developfrom
Nerixyz:cmake/unchecked-tests
Open

cmake-workflow: Always check test results if they run#34
Nerixyz wants to merge 2 commits into
alandefreitas:developfrom
Nerixyz:cmake/unchecked-tests

Conversation

@Nerixyz

@Nerixyz Nerixyz commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

I find the default configuration for run-tests in the CMake workflow a bit confusing. Unless the CMAKE_RUN_TESTS environment variable is set, the tests run, but they're not checked.

If you blindly take the example from https://alandefreitas.github.io/cpp-actions/cpp-actions/actions/cpp-matrix.html#_usage, you might never realize that the tests are not checked (that was the case for me 🙂). Moreover, even if you check the output when adding the action, you'll see that tests run. Since the tests should usually just pass, you'll expect the result to be checked.

In this PR I added a label to the group. Hopefully people will then check what it's about. I wanted to avoid changing the default, but I think by default the tests should be checked. However, --no-tests=ignore should still be passed to avoid failing if no tests are present.

@codecov

codecov Bot commented Jun 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.38%. Comparing base (456b4c6) to head (386ce15).

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff            @@
##           develop      #34   +/-   ##
========================================
  Coverage    95.37%   95.38%           
========================================
  Files          117      116    -1     
  Lines        10948    10938   -10     
  Branches      2659     2658    -1     
========================================
- Hits         10442    10433    -9     
+ Misses         474      473    -1     
  Partials        32       32           
Flag Coverage Δ
cpp ?
typescript 95.38% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
cmake-workflow/src/process-entry.ts 92.16% <100.00%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@alandefreitas

Copy link
Copy Markdown
Owner

Yes. I agree. The design at the time was a little more controversial than that, but it's time to change it. The thing is, there were people using this action to just build and install things. And then get blocked by tests they didn't ask for. That's why it became the default to check tests only if the person explicitly asked for tests. The reading was like "the user didn't ask for tests so there are no tests here but we can't stop cmake from doing it so we won't check it". Or something like that. Yes. It needs to change. Thanks for the PR.

@alandefreitas

Copy link
Copy Markdown
Owner

Sorry. I think I wasn't clear enough.

I think by default the tests should be checked

If you'd like, you can modify the PR to do exactly that, and I'll adjust whichever library I have accordingly so it doesn't depend on the old behavior.

@Nerixyz Nerixyz changed the title cmake-workflow: Show indicator when tests are unchecked cmake-workflow: Always check test results if they run Jun 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants