Skip to content

Stabilize Codecov coverage and test-result reporting in CI#53

Merged
dex3r merged 4 commits into
mainfrom
copilot/fix-codecov-issue
Mar 4, 2026
Merged

Stabilize Codecov coverage and test-result reporting in CI#53
dex3r merged 4 commits into
mainfrom
copilot/fix-codecov-issue

Conversation

Copilot AI commented Mar 4, 2026

Copy link
Copy Markdown
Contributor

Codecov reporting intermittently stopped attaching coverage to commits, resulting in missing commit-level status/details despite tests producing coverage files. The CI workflow was uploading in tokenless mode, which was rejected in failing runs.

  • Root cause

    • codecov-action@v5 was executing without a token/OIDC path and failed with Token required - not valid tokenless upload on affected runs.
    • CI was uploading coverage files, but test-result reporting upload was not configured.
    • After adding test-results upload, Codecov reported No JUnit XML reports found because the .NET test run did not produce JUnit files by default.
  • Workflow changes

    • Enabled GitHub OIDC token issuance for the CI job:
      • permissions.id-token: write
    • Configured Codecov coverage upload to use OIDC auth explicitly:
      • use_oidc: true
    • Added Codecov test-results upload at the end of the test run:
      • codecov/test-results-action@v1
      • if: ${{ !cancelled() }}
      • token: ${{ secrets.CODECOV_TOKEN }}
      • files: ./coverage/*.xml
      • disable_search: true
    • Updated test execution to emit TRX results:
      • --logger "trx;LogFilePrefix=test-results"
    • Added TRX → JUnit conversion step before test-results upload:
      • install/update pinned tool version: trx2junit 2.1.0
      • convert discovered *.trx files in ./coverage
      • gracefully handle the no-TRX case
    • Kept existing coverage upload behavior and failure policy intact.
  • Resulting CI shape

    jobs:
      test:
        permissions:
          contents: read
          id-token: write
        steps:
          - name: Test
            run: dotnet test EasySourceGenerators.sln --no-restore --verbosity normal --collect:"XPlat Code Coverage" --logger "trx;LogFilePrefix=test-results" --results-directory ./coverage
    
          - name: Upload coverage to Codecov
            uses: codecov/codecov-action@v5
            with:
              directory: ./coverage
              use_oidc: true
              fail_ci_if_error: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
    
          - name: Convert TRX to JUnit XML
            if: ${{ !cancelled() }}
            run: |
              if dotnet tool list --global | grep -q '^trx2junit '; then
                dotnet tool update --global trx2junit --version 2.1.0
              else
                dotnet tool install --global trx2junit --version 2.1.0
              fi
              TRX_FILES="$(find ./coverage -type f -name '*.trx')"
              if [ -n "$TRX_FILES" ]; then
                trx2junit $TRX_FILES
              else
                echo "No TRX files found in ./coverage."
              fi
    
          - name: Upload test results to Codecov
            if: ${{ !cancelled() }}
            uses: codecov/test-results-action@v1
            with:
              token: ${{ secrets.CODECOV_TOKEN }}
              files: ./coverage/*.xml
              disable_search: true

> [!WARNING]
>
>

Original prompt

This section details on the original issue you should resolve

<issue_title>Codecov doesnt seem to be working</issue_title>
<issue_description>Codecov doesnt seem to be working anymore. Figure out why.

Image</issue_description>

Comments on the Issue (you are @copilot in this section)


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: dex3r <3155725+dex3r@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix Codecov integration not working Stabilize Codecov uploads by switching CI to OIDC-authenticated reporting Mar 4, 2026
Copilot finished work on behalf of dex3r March 4, 2026 19:32
@dex3r

dex3r commented Mar 4, 2026

Copy link
Copy Markdown
Owner

Also add this @copilot

image

Copilot AI commented Mar 4, 2026

Copy link
Copy Markdown
Contributor Author

Also add this @copilot

image

...

I can add it, but I can’t access that second image content from this environment. Please paste the exact text/snippet from the screenshot (or describe the setting you want added), and I’ll apply it right away in a follow-up commit.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • github-production-user-asset-6210df.s3.amazonaws.com
    • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot finished work on behalf of dex3r March 4, 2026 19:36
@dex3r

dex3r commented Mar 4, 2026

Copy link
Copy Markdown
Owner

It says:

In your CI YAML file, add below scripts to the end of your test run.

- name: Upload test results to Codecov
  if: ${{ !cancelled() }}
  uses: codecov/test-results-action@v1
  with:
    token: ${{ secrets.CODECOV_TOKEN }}

@copilot

Co-authored-by: dex3r <3155725+dex3r@users.noreply.github.com>

Copilot AI commented Mar 4, 2026

Copy link
Copy Markdown
Contributor Author

It says:

In your CI YAML file, add below scripts to the end of your test run.
...

Implemented in commit 9d7ae9e. I added the codecov/test-results-action@v1 step at the end of the test job with if: ${{ !cancelled() }} and token: ${{ secrets.CODECOV_TOKEN }}.

Copilot AI changed the title Stabilize Codecov uploads by switching CI to OIDC-authenticated reporting Stabilize Codecov coverage and test-result reporting in CI Mar 4, 2026
Copilot finished work on behalf of dex3r March 4, 2026 19:56
@codecov

codecov Bot commented Mar 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!

@dex3r

dex3r commented Mar 4, 2026

Copy link
Copy Markdown
Owner
info - 2026-03-04 19:56:37,484 -- Found 0 test_results files to report
error - 2026-03-04 19:56:37,485 -- No JUnit XML reports found. Please review our documentation (https://docs.codecov.com/docs/test-result-ingestion-beta) to generate and upload the file.
info - 2026-03-04 19:56:37,802 -- No test results reports found. Triggering notifications without uploading.
info - 2026-03-04 19:56:37,802 -- Process Upload complete

it looks like codecov is expecting JUnit, can dotnet nunit be used here? @copilot

Co-authored-by: dex3r <3155725+dex3r@users.noreply.github.com>

Copilot AI commented Mar 4, 2026

Copy link
Copy Markdown
Contributor Author
info - 2026-03-04 19:56:37,484 -- Found 0 test_results files to report
error - 2026-03-04 19:56:37,485 -- No JUnit XML reports found. Please review our documentation (https://docs.codecov.com/do...

Yes — NUnit on .NET can be used here. I fixed this in 7ccb219 by making CI emit TRX from dotnet test, converting TRX to JUnit XML with trx2junit, and uploading those XML files to Codecov (files: ./coverage/*.xml). This addresses the No JUnit XML reports found message.

Copilot finished work on behalf of dex3r March 4, 2026 20:08
@dex3r dex3r marked this pull request as ready for review March 4, 2026 20:12
Copilot AI review requested due to automatic review settings March 4, 2026 20:12
@dex3r dex3r merged commit f1ad451 into main Mar 4, 2026
4 checks passed
@dex3r dex3r deleted the copilot/fix-codecov-issue branch March 4, 2026 20:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the CI workflow to make Codecov uploads more reliable by using GitHub OIDC for coverage and by adding test-result reporting for the .NET test run.

Changes:

  • Enable GitHub OIDC token issuance for the CI job and configure codecov-action@v5 to use OIDC.
  • Update dotnet test to emit TRX results and add a TRX → JUnit conversion step.
  • Add codecov/test-results-action@v1 to upload converted test results.
Comments suppressed due to low confidence (1)

.github/workflows/ci.yml:46

  • The workflow installs/updates the trx2junit global tool before checking whether any *.trx files exist. If tests fail before producing results (or result logging is disabled later), this adds an unnecessary network step. Consider checking for TRX files first and only installing/updating trx2junit when conversion is actually needed.
          if dotnet tool list --global | grep -q '^trx2junit '; then
            dotnet tool update --global trx2junit --version 2.1.0
          else
            dotnet tool install --global trx2junit --version 2.1.0
          fi
          TRX_FILES="$(find ./coverage -type f -name '*.trx')"
          if [ -n "$TRX_FILES" ]; then
            trx2junit $TRX_FILES

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/ci.yml
Comment on lines +51 to +57
- name: Upload test results to Codecov
if: ${{ !cancelled() }}
uses: codecov/test-results-action@v1
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./coverage/*.xml
disable_search: true

Copilot AI Mar 4, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

codecov/test-results-action is always executed, even when the previous step finds no TRX files (and therefore produces no JUnit XML). This can cause the workflow to fail or produce noisy Codecov errors like “No JUnit XML reports found”. Consider conditionally running this step only when converted JUnit files exist (e.g., have the conversion step set an output flag when it finds TRX/JUnit files and use that in the if: expression).

Copilot uses AI. Check for mistakes.
Comment thread .github/workflows/ci.yml
Comment on lines +54 to +56
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./coverage/*.xml

Copilot AI Mar 4, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step relies on secrets.CODECOV_TOKEN, but on pull_request workflows secrets are not provided to runs from forked repositories. If a fork PR triggers this workflow, the token will be empty and the action may fail. Consider guarding the step with a token-present condition (or switching to an auth method that works for fork PRs).

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Codecov doesnt seem to be working

3 participants