Coverage: five of six matrix reports are discarded, and the upload cannot fail the job

[thc1006]

test_pytest.yaml runs a 3x2 matrix and every leg ends with

      - name: Upload coverage to artifacts
        uses: actions/upload-artifact@main
        with:
          name: coverage
          path: coverage.xml
          overwrite: true

overwrite: true deletes the existing artifact and creates a new one rather than merging into it, so the six legs race and five results are dropped. This is visible from the API on any recent run without needing the logs. Taking 30239994837 on develop:

Pytest (ubuntu-latest, 3.10)   success
Pytest (ubuntu-latest, 3.14)   success
Pytest (macos-latest, 3.10)    success
Pytest (macos-latest, 3.14)    success
Pytest (windows-latest, 3.10)  success
Pytest (windows-latest, 3.14)  success
CodecovUpload                  success

$ gh api repos/RocketPy-Team/RocketPy/actions/runs/30239994837/artifacts --jq .total_count
1

One artifact, named coverage, 45494 bytes. CodecovUpload downloads that one, so whichever leg finished last is the entire picture Codecov receives.

The headline percentage is probably close to right, since the legs mostly agree. What the race decides is which platform's report arrives, so a line that only executes on Windows is counted or not depending on timing rather than on whether a test covered it.

The uploaded file is found by a fallback

          files: |
            coverage.xml

A YAML block literal keeps the trailing newline, so the name arrives as "coverage.xml\n" and is not matched. The upload still works because the uploader falls back to searching the workspace and happens to find the file.

I saw this in a fork running this file unchanged:

warning -- Some files were not found --- {"not_found_files": ["coverage.xml\n"]}
info    -- Found 1 coverage files to report

I cannot read this repository's Actions logs, so I am not claiming the same line appears here. The block literal is in the file either way, and the fallback is what the upload currently depends on.

Why neither has been noticed

fail_ci_if_error defaults to false, so nothing in the upload path can turn the job red. Whatever else happens, CodecovUpload reports success.

A shape that fixes both

Give each leg its own report name, download all of them into one directory, and pass the directory:

      - name: Run Acceptance Tests
        run: pytest tests/acceptance --cov=rocketpy --cov-append --cov-report=xml:coverage-${{ matrix.os }}-${{ matrix.python-version }}.xml

      - name: Upload coverage to artifacts
        uses: actions/upload-artifact@main
        with:
          name: coverage-${{ matrix.os }}-${{ matrix.python-version }}
          path: coverage-${{ matrix.os }}-${{ matrix.python-version }}.xml
          if-no-files-found: error

      - name: Download every coverage report
        uses: actions/download-artifact@main
        with:
          pattern: coverage-*
          merge-multiple: true
          path: coverage-reports
      - name: Refuse to upload nothing
        run: ls coverage-reports/*.xml
      - name: Upload to Codecov
        uses: codecov/codecov-action@main
        with:
          token: ${{ secrets.CODECOV_TOKEN }}
          directory: coverage-reports
          fail_ci_if_error: ${{ secrets.CODECOV_TOKEN != '' }}

Tying fail_ci_if_error to whether a token was available keeps pull requests from forks unaffected, since they get no secrets and a contributor should not see red for that.

This is running in a fork of this repository. actionlint is clean, --cov-report=xml:<name> writes the named file, and pattern, merge-multiple, directory and fail_ci_if_error all exist as inputs on the pinned actions.

Happy to open a PR with it if the shape looks right, or to adjust if you would rather have one canonical coverage job and let the matrix legs only prove the tests pass per platform.