Skip to content

Commit c15632a

Browse files
committed
Coverage that cannot report a green it did not earn
Two parts of the coverage path report success while doing nothing. Both are in the logs of any recent run. The six matrix legs all uploaded an artifact named `coverage` with `overwrite: true`, which deletes the previous artifact rather than merging into it. Run 30461091931 shows `Artifact 'coverage' (ID: 8727950028) deleted` and then finalized again by a later leg, so five of the six reports were thrown away and the surviving one was whichever leg happened to finish last. Each leg now writes and uploads a report named after itself, and the upload job downloads all of them. The headline number was not far off, since all six legs land on 80%, but which platform's report reached Codecov was decided by a race, and code that only runs on one platform was counted or not depending on it. `files:` was a YAML block literal, so the name arrived as "coverage.xml\n" and was not found. The upload only worked because the uploader falls back to searching. That fallback is now unnecessary: the reports are in one directory, and the directory is what is passed. `fail_ci_if_error` is tied to whether a token was there to use. A pull request from a fork gets no secrets, and a contributor should not see red for that. What this does not fix: `CODECOV_TOKEN` resolves to empty on this repository, so the upload is still rejected with "Token required - not valid tokenless upload", and with no token the run stays green. Setting that secret is the last step and it needs repository admin. Once it is set, this workflow fails when an upload fails, without another change. Upstream RocketPy carries both of these unchanged. Happy to send the same patch there so it comes back through the usual sync. Verified: actionlint clean, `--cov-report=xml:<name>` writes the named file, and `pattern` / `merge-multiple` / `directory` / `fail_ci_if_error` all exist as inputs on the pinned actions. Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
1 parent 6ec4a2c commit c15632a

1 file changed

Lines changed: 22 additions & 7 deletions

File tree

.github/workflows/test_pytest.yaml

Lines changed: 22 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -61,26 +61,41 @@ jobs:
6161
run: pytest tests/integration --cov=rocketpy --cov-append
6262

6363
- name: Run Acceptance Tests
64-
run: pytest tests/acceptance --cov=rocketpy --cov-append --cov-report=xml
64+
run: pytest tests/acceptance --cov=rocketpy --cov-append --cov-report=xml:coverage-${{ matrix.os }}-${{ matrix.python-version }}.xml
6565

6666
- name: Upload coverage to artifacts
6767
uses: actions/upload-artifact@main
6868
with:
69-
name: coverage
70-
path: coverage.xml
71-
overwrite: true
69+
# One name per leg. Six legs sharing a name with overwrite deleted each
70+
# other's artifact rather than merging, so five reports were discarded
71+
# and the surviving one was whichever leg happened to finish last.
72+
name: coverage-${{ matrix.os }}-${{ matrix.python-version }}
73+
path: coverage-${{ matrix.os }}-${{ matrix.python-version }}.xml
7274
if-no-files-found: error
7375

7476
CodecovUpload:
7577
needs: Pytest
7678
runs-on: ubuntu-latest
7779
steps:
7880
- uses: actions/checkout@main
79-
- name: Download latest coverage report
81+
- name: Download every coverage report
8082
uses: actions/download-artifact@main
83+
with:
84+
pattern: coverage-*
85+
merge-multiple: true
86+
path: coverage-reports
87+
- name: Refuse to upload nothing
88+
# `ls` exits non-zero on no match. Without this the upload has nothing to
89+
# send and still reports success, which is the failure being fixed here.
90+
run: ls coverage-reports/*.xml
8191
- name: Upload to Codecov
8292
uses: codecov/codecov-action@main
8393
with:
8494
token: ${{ secrets.CODECOV_TOKEN }}
85-
files: |
86-
coverage.xml
95+
# A directory rather than a filename: there is one report per matrix
96+
# leg now. The previous block literal passed "coverage.xml\n", which was
97+
# not found, and only the fallback search made the upload work at all.
98+
directory: coverage-reports
99+
# Only fail when a token was there to use. Pull requests from forks get
100+
# no secrets, and a contributor should not see red for that.
101+
fail_ci_if_error: ${{ secrets.CODECOV_TOKEN != '' }}

0 commit comments

Comments
 (0)