Skip to content

[codex] fix unit test coverage reporting#3319

Draft
JasonW404 wants to merge 2 commits into
developfrom
fix/test-coverage-on-cicd
Draft

[codex] fix unit test coverage reporting#3319
JasonW404 wants to merge 2 commits into
developfrom
fix/test-coverage-on-cicd

Conversation

@JasonW404

Copy link
Copy Markdown
Member

Summary

Fix the parallel unit-test runner so CI coverage reporting matches the pre-parallel behavior while preserving file-level parallel execution. Also fix Codecov uploader selection on ARM GitHub Actions runners.

Root Cause

PR #3285 changed unit tests from serial --cov-append into per-file coverage data files followed by coverage combine. The new flow only checked that coverage.xml existed and did not fully restore the previous coverage API reporting behavior. The combine/xml phase also needed to consistently use test/.coveragerc without relying on an unsupported --rcfile placement for the installed coverage version.

The unit-test workflow runs on ubuntu-24.04-arm, but codecov-action@v4 can mis-detect the uploader platform as plain linux. The uploader then fails during create-commit with a shell syntax error while the workflow stays green because fail_ci_if_error is false.

Changes

  • Pass COVERAGE_RCFILE to coverage combine/xml so both phases use test/.coveragerc.
  • Restore COVERAGE_PROCESS_START in per-file pytest subprocesses.
  • Restore coverage API report validation after combine.
  • Restore HTML coverage output under test/coverage_html.
  • Restore unit-test summary output: total, passed, failed, pass rate.
  • Restore detailed failed test-file reporting.
  • Upgrade Codecov upload from codecov-action@v4 to @v5 and set os: linux-arm64 for ARM runners.

Validation

  • python3 -m py_compile test/run_all_test.py
  • git diff --check
  • Full local CI setup matching workflow:
    • uv sync --extra data-process --extra test
    • uv pip install -e "../sdk[dev]"
  • Full local UT run completed before the coverage combine fix and produced all .coverage.* files.
  • Re-ran coverage combine/report against those files successfully: generated test/coverage.xml, emitted coverage report, total coverage about 78.8%.
  • Smoke test:
    • NEXENT_PYTEST_TARGETS='test/sdk/core/models/test_prompt_cache.py' NEXENT_PYTEST_WORKERS=1 NEXENT_PYTEST_FILE_TIMEOUT=120 backend/.venv/bin/python test/run_all_test.py
    • Passed and generated test/coverage.xml plus test/coverage_html/.

@codecov

codecov Bot commented Jun 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant