Skip to content

ci: separate PR coverage from test matrix#2180

Merged
Pouyanpi merged 2 commits into
developfrom
pouyanpi/untangle-coverage-pr-test
Jul 17, 2026
Merged

ci: separate PR coverage from test matrix#2180
Pouyanpi merged 2 commits into
developfrom
pouyanpi/untangle-coverage-pr-test

Conversation

@Pouyanpi

@Pouyanpi Pouyanpi commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Description

Separates Python 3.11 coverage from the PR test matrix so test timing and coverage reporting are tracked independently.

Related Issue(s)

Verification

AI Assistance

  • No AI tools were used.
  • AI tools were used; a human reviewed and can explain every change (tool: ___).

Checklist

  • I've read the CONTRIBUTING guidelines.
  • This PR links to a triaged issue assigned to me.
  • My PR title follows the project commit convention.
  • I've updated the documentation if applicable.
  • I've added tests if applicable.
  • I've noted any verification beyond CI and any checks I couldn't run.
  • I did not update generated changelog files manually.
  • I addressed all CodeRabbit, Greptile, and other review comments, or replied with why no change is needed.
  • @mentions of the person or team responsible for reviewing proposed changes.

Summary by CodeRabbit

  • New Features

    • Added automated pull request coverage checks with concurrency controls.
    • Added reusable coverage workflow support for consistent testing environments.
    • Added clear completion summaries for coverage checks.
  • Improvements

    • Separated coverage execution from standard pull request tests.
    • Added skip summaries when pull requests contain no code changes.

@Pouyanpi Pouyanpi self-assigned this Jul 17, 2026
@github-actions github-actions Bot added size: S status: needs triage New issues that have not yet been reviewed or categorized. labels Jul 17, 2026
@Pouyanpi Pouyanpi added status: triaged Triaged by a maintainer; eligible for automated review (CodeRabbit/Greptile). and removed status: needs triage New issues that have not yet been reviewed or categorized. labels Jul 17, 2026
@greptile-apps

greptile-apps Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR separates coverage reporting from the PR test matrix by introducing a reusable _coverage.yml workflow that always runs Python 3.11 with with-coverage: true, called from a new pr-coverage.yml for PRs and reused by the existing develop-coverage.yml. The pr-tests.yml matrix no longer carries coverage configuration or the CODECOV_TOKEN secret.

  • New _coverage.yml: centralizes the os=Ubuntu / python=3.11 / with-coverage=true configuration, eliminating the duplicated with: block that previously existed in both develop-coverage.yml and (via matrix include) pr-tests.yml.
  • New pr-coverage.yml: triggers on code-only PRs, delegates to _coverage.yml, and emits a pr-coverage-summary gate check. pr-tests-skip.yml gains a matching skip job so doc-only PRs satisfy the same required check without running tests.
  • Note: "3.11" remains in the pr-tests.yml matrix, so Python 3.11 tests will still execute twice per code PR — once without coverage in the matrix and once with coverage in pr-coverage.yml. This was flagged in a previous review and is a known trade-off of the current approach.

Confidence Score: 5/5

Safe to merge; the changes are a clean CI refactor that correctly wires up the new coverage workflow and skip gates without breaking any existing test paths.

All five workflow files have clear, consistent trigger rules, the reusable workflow nesting is within GitHub's supported depth, and the doc-only skip path correctly satisfies the new required check. No logic errors, missing guards, or broken secrets wiring were found in the changed code.

No files require special attention beyond the pre-existing Python 3.11 double-run noted in the previous review of pr-tests.yml.

Important Files Changed

Filename Overview
.github/workflows/_coverage.yml New reusable workflow that abstracts the Python 3.11 + coverage configuration, used by both develop and PR coverage workflows.
.github/workflows/pr-coverage.yml New PR-scoped coverage workflow that fires on code changes, delegates to _coverage.yml, and emits a pr-coverage-summary gate check.
.github/workflows/pr-tests.yml Removes coverage-specific matrix include and CODECOV_TOKEN secret; Python 3.11 remains in the matrix, running tests twice per PR alongside pr-coverage.yml (pre-existing noted issue).
.github/workflows/pr-tests-skip.yml Adds a pr-coverage-summary skip job so doc-only PRs satisfy the new required status check without running coverage.
.github/workflows/develop-coverage.yml Refactored to call the new _coverage.yml reusable workflow instead of duplicating hardcoded parameters; clean simplification.

Reviews (2): Last reviewed commit: "ci: centralize coverage workflow configu..." | Re-trigger Greptile

@coderabbitai

coderabbitai Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Coverage execution is moved into a reusable workflow, PR coverage is added with completion reporting, develop coverage is rerouted, and standard PR tests no longer pass coverage settings. Skipped PRs now report skipped coverage.

Changes

Coverage CI workflow

Layer / File(s) Summary
Dedicated coverage execution
.github/workflows/_coverage.yml, .github/workflows/develop-coverage.yml, .github/workflows/pr-coverage.yml
A reusable coverage workflow runs the shared test workflow with fixed Ubuntu, Python 3.11, and coverage settings; develop and pull-request workflows invoke it.
Test and skipped-job routing
.github/workflows/pr-tests.yml, .github/workflows/pr-tests-skip.yml
The PR test matrix stops forwarding Codecov settings, while skipped PRs receive a coverage summary job.

Estimated code review effort: 2 (Simple) | ~10 minutes

Sequence Diagram(s)

sequenceDiagram
  participant PullRequest as pull_request event
  participant PRCoverage as PR Coverage
  participant CoverageWorkflow as _coverage.yml
  participant TestWorkflow as _test.yml
  participant CoverageSummary as coverage summary
  PullRequest->>PRCoverage: trigger on eligible changes
  PRCoverage->>CoverageWorkflow: invoke with CODECOV_TOKEN
  CoverageWorkflow->>TestWorkflow: run with coverage enabled
  PRCoverage->>CoverageSummary: report completion after coverage
Loading
🚥 Pre-merge checks | ✅ 6
✅ Passed checks (6 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Test Results For Major Changes ✅ Passed The PR is workflow-only coverage/test CI refactoring, so it qualifies as a minor change; no test results are required in the description.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: separating PR coverage from the test matrix.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch pouyanpi/untangle-coverage-pr-test

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added size: M and removed size: S labels Jul 17, 2026
@Pouyanpi

Copy link
Copy Markdown
Collaborator Author

@greptileai

The duplicate execution is intentional. The regular matrix verifies uninstrumented compatibility across every supported Python version, while the separate workflow isolates coverage timing and reporting. Removing Python 3.11 from the matrix would make that matrix incomplete and couple supported-version maintenance to the coverage configuration. We prefer the extra runner cost over those semantics.

@greptile-apps

greptile-apps Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

That's a clear and reasonable trade-off — keeping the matrix complete across all supported versions is worth the extra runner cost, and coupling version maintenance to coverage config would be a real maintenance hazard down the line. Makes sense.

Shall I remember this as a rule: "In this repository, running Python 3.11 in both the PR test matrix and the coverage workflow is intentional — do not flag it as duplicate execution"?

@Pouyanpi
Pouyanpi merged commit 8cfaa12 into develop Jul 17, 2026
13 checks passed
@Pouyanpi
Pouyanpi deleted the pouyanpi/untangle-coverage-pr-test branch July 17, 2026 11:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size: M status: triaged Triaged by a maintainer; eligible for automated review (CodeRabbit/Greptile).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant