Skip to content

Migrate MyPy coverage from Codecov to Coveralls - #758

Merged
bdraco merged 1 commit into
aio-libs:masterfrom
aiolibsbot:koan/implement-756
May 17, 2026
Merged

Migrate MyPy coverage from Codecov to Coveralls#758
bdraco merged 1 commit into
aio-libs:masterfrom
aiolibsbot:koan/implement-756

Conversation

@aiolibsbot

@aiolibsbot aiolibsbot commented May 17, 2026

Copy link
Copy Markdown
Contributor

What do these changes do?

Port aio-libs/multidict#1231 to frozenlist: move the MyPy type preciseness coverage report to Coveralls instead of Codecov, and remove the typing flag from the Codecov configuration.

  • Swap the codecov/codecov-action step in reusable-linters.yml for coverallsapp/github-action@v2 with cobertura format and MyPy flag.
  • Drop the codecov-token secret plumbing from ci-cd.yml and reusable-linters.yml (Coveralls auth is via the GHA token).
  • Remove the typing patch/project flags from .codecov.yml and decrement after_n_builds from 20 to 19.
  • Add CHANGES/756.contrib.rst describing the move.

Are there changes in behavior for the user?

No end-user behaviour change. CI-only: MyPy type preciseness coverage now appears on Coveralls instead of Codecov.

Related issue number

Closes #756.

Checklist

  • I think the code is well written
  • Unit tests for the changes exist
  • Documentation reflects the changes
  • If you provide code modifications, please add yourself to CONTRIBUTORS.txt
    • The format is .
    • Please keep the list in alphabetical order, the file is sorted by name.
  • Add a new news fragment into the CHANGES folder
    • name it <issue_id>.<category> for example (588.bugfix)
    • if you don't have an issue_id change it to the pr id after creating the pr
    • ensure category is one of the following:
      • .bugfix: Signifying a bug fix.
      • .feature: Signifying a new feature.
      • .breaking: Signifying a breaking change or removal of something public.
      • .doc: Signifying a documentation improvement.
      • .packaging: Signifying a packaging or tooling change that may be relevant to downstreams.
      • .contrib: Signifying an improvement to the contributor/development experience.
      • .misc: Anything that does not fit the above; usually, something not of interest to users.
    • Make sure to use full sentences with correct case and punctuation, for example: "Fix issue with non-ascii contents in doctest text files."

Quality Report

Changes: 4 files changed, 14 insertions(+), 18 deletions(-)

Code scan: clean

Tests: failed (FAILED)

Branch hygiene: clean

Generated by Kōan post-mission quality pipeline

@bdraco

bdraco commented May 17, 2026

Copy link
Copy Markdown
Member

@aiolibsbot Use the pr template for aiolibs

@bdraco

bdraco commented May 17, 2026

Copy link
Copy Markdown
Member

@aiolibsbot Rr

@aiolibsbot

aiolibsbot commented May 17, 2026

Copy link
Copy Markdown
Contributor Author

PR Review — Move reporting MyPy type preciseness to Coveralls

Mechanically a clean port of aio-libs/multidict#1231: Codecov action swapped for Coveralls, codecov-token plumbing removed from ci-cd.yml and reusable-linters.yml, after_n_builds decremented by one, and a 756.contrib.rst changelog entry added. The shell-quoted runner.debug == '1' comparisons are already in canonical string form, so no concern there. The main thing to verify before merging is the loss of the 100% MyPy coverage gate that was enforced by the removed typing component in .codecov.yml — Coveralls won't enforce that automatically from this YAML alone, so either the gate needs to be recreated on the Coveralls side or the loss needs to be explicitly accepted. Secondary item: the CI-GHA flag is silently dropped because coverallsapp/github-action@v2 only takes a single flag-name. Also worth following @bdraco's request to align the PR description with the aio-libs template (and the rules referenced in PR #759).


🟡 Important

1. 100% MyPy coverage gate is dropped with no Coveralls equivalent (`.codecov.yml`, L40-44)

The removed typing component had target: 100% against the MyPy flag, which acted as a hard CI gate that would fail when MyPy type preciseness regressed. Moving the report to Coveralls is fine, but this diff does not recreate that gate anywhere: Coveralls' per-flag thresholds are configured in the Coveralls project settings or a .coveralls.yml, not in the GHA coverallsapp/github-action@v2 invocation. The Coveralls step's fail-on-error: true only catches upload failures, not coverage threshold violations.

Confirm one of the following: (a) the 100% gate has already been recreated on the Coveralls side (out-of-band project config), (b) it is intentional to drop the hard gate and rely on review-time inspection of the dashboard, or (c) a follow-up PR will add the gate. Without that, a future MyPy regression will silently produce a dashboard delta instead of failing CI.

-      typing:
-        flags:
-        - MyPy
-        target: 100%

🟢 Suggestions

1. `CI-GHA` flag is dropped in the migration (`.github/workflows/reusable-linters.yml`, L65-74)

The previous Codecov step uploaded under two flags: CI-GHA and MyPy. The new Coveralls step only sets flag-name: MyPy. coverallsapp/github-action@v2 does not accept a comma-separated flag-name list (only a single flag), so dropping CI-GHA is a forced consequence of the migration, not a bug — but it's worth being explicit about in the changelog or PR description in case any downstream dashboard view or org-wide convention relied on grouping by CI-GHA. If multidict's port (aio-libs/multidict#1231) accepted the same loss without follow-up, this is fine.

-        flags: >-
-          CI-GHA,
-          MyPy
+        flag-name: MyPy
2. Verify `after_n_builds: 19` matches the actual upload count (`.codecov.yml`, L5)

The comment on this line says "The number of test matrix+lint jobs uploading coverage". Decrementing 20→19 is consistent with removing exactly one Codecov upload (the MyPy/typing one from the lint job). Worth a quick sanity check that no other Codecov upload step elsewhere in the workflows was also removed or added in flight — if Codecov waits for 19 uploads but only 18 actually arrive, the report will hang for the full wait window before flushing.

    after_n_builds: 19  # The number of test matrix+lint jobs uploading coverage

Checklist

  • No hardcoded secrets or token leakage
  • CI gate parity preserved across migration — warning #1
  • Upload count (after_n_builds) matches actual uploaders — suggestion #3
  • Action input syntax valid for coverallsapp/github-action@v2
  • Changelog fragment present and correctly categorized
  • PR description follows aio-libs template

Summary

Mechanically a clean port of aio-libs/multidict#1231: Codecov action swapped for Coveralls, codecov-token plumbing removed from ci-cd.yml and reusable-linters.yml, after_n_builds decremented by one, and a 756.contrib.rst changelog entry added. The shell-quoted runner.debug == '1' comparisons are already in canonical string form, so no concern there. The main thing to verify before merging is the loss of the 100% MyPy coverage gate that was enforced by the removed typing component in .codecov.yml — Coveralls won't enforce that automatically from this YAML alone, so either the gate needs to be recreated on the Coveralls side or the loss needs to be explicitly accepted. Secondary item: the CI-GHA flag is silently dropped because coverallsapp/github-action@v2 only takes a single flag-name. Also worth following @bdraco's request to align the PR description with the aio-libs template (and the rules referenced in PR #759).


Automated review by Kōan38db447
86a8eaa

@bdraco

bdraco commented May 17, 2026

Copy link
Copy Markdown
Member

Follow the rules in #759

@bdraco

bdraco commented May 17, 2026

Copy link
Copy Markdown
Member

@aiolibsbot Rr

@aiolibsbot

Copy link
Copy Markdown
Contributor Author

Rebase with requested adjustments

Branch koan/implement-756 was rebased onto master and review feedback was applied.

Changes applied

  • Done. Summary:
  • .github/workflows/reusable-linters.yml: Changed runner.debug == 1 to runner.debug == '1' (string comparison) on all three Coveralls action inputs (debug, fail-on-error, measure). Per review suggestion: GitHub Actions sets runner.debug to string '1'; explicit string comparison matches the canonical GHA documented pattern instead of relying on implicit coercion.
  • PR description: Rewrote PR body to follow the aio-libs PULL_REQUEST_TEMPLATE.md structure (What do these changes do? / Behavior changes / Related issue / Checklist) per @bdraco's request to use the aiolibs PR template.

Stats

4 files changed, 14 insertions(+), 18 deletions(-)
Actions performed
  • Already-solved check: negative (confidence=high, reasoning=Master still uses codecov-action for MyPy coverage with typing flag in .codecov.yml; no commit migra)
  • Rebased koan/implement-756 onto upstream/master
  • Applied review feedback
  • Pre-push CI check: no CI runs found
  • Force-pushed koan/implement-756 to origin
  • CI check enqueued in ## CI (async)

CI status

CI will be checked asynchronously.


Automated by Kōan

aiolibsbot added a commit to aiolibsbot/frozenlist that referenced this pull request May 17, 2026
@aiolibsbot

Copy link
Copy Markdown
Contributor Author

Rebase with requested adjustments

Branch koan/implement-756 was rebased onto master and review feedback was applied.

Stats

4 files changed, 14 insertions(+), 18 deletions(-)
Actions performed
  • Already-solved check: negative (confidence=high, reasoning=Master still uses codecov-action with MyPy flag and retains typing block in .codecov.yml; no Coveral)
  • Rebased koan/implement-756 onto upstream/master
  • Pre-push CI check: no CI runs found
  • Force-pushed koan/implement-756 to origin
  • CI check enqueued in ## CI (async)

CI status

CI will be checked asynchronously.


Automated by Kōan

@bdraco
bdraco marked this pull request as ready for review May 17, 2026 03:45
@bdraco
bdraco enabled auto-merge (squash) May 17, 2026 03:45
Move type coverage reporting to Coveralls per aio-libs/multidict#1231.
Updates workflows to use coverallsapp/github-action, removes codecov-token
secret, and drops typing config from .codecov.yml.
@aiolibsbot aiolibsbot changed the title Move reporting MyPy type preciseness to Coveralls Migrate MyPy coverage from Codecov to Coveralls May 17, 2026
auto-merge was automatically disabled May 17, 2026 04:09

Head branch was pushed to by a user without write access

@aiolibsbot
aiolibsbot force-pushed the koan/implement-756 branch from dc7e2cd to aef89a2 Compare May 17, 2026 04:09
@aiolibsbot

Copy link
Copy Markdown
Contributor Author

Squash: 3 commits → 1

Branch koan/implement-756 was squashed and force-pushed.

Commit message

ci: migrate MyPy coverage from Codecov to Coveralls

Move type coverage reporting to Coveralls per aio-libs/multidict#1231.
Updates workflows to use coverallsapp/github-action, removes codecov-token
secret, and drops typing config from .codecov.yml.

Actions

  • Squashed 3 commits into 1
  • Force-pushed koan/implement-756 to origin
  • Updated PR title
  • Updated PR description

Automated by Koan

@bdraco
bdraco enabled auto-merge (squash) May 17, 2026 04:10
@codecov

codecov Bot commented May 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (2392a08) to head (aef89a2).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff             @@
##           master      #758      +/-   ##
===========================================
+ Coverage   95.01%   100.00%   +4.98%     
===========================================
  Files          10         2       -8     
  Lines         763       370     -393     
  Branches       51        26      -25     
===========================================
- Hits          725       370     -355     
+ Misses         13         0      -13     
+ Partials       25         0      -25     
Flag Coverage Δ
CI-GHA 99.72% <ø> (+4.84%) ⬆️
MyPy ?
OS-Linux 99.72% <ø> (ø)
OS-Windows 99.72% <ø> (ø)
OS-macOS 99.72% <ø> (ø)
Py-3.10.11 99.72% <ø> (ø)
Py-3.10.20 99.72% <ø> (ø)
Py-3.11.15 99.72% <ø> (ø)
Py-3.11.9 99.72% <ø> (ø)
Py-3.12.10 99.72% <ø> (ø)
Py-3.12.13 99.72% <ø> (ø)
Py-3.13.13 99.72% <ø> (ø)
Py-3.13.13t 99.72% <ø> (ø)
Py-3.14.4 99.72% <ø> (ø)
Py-3.14.5 99.72% <ø> (ø)
Py-3.14.5t 99.72% <ø> (ø)
Py-3.9.13 99.72% <ø> (ø)
Py-3.9.25 99.72% <ø> (ø)
Py-pypy3.10.16-7.3.19 98.91% <ø> (ø)
Py-pypy3.9.19-7.3.16 98.91% <ø> (ø)
VM-macos-latest 99.72% <ø> (ø)
VM-ubuntu-latest 99.72% <ø> (ø)
VM-windows-11-arm 99.72% <ø> (ø)
VM-windows-latest 99.72% <ø> (ø)
pytest 99.72% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@aiolibsbot

Copy link
Copy Markdown
Contributor Author

❌ Unknown command use. Did you mean usage?

Code & Development

  • 🔎 @aiolibsbot audit — Audit a project for optimizations, simplifications, and issues — creates GitHub issues for findings
  • 🧠 @aiolibsbot brainstorm — Break down a topic into detailed sub-issues grouped under a master issue
  • 🧠 @aiolibsbot deepplan (alias: deeplan) — Deep design an idea — explores approaches, posts spec as GitHub issue, queues /plan
  • 🐞 @aiolibsbot fix — "Queue a fix mission for a GitHub issue — understand, plan, test, implement, and submit a PR. Can also batch-queue all open issues from a repo URL. Use --now to queue at the top."
  • 🔨 @aiolibsbot implement (alias: impl) — "Queue an implementation mission for a GitHub issue"
  • 🧠 @aiolibsbot plan — Plan an idea or iterate on an existing GitHub issue
  • 📊 @aiolibsbot profile (alias: perf, benchmark) — Queue a performance profiling mission
  • 🛠️ @aiolibsbot refactor (alias: rf) — "Queue a refactoring mission for a PR, issue, or file"
  • 🔍 @aiolibsbot review (alias: rv) — "Queue a code review for a PR or issue. Use --now to queue at the top."
  • 🛡️ @aiolibsbot security_audit (alias: security, secu) — SDLC security audit — finds critical vulnerabilities and creates GitHub issues for each

Pull Requests

  • @aiolibsbot ask — "Ask a question about a PR or issue and get an AI reply posted to GitHub"
  • 🔀 @aiolibsbot gh_request — "Route a natural-language GitHub request to the appropriate action (fix, rebase, review, reply, etc.)"
  • 🔄 @aiolibsbot rebase (alias: rb) — "Queue a PR rebase (ex: /rebase https://github.com/owner/repo/pull/42). Use --now to queue at the top."
  • 🔁 @aiolibsbot recreate (alias: rc) — "Recreate a diverged PR from scratch on current upstream (ex: /recreate https://github.com/owner/repo/pull/42)"
  • 🔍 @aiolibsbot reviewrebase (alias: rr) — "Queue /review then /rebase for a PR — review insights feed the rebase"
  • 🔄 @aiolibsbot squash (alias: sq) — "Squash PR commits into one (ex: /squash https://github.com/owner/repo/pull/42)"

Usage: @aiolibsbot <command> in any PR or issue comment.

@bdraco
bdraco merged commit a84747f into aio-libs:master May 17, 2026
63 of 64 checks passed
@aiolibsbot
aiolibsbot deleted the koan/implement-756 branch May 17, 2026 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Switch typing coverage tracking to use coveralls like aio-libs/multidict#1231

2 participants