Skip to content

ci: Upload coverage to GitHub Code Quality#1036

Merged
Aaron ("AJ") Steers (aaronsteers) merged 10 commits into
mainfrom
devin/1779860596-github-coverage-upload
May 27, 2026
Merged

ci: Upload coverage to GitHub Code Quality#1036
Aaron ("AJ") Steers (aaronsteers) merged 10 commits into
mainfrom
devin/1779860596-github-coverage-upload

Conversation

@aaronsteers

@aaronsteers Aaron ("AJ") Steers (aaronsteers) commented May 27, 2026

Copy link
Copy Markdown
Member

Summary

  • Add actions/upload-code-coverage@v1 uploads for PyAirbyte's existing Cobertura XML reports from Pytest Fast, Pytest No Creds, and the representative full pytest matrix job.
  • Scope code-quality: write to only the pytest jobs that upload coverage while keeping workflow-level read permissions for contents and pull-requests.
  • Limit uploaded coverage XML to airbyte/* package files so GitHub Code Quality does not report stdlib or test-helper paths.
  • Keep existing GitHub artifact uploads intact and make Code Quality uploads best-effort with fail-on-error: false.
  • Add an actionlint ignore for GitHub's new public-preview code-quality permission scope until actionlint recognizes it upstream.

Review & Testing Checklist for Human

  • Confirm GitHub Code Quality is enabled for this repository so uploaded coverage appears on PRs.
  • Confirm the three coverage labels are useful and not too noisy in the PR coverage UI.
  • Let the Run Tests and actionlint workflows run once on this PR and confirm the Code Quality upload steps complete or produce actionable preview-period errors.

Notes

  • Requested by AJ Steers in Slack.
  • Local validation: yq e '.' .github/workflows/python_pytest.yml, yq e '.' .github/workflows/actionlint.yml, and git diff --check.

Devin session
Requested by: Aaron ("AJ") Steers (@aaronsteers)

Summary by CodeRabbit

  • Chores
    • Enhanced code coverage workflow: limited coverage collection to project code and adjusted when reports are uploaded (always for fast/no-creds runs; full report from one specified environment).
    • Tightened CI permissions with explicit top-level and job-level permission settings.
    • Updated CI linting to suppress a specific workflow lint warning.

Review Change Stack

@devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@github-actions

Copy link
Copy Markdown

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This PyAirbyte Version

You can test this version of PyAirbyte using the following:

# Run PyAirbyte CLI from this branch:
uvx --from 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1779860596-github-coverage-upload' pyairbyte --help

# Install PyAirbyte from this branch for development:
pip install 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1779860596-github-coverage-upload'

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /fix-pr - Fixes most formatting and linting issues
  • /uv-lock - Updates uv.lock file
  • /test-pr - Runs tests with the updated PyAirbyte
  • /prerelease - Builds and publishes a prerelease version to PyPI
📚 Show Repo Guidance

Helpful Resources

Community Support

Questions? Join the #pyairbyte channel in our Slack workspace.

📝 Edit this welcome message.

@coderabbitai

coderabbitai Bot commented May 27, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@devin-ai-integration[bot], we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 3 minutes and 44 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: ac8f1a73-65da-4f46-9b54-37225bd1c38f

📥 Commits

Reviewing files that changed from the base of the PR and between 2e5f272 and 8d18182.

📒 Files selected for processing (1)
  • .github/workflows/python_pytest.yml
📝 Walkthrough

Walkthrough

The workflows declare explicit workflow- and job-level permissions (adding code-quality: write to test jobs), narrow coverage XML to --include="airbyte/*" in test jobs, add actions/upload-code-coverage@v1 upload steps guarded with if: always() and fail-on-error: false, and silence actionlint's code-quality permission warning.

Changes

Code Coverage Reporting

Layer / File(s) Summary
Workflow & Job Permissions and coverage include flags
.github/workflows/python_pytest.yml
Declares explicit workflow- and job-level permissions (adds code-quality: write to test jobs) and updates coverage XML generation to include only airbyte/* for pytest-fast, pytest-no-creds, and pytest jobs.
Coverage upload steps
.github/workflows/python_pytest.yml
Adds actions/upload-code-coverage@v1 steps to upload htmlcov/coverage.xml for pytest-fast, pytest-no-creds, and the full pytest matrix (restricted to Python 3.10 on Ubuntu), each with if: always() and fail-on-error: false.

Actionlint configuration

Layer / File(s) Summary
actionlint flag
.github/workflows/actionlint.yml
Adds actionlint_flags and a TODO comment to ignore unknown permission scope "code-quality" warnings from actionlint.

🎯 3 (Moderate) | ⏱️ ~20 minutes

Would you like a suggested short commit message for these workflow edits, wdyt?

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately reflects the main change: adding GitHub Code Quality coverage uploads to the CI workflow across multiple pytest jobs.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch devin/1779860596-github-coverage-upload

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
.github/workflows/python_pytest.yml (1)

22-25: ⚡ Quick win

Could we scope code-quality: write to only the three coverage-upload jobs instead of workflow-wide, wdyt?

Right now code-quality: write is granted to every job, including dependency-analysis, which doesn’t need it. Moving that permission to job-level would tighten least-privilege with minimal churn.

Suggested diff
 permissions:
   contents: read
-  code-quality: write
   pull-requests: read

 jobs:
   pytest-fast:
+    permissions:
+      code-quality: write
     name: Pytest (Fast)
@@
   pytest-no-creds:
+    permissions:
+      code-quality: write
     name: Pytest (No Creds)
@@
   pytest:
+    permissions:
+      code-quality: write
     name: Pytest (All, Python ${{ matrix.python-version }}, ${{ matrix.os }})
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/python_pytest.yml around lines 22 - 25, The workflow
currently grants code-quality: write at the top-level permissions block,
applying to every job; change this by removing or setting code-quality to read
in the top-level permissions and instead add permissions: contents: read
code-quality: write pull-requests: read under only the three coverage-upload job
definitions (the jobs named coverage-upload-*, or whichever three
coverage-upload job names appear) so that dependency-analysis and other jobs no
longer get code-quality: write; update each coverage-upload job's job-level
permissions field accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In @.github/workflows/python_pytest.yml:
- Around line 22-25: The workflow currently grants code-quality: write at the
top-level permissions block, applying to every job; change this by removing or
setting code-quality to read in the top-level permissions and instead add
permissions: contents: read code-quality: write pull-requests: read under only
the three coverage-upload job definitions (the jobs named coverage-upload-*, or
whichever three coverage-upload job names appear) so that dependency-analysis
and other jobs no longer get code-quality: write; update each coverage-upload
job's job-level permissions field accordingly.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 320aa104-cc9b-44e8-9978-6891d64d36b0

📥 Commits

Reviewing files that changed from the base of the PR and between 8427840 and 6d052d2.

📒 Files selected for processing (1)
  • .github/workflows/python_pytest.yml

@github-code-quality

github-code-quality Bot commented May 27, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: Python

Python / code-coverage/pytest-fast

The overall coverage in the branch is 65%. Coverage data for the branch is not yet available.

Show a code coverage summary of the most covered files.
File 8d18182 +/-
airbyte/progress.py 88%
airbyte/_connector_base.py 85%
airbyte/exceptions.py 84%
airbyte/_proces...ql/snowflake.py 80%
airbyte/shared/...ql_processor.py 77%
airbyte/caches/base.py 72%
airbyte/registry.py 70%
airbyte/sources/base.py 68%
airbyte/mcp/cloud.py 52%
airbyte/_util/api_util.py 36%

Python / code-coverage/pytest-no-creds

The overall coverage in the branch is 65%. Coverage data for the branch is not yet available.

Show a code coverage summary of the most covered files.
File 8d18182 +/-
airbyte/progress.py 88%
airbyte/_connector_base.py 85%
airbyte/exceptions.py 84%
airbyte/_proces...ql/snowflake.py 80%
airbyte/shared/...ql_processor.py 78%
airbyte/caches/base.py 72%
airbyte/sources/base.py 70%
airbyte/registry.py 70%
airbyte/mcp/cloud.py 52%
airbyte/_util/api_util.py 36%

Python / code-coverage/pytest

The overall coverage in the branch is 71%. Coverage data for the branch is not yet available.

Show a code coverage summary of the most covered files.
File 8d18182 +/-
airbyte/_proces...ql/snowflake.py 98%
airbyte/progress.py 89%
airbyte/_connector_base.py 85%
airbyte/exceptions.py 84%
airbyte/shared/...ql_processor.py 82%
airbyte/_util/api_util.py 72%
airbyte/sources/base.py 71%
airbyte/registry.py 70%
airbyte/mcp/cloud.py 52%
airbyte/cloud/connectors.py 51%

Updated May 27, 2026 07:20 UTC
Code Coverage is in Public Preview. Learn more and provide us with your feedback.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
.github/workflows/actionlint.yml (1)

22-22: ⚡ Quick win

Consider adding a comment explaining this is temporary, wdyt?

Since the code-quality permission scope is in public preview and not yet recognized by actionlint, it might be helpful to add an inline comment noting this ignore is temporary until actionlint adds support. This would help future maintainers understand why the ignore exists and when it can be removed.

📝 Suggested comment addition
      - name: Run actionlint
        uses: reviewdog/action-actionlint@95395aac8c053577d0bc67eb7b74936c660c6f66 # v1.67.0
        with:
+         # Ignore code-quality permission warning until actionlint supports GitHub's public-preview scope
          actionlint_flags: -ignore 'unknown permission scope "code-quality"'
          github_token: ${{ secrets.GITHUB_TOKEN }}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/actionlint.yml at line 22, Add a brief inline comment
explaining that the -ignore 'unknown permission scope "code-quality"' in the
actionlint_flags is temporary because the code-quality permission is in public
preview and currently unrecognized by actionlint; update the
.github/workflows/actionlint.yml near the actionlint_flags entry (referencing
actionlint_flags and the exact ignore string) to include a # comment indicating
this can be removed once actionlint adds support and optionally include a link
or note about when to re-evaluate.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In @.github/workflows/actionlint.yml:
- Line 22: Add a brief inline comment explaining that the -ignore 'unknown
permission scope "code-quality"' in the actionlint_flags is temporary because
the code-quality permission is in public preview and currently unrecognized by
actionlint; update the .github/workflows/actionlint.yml near the
actionlint_flags entry (referencing actionlint_flags and the exact ignore
string) to include a # comment indicating this can be removed once actionlint
adds support and optionally include a link or note about when to re-evaluate.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: a0dda7b0-6a3b-41f5-a585-baecaab8061b

📥 Commits

Reviewing files that changed from the base of the PR and between 6d052d2 and 0d15f5c.

📒 Files selected for processing (2)
  • .github/workflows/actionlint.yml
  • .github/workflows/python_pytest.yml
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/workflows/python_pytest.yml

@aaronsteers Aaron ("AJ") Steers (aaronsteers) marked this pull request as ready for review May 27, 2026 06:36
Copilot AI review requested due to automatic review settings May 27, 2026 06:36

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

View 3 additional findings in Devin Review.

Open in Devin Review

Comment thread .github/workflows/python_pytest.yml

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR integrates the existing pytest coverage output with GitHub Code Quality while keeping the existing coverage artifacts.

Changes:

  • Adds Code Quality coverage upload steps to pytest CI jobs.
  • Narrows generated Cobertura XML reports to the airbyte/* package scope.
  • Adds a temporary actionlint ignore for the preview code-quality permission.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
.github/workflows/python_pytest.yml Adds workflow/job permissions, filtered coverage XML generation, and Code Quality upload steps.
.github/workflows/actionlint.yml Suppresses actionlint’s unsupported preview permission warning for code-quality.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/python_pytest.yml
Comment thread .github/workflows/python_pytest.yml
@aaronsteers Aaron ("AJ") Steers (aaronsteers) merged commit 2047741 into main May 27, 2026
21 checks passed
@aaronsteers Aaron ("AJ") Steers (aaronsteers) deleted the devin/1779860596-github-coverage-upload branch May 27, 2026 14:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants