ci: replace archived codeclimate-action with qlty-action#49
Conversation
paambaati/codeclimate-action is archived and the CodeClimate test-reporter download now returns 404, breaking coverage upload on Ruby 3.1. Migrate to qltysh/qlty-action/coverage@v2 per the qlty migration guide (https://docs.qlty.sh/migration/coverage). Uses OIDC for auth so no coverage token secret is required. Points at the SimpleCov JSON formatter output already produced by the test suite.
|
Note Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported. |
WalkthroughGitHub Actions workflow for the Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~3 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
Migrates CI coverage upload in the Ruby test workflow away from the archived paambaati/codeclimate-action to Qlty’s coverage action, using OIDC authentication.
Changes:
- Adds
id-token: write(and explicitcontents: read) permissions to thetestjob for OIDC-based auth. - Replaces the CodeClimate action step with
qltysh/qlty-action/coverage@v2targetingcoverage/coverage.jsonwithformat: simplecov.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| CC_TEST_REPORTER_ID: "${{ secrets.CC_TEST_REPORTER_ID }}" | ||
| with: | ||
| oidc: true | ||
| files: coverage/coverage.json |
There was a problem hiding this comment.
The workflow uploads coverage/coverage.json, but the repo’s SimpleCov setup (spec/spec_helper.rb) only calls SimpleCov.start and does not configure the JSON formatter, so coverage/coverage.json is unlikely to be generated. Either update the SimpleCov configuration to emit coverage/coverage.json via simplecov_json_formatter, or point files: at the SimpleCov output file that is actually produced (e.g., .resultset.json if that’s what Qlty expects for format: simplecov).
| files: coverage/coverage.json | |
| files: coverage/.resultset.json |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/rubygems.yml:
- Around line 59-65: The "Upload coverage to Qlty Cloud" step can fail the job
when Qlty/OIDC or project setup is incomplete; make it non-blocking by adding
continue-on-error: true and pass the action input skip-errors: true to
qltysh/qlty-action/coverage@v2, and avoid running it when the artifact is
missing by adding a prior step (e.g., "check-coverage-artifact") that tests for
coverage/coverage.json and sets outputs.coverage_exists, then change this step's
if to: matrix.ruby-version == '3.1' &&
steps.check-coverage-artifact.outputs.coverage_exists == 'true' so the upload
only runs when the file exists and never fails the workflow.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 5754bcc8-9d17-4d64-9996-c1c79ef38334
📒 Files selected for processing (1)
.github/workflows/rubygems.yml
| - name: Upload coverage to Qlty Cloud | ||
| uses: qltysh/qlty-action/coverage@v2 | ||
| if: matrix.ruby-version == '3.1' | ||
| env: | ||
| CC_TEST_REPORTER_ID: "${{ secrets.CC_TEST_REPORTER_ID }}" | ||
| with: | ||
| oidc: true | ||
| files: coverage/coverage.json | ||
| format: simplecov |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
For qltysh/qlty-action/coverage@v2, what is the default behavior when upload/auth fails, and is there a supported input like skip-errors to avoid failing CI?
💡 Result:
The default behavior of qltysh/qlty-action/coverage@v2 when upload/auth fails is to fail the GitHub Action step, causing the CI workflow to fail. This is evidenced by GitHub issue #145 where the action crashes on upload attempt with an authentication-related panic/error, and official docs describe authentication failures (e.g., invalid token, OIDC issues on forks) as errors that prevent successful uploads. There is a supported input skip-errors that can be set to true to avoid failing CI on certain errors. In issue #145, a user explicitly used skip-errors: false (implying default may be true or configurable), and the action continued processing files before failing on upload/parsing. Setting skip-errors: true likely skips upload failures without failing the step, though official action.yml/docs do not explicitly list all inputs (action is a wrapper around qlty coverage publish, which has --no-validate but no direct skip-auth flag).
Citations:
- 1: Crash on upload attempt qltysh/qlty-action#145
- 2: https://docs.qlty.sh/coverage/quickstart
- 3: https://docs.qlty.sh/coverage/troubleshooting
- 4: https://docs.qlty.sh/cli/commands/coverage-publish
🏁 Script executed:
# First, let's examine the actual workflow file to see the current state
cat -n .github/workflows/rubygems.yml | sed -n '59,65p'Repository: bskim45/fastlane-plugin-google_drive
Length of output: 361
🏁 Script executed:
# Check if there's any documentation or action.yml in the repo related to qlty-action
find . -type f -name "*.yml" -o -name "*.yaml" -o -name "action.yml" | xargs grep -l "qlty" 2>/dev/null | head -20Repository: bskim45/fastlane-plugin-google_drive
Length of output: 112
🌐 Web query:
qltysh/qlty-action/coverage@v2 skip-errors input documentation
💡 Result:
The qltysh/qlty-action/coverage@v2 GitHub Action is a reusable action for uploading code coverage data to Qlty Cloud. It requires a free Qlty Cloud account. Usage example from official Qlty documentation: - uses: qltysh/qlty-action/coverage@v2 with: oidc: true files: coverage/lcov.info Or with token: - uses: qltysh/qlty-action/coverage@v2 with: token: ${{ secrets.QLTY_COVERAGE_TOKEN }} files: coverage/lcov.info The skip-errors input is shown in a real-world usage example from GitHub issue #145 in the repository: - uses: qltysh/qlty-action/coverage@v2 with: oidc: true files: app/coverage.lcov skip-errors: false add-prefix: app/ This suggests skip-errors is a boolean input (default likely true) that controls whether to skip (continue on) errors during coverage processing or upload, such as parsing failures. Setting it to false makes the action fail on errors, as seen in the crash report where a parsing panic occurred. Other common inputs from examples and CLI mapping (qlty coverage publish): - token: Qlty coverage token (alternative to oidc) - oidc: Use OIDC authentication (boolean) - files: Path(s) to coverage report file(s) - tag: Coverage tag for organization - add-prefix: Prefix to add to file paths - total-parts-count: For server-side merging - format: Coverage format (inferred or specified) The action wraps the qlty coverage publish CLI command, passing inputs as flags. Full inputs are defined in the action's action.yml file (not directly surfaced in search snippets), but skip-errors allows error-tolerant uploads, useful for malformed reports. For complete documentation, see Qlty Docs: https://docs.qlty.sh/coverage/quickstart and repository: https://github.com/qltysh/qlty-action
Citations:
- 1: https://github.com/qltysh/qlty-action/blob/main/README.md
- 2: https://docs.qlty.sh/coverage/troubleshooting
- 3: https://docs.qlty.sh/coverage/tags
- 4: Crash on upload attempt qltysh/qlty-action#145
- 5: https://docs.qlty.sh/cli/commands/coverage-publish
- 6: https://docs.qlty.sh/coverage/quickstart
🌐 Web query:
qltysh/qlty-action action.yml inputs skip-errors default value
💡 Result:
The default value for the 'skip-errors' input in qltysh/qlty-action's action.yml is false. This input is used in the qltysh/qlty-action/coverage action to control whether to skip errors during coverage upload (e.g., parsing failures). It is explicitly set to skip-errors: false in usage examples from issue reports.
Citations:
🌐 Web query:
GitHub Actions continue-on-error vs skip-errors best practice workflow reliability
💡 Result:
skip-errors is not a recognized GitHub Actions feature. The standard mechanism for handling errors without halting execution is continue-on-error: true, available at both job and step levels. Key differences and behaviors from official documentation and community sources: - Step level (jobs.<job_id>.steps[*].continue-on-error): When true, a step failure does not fail the job. The step's outcome remains 'failure' (accessible via steps.<step_id>.outcome), but conclusion is 'success'. Subsequent steps run normally. The job shows as failed if any non-continued step fails. Use for non-critical steps like optional coverage checks or notifications, where you want to upload artifacts/logs despite failure. Example: - name: Optional check id: check continue-on-error: true run: exit 1 - name: Upload if failed if: steps.check.outcome == 'failure' run: echo "Failed but continuing" - Job level (jobs.<job_id>.continue-on-error): When true, a job failure does not fail the workflow. The job shows as failed (red X), but dependent jobs (via needs) can run if their if allows (e.g., if: always). Use for experimental matrix jobs. Example for matrix: test: continue-on-error: ${{ matrix.experimental }} strategy: matrix: experimental: [true, false] For workflow reliability best practices: - Prefer if: success || failure or if: !cancelled on cleanup/upload steps over continue-on-error at step level. This runs always (except cancel), keeps failure visible, no special outcome checking needed. Sources confirm this shows correct failure status while continuing. - For matrices: strategy.fail-fast: false + no continue-on-error lets all jobs run, workflow fails if any does (preferred for full visibility). - Use steps..outcome to check true failure after continue-on-error. - Avoid overusing continue-on-error as it can mask issues; combine with notifications (e.g., PR comments on failure). - Set timeout-minutes and retry for resilience. This ensures reliability: failures are visible, workflows continue for cleanup/completeness, no silent ignores.
Citations:
- 1: https://docs.github.com/actions/using-workflows/workflow-syntax-for-github-actions
- 2: https://docs.github.com/actions/reference/workflows-and-actions/workflow-syntax
- 3: https://stackoverflow.com/questions/58858429/how-to-run-a-github-actions-step-even-if-the-previous-step-fails-while-still-f
- 4: https://stackoverflow.com/questions/62045967/is-there-a-way-to-continue-on-error-while-still-getting-correct-feedback
- 5: https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions
- 6: https://notes.kodekloud.com/docs/GitHub-Actions/Continuous-Integration-with-GitHub-Actions/Using-continue-on-error-expression/page
- 7: https://www.kenmuse.com/blog/how-to-handle-step-and-job-errors-in-github-actions/
- 8: https://docs.github.com/en/actions/learn-github-actions/contexts
- 9: https://docs.github.com/en/enterprise-server@3.0/actions/reference/workflow-syntax-for-github-actions
Make coverage upload explicitly non-blocking (per PR objective).
On lines 59-65, this step can still fail the job when Qlty Cloud/OIDC/project setup is incomplete. The action defaults to failing on errors (skip-errors defaults to false), so add a hard non-blocking guard at the GitHub Actions level and skip when the coverage artifact is missing.
Proposed workflow hardening
- name: Upload coverage to Qlty Cloud
uses: qltysh/qlty-action/coverage@v2
- if: matrix.ruby-version == '3.1'
+ if: matrix.ruby-version == '3.1' && hashFiles('coverage/coverage.json') != ''
+ continue-on-error: true
with:
oidc: true
files: coverage/coverage.json
format: simplecov🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/workflows/rubygems.yml around lines 59 - 65, The "Upload coverage to
Qlty Cloud" step can fail the job when Qlty/OIDC or project setup is incomplete;
make it non-blocking by adding continue-on-error: true and pass the action input
skip-errors: true to qltysh/qlty-action/coverage@v2, and avoid running it when
the artifact is missing by adding a prior step (e.g., "check-coverage-artifact")
that tests for coverage/coverage.json and sets outputs.coverage_exists, then
change this step's if to: matrix.ruby-version == '3.1' &&
steps.check-coverage-artifact.outputs.coverage_exists == 'true' so the upload
only runs when the file exists and never fails the workflow.
Summary
paambaati/codeclimate-actionis archived and the CodeClimate test-reporter download now returns 404, breaking coverage upload on Ruby 3.1 (see chore(deps): update actions/checkout action to v6 #47 / chore(deps): update actions/github-script action to v9 #48 CI logs).qltysh/qlty-action/coverage@v2per the qlty migration guide.id-token: writepermission to thetestjob.coverage/coverage.json(SimpleCov JSON formatter output) withformat: simplecov.Follow-ups
skip-errors: trueby default, so failures here won't break CI.CC_TEST_REPORTER_IDsecret can be removed from the repo settings.ostructGemfile mismatch — tracked separately.Test plan
Summary by CodeRabbit