Skip to content

Add JaCoCo test coverage configuration and CI reporting#448

Open
devin-ai-integration[bot] wants to merge 5 commits into
masterfrom
choikh-test-coverage-demo
Open

Add JaCoCo test coverage configuration and CI reporting#448
devin-ai-integration[bot] wants to merge 5 commits into
masterfrom
choikh-test-coverage-demo

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Mar 13, 2026

Copy link
Copy Markdown

Summary

Adds the JaCoCo Gradle plugin to build.gradle to enable test coverage reporting and enforcement, and adds a GitHub Actions workflow that posts before/after coverage percentages as a PR comment.

build.gradle changes

  • Adds jacoco plugin (toolVersion 0.8.8)
  • Configures jacocoTestReport to generate HTML and XML reports after each test run
  • Configures jacocoTestCoverageVerification with a 34% line coverage minimum threshold (matches current actual coverage)
  • Wires testjacocoTestReport (via finalizedBy) and checkjacocoTestCoverageVerification (via dependsOn)
  • Replaces the existing test task block (merging useJUnitPlatform() into the new block; also fixes tab→space indentation)

.github/workflows/coverage.yml (new)

  • Runs on PRs targeting master
  • Builds and tests both the PR branch and the base branch, then parses JaCoCo XML to extract line coverage percentages for each
  • Posts a before/after comparison table as a PR comment via actions/github-script@v6 (with emoji diff indicator: 🟢 up / 🔴 down / ⚪ unchanged). Comment is upserted using a hidden HTML marker so re-runs update in place.
  • Also posts per-file coverage details via madrapps/jacoco-report@v1.6.1
  • Includes Gradle caching for faster CI runs
  • If the base branch does not have JaCoCo configured (e.g. on the first run before this PR merges), the workflow gracefully skips base coverage and shows "No JaCoCo configured" instead

Review & Testing Checklist for Human

  • Verify XML parsing robustness. The workflow extracts the report-level counter type="LINE" from JaCoCo XML using grep -oP ... | tail -1. This relies on the last <counter type="LINE" .../> in the file being the report-level summary. Confirm this holds for the current JaCoCo version (0.8.8) and consider whether a more robust parser (e.g. xmllint) would be preferable.
  • Test the before/after comparison end-to-end. Since master does not currently have JaCoCo, the first run of this workflow will show "No JaCoCo configured" for the base branch. After merging, open a trivial follow-up PR to confirm the workflow posts a proper before/after table with a numeric diff.
  • Check the min-coverage-changed-files: 60 threshold in the madrapps/jacoco-report step. This requires 60% line coverage on files changed in a PR, which may be aggressive depending on which files are touched.
  • Confirm ./gradlew jacocoTestCoverageVerification passes locally. The 34% threshold was set to match measured coverage — verify it still holds.

Suggested test plan: Merge this PR, then open a small follow-up PR that touches a Java file. Confirm the workflow posts two comments: (1) a before/after coverage table with a numeric diff, and (2) per-file coverage details from madrapps/jacoco-report.

Notes

  • The pre-existing spotlessCheck failure on master is not introduced by this PR.
  • The 34% coverage threshold was set to match the current actual line coverage. A // TODO comment in build.gradle notes it should be raised over time.
  • The workflow uses actions/checkout@v3, actions/setup-java@v3, and actions/cache@v3, which run on Node.js 20 (deprecated by GitHub starting June 2026). Consider upgrading to v4 in a follow-up.

Requested by: @choikh0423
Link to Devin session


Open with Devin

Co-Authored-By: Kyu Choi <kyuhwanchoi0423@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

Co-Authored-By: Kyu Choi <kyuhwanchoi0423@gmail.com>

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 2 additional findings.

Open in Devin Review

Co-Authored-By: Kyu Choi <kyuhwanchoi0423@gmail.com>
@devin-ai-integration devin-ai-integration Bot changed the title Add JaCoCo test coverage configuration Add JaCoCo test coverage configuration and CI reporting Mar 13, 2026
@github-actions

Copy link
Copy Markdown

Test Coverage Report

Overall Project 33.21% 🔴

There is no coverage information present for the Files changed

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration Bot and others added 2 commits March 13, 2026 01:46
Co-Authored-By: Kyu Choi <kyuhwanchoi0423@gmail.com>
Co-Authored-By: Kyu Choi <kyuhwanchoi0423@gmail.com>
@github-actions

Copy link
Copy Markdown

Test Coverage Report

Metric Coverage
Base branch (master) 34.06%
PR branch (choikh-test-coverage-demo) 34.06%
Diff ⚪ ±0.00%

@github-actions

Copy link
Copy Markdown

Per-File Coverage Details

Overall Project 33.21% 🔴

There is no coverage information present for the Files changed

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 new potential issues.

View 10 additional findings in Devin Review.

Open in Devin Review

} else {
body += `| **Base branch** (\`${{ github.base_ref }}\`) | _No JaCoCo configured_ |\n`;
}
body += `| **PR branch** (\`${{ github.head_ref }}\`) | ${prCoverage}% |\n`;

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Script injection via github.head_ref in JavaScript template literal

On line 127, ${{ github.head_ref }} is interpolated directly into a JavaScript template literal (backtick string). Since github.head_ref is attacker-controlled (the PR author chooses their branch name), and Git branch names can contain $, {, and } characters, an attacker can craft a branch name like exploit${require('child_process').execSync('malicious command')} which would be evaluated as JavaScript code inside the template literal. The workflow has pull-requests: write permissions, so the attacker could use this to post arbitrary comments, exfiltrate the GITHUB_TOKEN, or read repository contents. The fix is to pass github.head_ref as an environment variable and reference it via process.env inside the script instead of using ${{ }} interpolation.

Vulnerable interpolation pattern
# Line 127 - github.head_ref injected into JS template literal
body += `| **PR branch** (\`${{ github.head_ref }}\`) | ${prCoverage}% |\n`;

Should instead use:

env:
  HEAD_REF: ${{ github.head_ref }}
  BASE_REF: ${{ github.base_ref }}
script: |
  const headRef = process.env.HEAD_REF;
  ...
  body += `| **PR branch** (\`${headRef}\`) | ${prCoverage}% |\n`;
Prompt for agents
In .github/workflows/coverage.yml, the actions/github-script step at line 96-157 directly interpolates github.head_ref and github.base_ref using ${{ }} syntax inside JavaScript template literals, which allows script injection. Fix by:

1. Add an `env` block to the step (after `with:` at line 97):
   env:
     HEAD_REF: ${{ github.head_ref }}
     BASE_REF: ${{ github.base_ref }}

2. Inside the script block, replace all uses of ${{ github.head_ref }} with process.env.HEAD_REF and all uses of ${{ github.base_ref }} with process.env.BASE_REF.

Specifically, change lines 123, 125, and 127 from:
  body += `| **Base branch** (\`${{ github.base_ref }}\`) | ${baseCoverage}% |\n`;
to:
  body += `| **Base branch** (\`${process.env.BASE_REF}\`) | ${baseCoverage}% |\n`;

And line 127 from:
  body += `| **PR branch** (\`${{ github.head_ref }}\`) | ${prCoverage}% |\n`;
to:
  body += `| **PR branch** (\`${process.env.HEAD_REF}\`) | ${prCoverage}% |\n`;
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

echo "PR branch line coverage: $pct% (missed=$missed, covered=$covered)"

- name: Checkout base branch
run: git checkout ${{ github.base_ref }}

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Shell injection via unquoted github.base_ref in run: block

On line 58, ${{ github.base_ref }} is interpolated directly into a shell run: command without quotes: git checkout ${{ github.base_ref }}. While the workflow restricts to PRs targeting master (mitigating this somewhat), directly interpolating GitHub context values into shell commands is a security anti-pattern. If the workflow trigger was ever broadened to allow other base branches, a branch name containing shell metacharacters could execute arbitrary commands. The safe pattern is to use an environment variable: env: BASE_REF: ${{ github.base_ref }} and then git checkout "$BASE_REF".

Prompt for agents
In .github/workflows/coverage.yml line 57-58, change the step to use an environment variable instead of direct interpolation:

  - name: Checkout base branch
    run: git checkout "$BASE_REF"
    env:
      BASE_REF: ${{ github.base_ref }}

This prevents shell injection if the workflow trigger is ever broadened to allow base branches with special characters.
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant