chore(ci): enhance Python E2E and SonarCloud workflows with unit and and integration tests by SumanMaharana · Pull Request #26481 · open-metadata/OpenMetadata

SumanMaharana · 2026-03-13T12:34:09Z

Describe your changes:

This pull request updates the Python test workflows in GitHub Actions to improve test coverage reporting, enable test sharding for integration tests, and streamline the execution and artifact handling for both unit and integration tests. The changes introduce separate jobs for unit and integration tests, shard integration tests for better parallelism, and add a new job to combine coverage data before pushing results to SonarCloud.

Test Workflow Improvements

Split the Python test workflow into separate jobs for unit tests (py-unit-tests) and integration tests (py-integration-tests), with the integration tests now sharded into two groups for parallel execution.
Updated test execution steps to use nox for running both unit and integration tests, replacing previous make commands and ensuring consistent test environments. [1] [2]

Coverage Collection and Aggregation

Modified artifact upload steps to ensure coverage files from both unit and integration tests are correctly collected from the ingestion/.coverage path and include hidden files. [1] [2]
Added a new py-combine-coverage job to download all coverage artifacts, prepare and rename them as needed, and then combine the results using a dedicated nox session before pushing to SonarCloud.

These changes will result in more reliable and granular test reporting, improved parallelism for integration tests, and accurate combined coverage metrics for SonarCloud analysis.

Type of change:

Checklist:

I have read the CONTRIBUTING document.
My PR title is Fixes <issue-number>: <short explanation>
I have commented on my code, particularly in hard-to-understand areas.
For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

…integration tests

Copilot

Pull request overview

This PR refactors the GitHub Actions Python testing workflows to improve SonarCloud coverage reporting and increase parallelism by splitting unit vs. integration execution (including sharded integration runs), then aggregating coverage before uploading to SonarCloud.

Changes:

Split the nightly SonarCloud ingestion workflow into separate unit + sharded integration jobs, plus a coverage-combine job.
Updated the CLI E2E workflow matrix to replace the single python entry with python-unittests and python-integration, and adjusted artifact uploads/conditions accordingly.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
.github/workflows/py-sonarcloud-nightly.yml	Splits unit/integration testing, shards integration tests, and adds a job to combine coverage before SonarCloud scan.
.github/workflows/py-cli-e2e-tests.yml	Updates the E2E matrix to separate Python unit vs. integration execution and changes coverage artifact handling.

You can also share your feedback on Copilot code review. Take the survey.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR updates GitHub Actions workflows to improve Python CI signal for ingestion by splitting unit vs. integration test execution, adding integration test sharding for parallelism, and aggregating coverage before publishing to SonarCloud.

Changes:

Split SonarCloud nightly ingestion workflow into separate unit + sharded integration jobs, with a dedicated coverage-combine job.
Switch Python test execution to nox sessions for unit/integration runs.
Update the CLI E2E workflow matrix to run Python unit tests and Python integration tests as separate entries and upload their coverage artifacts.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
.github/workflows/py-sonarcloud-nightly.yml	Splits unit/integration testing, shards integration tests, downloads artifacts, combines coverage via `nox`, then runs Sonar scan.
.github/workflows/py-cli-e2e-tests.yml	Replaces `python` entry with `python-unittests` + `python-integration`, runs corresponding `nox` sessions, and uploads coverage artifacts.

You can also share your feedback on Copilot code review. Take the survey.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR updates GitHub Actions workflows to run Python unit and integration tests via nox, shard integration tests for parallelism, and aggregate coverage artifacts before publishing results to SonarCloud.

Changes:

Split the nightly Sonar workflow into dedicated unit-test, sharded integration-test, and coverage-combine/Sonar upload jobs.
Updated the CLI E2E workflow matrix to run python-unittests and python-integration as distinct entries using nox.
Adjusted coverage artifact handling (including hidden .coverage* files) to support downstream aggregation.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
.github/workflows/py-sonarcloud-nightly.yml	Introduces separate unit/integration jobs, shards integration tests, and adds a combine-coverage job before Sonar scan.
.github/workflows/py-cli-e2e-tests.yml	Splits the prior “python” entry into `python-unittests` + `python-integration` and updates coverage artifact naming/conditions.

You can also share your feedback on Copilot code review. Take the survey.


-    - name: Run Python Tests & record coverage
-      if: matrix.e2e-test == 'python'
-      id: python-e2e-test
+    - name: Run Python Integration Tests
+      if: matrix.e2e-test == 'python-integration'
+      id: python-integration-test
+      continue-on-error: true
      run: |
        source env/bin/activate
-        make coverage
+        cd ingestion
+        nox --no-venv -s integration-tests -- --standalone --durations=5
+      env:


+    - name: Run Python Unit Tests
+      if: matrix.e2e-test == 'python-unittests'
+      id: python-unittest
+      continue-on-error: true
+      run: |
+        source env/bin/activate
+        cd ingestion
+        nox --no-venv -s unit-tests


Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR updates GitHub Actions workflows to improve Python test execution and SonarCloud coverage reporting by splitting unit vs. integration testing, adding integration test sharding, and combining coverage artifacts before publishing results.

Changes:

Split Python SonarCloud nightly workflow into dedicated unit/integration jobs and added a coverage-combine job.
Sharded integration tests into two matrix shards for parallel execution.
Updated CLI E2E workflow matrix defaults to run Python unit tests and Python integration tests as separate entries and upload their coverage artifacts.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
.github/workflows/py-sonarcloud-nightly.yml	Adds unit + sharded integration jobs and a combine-coverage job before SonarCloud scan
.github/workflows/py-cli-e2e-tests.yml	Replaces prior single “python” entry with separate python unit/integration entries and updates coverage artifact handling

You can also share your feedback on Copilot code review. Take the survey.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR refactors the GitHub Actions Python test pipelines to improve parallelism and coverage reporting by splitting unit vs. integration tests, sharding integration runs, and aggregating coverage artifacts for SonarCloud analysis.

Changes:

Split SonarCloud nightly ingestion workflow into separate unit and sharded integration jobs, plus a downstream coverage-combine + scan job.
Update CLI E2E workflow to run Python unit tests and Python integration tests as distinct matrix entries and upload their coverage artifacts separately.
Standardize Python test execution on nox sessions for unit/integration runs.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
.github/workflows/py-sonarcloud-nightly.yml	Splits unit/integration tests, shards integration suite, and adds a combine+Sonar scan job (currently missing required Sonar inputs).
.github/workflows/py-cli-e2e-tests.yml	Replaces the single “python” entry with separate “python-unittests” and “python-integration” matrix runs and uploads coverage artifacts accordingly.

Copilot · 2026-03-20T06:37:36Z

+          name: coverage-unit
+          path: ingestion/.coverage
+          include-hidden-files: true
+


The unit test job produces JUnit XML (via the nox session) but only uploads the coverage file. Since the Sonar scan runs in a separate job, the JUnit report(s) need to be uploaded as an artifact here (e.g., ingestion/junit/test-results-.xml) and later downloaded in the combine/scan job, otherwise Sonar will not find any xUnit reports (sonar.python.xunit.reportPath=junit/test-results-.xml).

Suggested change

- name: Upload JUnit test results

if: ${{ !cancelled() }}

uses: actions/upload-artifact@v4

with:

name: junit-unit

path: ingestion/junit/test-results-*.xml

Copilot · 2026-03-20T06:37:36Z

+          name: coverage-integration-${{ matrix.shard.name }}
+          path: ingestion/.coverage
+          include-hidden-files: true



Same as unit tests: the integration nox session generates JUnit XML under ingestion/junit, but this job only uploads the coverage file. To keep Sonar test reporting intact when scanning in a downstream job, upload the JUnit XML as an artifact from here as well.

Suggested change

- name: Upload JUnit XML artifact

if: ${{ !cancelled() }}

uses: actions/upload-artifact@v4

with:

name: junit-integration-${{ matrix.shard.name }}

path: ingestion/junit

Copilot · 2026-03-20T06:37:37Z

+          source env/bin/activate
+          cd ingestion
+          coverage combine
+          coverage xml -i


This combine step generates coverage.xml (default output), but ingestion/sonar-project.properties is configured to read coverage from ci-coverage.xml (sonar.python.coverage.reportPaths=ci-coverage.xml). As written, the Sonar scan job will miss coverage. Update this step to produce ci-coverage.xml (and apply the same path-rewrite as the existing Makefile/nox session), e.g., run the existing nox session "combine-coverage" or replicate its sed + rename logic.

Suggested change

source env/bin/activate

cd ingestion

coverage combine

coverage xml -i

cd ingestion

source ../env/bin/activate

nox -s combine-coverage

Copilot · 2026-03-20T06:37:37Z

+        with:
+          pattern: coverage-*
+          path: ingestion/coverage-data/
+


The combine/scan job only downloads coverage artifacts (pattern: coverage-). If unit/integration jobs upload JUnit XML artifacts for Sonar, this job also needs to download them and place them under ingestion/junit/ before running the Sonar scan; otherwise sonar.python.xunit.reportPath=junit/test-results-.xml won't resolve.

Suggested change

- name: Download JUnit artifacts

uses: actions/download-artifact@v4

with:

pattern: junit-*

path: ingestion/junit/

gitar-bot · 2026-03-23T03:18:58Z

Code Review 👍 Approved with suggestions 1 resolved / 2 findings

Enhances Python E2E and SonarCloud workflows with comprehensive unit and integration test coverage, addressing the missing Slack failure notification for python test jobs. Consider preventing py-combine-coverage from running when all test jobs fail or are cancelled by tightening its execution condition.

💡 Edge Case: py-combine-coverage runs even if all test jobs fail/are cancelled

📄 .github/workflows/py-sonarcloud-nightly.yml:146 📄 .github/workflows/py-sonarcloud-nightly.yml:174

The py-combine-coverage job uses if: ${{ !cancelled() }} which means it will execute even when both py-unit-tests and py-integration-tests fail before uploading any coverage artifacts. In that scenario, actions/download-artifact with pattern: coverage-* may find no matching artifacts (causing a warning or error), and subsequently nox --no-venv -s combine-coverage will fail because no .coverage.* files exist to combine. This produces noisy failures in an already-failing pipeline.

Consider adding a condition that checks at least one upstream job succeeded, or add continue-on-error: true on the download/prepare steps, or gate the combine step on the existence of coverage files.

Suggested fix

py-combine-coverage:
  if: ${{ !cancelled() && (needs.py-unit-tests.result == 'success' || needs.py-integration-tests.result == 'success') }}
  needs: [py-unit-tests, py-integration-tests]

✅ 1 resolved

✅ Bug: Slack failure notification missing for python test jobs

📄 .github/workflows/py-cli-e2e-tests.yml:225
The Slack notification step (line 225) was changed from steps.e2e-test.outcome != 'success' && steps.python-e2e-test.outcome != 'success' to only steps.e2e-test.outcome == 'failure'. This means failures in python-unittests or python-integration jobs will no longer trigger a Slack alert. The Force failure step (line 237) correctly checks all three step outcomes, but the Slack step was not updated to match.

🤖 Prompt for agents

Code Review: Enhances Python E2E and SonarCloud workflows with comprehensive unit and integration test coverage, addressing the missing Slack failure notification for python test jobs. Consider preventing `py-combine-coverage` from running when all test jobs fail or are cancelled by tightening its execution condition.

1. 💡 Edge Case: py-combine-coverage runs even if all test jobs fail/are cancelled
   Files: .github/workflows/py-sonarcloud-nightly.yml:146, .github/workflows/py-sonarcloud-nightly.yml:174

   The `py-combine-coverage` job uses `if: ${{ !cancelled() }}` which means it will execute even when both `py-unit-tests` and `py-integration-tests` fail before uploading any coverage artifacts. In that scenario, `actions/download-artifact` with `pattern: coverage-*` may find no matching artifacts (causing a warning or error), and subsequently `nox --no-venv -s combine-coverage` will fail because no `.coverage.*` files exist to combine. This produces noisy failures in an already-failing pipeline.
   
   Consider adding a condition that checks at least one upstream job succeeded, or add `continue-on-error: true` on the download/prepare steps, or gate the combine step on the existence of coverage files.

   Suggested fix:
   py-combine-coverage:
     if: ${{ !cancelled() && (needs.py-unit-tests.result == 'success' || needs.py-integration-tests.result == 'success') }}
     needs: [py-unit-tests, py-integration-tests]

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.

Comment with these commands to change:

`Auto-apply`	`Compact`
`gitar auto-apply:on`	`gitar display:verbose`

_{Was this helpful? React with 👍 / 👎 | Gitar}

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

…and integration tests (#26481) * chore(ci): enhance Python E2E and SonarCloud workflows with unit and integration tests * seperate the unit and integration test * address commensts * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * address comments --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: ulixius9 <mayursingal9@gmail.com>

chore(ci): enhance Python E2E and SonarCloud workflows with unit and …

6a136d2

…integration tests

SumanMaharana requested review from akash-jain-10, harshach and tutte as code owners March 13, 2026 12:34

github-actions Bot added Ingestion safe to test Add this label to run secure Github workflows on PRs labels Mar 13, 2026

SumanMaharana had a problem deploying to test March 13, 2026 12:34 — with GitHub Actions Failure

SumanMaharana temporarily deployed to test March 13, 2026 12:34 — with GitHub Actions Inactive

SumanMaharana had a problem deploying to test March 13, 2026 12:34 — with GitHub Actions Failure

SumanMaharana temporarily deployed to test March 13, 2026 12:34 — with GitHub Actions Inactive

SumanMaharana had a problem deploying to test March 13, 2026 12:34 — with GitHub Actions Failure

SumanMaharana temporarily deployed to test March 13, 2026 12:34 — with GitHub Actions Inactive

gitar-bot Bot reviewed Mar 13, 2026

View reviewed changes

Comment thread .github/workflows/py-sonarcloud-nightly.yml

SumanMaharana had a problem deploying to test March 13, 2026 14:19 — with GitHub Actions Failure

SumanMaharana had a problem deploying to test March 13, 2026 15:04 — with GitHub Actions Failure

SumanMaharana added 2 commits March 16, 2026 23:29

Merge branch 'main' into fix-python-e2e

fb14dd3

seperate the unit and integration test

f257e5e

gitar-bot Bot reviewed Mar 16, 2026

View reviewed changes

Comment thread .github/workflows/py-cli-e2e-tests.yml Outdated

SumanMaharana temporarily deployed to test March 16, 2026 19:03 — with GitHub Actions Inactive

Copilot AI reviewed Mar 19, 2026

View reviewed changes

Comment thread .github/workflows/py-cli-e2e-tests.yml

Comment thread .github/workflows/py-sonarcloud-nightly.yml

Comment thread .github/workflows/py-cli-e2e-tests.yml

Comment thread .github/workflows/py-cli-e2e-tests.yml Outdated

Potential fix for pull request finding

f7253e4

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings March 19, 2026 10:48

Copilot AI reviewed Mar 19, 2026

View reviewed changes

Comment thread .github/workflows/py-cli-e2e-tests.yml Outdated

Comment thread .github/workflows/py-cli-e2e-tests.yml

Comment thread .github/workflows/py-cli-e2e-tests.yml

Potential fix for pull request finding

66a2d52

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings March 19, 2026 10:59

SumanMaharana and others added 2 commits March 19, 2026 16:30

Potential fix for pull request finding

efa49dd

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

5fa43f1

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI reviewed Mar 19, 2026

View reviewed changes

Potential fix for pull request finding

60f9039

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings March 19, 2026 11:17

Copilot AI reviewed Mar 19, 2026

View reviewed changes

Comment thread .github/workflows/py-sonarcloud-nightly.yml

Potential fix for pull request finding

8fcb728

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings March 20, 2026 06:33

Copilot started reviewing on behalf of SumanMaharana March 20, 2026 06:33 View session

Copilot AI reviewed Mar 20, 2026

View reviewed changes

SumanMaharana temporarily deployed to test March 20, 2026 06:59 — with GitHub Actions Inactive

SumanMaharana and others added 2 commits March 20, 2026 20:34

Merge branch 'main' into fix-python-e2e

b2cbd1b

address comments

38f413e

Copilot AI review requested due to automatic review settings March 23, 2026 03:17

Copilot started reviewing on behalf of ulixius9 March 23, 2026 03:18 View session

ulixius9 temporarily deployed to test March 23, 2026 03:19 — with GitHub Actions Inactive

Copilot AI reviewed Mar 23, 2026

View reviewed changes

ulixius9 approved these changes Mar 23, 2026

View reviewed changes

ulixius9 merged commit 108cfe7 into main Mar 23, 2026
39 checks passed

ulixius9 deleted the fix-python-e2e branch March 23, 2026 05:17

+      - name: Upload JUnit test results
+        if: ${{ !cancelled() }}
+        uses: actions/upload-artifact@v4
+        with:
+          name: junit-unit
+          path: ingestion/junit/test-results-*.xml

+      - name: Download JUnit artifacts
+        uses: actions/download-artifact@v4
+        with:
+          pattern: junit-*
+          path: ingestion/junit/

Conversation

SumanMaharana commented Mar 13, 2026

Describe your changes:

Type of change:

Checklist:

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

gitar-bot Bot commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gitar-bot Bot commented Mar 23, 2026 •

edited

Loading