Speed up Code Coverage Jobs

[paulmedynski]

Description

The code coverage pipeline job is inefficient, and can be greatly streamlined. We don't need to publish separate uploads to CodeCov for MDS .NET, .NET Framework, and AKV.

Testing

• PR runs will validate the new consolidation steps.
• I will manually verify the Azure DevOps test results and coverage, and CodeCov reports.

[Copilot]

Pull request overview

This pull request optimizes CI/CD pipeline execution by separating unit and integration tests to reduce redundant test execution. The PR introduces a testType parameter that controls which test suites run in different pipeline contexts.

Changes:

• Added testType parameter ('All', 'Unit', or 'Integration') to pipeline templates
• Modified PR pipelines to run only relevant test suites (Unit tests in Project builds, Integration tests in Package builds)
• Updated test execution conditionals to respect the new testType parameter

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
eng/pipelines/sqlclient-pr-project-ref-pipeline.yml	Sets testType to 'Unit' for project reference builds, restricting to unit and functional tests
eng/pipelines/sqlclient-pr-package-ref-pipeline.yml	Sets testType to 'Integration' for package reference builds, restricting to manual tests
eng/pipelines/dotnet-sqlclient-ci-core.yml	Defines testType parameter with default 'All' for CI pipelines to continue running all tests
eng/pipelines/common/templates/steps/run-all-tests-step.yml	Implements conditional test execution based on testType parameter
eng/pipelines/common/templates/stages/ci-run-tests-stage.yml	Propagates testType parameter through stage template
eng/pipelines/common/templates/jobs/ci-run-tests-job.yml	Propagates testType parameter through job template

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 0.00%. Comparing base (a68e00f) to head (5d2d5f4).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #3907       +/-   ##
==========================================
- Coverage   75.22%       0   -75.23%     
==========================================
  Files         266       0      -266     
  Lines       42932       0    -42932     
==========================================
- Hits        32294       0    -32294     
+ Misses      10638       0    -10638     

Flag	Coverage Δ
addons	?
netcore	?
netfx	?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[benrr101]

Overall, probably a good change. The main issue I have right now is the lumping of functional tests with unit tests. My understanding of them were that they were not unit tests in theory, they were just the simple integration tests that validate they do the most basic stuff. As it turns out a lot of those are unit tests, but not the majority. I was working on the side a bit to move the unit tests from the functional tests and into the unit test project. Ultimately I think we're aligned that we want: unit test, local integration test, remote integration test, stress test, and fuzz test projects. And this change is kinda orthogonal to those goals, without getting in the way of them. So go for it 😆

I think another low-ish hanging fruit is to separate the unit tests and functional tests as separate jobs, like the test sets. That way we don't need to repeat unit and functional tests four times for each platform. They're fast, yes, but not instantaneous, and I imagine we could shave a decent amount of time off a build doing it.

[paulmedynski]

You're correct on all counts. I chose to combine the running of Unit and Functional tests because they're mostly unit with some local integration sprinkled in, but mainly because their combined run time is close to the Manual ones. This gives us the most bang-for-the-buck in terms of speeding up the PR runs right now.

We also shouldn't be running Unit and Functional tests in any of the Azure SQL configurations - another future optimization.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

[priyankatiwari08]

Looks good to me. Manual tests in itself takes quite significant amount of time to run. So, this segregation should help us in longer run.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 25 out of 25 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 18 out of 21 changed files in this pull request and generated 5 comments.

[Copilot]

Pull request overview

Copilot reviewed 21 out of 24 changed files in this pull request and generated no new comments.