Skip to content

Commit 44e1b72

Browse files
benrr101benrr101
authored
PR Pipeline - Code Coverage and Triggers (#4360)
* First crack at adding test result publishing to non-manual test jobs. # Conflicts: # eng/pipelines/pr/jobs/test-buildproj-job.yml # eng/pipelines/pr/stages/test-stages.yml * If statement syntax? * Ok forget the if statement. * Version * Unique job names... could be better, but let's start here. * Tree in linux? * Ok forget it, we don't need do emit the tree.... * Wire up test result uploading to sqlclient manual test jobs * Give a test display name to sql manual tests * First attempt at code coverage report publishing * 🤖 Code coverage takes a dynamic dependency on the platforms to ensure all tests are first. * We only need *one* set of parameters * We only need *one* pool name * Add triggers and fix code coverage verbosity thingy * 🤖 fixes for codecov CLI * 👨‍🦱 and 🤖 comments * 🤖 Use dotnet-tools.json * Comments for general variables Rename testResultsArtifactsName to testResultsArtifactsBaseName Remove codecov uploads Ensure folder exists
1 parent c0f341a commit 44e1b72

8 files changed

Lines changed: 344 additions & 94 deletions

File tree

‎eng/pipelines/pr/jobs/test-buildproj-job.yml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,10 @@ parameters:
1818
- name: buildSuffix
1919
type: string
2020

21+
# Name of the artifact to upload the test results (including code coverage) to.
22+
- name: testResultsArtifactBaseName
23+
type: string
24+
2125
# Dotnet CLI verbosity level.
2226
- name: dotnetVerbosity
2327
type: string
@@ -78,6 +82,10 @@ jobs:
7882
- job: "test_${{ parameters.platformDisplayName }}_${{ parameters.testDisplayName }}"
7983
displayName: "${{ parameters.testDisplayName }}_${{ parameters.platformDisplayName }}"
8084

85+
variables:
86+
- name: testResultsPath
87+
value: "$(REPO_ROOT)/test_results/${{ parameters.platformDisplayName }}_${{ parameters.testDisplayName }}"
88+
8189
pool:
8290
name: ${{ parameters.poolName }}
8391
demands:
@@ -107,3 +115,13 @@ jobs:
107115
-p:BuildNumber='$(Build.BuildNumber)'
108116
-p:BuildSuffix='${{ parameters.buildSuffix }}'
109117
-p:TestFramework=${{ parameters.platformDotnet }}
118+
-p:TestResultsFolderPath=${{ variables.testResultsPath }}
119+
120+
# Publish test results
121+
- template: /eng/pipelines/pr/steps/publish-test-results-step.yml@self
122+
parameters:
123+
buildConfiguration: ${{ parameters.buildConfiguration }}
124+
platformDisplayName: ${{ parameters.platformDisplayName }}
125+
testDisplayName: ${{ parameters.testDisplayName }}
126+
testResultsArtifactBaseName: ${{ parameters.testResultsArtifactBaseName }}
127+
testResultsPath: ${{ variables.testResultsPath }}

‎eng/pipelines/pr/jobs/test-sqlclientmanual-job.yml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,10 @@ parameters:
3333
- name: stageNameSecrets
3434
type: string
3535

36+
# Name of the artifact to upload the test results (including code coverage) to.
37+
- name: testResultsArtifactBaseName
38+
type: string
39+
3640
# Platform Parameters ====================================================
3741

3842
# Display name of the platform. This will be formatted into the job's official name as well as
@@ -128,6 +132,12 @@ jobs:
128132
- name: saPassword
129133
value: $[stageDependencies.${{ parameters.stageNameSecrets }}.secrets_job.outputs['SaPassword.Value']]
130134

135+
- name: testDisplayName
136+
value: "sqlclient_manual_${{ parameters.configDisplayName}}_${{ parameters.testSet }}"
137+
138+
- name: testResultsPath
139+
value: "$(REPO_ROOT)/test_results/${{ parameters.platformDisplayName }}_$(testDisplayName)"
140+
131141
steps:
132142
# Install dotnet and the runtime that will run the tests (if it is not netframework)
133143
- template: /eng/pipelines/common/steps/install-dotnet.yml@self
@@ -201,4 +211,13 @@ jobs:
201211
-p:BuildSuffix='${{ parameters.buildSuffix }}'
202212
-p:TestFramework=${{ parameters.platformDotnet }}
203213
-p:TestSet=${{ parameters.testSet }}
214+
-p:TestResultsFolderPath=${{ variables.testResultsPath }}
204215
216+
# Publish test results
217+
- template: /eng/pipelines/pr/steps/publish-test-results-step.yml@self
218+
parameters:
219+
buildConfiguration: ${{ parameters.buildConfiguration }}
220+
platformDisplayName: ${{ parameters.platformDisplayName }}
221+
testDisplayName: ${{ variables.testDisplayName }}
222+
testResultsArtifactBaseName: ${{ parameters.testResultsArtifactBaseName }}
223+
testResultsPath: ${{ variables.testResultsPath }}

‎eng/pipelines/pr/pr-pipeline.yml‎

Lines changed: 0 additions & 94 deletions
This file was deleted.

‎eng/pipelines/pr/sqlclient-pr-pipeline.yml‎

Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
#################################################################################
2+
# Licensed to the .NET Foundation under one or more agreements. #
3+
# The .NET Foundation licenses this file to you under the MIT license. #
4+
# See the LICENSE file in the project root for more information. #
5+
#################################################################################
6+
7+
# This pipeline builds and tests all products that we release. It is intended to run as validation
8+
# for every PR. The set of tests it executes is not exhaustive, as many tests require special
9+
# environments to provide complete coverage. Instead, this PR strikes a balance between complete
10+
# confidence and speed of PR validation.
11+
#
12+
# It is triggered by pushes to PRs that target the main, dev/*, feat/*, and release/* branches in
13+
# GitHub. The dev/* pattern includes dev/automation/* branches created by AI agents.
14+
#
15+
# It maps to the "sqlclient-pr" pipeline in the Public ADO project:
16+
# https://sqlclientdrivers.visualstudio.com/public/_build?definitionId=2281
17+
# And the "sqlclient-pr" pipeline in the internal ADO project:
18+
# https://dev.azure.com/SqlClientDrivers/ADO.Net/_build?definitionId=2271
19+
20+
# Set the pipeline run name to the day-of-year and the daily run counter.
21+
name: $(DayOfYear)$(Rev:rr)
22+
23+
# Trigger PR validation runs for all pushes to PRs that target the specified branches.
24+
# https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/github?view=azure-devops&tabs=yaml#pr-triggers
25+
pr:
26+
branches:
27+
include:
28+
# GitHub repo branch targets that will trigger PR validation builds.
29+
- dev/*
30+
- feat/*
31+
- main
32+
- release/*
33+
34+
paths:
35+
include:
36+
- .azuredevops/*
37+
- .config/*
38+
- doc/*
39+
- eng/pipelines/common/*
40+
- eng/pipelines/pr/*
41+
- src/*
42+
- tools/*
43+
- azurepipelines-coverage.yml
44+
- build.proj
45+
- Directory.Packages.props
46+
- dotnet-tools.json
47+
- global.json
48+
- NuGet.config
49+
50+
# Do not trigger commit or schedule runs for this pipeline.
51+
trigger: none
52+
53+
parameters:
54+
# General Parameters =====================================================
55+
56+
# Dotnet CLI verbosity level.
57+
- name: dotnetVerbosity
58+
displayName: dotnet CLI Verbosity
59+
type: string
60+
default: normal
61+
values:
62+
- quiet
63+
- minimal
64+
- normal
65+
- detailed
66+
- diagnostic
67+
68+
- name: platforms
69+
type: object
70+
default:
71+
- displayName: "windows_net462"
72+
dotnet: "net462"
73+
image: "ADO-MMS22-SQL22"
74+
operatingSystem: "Windows"
75+
- displayName: "windows_net8"
76+
dotnet: "net8.0"
77+
image: "ADO-MMS22-SQL22"
78+
operatingSystem: "Windows"
79+
- displayName: "windows_net9"
80+
dotnet: "net9.0"
81+
image: "ADO-MMS22-SQL22"
82+
operatingSystem: "Windows"
83+
- displayName: "windows_net10"
84+
dotnet: "net10.0"
85+
image: "ADO-MMS22-SQL22"
86+
operatingSystem: "Windows"
87+
88+
- displayName: "linux_net8"
89+
dotnet: "net8.0"
90+
image: "ADO-UB22-SQL22"
91+
operatingSystem: "Linux"
92+
- displayName: "linux_net9"
93+
dotnet: "net9.0"
94+
image: "ADO-UB22-SQL22"
95+
operatingSystem: "Linux"
96+
- displayName: "linux_net10"
97+
dotnet: "net10.0"
98+
image: "ADO-UB22-SQL22"
99+
operatingSystem: "Linux"
100+
101+
variables:
102+
- template: /eng/pipelines/common/variables/common-variables.yml@self
103+
- template: /eng/pipelines/pr/variables/pr-variables.yml@self
104+
105+
stages:
106+
# Stage 1a: Build and pack all projects in the repository
107+
- template: /eng/pipelines/pr/stages/pack-stage.yml@self
108+
parameters:
109+
buildConfiguration: Debug
110+
buildSuffix: pr
111+
stageName: ${{ variables.stageNamePack }}
112+
113+
# Stage 1b: Generate secrets
114+
- template: /eng/pipelines/pr/stages/generate-secrets-stage.yml@self
115+
parameters:
116+
stageName: ${{ variables.stageNameSecrets }}
117+
118+
# Stage 2: Execute tests and collect code coverage
119+
- template: /eng/pipelines/pr/stages/test-stages.yml@self
120+
parameters:
121+
buildConfiguration: Debug
122+
buildSuffix: pr
123+
dotnetVerbosity: ${{ parameters.dotnetVerbosity }}
124+
poolName: $(PoolNameDefault)
125+
stageNamePack: ${{ variables.stageNamePack }}
126+
stageNameSecrets: ${{ variables.stageNameSecrets }}
127+
testResultsArtifactBaseName: ${{ variables.testResultsArtifactBaseName }}
128+
129+
platforms: ${{ parameters.platforms }}
130+
131+
manualTestAzureKeyVaultUrl: $(AzureKeyVaultUrl)
132+
manualTestAzureKeyVaultTenantId: $(AzureKeyVaultTenantId)
133+
manualTestConnectionStringNpLocalhost: $(ConnectionStringNp_LocalhostDefault_UsernamePassword)
134+
manualTestConnectionStringTcpLocalhost: $(ConnectionStringTcp_LocalhostDefault_UsernamePassword)
135+
manualTestConnectionStringNpAzure: $(ConnectionStringNp_Azure_ManagedIdentity)
136+
manualTestConnectionStringTcpAzure: $(ConnectionStringTcp_Azure_ManagedIdentity)
137+
manualTestFileStreamDirectory: "$(Pipeline.Workspace)/filestream"
138+
manualTestLocalDbAppName: $(LocalDbAppName)
139+
manualTestLocalDbSharedInstanceName: $(LocalDbSharedInstanceName)
140+
manualTestUserManagedIdentityClientId: $(UserManagedIdentityClientId)
141+
142+
# Stage 3: Collect code coverage
143+
- template: /eng/pipelines/pr/stages/collect-coverage-stage.yml@self
144+
parameters:
145+
dependsOn:
146+
- ${{ each platform in parameters.platforms }}:
147+
- "test_${{ platform.displayName }}"

‎eng/pipelines/pr/stages/collect-coverage-stage.yml‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
#################################################################################
2+
# Licensed to the .NET Foundation under one or more agreements. #
3+
# The .NET Foundation licenses this file to you under the MIT license. #
4+
# See the LICENSE file in the project root for more information. #
5+
#################################################################################
6+
7+
parameters:
8+
9+
# Names of the test stages that must complete before collecting coverage.
10+
- name: dependsOn
11+
type: object
12+
default: []
13+
14+
stages:
15+
- stage: collect_code_coverage
16+
displayName: "Collect code coverage"
17+
dependsOn: ${{ parameters.dependsOn }}
18+
19+
jobs:
20+
- job: collect_code_coverage
21+
displayName: "Collect Code Coverage"
22+
23+
pool:
24+
vmImage: ubuntu-latest
25+
26+
variables:
27+
28+
# Set up a temporary directory that is cleaned up after each job run. This helps avoid
29+
# disk space issues on pooled agents that may run many jobs before being retired.
30+
- name: workingDir
31+
value: "$(Agent.TempDirectory)/coverage"
32+
33+
steps:
34+
35+
# Part 1) Merge coverage reports =================================
36+
37+
# Install .NET SDK
38+
- template: /eng/pipelines/common/steps/install-dotnet.yml@self
39+
40+
# Restore additional dotnet tools
41+
- template: /eng/pipelines/common/steps/restore-dotnet-tools.yml@self
42+
43+
# Download coverage reports from the test jobs.
44+
# NOTE: The artifact name is not specified, so *all* artifacts of the build will be
45+
# checked for the coverage reports.
46+
- task: DownloadPipelineArtifact@2
47+
displayName: Download coverage reports
48+
inputs:
49+
itemPattern: '**/*.coverage'
50+
targetPath: "${{ variables.workingDir }}/originals"
51+
52+
# Merge original files into a single Cobertura XML file
53+
- script: >-
54+
dotnet tool run dotnet-coverage -- merge "${{ variables.workingDir }}/originals/**/*.coverage"
55+
--output "${{ variables.workingDir }}/merge/cobertura.xml"
56+
--output-format cobertura
57+
--log-file "${{ variables.workingDir }}/merge/merge.log"
58+
--log-level Verbose
59+
displayName: Merge coverage files
60+
workingDirectory: $(REPO_ROOT)
61+
62+
# Part 2) Publish merged results to the pipeline results/artifacts
63+
64+
# Publish the merged coverage file as a pipeline artifact
65+
- task: PublishPipelineArtifact@1
66+
displayName: Publish coverage artifact
67+
inputs:
68+
artifact: merged_coverage
69+
targetPath: "${{ variables.workingDir }}/merge"
70+
71+
# Publish the merged coverage file as coverage results so they can be viewed in ADO UI.
72+
- task: PublishCodeCoverageResults@2
73+
displayName: Publish coverage results
74+
inputs:
75+
summaryFileLocation: "${{ variables.workingDir }}/merge/cobertura.xml"

0 commit comments

Comments
 (0)