-
Notifications
You must be signed in to change notification settings - Fork 3
Expand file tree
/
Copy pathassess-update-readiness.yml
More file actions
198 lines (184 loc) · 9.51 KB
/
Copy pathassess-update-readiness.yml
File metadata and controls
198 lines (184 loc) · 9.51 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
# AZLOCAL-PIPELINE-ID: assess-update-readiness
# Step.05 - Assess Update Readiness (Pre-flight go/no-go gate)
# --------------------------------------------------
# Runs Get-AzLocalClusterUpdateReadiness and Test-AzLocalClusterHealth -BlockingOnly
# against a target UpdateRing (or the whole fleet) BEFORE apply-updates.yml runs.
#
# Publishes two JUnit XML diagnostic results to the Azure DevOps Tests tab:
# - readiness.xml (one test per cluster; fails if ReadyForUpdate = $false)
# - health-blocking.xml (one test per cluster; fails if any Critical health failure exists)
#
# CSV artifacts are also attached for spreadsheet triage.
#
# This pipeline is REPORT-ONLY and always succeeds. It surfaces not-ready / unhealthy
# clusters via the Tests tab (JUnit diagnostics), the CSV artifacts, and the job summary - but it
# does NOT block downstream runs. In large fleets, day-to-day environmental issues
# (transient storage noise, a single node out, etc.) routinely affect a small subset of
# clusters; blocking the entire wave for one unhealthy cluster is rarely the desired
# behavior. apply-updates.yml is itself per-cluster scoped, so clusters that are not
# ready will simply no-op there too.
#
# If you want a hard pass/fail signal for a chained gate, read the output variables
# (not_ready, critical_failures - lowercase since v0.8.5) from a downstream pipeline via
# a pipeline resource and apply your own tolerance threshold there.
#
# Remediation of Critical health failures is out of scope for this module - see the module
# README "Assess Readiness and Health BEFORE Applying Updates" section for pointers.
#
# v0.8.5 thin-YAML: the inline run body (inventory + scope-param construction +
# Get-AzLocalClusterUpdateReadiness + Test-AzLocalClusterHealth + combined JUnit XML
# merge + 8-section markdown summary + output variables) is now the
# Export-AzLocalClusterUpdateReadinessReport Public cmdlet. This yml is condensed
# to a few lines per step; the full workload (and its Pester tests) live in the module.
# BEGIN-AZLOCAL-CUSTOMIZE:schedule-triggers
# Content between BEGIN/END markers is preserved by Update-AzLocalPipelineExample
# across module upgrades.
trigger: none # Manual or scheduled only
schedules:
# Daily at 07:00 UTC
- cron: '0 7 * * *'
displayName: 'Daily Fleet Update Readiness Check'
branches:
include:
- main
always: true
# END-AZLOCAL-CUSTOMIZE:schedule-triggers
parameters:
- name: scope
displayName: 'Scope of clusters to check'
type: string
default: 'by-update-ring'
values:
- 'by-update-ring'
- 'all'
- name: updateRing
# accepts single ring, 'Prod;Ring2' list, or '***' wildcard (three stars - deliberate).
displayName: "UpdateRing tag value (only used when scope=by-update-ring). Single, 'Prod;Ring2', or '***'."
type: string
default: 'Wave1'
- name: moduleVersion
displayName: 'Pin AzLocal.UpdateManagement version (empty = latest from PSGallery). See Automation-Pipeline-Examples/README.md section 5 "Optional configuration".'
type: string
default: ''
variables:
# Module version this YAML was generated against. The install step compares this to
# the version actually installed and to the latest on PSGallery, and emits a warning
# log if the YAML appears stale - prompting you to refresh via
# Copy-AzLocalPipelineExample -Update. See Automation-Pipeline-Examples/README.md section 5.
GENERATED_AGAINST_MODULE_VERSION: '0.8.73'
# Resolution order for the module version pin (leave all unset to install the latest,
# which is the default "fix-forward" behaviour): queue-time parameter > pipeline variable
# 'REQUIRED_MODULE_VERSION' overridden at queue time > empty (latest).
REQUIRED_MODULE_VERSION: '${{ parameters.moduleVersion }}'
artifactsPath: '$(Build.ArtifactStagingDirectory)/readiness-assessment'
pool:
vmImage: 'windows-latest'
stages:
- stage: Assess
displayName: 'Assess Update Readiness'
jobs:
- job: AssessReadiness
displayName: 'Readiness + blocking health gate'
steps:
- checkout: self
displayName: 'Checkout repository'
- task: PowerShell@2
displayName: 'Install AzLocal.UpdateManagement from PSGallery'
# v0.8.5 thin-YAML: drift detection + banner + output variables are
# all produced by Add-AzLocalPipelineVersionBanner (Public cmdlet).
name: moduleVersion
env:
REQUIRED_MODULE_VERSION: $(REQUIRED_MODULE_VERSION)
GENERATED_AGAINST_MODULE_VERSION: $(GENERATED_AGAINST_MODULE_VERSION)
inputs:
targetType: 'inline'
pwsh: true
script: |
$ErrorActionPreference = 'Stop'
$installArgs = @{ Name = 'AzLocal.UpdateManagement'; Scope = 'CurrentUser'; Force = $true; AllowClobber = $true }
if ($env:REQUIRED_MODULE_VERSION) {
$installArgs.RequiredVersion = $env:REQUIRED_MODULE_VERSION
Write-Host "REQUIRED_MODULE_VERSION is set - pinning install to v$($env:REQUIRED_MODULE_VERSION)."
} else {
Write-Host "REQUIRED_MODULE_VERSION is empty - installing the latest version from PSGallery (default fix-forward behaviour)."
}
Install-Module @installArgs
Import-Module AzLocal.UpdateManagement -Force
Add-AzLocalPipelineVersionBanner `
-GeneratedAgainstVersion $env:GENERATED_AGAINST_MODULE_VERSION `
-PinnedVersion $env:REQUIRED_MODULE_VERSION
- task: AzureCLI@2
displayName: 'Run readiness + blocking health checks'
# v0.8.5 thin-YAML: the inline run block (inventory + scope param
# construction + Get-AzLocalClusterUpdateReadiness CSV/XML +
# Test-AzLocalClusterHealth -BlockingOnly CSV/XML + combined JUnit
# merge + 8-section markdown step summary + the two output
# variables (NOT_READY / CRITICAL_FAILURES, now lowercase
# not_ready / critical_failures) has been condensed into the
# Public cmdlet Export-AzLocalClusterUpdateReadinessReport. The
# cmdlet writes $(artifactsPath)/{readiness,health-blocking,assess-readiness}.{csv,xml},
# emits the markdown summary via task.uploadsummary, and sets the
# two output variables.
name: gate
env:
INPUT_SCOPE: ${{ parameters.scope }}
INPUT_UPDATE_RING: ${{ parameters.updateRing }}
INSTALLED_MODULE_VERSION: $(moduleVersion.installed_module_version)
ARTIFACTS_PATH: $(artifactsPath)
inputs:
# Replace with your service connection name
azureSubscription: 'AzureLocal-ServiceConnection' # Update with your service connection name
scriptType: 'pscore'
scriptLocation: 'inlineScript'
inlineScript: |
$ErrorActionPreference = 'Stop'
az extension add --name resource-graph --yes
Import-Module AzLocal.UpdateManagement -Force
$params = @{
Scope = if ($env:INPUT_SCOPE) { $env:INPUT_SCOPE } else { 'all' }
OutputDirectory = $env:ARTIFACTS_PATH
InstalledModuleVersion = $env:INSTALLED_MODULE_VERSION
}
if ($env:INPUT_UPDATE_RING) { $params['UpdateRing'] = $env:INPUT_UPDATE_RING }
Export-AzLocalClusterUpdateReadinessReport @params
- task: PublishTestResults@2
displayName: 'Publish Update Readiness Assessment (combined)'
condition: always()
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: '$(artifactsPath)/assess-readiness.xml'
testRunTitle: 'Update Readiness Assessment'
mergeTestResults: false
failTaskOnFailedTests: false
- task: PublishTestResults@2
displayName: 'Publish Readiness JUnit Diagnostics'
condition: always()
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: '$(artifactsPath)/readiness.xml'
testRunTitle: '[JUnit Debug] Readiness (one test per cluster)'
mergeTestResults: false
failTaskOnFailedTests: false
- task: PublishTestResults@2
displayName: 'Publish Blocking-Health JUnit Diagnostics'
condition: always()
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: '$(artifactsPath)/health-blocking.xml'
testRunTitle: '[JUnit Debug] Blocking Health Checks (one test per cluster)'
mergeTestResults: false
failTaskOnFailedTests: false
# compute a UTC timestamp variable so every downloadable artifact name is unique per run.
- pwsh: |
$stamp = (Get-Date).ToUniversalTime().ToString('yyyyMMdd_HHmmss')
Write-Host "##vso[task.setvariable variable=artifactStamp;isOutput=true]$stamp"
Write-Host "Artifact timestamp: $stamp"
displayName: 'Compute Artifact Timestamp'
condition: always()
name: stamp
- task: PublishPipelineArtifact@1
displayName: 'Publish readiness assessment artifacts'
condition: always()
inputs:
targetPath: '$(artifactsPath)'
artifact: 'azlocal-step.5-readiness-assessment-report_$(stamp.artifactStamp)'