Skip to content

Commit 52af86e

Browse files
🌟 [Major]: Introducing Get-PesterTestResults (#2)
## Description This pull request introduces a new GitHub Action, `Get-PesterTestResults`, which processes Pester test results, validates test execution, and provides a summary of results. ### Implementation of Core Functionality * `action.yml`: Added inputs (`SourceCodeTestSuites`, `PSModuleTestSuites`, `ModuleTestSuites`) to specify test suite configurations. Updated the default value of `WorkingDirectory` to `'.'`. Defined environment variables to pass input values to the script. * `scripts/main.ps1`: Download and process test artifacts, validate test results, and generate a summary. Key features include: - Parsing JSON test suite configurations and expected test results. - Logging detailed outputs for executed, failed, and unexecuted tests. - Marking the action as failed if any tests are unsuccessful or missing. # Testing * `.github/workflows/Action-Test.yml`: Added a test workflow to validate the action's functionality. * `tests/*`: Added test data for the action to process and validate test results. ### Documentation * `README.md`: Updated to describe the purpose, usage, inputs, outputs, and failure conditions of the `Get-PesterTestResults` action. Added detailed examples and notes for user guidance. ## Type of change <!-- Use the check-boxes [x] on the options that are relevant. --> - [ ] 📖 [Docs] - [ ] 🪲 [Fix] - [ ] 🩹 [Patch] - [ ] ⚠️ [Security fix] - [ ] 🚀 [Feature] - [x] 🌟 [Breaking change] ## Checklist <!-- Use the check-boxes [x] on the options that are relevant. --> - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas
1 parent 503222e commit 52af86e

15 files changed

Lines changed: 623 additions & 19 deletions

.github/workflows/Action-Test.yml

Lines changed: 44 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,50 @@ jobs:
2525
- name: Checkout repo
2626
uses: actions/checkout@v4
2727

28+
# Upload artifact from tests:
29+
- name: Upload artifact [PATH-Windows-TestResults]
30+
uses: actions/upload-artifact@v4
31+
with:
32+
name: PATH-Windows-TestResults
33+
path: ./tests/TestResults/PATH-Windows-TestResults
34+
retention-days: 1
35+
if-no-files-found: error
36+
37+
- name: Upload artifact [PSModuleLint-Module-Windows-TestResults]
38+
uses: actions/upload-artifact@v4
39+
with:
40+
name: PSModuleLint-Module-Windows-TestResults
41+
path: ./tests/TestResults/PSModuleLint-Module-Windows-TestResults
42+
retention-days: 1
43+
if-no-files-found: error
44+
45+
- name: Upload artifact [PSModuleLint-SourceCode-Windows-TestResults]
46+
uses: actions/upload-artifact@v4
47+
with:
48+
name: PSModuleLint-SourceCode-Windows-TestResults
49+
path: ./tests/TestResults/PSModuleLint-SourceCode-Windows-TestResults
50+
retention-days: 1
51+
if-no-files-found: error
52+
53+
- name: Upload artifact [PSModuleTest-Module-Windows-TestResults]
54+
uses: actions/upload-artifact@v4
55+
with:
56+
name: PSModuleTest-Module-Windows-TestResults
57+
path: ./tests/TestResults/PSModuleTest-Module-Windows-TestResults
58+
retention-days: 1
59+
if-no-files-found: error
60+
61+
- name: Upload artifact [PSModuleTest-SourceCode-Windows-TestResults]
62+
uses: actions/upload-artifact@v4
63+
with:
64+
name: PSModuleTest-SourceCode-Windows-TestResults
65+
path: ./tests/TestResults/PSModuleTest-SourceCode-Windows-TestResults
66+
retention-days: 1
67+
if-no-files-found: error
68+
2869
- name: Action-Test
2970
uses: ./
3071
with:
31-
WorkingDirectory: ./tests
72+
SourceCodeTestSuites: '[{"OSName": "Windows"}'
73+
PSModuleTestSuites: '[{"OSName": "Windows"}]'
74+
ModuleTestSuites: '[{"TestName": "PATH", "OSName": "Windows"}]'

.github/workflows/Auto-Release.yml

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,5 +30,4 @@ jobs:
3030

3131
- name: Auto-Release
3232
uses: PSModule/Auto-Release@v1
33-
env:
34-
GITHUB_TOKEN: ${{ github.token }}
33+

README.md

Lines changed: 40 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,54 @@
1-
# Template-Action
1+
# Get-PesterTestResults Action
22

3-
A template repository for GitHub Actions
3+
A GitHub Action that gathers Pester test results for the PSModule process by analyzing test artifacts from the workflow run.
4+
It validates test execution and results, providing a summary and failing if any tests are unsuccessful.
5+
6+
This GitHub Action is a part of the [PSModule framework](https://github.com/PSModule). It is recommended to use the
7+
[Process-PSModule workflow](https://github.com/PSModule/Process-PSModule) to automate the whole process of managing the PowerShell module.
48

59
## Usage
610

11+
This action retrieves test artifacts named `*-TestResults`, processes the contained JSON files, and checks for test failures, unexecuted tests,
12+
or missing results. It supports three categories of test suites: Source Code, PSModule, and Module tests.
13+
714
### Inputs
815

16+
| Input | Description | Required | Default |
17+
|-------------------------|-------------------------------------------------------------------------------------------------------------------------------|----------|-----------|
18+
| `SourceCodeTestSuites` | JSON array specifying OS names for Source Code test suites. Example: `[{"OSName": "Windows"}]` | Yes | |
19+
| `PSModuleTestSuites` | JSON array specifying OS names for PSModule test suites. Example: `[{"OSName": "Linux"}]` | Yes | |
20+
| `ModuleTestSuites` | JSON array specifying TestName and OSName for Module test suites. Example: `[{"TestName": "Integration", "OSName": "MacOS"}]` | Yes | |
21+
| `Debug` | Enable debug output (`true`/`false`). | No | `false` |
22+
| `Verbose` | Enable verbose output (`true`/`false`). | No | `false` |
23+
| `Version` | Exact version of the GitHub module to install (e.g., `1.0.0`). | No | Latest |
24+
| `Prerelease` | Allow installing prerelease module versions (`true`/`false`). | No | `false` |
25+
| `WorkingDirectory` | Working directory for the script. | No | `.` |
26+
927
### Secrets
1028

29+
No secrets are required if the action runs in the same repository. The action uses the default `GITHUB_TOKEN` provided by GitHub Actions to access workflow artifacts.
30+
1131
### Outputs
1232

33+
This action does not define explicit outputs. Instead:
34+
35+
- If any tests fail or errors occur, the action exits with a non-zero code, marking the workflow step as failed.
36+
- Detailed results are logged in the workflow run's output.
37+
1338
### Example
1439

1540
```yaml
16-
Example here
41+
- name: Run and Collect Pester Tests
42+
uses: PSModule/Get-PesterTestResults@v1
43+
with:
44+
SourceCodeTestSuites: '[{"OSName": "Windows"}, {"OSName": "Linux"}]'
45+
PSModuleTestSuites: '[{"OSName": "Windows"}]'
46+
ModuleTestSuites: '[{"TestName": "Integration", "OSName": "Windows"}]'
1747
```
48+
49+
### Notes
50+
- **Test Suite Inputs**: Must be valid JSON arrays.
51+
- `SourceCodeTestSuites` and `PSModuleTestSuites` require `OSName`.
52+
- `ModuleTestSuites` requires both `TestName` and `OSName`.
53+
- **Artifact Names**: The action expects artifacts named `*-TestResults` containing Pester JSON reports.
54+
- **Failure Conditions**: The action fails if tests are unexecuted, explicitly failed, or if result files are missing.

action.yml

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,15 @@ branding:
66
color: white
77

88
inputs:
9+
SourceCodeTestSuites:
10+
description: The test suites to run for the source code.
11+
required: true
12+
PSModuleTestSuites:
13+
description: The test suites to run for the PSModule.
14+
required: true
15+
ModuleTestSuites:
16+
description: The test suites to run for the module.
17+
required: true
918
Debug:
1019
description: Enable debug output.
1120
required: false
@@ -24,19 +33,24 @@ inputs:
2433
WorkingDirectory:
2534
description: The working directory where the script will run from.
2635
required: false
27-
default: ${{ github.workspace }}
36+
default: '.'
2837

2938
runs:
3039
using: composite
3140
steps:
3241
- name: Get-PesterTestResults
3342
uses: PSModule/GitHub-Script@v1
43+
env:
44+
PSMODULE_GET_PESTERTESTRESULTS_INPUT_SourceCodeTestSuites: ${{ inputs.SourceCodeTestSuites }}
45+
PSMODULE_GET_PESTERTESTRESULTS_INPUT_PSModuleTestSuites: ${{ inputs.PSModuleTestSuites }}
46+
PSMODULE_GET_PESTERTESTRESULTS_INPUT_ModuleTestSuites: ${{ inputs.ModuleTestSuites }}
3447
with:
48+
Name: Get-PesterTestResults
3549
Debug: ${{ inputs.Debug }}
3650
Prerelease: ${{ inputs.Prerelease }}
3751
Verbose: ${{ inputs.Verbose }}
3852
Version: ${{ inputs.Version }}
3953
WorkingDirectory: ${{ inputs.WorkingDirectory }}
54+
ShowInfo: false
4055
Script: |
41-
# Get-PesterTestResults
4256
${{ github.action_path }}/scripts/main.ps1

scripts/main.ps1

Lines changed: 176 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,186 @@
1-
#Requires -Modules GitHub
1+
#Requires -Modules GitHub
22

3+
[Diagnostics.CodeAnalysis.SuppressMessageAttribute(
4+
'PSAvoidUsingWriteHost', '',
5+
Justification = 'Outputs to GitHub Actions logs.'
6+
)]
37
[CmdletBinding()]
48
param()
59

6-
begin {
7-
$scriptName = $MyInvocation.MyCommand.Name
8-
Write-Debug "[$scriptName] - Start"
10+
$owner = $env:GITHUB_REPOSITORY_OWNER
11+
$repo = $env:GITHUB_REPOSITORY_NAME
12+
$runId = $env:GITHUB_RUN_ID
13+
14+
$files = Get-GitHubArtifact -Owner $owner -Repository $repo -WorkflowRunID $runId -Name '*-TestResults' |
15+
Save-GitHubArtifact -Path 'TestResults' -Force -Expand -PassThru | Get-ChildItem -Recurse -Filter *.json | Sort-Object Name -Unique
16+
17+
LogGroup 'List files' {
18+
$files.Name | Out-String
919
}
1020

11-
process {
12-
try {
13-
Write-Output "Hello, $Subject!"
14-
} catch {
15-
throw $_
21+
$sourceCodeTestSuites = $env:PSMODULE_GET_PESTERTESTRESULTS_INPUT_SourceCodeTestSuites | ConvertFrom-Json
22+
$psModuleTestSuites = $env:PSMODULE_GET_PESTERTESTRESULTS_INPUT_PSModuleTestSuites | ConvertFrom-Json
23+
$moduleTestSuites = $env:PSMODULE_GET_PESTERTESTRESULTS_INPUT_ModuleTestSuites | ConvertFrom-Json
24+
25+
LogGroup 'Expected test suites' {
26+
27+
# Build an array of expected test suite objects
28+
$expectedTestSuites = @()
29+
30+
# SourceCodeTestSuites: expected file names start with "SourceCode-"
31+
foreach ($suite in $sourceCodeTestSuites) {
32+
$expectedTestSuites += [pscustomobject]@{
33+
Name = "PSModuleTest-SourceCode-$($suite.OSName)-TestResult-Report"
34+
Category = 'SourceCode'
35+
OSName = $suite.OSName
36+
TestName = $null
37+
}
38+
$expectedTestSuites += [pscustomobject]@{
39+
Name = "PSModuleLint-SourceCode-$($suite.OSName)-TestResult-Report"
40+
Category = 'SourceCode'
41+
OSName = $suite.OSName
42+
TestName = $null
43+
}
44+
}
45+
46+
# PSModuleTestSuites: expected file names start with "Module-"
47+
foreach ($suite in $psModuleTestSuites) {
48+
$expectedTestSuites += [pscustomobject]@{
49+
Name = "PSModuleTest-Module-$($suite.OSName)-TestResult-Report"
50+
Category = 'PSModuleTest'
51+
OSName = $suite.OSName
52+
TestName = $null
53+
}
54+
$expectedTestSuites += [pscustomobject]@{
55+
Name = "PSModuleLint-Module-$($suite.OSName)-TestResult-Report"
56+
Category = 'PSModuleTest'
57+
OSName = $suite.OSName
58+
TestName = $null
59+
}
60+
}
61+
62+
# ModuleTestSuites: expected file names use the TestName as prefix
63+
foreach ($suite in $moduleTestSuites) {
64+
$expectedTestSuites += [pscustomobject]@{
65+
Name = "$($suite.TestName)-$($suite.OSName)-TestResult-Report"
66+
Category = 'ModuleTest'
67+
OSName = $suite.OSName
68+
TestName = $suite.TestName
69+
}
70+
}
71+
72+
$expectedTestSuites = $expectedTestSuites | Sort-Object Category, Name
73+
$expectedTestSuites | Format-Table | Out-String
74+
}
75+
$isFailure = $false
76+
77+
$testResults = [System.Collections.Generic.List[psobject]]::new()
78+
$failedTests = [System.Collections.Generic.List[psobject]]::new()
79+
$unexecutedTests = [System.Collections.Generic.List[psobject]]::new()
80+
$totalErrors = 0
81+
82+
foreach ($expected in $expectedTestSuites) {
83+
$file = $files | Where-Object { $_.BaseName -eq $expected.Name }
84+
$result = if ($file) {
85+
$object = $file | Get-Content | ConvertFrom-Json
86+
[pscustomobject]@{
87+
Result = $object.Result
88+
Executed = $object.Executed
89+
ResultFilePresent = $true
90+
Tests = [int]([math]::Round(($object | Measure-Object -Sum -Property TotalCount).Sum))
91+
Passed = [int]([math]::Round(($object | Measure-Object -Sum -Property PassedCount).Sum))
92+
Failed = [int]([math]::Round(($object | Measure-Object -Sum -Property FailedCount).Sum))
93+
NotRun = [int]([math]::Round(($object | Measure-Object -Sum -Property NotRunCount).Sum))
94+
Inconclusive = [int]([math]::Round(($object | Measure-Object -Sum -Property InconclusiveCount).Sum))
95+
Skipped = [int]([math]::Round(($object | Measure-Object -Sum -Property SkippedCount).Sum))
96+
}
97+
} else {
98+
[pscustomobject]@{
99+
Result = $null
100+
Executed = $null
101+
ResultFilePresent = $false
102+
Tests = $null
103+
Passed = $null
104+
Failed = $null
105+
NotRun = $null
106+
Inconclusive = $null
107+
Skipped = $null
108+
}
109+
}
110+
111+
# Determine if there’s any failure for this single test file
112+
$testFailure = (
113+
$result.Result -ne 'Passed' -or
114+
$result.Executed -ne $true -or
115+
$result.ResultFilePresent -eq $false -or
116+
$result.Tests -eq 0 -or
117+
$result.Passed -eq 0 -or
118+
$result.Failed -gt 0 -or
119+
$result.Inconclusive -gt 0
120+
)
121+
122+
if ($testFailure) {
123+
$conclusion = 'Failed'
124+
$color = $PSStyle.Foreground.Red
125+
$isFailure = $true
126+
} else {
127+
$conclusion = 'Passed'
128+
$color = $PSStyle.Foreground.Green
129+
}
130+
$result | Add-Member -NotePropertyName 'Conclusion' -NotePropertyValue $conclusion
131+
132+
$reset = $PSStyle.Reset
133+
$logGroupName = $expected.Name -replace '-TestResult-Report.*', ''
134+
135+
LogGroup " - $color$logGroupName$reset" {
136+
if ($result.Executed -eq $false) {
137+
$unexecutedTests.Add($expected.Name)
138+
Write-GitHubError "Test was not executed as reported in file: $($expected.Name)"
139+
$totalErrors++
140+
} elseif ($result.Result -eq 'Failed') {
141+
$failedTests.Add($expected.Name)
142+
Write-GitHubError "Test result explicitly marked as Failed in file: $($expected.Name)"
143+
$totalErrors++
144+
}
145+
$result | Format-Table | Out-String
146+
}
147+
148+
if ($result.ResultFilePresent) {
149+
$testResults.Add($result)
16150
}
17151
}
18152

19-
end {
20-
Write-Debug "[$scriptName] - End"
153+
Write-Output ('' * 50)
154+
$total = [pscustomobject]@{
155+
Tests = [int]([math]::Round(($testResults | Measure-Object -Sum -Property Tests).Sum))
156+
Passed = [int]([math]::Round(($testResults | Measure-Object -Sum -Property Passed).Sum))
157+
Failed = [int]([math]::Round(($testResults | Measure-Object -Sum -Property Failed).Sum))
158+
NotRun = [int]([math]::Round(($testResults | Measure-Object -Sum -Property NotRun).Sum))
159+
Inconclusive = [int]([math]::Round(($testResults | Measure-Object -Sum -Property Inconclusive).Sum))
160+
Skipped = [int]([math]::Round(($testResults | Measure-Object -Sum -Property Skipped).Sum))
21161
}
162+
163+
164+
$color = if ($isFailure) { $PSStyle.Foreground.Red } else { $PSStyle.Foreground.Green }
165+
$reset = $PSStyle.Reset
166+
LogGroup " - $color`Summary$reset" {
167+
$total | Format-Table | Out-String
168+
if ($total.Failed -gt 0) {
169+
Write-GitHubError "There are $($total.Failed) failed tests of $($total.Tests) tests"
170+
$totalErrors += $total.Failed
171+
}
172+
if ($total.Inconclusive -gt 0) {
173+
Write-GitHubError "There are $($total.Inconclusive) inconclusive tests of $($total.Tests) tests"
174+
$totalErrors += $total.Inconclusive
175+
}
176+
if ($failedTests.Count -gt 0) {
177+
Write-Host 'Failed Test Files'
178+
$failedTests | ForEach-Object { Write-Host " - $_" }
179+
}
180+
if ($unexecutedTests.Count -gt 0) {
181+
Write-Host 'Unexecuted Test Files'
182+
$unexecutedTests | ForEach-Object { Write-Host " - $_" }
183+
}
184+
}
185+
186+
exit $totalErrors

0 commit comments

Comments
 (0)