🪲 [Fix]: Fix test and container loading function to also process tests root folder (#35)

MariusStorhaug · web-flow · commit 5dfa6c6b790b · 2025-03-09T16:31:51.000+01:00
## Description This pull request includes several improvements and new features to enhance the robustness and functionality of the PowerShell scripts. The key changes involve adding retry mechanisms for module installations, introducing a new function to process test directories, and updating existing scripts to use these new functionalities. ### Enhancements to module installation: * [`.github/workflows/Action-Test.yml`](diffhunk://#diff-a12ae5c885b0673c0ff6f70c2670886907590d624626e07da4c52e01aeaf56a4L232-R251): Added a retry mechanism for installing the `Markdown` module to handle potential installation failures more gracefully. * [`scripts/exec.ps1`](diffhunk://#diff-9565c9ed6189efecf687222ac63d0ec9702c9708f75a02f7d9d9de9f86ae37a0L7-R8): Replaced direct module installations with the new `Install-PSResourceWithRetry` function to ensure reliable installations. [[1]](diffhunk://#diff-9565c9ed6189efecf687222ac63d0ec9702c9708f75a02f7d9d9de9f86ae37a0L7-R8) [[2]](diffhunk://#diff-9565c9ed6189efecf687222ac63d0ec9702c9708f75a02f7d9d9de9f86ae37a0L61-R58) * [`scripts/init.ps1`](diffhunk://#diff-f47ceebe9ade2bb55cede031de8267e9c87b09336a93fcd557c02c1f488554b6L5-R6): Updated to use `Install-PSResourceWithRetry` for installing required modules, ensuring retries on failure. ### New function for processing test directories: * [`scripts/Helpers.psm1`](diffhunk://#diff-acb1351e3ba396afa6a397b0bd44b5d8ed3ffeb97a3f3ef3633e9cfd6cacf11aR1113-R1318): Introduced `Invoke-ProcessTestDirectory` to scan directories for container and test files, process them accordingly, and support recursive processing. ### Directory and path handling improvements: * [`scripts/exec.ps1`](diffhunk://#diff-9565c9ed6189efecf687222ac63d0ec9702c9708f75a02f7d9d9de9f86ae37a0L24-R21): Changed the temporary directory name from 'temp' to '.temp' for better organization. * [`scripts/init.ps1`](diffhunk://#diff-f47ceebe9ade2bb55cede031de8267e9c87b09336a93fcd557c02c1f488554b6L184-R198): Updated the script to create a '.temp' directory for container output and utilize `Invoke-ProcessTestDirectory` for processing test directories. ### Additional changes: * [`tests/1-Simple-Failure/data/Test.Data.ps1`](diffhunk://#diff-170528ba3d6e6898a263b83dd935f3bc4954ed8f98435b8e66f23df7772fd20fR1): Added a mock test data file. ## Type of change  - [ ] 📖 [Docs] - [x] 🪲 [Fix] - [ ] 🩹 [Patch] - [ ] ⚠️ [Security fix] - [ ] 🚀 [Feature] - [ ] 🌟 [Breaking change] ## Checklist  - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas
diff --git a/.github/workflows/Action-Test.yml b/.github/workflows/Action-Test.yml
@@ -229,7 +229,26 @@ jobs:
         uses: PSModule/Github-Script@v1
         with:
           Script: |
-            Install-PSResource -Name Markdown -Repository PSGallery -TrustRepository
+            'Markdown' | ForEach-Object {
+                $name = $_
+                Write-Output "Installing module: $name"
+                $retryCount = 5
+                $retryDelay = 10
+                for ($i = 0; $i -lt $retryCount; $i++) {
+                    try {
+                        Install-PSResource -Name $name -WarningAction SilentlyContinue -TrustRepository -Repository PSGallery
+                        break
+                    } catch {
+                        Write-Warning "Installation of $name failed with error: $_"
+                        if ($i -eq $retryCount - 1) {
+                            throw
+                        }
+                        Write-Warning "Retrying in $retryDelay seconds..."
+                        Start-Sleep -Seconds $retryDelay
+                    }
+                }
+                Import-Module -Name $name
+            }
 
             # Build an array of objects for each job
             $ActionTest1SimpleOutcome = '${{ needs.ActionTest1Simple.outputs.outcome }}'
diff --git a/scripts/Helpers.psm1 b/scripts/Helpers.psm1
@@ -1110,3 +1110,209 @@ filter Show-Input {
     }
     [pscustomobject]$new | Format-List | Out-String
 }
+
+function Invoke-ProcessTestDirectory {
+    <#
+        .SYNOPSIS
+        Processes a directory to find and handle test and container files.
+
+        .DESCRIPTION
+        This function scans a given directory for container and test files, processing them accordingly.
+        If container files are found, they are imported and exported to the specified output path.
+        If no container files exist, test files are used to generate new container files.
+        The function supports recursive processing of subdirectories.
+
+        .EXAMPLE
+        Invoke-ProcessTestDirectory -Directory 'C:\Tests' -OutputPath 'C:\Output'
+
+        Output:
+        ```powershell
+        === Examining directory: [C:\Tests] (Level: 0) ===
+        Looking for container files in current directory (non-recursive)...
+        Container files found in [C:\Tests]: [2]
+        Processing container file: [Test1.Container.ps1]
+        Processing container file: [Test2.Container.ps1]
+        Exporting container [C:\Output\Test1.Container.ps1]
+        Exporting container [C:\Output\Test2.Container.ps1]
+        === Completed processing directory: [C:\Tests] ===
+        ```
+
+        Processes test container files in 'C:\Tests' and exports them to 'C:\Output'.
+
+        .EXAMPLE
+        Invoke-ProcessTestDirectory -Directory 'C:\Tests' -OutputPath 'C:\Output' -RecursionLevel 1
+
+        Output:
+        ```powershell
+        === Examining directory: [C:\Tests] (Level: 1) ===
+        Looking for container files in current directory (non-recursive)...
+        No container files found - looking for test files...
+        Test files found in [C:\Tests]: [3]
+        Creating container for test file: [TestA.Tests.ps1]
+        Creating container for test file: [TestB.Tests.ps1]
+        Exporting container [C:\Output\TestA.Container.ps1]
+        Exporting container [C:\Output\TestB.Container.ps1]
+        === Completed processing directory: [C:\Tests] ===
+        ```
+
+        Generates and exports container files for test scripts found in 'C:\Tests' at recursion level 1.
+
+        .OUTPUTS
+        array
+
+        .NOTES
+        A list of processed container hashtables.
+        Each container represents a test file or an imported container configuration.
+    #>
+    [Diagnostics.CodeAnalysis.SuppressMessageAttribute(
+        'PSAvoidUsingWriteHost', '',
+        Justification = 'Log to the GitHub Action runner'
+    )]
+    [CmdletBinding()]
+    param(
+        # The directory to process for container and test files.
+        [Parameter(Mandatory)]
+        [string]$Directory,
+
+        # The path where container files should be exported.
+        [Parameter(Mandatory)]
+        [string]$OutputPath,
+
+        # A collection of processed containers, used for recursive directory processing.
+        [Parameter()]
+        [array]$Containers = @(),
+
+        # The current recursion level, used for logging indentation.
+        [Parameter()]
+        [int]$RecursionLevel = 0
+    )
+
+    # Create indent for better log readability based on recursion level
+    $indent = '  ' * $RecursionLevel
+    Write-Host "${indent}=== Examining directory: [$Directory] (Level: $RecursionLevel) ==="
+
+    # First check for container files in this directory (non-recursive)
+    Write-Host "${indent}Looking for container files in current directory (non-recursive)..."
+    $containerFiles = Get-ChildItem -Path $Directory -Filter *.Container.* -File
+    $containerFilesFound = $containerFiles.Count -gt 0
+
+    Write-Host "${indent}Container files found in [$Directory]: [$($containerFiles.Count)]"
+    if ($containerFilesFound) {
+        Write-Host "${indent}Container files detected - will process them directly"
+    }
+
+    if ($containerFilesFound) {
+        # If container files exist, use those for this directory
+        foreach ($containerFile in $containerFiles) {
+            Write-Host "${indent}Processing container file: [$($containerFile.Name)]"
+            $container = Import-Hashtable $containerFile
+            $containerFileName = $containerFile | Split-Path -Leaf
+            Write-Host "${indent}Init - Export containers - $containerFileName"
+            Write-Host "${indent}Container configuration:"
+            Write-Host (Format-Hashtable -Hashtable $container)
+            Write-Host "${indent}Exporting container [$OutputPath/$containerFileName]"
+            Export-Hashtable -Hashtable $container -Path "$OutputPath/$containerFileName"
+            Write-Host "${indent}Added container from $containerFileName to collection"
+            $Containers += $container
+        }
+    } else {
+        # If no container files, look for test files in this directory only (non-recursive)
+        Write-Host "${indent}No container files found - looking for test files..."
+        $testFiles = Get-ChildItem -Path $Directory -Filter *.Tests.ps1 -File
+        Write-Host "${indent}Test files found in [$Directory]: [$($testFiles.Count)]"
+
+        if ($testFiles.Count -gt 0) {
+            Write-Host "${indent}Will generate containers for each test file"
+        } else {
+            Write-Host "${indent}No test files found in this directory"
+        }
+
+        # Create containers for test files in this directory
+        foreach ($testFile in $testFiles) {
+            Write-Host "${indent}Creating container for test file: [$($testFile.Name)]"
+            $container = @{
+                Path = $testFile.FullName
+            }
+            $containerFileName = ($testFile | Split-Path -Leaf).Replace('.Tests.ps1', '.Container.ps1')
+            Write-Host "${indent}Init - Export containers - Generated - $containerFileName"
+            Write-Host "${indent}Container configuration:"
+            Write-Host (Format-Hashtable -Hashtable $container)
+            Write-Host "${indent}Exporting container [$OutputPath/$containerFileName]"
+            Export-Hashtable -Hashtable $container -Path "$OutputPath/$containerFileName"
+            Write-Host "${indent}Added generated container for $($testFile.Name) to collection"
+            $Containers += $container
+        }
+    }
+
+    # Now process subdirectories recursively
+    Write-Host "${indent}Checking for subdirectories in [$Directory]..."
+    $subdirectories = Get-ChildItem -Path $Directory -Directory
+    $subdirCount = $subdirectories.Count
+    Write-Host "${indent}Found $subdirCount subdirectories to process"
+
+    if ($subdirCount -gt 0) {
+        Write-Host "${indent}Beginning recursive processing of $subdirCount subdirectories..."
+    }
+
+    foreach ($subdir in $subdirectories) {
+        Write-Host "${indent}Processing subdirectory - [$($subdir.Name)]"
+        $params = @{
+            Directory      = $subdir.FullName
+            OutputPath     = $OutputPath
+            Containers     = $Containers
+            RecursionLevel = ($RecursionLevel + 1)
+        }
+        $Containers = Invoke-ProcessTestDirectory @params
+    }
+
+    Write-Host "${indent}=== Completed processing directory: [$Directory] ==="
+    Write-Host "${indent}Total containers after processing [$Directory]: [$($Containers.Count)]"
+
+    $Containers
+}
+
+function Install-PSResourceWithRetry {
+    <#
+        .SYNOPSIS
+        Installs a PowerShell module with retry mechanism
+
+        .DESCRIPTION
+        Attempts to install a PowerShell module multiple times in case of failure
+    #>
+    [CmdletBinding()]
+    param (
+        # Name of the module to install
+        [Parameter(
+            Mandatory,
+            Position = 0,
+            ValueFromPipeline
+        )]
+        [string] $Name,
+
+        # Number of times to retry installation, default is 5
+        [Parameter()]
+        [int] $RetryCount = 5,
+
+        # Delay in seconds between retries, default is 10
+        [Parameter()]
+        [int] $RetryDelay = 10
+    )
+
+    process {
+        Write-Output "Installing module: $Name"
+        for ($i = 0; $i -lt $RetryCount; $i++) {
+            try {
+                Install-PSResource -Name $Name -WarningAction SilentlyContinue -TrustRepository -Repository PSGallery
+                break
+            } catch {
+                Write-Warning "Installation of $Name failed with error: $_"
+                if ($i -eq $RetryCount - 1) {
+                    throw
+                }
+                Write-Warning "Retrying in $RetryDelay seconds..."
+                Start-Sleep -Seconds $RetryDelay
+            }
+        }
+        Import-Module -Name $Name
+    }
+}
diff --git a/scripts/exec.ps1 b/scripts/exec.ps1
@@ -4,11 +4,8 @@ param()
 $PSStyle.OutputRendering = 'Ansi'
 
 '::group::Exec - Setup prerequisites'
-'Pester' | ForEach-Object {
-    Install-PSResource -Name $_ -WarningAction SilentlyContinue -TrustRepository -Repository PSGallery
-    Import-Module -Name $_
-}
 Import-Module "$PSScriptRoot/Helpers.psm1"
+'Pester' | Install-PSResourceWithRetry
 '::endgroup::'
 
 '::group::Exec - Get test kit versions'
@@ -21,7 +18,7 @@ $pesterModule = Get-PSResource -Name Pester -Verbose:$false | Sort-Object Versio
 '::endgroup::'
 
 '::group::Exec - Info about environment'
-$path = Join-Path -Path $pwd.Path -ChildPath 'temp'
+$path = Join-Path -Path $pwd.Path -ChildPath '.temp'
 Test-Path -Path $path
 Get-ChildItem -Path $path -Recurse | Sort-Object FullName | Format-Table -AutoSize | Out-String
 
@@ -58,11 +55,7 @@ $configuration = New-PesterConfiguration -Hashtable $configuration
 $testResults = Invoke-Pester -Configuration $configuration
 
 LogGroup 'Eval - Setup prerequisites' {
-    'Pester', 'Hashtable', 'TimeSpan', 'Markdown' | ForEach-Object {
-        Install-PSResource -Name $_ -Verbose:$false -WarningAction SilentlyContinue -TrustRepository -Repository PSGallery
-        Import-Module -Name $_ -Verbose:$false
-    }
-    Import-Module "$PSScriptRoot/Helpers.psm1"
+    'Pester', 'Hashtable', 'TimeSpan', 'Markdown' | Install-PSResourceWithRetry
 }
 
 LogGroup 'Eval - Get test kit versions' {
diff --git a/scripts/init.ps1 b/scripts/init.ps1
@@ -2,11 +2,8 @@
 param()
 
 LogGroup 'Init - Setup prerequisites' {
-    'Pester', 'Hashtable', 'TimeSpan', 'Markdown' | ForEach-Object {
-        Install-PSResource -Name $_ -WarningAction SilentlyContinue -TrustRepository -Repository PSGallery
-        Import-Module -Name $_
-    }
     Import-Module "$PSScriptRoot/Helpers.psm1"
+    'Pester', 'Hashtable', 'TimeSpan', 'Markdown' | Install-PSResourceWithRetry
 }
 
 LogGroup 'Init - Get test kit versions' {
@@ -181,61 +178,24 @@ LogGroup 'Init - Export containers' {
         }
     }
     Write-Output "Containers from configuration: [$($containers.Count)]"
-    # Search for "*.Container.*" files in each Run.Path directory
-    Write-Output 'Searching for containers in same location as config.'
-    $path = New-Item -Path . -ItemType Directory -Name 'temp' -Force
+
+    # Create temp directory for container output
+    $path = New-Item -Path . -ItemType Directory -Name '.temp' -Force
+
+    # Process each input path
     foreach ($testDir in $inputs.Path) {
-        #If testDir is a file, get the directory
+        # If testDir is a file, get the directory
         $testItem = Get-Item -Path $testDir
         if ($testItem.PSIsContainer -eq $false) {
             $testDir = $testItem.DirectoryName
         }
 
-        $containerFiles = Get-ChildItem -Path $testDir -Filter *.Container.* -Recurse
-        Write-Output "Containers found in [$testDir]: [$($containerFiles.Count)]"
-        if ($containerFiles.Count -eq 0) {
-            # First, look for test files directly in the test directory (non-recursive)
-            $rootTestFiles = Get-ChildItem -Path $testDir -Filter *.Tests.ps1 -File
-            Write-Output "Root level test files found in [$testDir]: [$($rootTestFiles.Count)]"
-
-            # Then, look for test files in subdirectories
-            $subfolderTestFiles = Get-ChildItem -Path $testDir -Filter *.Tests.ps1 -Recurse -File |
-                Where-Object { $_.DirectoryName -ne $testDir }
-            Write-Output "Subfolder test files found in [$testDir]: [$($subfolderTestFiles.Count)]"
-
-            # Combine all test files using a generic List
-            $testFiles = [System.Collections.Generic.List[System.IO.FileInfo]]::new()
-            if ($rootTestFiles) {
-                $rootTestFiles | ForEach-Object { $testFiles.Add($_) }
-            }
-            if ($subfolderTestFiles) {
-                $subfolderTestFiles | ForEach-Object { $testFiles.Add($_) }
-            }
-            Write-Output "Total test files found in [$testDir]: [$($testFiles.Count)]"
-
-            foreach ($testFile in $testFiles) {
-                $container = @{
-                    Path = $testFile.FullName
-                }
-                $containerFileName = ($testFile | Split-Path -Leaf).Replace('.Tests.ps1', '.Container.ps1')
-                LogGroup "Init - Export containers - Generated - $containerFileName" {
-                    Write-Output "Exporting container [$path/$containerFileName]"
-                    Export-Hashtable -Hashtable $container -Path "$path/$containerFileName"
-                }
-                $containers += $container
-            }
-            Write-Output "Containers created from test files: [$($containers.Count)]"
-        }
-        foreach ($containerFile in $containerFiles) {
-            $container = Import-Hashtable $containerFile
-            $containerFileName = $containerFile | Split-Path -Leaf
-            LogGroup "Init - Export containers - $containerFileName" {
-                Format-Hashtable -Hashtable $container
-                Write-Output "Exporting container [$path/$containerFileName]"
-                Export-Hashtable -Hashtable $container -Path "$path/$containerFileName"
-            }
-            $containers += $container
-        }
+        Write-Output "Processing test directory: [$testDir]"
+
+        # Process the root directory and all subdirectories recursively
+        $containers += Invoke-ProcessTestDirectory -Directory $testDir -OutputPath $path
+
+        Write-Output "Total containers after processing [$testDir]: [$($containers.Count)]"
     }
     $configuration.Run.Container = @()
 }
diff --git a/tests/1-Simple-Failure/data/Test.Data.ps1 b/tests/1-Simple-Failure/data/Test.Data.ps1
@@ -0,0 +1 @@
+﻿"This is some mock test data"
