(PA-7407) Add more detailed error information when upgrade fails

[AriaXLi]

Previously when puppet_agent fails to upgrade, only the error is given. This commit updates install_puppet.ps1 and puppet_agent_upgrade_error provider to collect & provide more information about the error. Specifically, the last 10 lines of output from the installer.log and any other error(s) that caused the failure are now included in the puppet_agent_upgrade_failure.log and are output by the puppet_agent_upgrade_error type and provider. Any lines with
PUPPET_AGENT_ACCOUNT_PASSWORD are excluded from the output and failure.log.

[AriaXLi]

I am able to recreate the ScriptHalted error using this upgrade.pp (with an invalid dir given for install_dir):

  class { 'puppet_agent':
    package_version => "8.11.0",
    # Upgrades in Docker cannot start the puppet service due to systemd
    # incompatibilities. Essentially, Docker expects an init system for
    # services, but puppet installs services as systemd services. The line below
    # ensures that no services are started, resulting in a proper upgrade
    # process.
    collection      => "puppetcore8",
    username        => "forge-key",
    password        => Sensitive("XXX"),
    install_dir => 'D:\Program Files\Puppet Labs'
  }
include puppet_agent

The output before my changes:

$ puppet apply --debug uprade.pp
...
Debug: Reading Error Log C:/ProgramData/PuppetLabs/puppet/cache/state/puppet_agent_upgrade_failure.log
Debug: Deleting Error Log
Error: Failed previous installation with: ScriptHalted

Error: /Stage[main]/Puppet_agent::Install::Windows/Puppet_agent_upgrade_error[puppet_agent_upgrade_failure.log]/ensure_notexist: change from false to true failed: Failed previous installation with: ScriptHalted

The output after my changes:

puppet apply --debug uprade.pp
...
Debug: Reading Error Log C:/ProgramData/PuppetLabs/puppet/cache/state/puppet_agent_upgrade_failure.log
Debug: Deleting Error Log
Error: Failed previous installation with: ScriptHalted
MSI (s) (40:88) [18:26:53:231]: Decrementing counter to disable shutdown. If counter >= 0, shutdown will be denied.  Counter after decrement: -1
MSI (s) (40:88) [18:26:53:231]: Destroying RemoteAPI object.
MSI (s) (40:B0) [18:26:53:231]: Custom Action Manager thread ending.
MSI (c) (58:08) [18:26:53:231]: Decrementing counter to disable shutdown. If counter >= 0, shutdown will be denied.  Counter after decrement: -1
MSI (c) (58:08) [18:26:53:231]: MainEngineThread is returning 1603
=== Verbose logging stopped: 10/2/2025  18:26:53 ===

10/2/2025 6:26 PM ****************************** End msiexec.exe output ******************************
10/2/2025 6:26 PM ERROR: msiexec.exe installation failed!!! Return code 1603
10/2/2025 6:26 PM ERROR: ScriptHalted

Error: /Stage[main]/Puppet_agent::Install::Windows/Puppet_agent_upgrade_error[puppet_agent_upgrade_failure.log]/ensure_notexist: change from false to true failed: Failed previous installation with: ScriptHalted
MSI (s) (40:88) [18:26:53:231]: Decrementing counter to disable shutdown. If counter >= 0, shutdown will be denied.  Counter after decrement: -1
MSI (s) (40:88) [18:26:53:231]: Destroying RemoteAPI object.
MSI (s) (40:B0) [18:26:53:231]: Custom Action Manager thread ending.
MSI (c) (58:08) [18:26:53:231]: Decrementing counter to disable shutdown. If counter >= 0, shutdown will be denied.  Counter after decrement: -1
MSI (c) (58:08) [18:26:53:231]: MainEngineThread is returning 1603
=== Verbose logging stopped: 10/2/2025  18:26:53 ===

10/2/2025 6:26 PM ****************************** End msiexec.exe output ******************************
10/2/2025 6:26 PM ERROR: msiexec.exe installation failed!!! Return code 1603
10/2/2025 6:26 PM ERROR: ScriptHalted

[AriaXLi]

I modified installer.log (which maps to $logfile) to contain lines with PUPPET_AGENT_ACCOUNT_PASSWORD

PS C:\Users\Administrator> echo $logfile
C:\Users\ADMINI~1\AppData\Local\Temp\1\puppet-2025_10_02-18_09-installer.log

Without Select-String -NotMatch for last 10 lines of logfile

PS C:\Users\Administrator> Get-Content -Path $logfile -Tail 10
10/2/2025 6:09 PM ERROR: ScriptHalted
10/2/2025 6:09 PM Restoring service state for puppet
10/2/2025 6:09 PM Restoring service state for pxp-agent
10/2/2025 6:09 PM Unlocking installation
10/2/2025 6:09 PM Unlocked
10/2/2025 6:09 PM Checking puppet-agent.msi version and expected puppet-agent version..
10/2/2025 6:09 PM .msi file version and expected puppet-agent version match (8.11.0)
MSI (s) (C4:10) [23:58:05:530]: PROPERTY CHANGE: Adding PUPPET_AGENT_ACCOUNT_PASSWORD property. Its value is 'PuppetLabs!'.
Property(S): PUPPET_AGENT_ACCOUNT_PASSWORD = PuppetLabs!

With Select-String -NotMatch for last 10 lines of logfile

PS C:\Users\Administrator> Get-Content -Path $logfile -Tail 10 | Select-String -Pattern "^.*PUPPET_AGENT_ACCOUNT_PASSWORD.*$" -NotMatch

10/2/2025 6:09 PM ERROR: ScriptHalted
10/2/2025 6:09 PM Restoring service state for puppet
10/2/2025 6:09 PM Restoring service state for pxp-agent
10/2/2025 6:09 PM Unlocking installation
10/2/2025 6:09 PM Unlocked
10/2/2025 6:09 PM Checking puppet-agent.msi version and expected puppet-agent version..
10/2/2025 6:09 PM .msi file version and expected puppet-agent version match (8.11.0)
[4:57](https://perforce.slack.com/archives/D041EHTQGUE/p1759967878416829)

Without Select-String -NotMatch for lines with Error or -- Error in logfile

PS C:\Users\Administrator> Get-Content -Path $logfile | Select-String -Pattern "^Error.*$|-- Error"

MSI (s) (78:18) [18:09:39:208]: Product: Puppet Agent (64-bit) -- Error 1327. Invalid Drive: F:\ Property(S): PUPPET_AGENT_ACCOUNT_PASSWORD = PuppetLabs!
Error 1327. Invalid Drive: F:\

With Select-String -NotMatch for lines with Error or -- Error in logfile

PS C:\Users\Administrator> Get-Content -Path $logfile | Select-String -Pattern "^Error.*$|-- Error"| Select-String -Pattern "^.*PUPPET_AGENT_ACCOUNT_PASSWORD.*$" -NotMatch

Error 1327. Invalid Drive: F:\