Windows Helix images are failing to build and are blocking builds

[MichaelSimons]

Failing build (Microsoft internal link)

The failures are related to installing python.

The command 'cmd /S /C powershell -Command "         [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12;         $apiUrl = 'https://api.nuget.org/v3-flatcontainer/python/index.json';         $response = Invoke-RestMethod -Uri $apiUrl;         $versions = $response.versions | Where-Object { $_ -notmatch '-' } | Sort-Object { [version]$_ } -Descending;         $latestVersion = $versions[0];         echo \"Downloading Python $latestVersion\";         $pythonZip = \"$env:TEMP\python.zip\";         Invoke-WebRequest -Uri https://www.nuget.org/api/v2/package/python/$latestVersion -OutFile $pythonZip;         $pythonTemp = 'C:\PythonTemp';         md $pythonTemp;         tar -zxf $pythonZip -C $pythonTemp;         if ($LASTEXITCODE -ne 0) {             echo 'Failed to extract Python';             exit 1;         }         xcopy /s /i c:\PythonTemp\tools C:\Python;         if ($LASTEXITCODE -ne 0) {             echo 'Failed to copy Python';             exit 1;         }         Remove-Item -Recurse -Force $pythonTemp;         Remove-Item -Force $pythonZip;"' returned a non-zero code: 1
Retry 5/5, retrying in 625 seconds...
Sending build context to Docker daemon  5.632kB

Step 1/6 : FROM mcr.microsoft.com/windows/servercore:ltsc2025
 ---> d2be4b0d11c6
Step 2/6 : RUN setx /M VIRTUAL_ENV "C:\Python-env" &&     setx /M PATH "%PATH%;C:\Program Files\PowerShell\;%VIRTUAL_ENV%\Scripts;C:\Python;C:\python\scripts" &&     setx /M PYTHONPATH "C:\Python\Lib;C:\Python\DLLs;"
 ---> Using cache
 ---> cda306a81e3d
Step 3/6 : RUN powershell -Command "         [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12;         $apiUrl = 'https://api.nuget.org/v3-flatcontainer/python/index.json';         $response = Invoke-RestMethod -Uri $apiUrl;         $versions = $response.versions | Where-Object { $_ -notmatch '-' } | Sort-Object { [version]$_ } -Descending;         $latestVersion = $versions[0];         echo \"Downloading Python $latestVersion\";         $pythonZip = \"$env:TEMP\python.zip\";         Invoke-WebRequest -Uri https://www.nuget.org/api/v2/package/python/$latestVersion -OutFile $pythonZip;         $pythonTemp = 'C:\PythonTemp';         md $pythonTemp;         tar -zxf $pythonZip -C $pythonTemp;         if ($LASTEXITCODE -ne 0) {             echo 'Failed to extract Python';             exit 1;         }         xcopy /s /i c:\PythonTemp\tools C:\Python;         if ($LASTEXITCODE -ne 0) {             echo 'Failed to copy Python';             exit 1;         }         Remove-Item -Recurse -Force $pythonTemp;         Remove-Item -Force $pythonZip;"
 ---> Running in 599838e472eb
Invoke-RestMethod : Unable to connect to the remote server
At line:1 char:189
+ ... dex.json';         $response = Invoke-RestMethod -Uri $apiUrl;        ...
+                                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:Htt 
   pWebRequest) [Invoke-RestMethod], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShe 
   ll.Commands.InvokeRestMethodCommand
 
Cannot index into a null array.
At line:1 char:347
+ ... rsion]$_ } -Descending;         $latestVersion = $versions[0];        ...
+                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : NullArray
 
Downloading Python 
Invoke-WebRequest : Unable to connect to the remote server
At line:1 char:481
+ ... p";         Invoke-WebRequest -Uri https://www.nuget.org/api/v2/packa ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:Htt 
   pWebRequest) [Invoke-WebRequest], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShe 
   ll.Commands.InvokeWebRequestCommand
 


    Directory: C:\


Mode                 LastWriteTime         Length Name                         
----                 -------------         ------ ----                         
d-----        10/20/2025   3:05 PM                PythonTemp                   
tar.exe: Error opening archive: Failed to open 'C:\Users\ContainerAdministrator\AppData\Local\Temp\python.zip'
Failed to extract Python

[mthalman]

This was supposed to be fixed by #1518 but it doesn't seem to be helping for some reason. The test build (internal link) for that change did pass, strangely enough.

[mthalman]

This is still working in the unofficial pipeline. No idea why it would only fail in the official pipeline.