[CI] 'extract workload packs' failures should fail the job fast (currently succeedsWithIssues, then runs the whole suite against a broken SDK)

[simonrozsival]

Summary

When the extract workload packs setup step fails (e.g. a transient dotnet workload install network error), the CI job does not fail fast. The step is marked succeededWithIssues, the pipeline continues to boot the emulator and run the entire device-test suite against a broken/rolled-back local SDK, every dotnet build fails, and the job only finally fails at the very end via fail if any issues occurred — after wasting the full emulator test run and producing a pile of misleading "build failure" results.

If we cannot install the SDK/workloads, we cannot run a single test, so the job should stop immediately with a clear error instead of soldiering on.

Real-world example

Build 1486425 (PR #11804), shard macOS > Tests > MSBuild+Emulator 10 (agent Azure Pipelines 47):

extract workload packs step:

Workload installation failed. Rolling back installed packs...
Workload installation failed: One or more errors occurred.
  (Unable to read data from the transport connection: Connection reset by peer.)
.../build-tools/create-packs/Directory.Build.targets(140,5): error MSB3073:
  The command ""/Users/runner/work/1/s/bin/Release/dotnet/dotnet" workload ..." exited ...
##[error]PowerShell exited with code '1'.

Step result: succeededWithIssues (not failed).

Consequence — the job ran the whole suite anyway and reported 19 failed tests, all on this one shard, all cascade symptoms of the missing SDK:

• ~16× Xamarin.ProjectTools.FailedBuildException : Build failure: <project>.csproj across unrelated tests (BundleTool/Localization, Styleable.Library, marshal2, テスト, UnnamedProject, …)
• 2× System.IO.DirectoryNotFoundException : Could not find a part of the path '.../MSBuildDeviceIntegration/net10.0/data'
• a few dotnet build should succeed / Build should have succeeded

None of these are real test failures — they are the symptom of extract workload packs having failed. The misleading results cost triage time (looks like 19 product failures on a PR that changed unrelated code) and the run burned the full emulator-test duration before failing.

Root cause (pipeline config)

extract workload packs calls the run-dotnet-preview.yaml template and overrides none of its error-handling defaults:

build-tools/automation/yaml-templates/setup-test-environment-steps.yaml:

- template: /build-tools/automation/yaml-templates/run-dotnet-preview.yaml
  parameters:
    displayName: extract workload packs
    project: .../build-tools/create-packs/Microsoft.Android.Sdk.proj
    arguments: -t:ExtractWorkloadPacks -c ${{ parameters.configuration }} -v:n -bl:...

build-tools/automation/yaml-templates/run-dotnet-preview.yaml defaults:

useExitCodeForErrors: false
continueOnError: true          # <-- failure becomes succeededWithIssues
retryCountOnTaskFailure: 0     # <-- no retry on transient network errors

So a failed SDK/workload install is swallowed (continueOnError: true), never retried (retryCountOnTaskFailure: 0), and the job proceeds to run tests it has no chance of passing.

Proposed fix

Treat workload-pack extraction as a hard prerequisite of the test job:

1. Fail fast — for the extract workload packs step, set continueOnError: false (and/or useExitCodeForErrors: true) so the job stops immediately with a clear error when the install fails, instead of running the whole suite and failing at the end.
2. Retry transient failures — set retryCountOnTaskFailure (e.g. 2) on this step so a one-off Connection reset by peer during the workload download is retried rather than poisoning the run.

This keeps genuine setup failures obvious and fast, retries flaky network blips, and prevents a single broken SDK install from masquerading as ~19 product test failures.

Affected files

• build-tools/automation/yaml-templates/setup-test-environment-steps.yaml (the extract workload packs step, ~line 101)
• build-tools/automation/yaml-templates/run-dotnet-preview.yaml (error-handling defaults)