Skip to content

Fix template engine test parallelism issue with GetVisualStudioInstances#55045

Merged
MichaelSimons merged 3 commits into
mainfrom
michaelsimons/fix-kbe-44878-vs-instances-parallelism
Jun 30, 2026
Merged

Fix template engine test parallelism issue with GetVisualStudioInstances#55045
MichaelSimons merged 3 commits into
mainfrom
michaelsimons/fix-kbe-44878-vs-instances-parallelism

Conversation

@MichaelSimons

Copy link
Copy Markdown
Member

Summary

The VS Setup Configuration COM API (ISetupConfiguration2) has a known concurrency bug that causes failures (exit code 57005/0xDEAD) when multiple processes enumerate VS instances simultaneously. This intermittently hits template engine integration tests that run dotnet new in parallel on CI.

Root Cause

When multiple dotnet new processes run concurrently (as happens in parallel test execution), each process calls VisualStudioWorkloads.GetInstalledWorkloads → COM ISetupConfiguration2::EnumInstances(). The COM API has an unintended concurrent access issue in a critical section that wasn't locked (tracked internally at https://dev.azure.com/devdiv/DevDiv/_workitems/edit/2241752).

The previous fix (PR #44930) added an in-process lock (s_guard) but that doesn't protect against cross-process concurrent access.

Fix

This PR adds two layers of protection:

  1. Named system mutex (Global\DotNetSdk_VSSetupConfiguration) — serializes cross-process access to the COM API. All dotnet processes on the same machine will take turns calling the VS enumeration API.

  2. Retry with exponential backoff (3 attempts, 100ms/200ms/400ms delays) — safety net for cases where external processes (not using our mutex) also call the API concurrently.

Testing

The fix was verified to compile successfully. The intermittent nature of this issue means it can only be validated by observing reduced failure rates on CI over time.

Fixes #44878

…M API

The VS Setup Configuration COM API (ISetupConfiguration2) has a known
concurrency bug that causes failures (exit code 57005/0xDEAD) when
multiple processes enumerate VS instances simultaneously. This hits
template engine integration tests that run dotnet new in parallel.

The previous fix (PR #44930) added an in-process lock (s_guard) but
that doesn't help when multiple test processes call the API concurrently.

This fix adds:
- A named system mutex (Global\DotNetSdk_VSSetupConfiguration) to
  serialize cross-process access to the COM API
- Retry logic with exponential backoff (3 attempts) as a safety net
  for cases where external processes also call the API without our mutex

Fixes #44878

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@MichaelSimons MichaelSimons marked this pull request as ready for review June 30, 2026 14:46
@MichaelSimons MichaelSimons requested review from a team and Copilot June 30, 2026 14:46
@MichaelSimons

Copy link
Copy Markdown
Member Author

@joeloff - are you familiar with this space? Looking for an SME to validate this is a reasonable approach. This is being hit consistently in the SDK tests.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR mitigates intermittent failures in template engine / dotnet new integration tests caused by concurrent enumeration of Visual Studio instances via the VS Setup Configuration COM API (ISetupConfiguration2::EnumInstances). It does so by serializing access across processes and adding a retry safety net around the COM enumeration.

Changes:

  • Add a named system-wide mutex (Global\DotNetSdk_VSSetupConfiguration) to serialize COM access across concurrently running dotnet processes.
  • Add retry logic with exponential backoff around the Visual Studio instance enumeration call path.
  • Refactor the existing workload enumeration into a new GetInstalledWorkloadsCore helper so it can be invoked within the mutex/retry wrapper.

Comment thread src/Cli/dotnet/Commands/Workload/List/VisualStudioWorkloads.cs Outdated
Comment thread src/Cli/dotnet/Commands/Workload/List/VisualStudioWorkloads.cs

@baronfel baronfel left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks reasonable - nitpick idea: any care for jitter to prevent repeated collisions?

@MichaelSimons MichaelSimons force-pushed the michaelsimons/fix-kbe-44878-vs-instances-parallelism branch from 9dd6c43 to 5b85111 Compare June 30, 2026 15:13
- Remove unused s_guard field (would cause CS0414 warning/build failure)
- Add random jitter (0-50ms) to retry delays to prevent thundering-herd
  collisions between processes retrying simultaneously
- Clarify MaxRetryAttempts doc to distinguish retries from total attempts

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@MichaelSimons MichaelSimons force-pushed the michaelsimons/fix-kbe-44878-vs-instances-parallelism branch from 5b85111 to a4882ff Compare June 30, 2026 15:14
@MichaelSimons

Copy link
Copy Markdown
Member Author

This looks reasonable - nitpick idea: any care for jitter to prevent repeated collisions?

Great feedback - copilot address by adding a random offset.

@MichaelSimons MichaelSimons requested a review from baronfel June 30, 2026 15:15
@MichaelSimons MichaelSimons merged commit 2e6a880 into main Jun 30, 2026
25 checks passed
@MichaelSimons MichaelSimons deleted the michaelsimons/fix-kbe-44878-vs-instances-parallelism branch June 30, 2026 21:31
@dotnet-milestone-bot dotnet-milestone-bot Bot added this to the 11.0-preview7 milestone Jul 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Template engine tests hitting known parallelism issue in GetVisualStudioInstances

3 participants