Fix intermittent SDK resolution failure in tests#55043
Closed
MichaelSimons wants to merge 1 commit into
Closed
Conversation
Improve the transient SDK resolution error detection in two ways: 1. Check both stdout AND stderr for transient errors in TestCommand's retry logic. MSBuild may emit error diagnostics to either stream depending on logger configuration, so checking only stdout missed errors written to stderr. 2. Add detection for MSB4276 (default SDK resolver directory probe failure) in addition to the existing MSB4236 + workload resolver pattern. Under heavy parallel I/O, the default resolver can also fail to find the Sdks/<name>/Sdk directory even though it exists on disk. This pattern is constrained to Microsoft.NET.Sdk family SDKs to avoid retrying genuinely missing custom SDKs. Fixes #51530 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes intermittent "Could not resolve SDK" test failures that occur under heavy parallel I/O in CI.
Fixes #51530
Changes
1. Check both stdout and stderr for transient errors (
TestCommand.cs)The
ShouldStopRetrymethod previously only checkedresult.StdOutfor transient SDK resolution errors. However, MSBuild may emit error diagnostics to stderr depending on the logger configuration and hosting context. This meant transient errors written to stderr were not detected, causing the test to fail without retrying.2. Detect MSB4276 default resolver directory probe failures (
TransientSdkResolutionErrorDetector.cs)Added a second detection pattern for MSB4276 errors where the default SDK resolver fails to find the
Sdks/<name>/Sdkdirectory under heavy parallel I/O. This is a race condition where the directory exists on disk but the filesystem probe fails transiently under load. The detection is constrained toMicrosoft.NET.Sdkfamily SDKs to avoid retrying genuinely missing custom SDKs.3. Test coverage (
TransientSdkResolutionErrorDetectorTests.cs)Added tests for the new MSB4276 pattern, including a negative test ensuring custom (non-Microsoft.NET.Sdk) SDKs are not retried.
Context
See also: #51525 (related testfx issue where SDK resolution fails due to PATH/environment issues after image updates — a different root cause but similar symptom).