Skip to content

C#: Only use reachable feeds when private registries are configured #21385

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
mbg wants to merge 7 commits into
base: main
Choose a base branch
from
+89 −54
Draft

C#: Only use reachable feeds when private registries are configured #21385

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
1ceb420
Refactor `GetReachableNuGetFeeds` out of `GetReachableFallbackNugetFe…
mbg Feb 27, 2026
ddcd9d5
Use `GetReachableNuGetFeeds` in `CheckSpecifiedFeeds`
mbg Feb 27, 2026
acba599
Inline `CheckFeeds`
mbg Feb 27, 2026
27ff77e
Use `explicitFeeds` directly
mbg Feb 27, 2026
5415bb7
Divide up `CheckSpecifiedFeeds`
mbg Feb 27, 2026
7a8e10e
Check reachability of inherited feeds
mbg Feb 27, 2026
791c1fa
Only use reachable feeds when private registries are configured
mbg Feb 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
143 changes: 89 additions & 54 deletions csharp/extractor/Semmle.Extraction.CSharp.DependencyFetching/NugetPackageRestorer.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,16 +116,41 @@ public HashSet<AssemblyLookupLocation> Restore()

HashSet<string>? explicitFeeds = null;
HashSet<string>? allFeeds = null;
HashSet<string>? reachableFeeds = [];

try
{
if (checkNugetFeedResponsiveness && !CheckFeeds(out explicitFeeds, out allFeeds))
if (checkNugetFeedResponsiveness)
{
// todo: we could also check the reachability of the inherited nuget feeds, but to use those in the fallback we would need to handle authentication too.
var unresponsiveMissingPackageLocation = DownloadMissingPackagesFromSpecificFeeds([], explicitFeeds);
return unresponsiveMissingPackageLocation is null
? []
: [unresponsiveMissingPackageLocation];
// Find feeds that are configured in NuGet.config files and divide them into ones that
// are explicitly configured for the project, and "all feeds" (including inherited ones)
// from other locations on the host outside of the working directory.
(explicitFeeds, allFeeds) = GetAllFeeds();
var inheritedFeeds = allFeeds.Except(explicitFeeds).ToHashSet();

// If private package registries are configured for C#, then consider those
// in addition to the ones that are configured in `nuget.config` files.
this.dependabotProxy?.RegistryURLs.ForEach(url => explicitFeeds.Add(url));
Copy link
Copy Markdown
Contributor

@michaelnebel michaelnebel Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Remove all the this prefixes when calling instance methods and accessing instance variables (this is the pattern used in the extractor).


var (explicitFeedsReachable, reachableExplicitFeeds) =
this.CheckSpecifiedFeeds(explicitFeeds);
reachableFeeds.UnionWith(reachableExplicitFeeds);

reachableFeeds.UnionWith(this.GetReachableNuGetFeeds(inheritedFeeds, isFallback: false));

if (inheritedFeeds.Count > 0)
{
compilationInfoContainer.CompilationInfos.Add(("Inherited NuGet feed count", inheritedFeeds.Count.ToString()));
}

if (!explicitFeedsReachable)
{
// todo: we could also check the reachability of the inherited nuget feeds, but to use those in the fallback we would need to handle authentication too.
var unresponsiveMissingPackageLocation = DownloadMissingPackagesFromSpecificFeeds([], explicitFeeds);
return unresponsiveMissingPackageLocation is null
? []
: [unresponsiveMissingPackageLocation];
}
}

using (var nuget = new NugetExeWrapper(fileProvider, legacyPackageDirectory, logger))
Copy link
Copy Markdown
Contributor

@michaelnebel michaelnebel Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The NugetExeWrapper also explicitly add the "default" NuGet feed in some cases. Maybe we should consider to add a feed reachability check (if checking nuget feed reponsiveness)?

Expand Down Expand Up @@ -167,9 +192,10 @@ public HashSet<AssemblyLookupLocation> Restore()
logger.LogError($"Failed to restore NuGet packages with nuget.exe: {exc.Message}");
}

// Restore project dependencies with `dotnet restore`.
var restoredProjects = RestoreSolutions(out var container);
Copy link
Copy Markdown
Contributor

@michaelnebel michaelnebel Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also add the reachableFeeds logic for restoring solution files (and also make use of the private registries in this case)?

var projects = fileProvider.Projects.Except(restoredProjects);
RestoreProjects(projects, allFeeds, out var containers);
RestoreProjects(projects, reachableFeeds, out var containers);

var dependencies = containers.Flatten(container);

Expand All @@ -192,6 +218,34 @@ public HashSet<AssemblyLookupLocation> Restore()
return assemblyLookupLocations;
}

/// <summary>
/// Tests which of the feeds given by <paramref name="feedsToCheck"/> are reachable.
/// </summary>
/// <param name="feedsToCheck">The feeds to check.</param>
/// <param name="isFallback">Whether the feeds are fallback feeds or not.</param>
/// <returns>The list of feeds that could be reached.</returns>
private List<string> GetReachableNuGetFeeds(HashSet<string> feedsToCheck, bool isFallback)
{
var fallbackStr = isFallback ? "fallback " : "";
logger.LogInfo($"Checking {fallbackStr}NuGet feed reachability on feeds: {string.Join(", ", feedsToCheck.OrderBy(f => f))}");

var (initialTimeout, tryCount) = GetFeedRequestSettings(isFallback);
var reachableFeeds = feedsToCheck
.Where(feed => IsFeedReachable(feed, initialTimeout, tryCount, allowExceptions: false))
.ToList();

if (reachableFeeds.Count == 0)
{
logger.LogWarning($"No {fallbackStr}NuGet feeds are reachable.");
}
else
{
logger.LogInfo($"Reachable {fallbackStr}NuGet feeds: {string.Join(", ", reachableFeeds.OrderBy(f => f))}");
}

return reachableFeeds;
}

private List<string> GetReachableFallbackNugetFeeds(HashSet<string>? feedsFromNugetConfigs)
{
var fallbackFeeds = EnvironmentVariables.GetURLs(EnvironmentVariableNames.FallbackNugetFeeds).ToHashSet();
Expand All @@ -212,21 +266,7 @@ private List<string> GetReachableFallbackNugetFeeds(HashSet<string>? feedsFromNu
}
}

logger.LogInfo($"Checking fallback NuGet feed reachability on feeds: {string.Join(", ", fallbackFeeds.OrderBy(f => f))}");
var (initialTimeout, tryCount) = GetFeedRequestSettings(isFallback: true);
var reachableFallbackFeeds = fallbackFeeds.Where(feed => IsFeedReachable(feed, initialTimeout, tryCount, allowExceptions: false)).ToList();
if (reachableFallbackFeeds.Count == 0)
{
logger.LogWarning("No fallback NuGet feeds are reachable.");
}
else
{
logger.LogInfo($"Reachable fallback NuGet feeds: {string.Join(", ", reachableFallbackFeeds.OrderBy(f => f))}");
}

compilationInfoContainer.CompilationInfos.Add(("Reachable fallback NuGet feed count", reachableFallbackFeeds.Count.ToString()));

return reachableFallbackFeeds;
return GetReachableNuGetFeeds(fallbackFeeds, isFallback: true);
}

/// <summary>
Expand Down Expand Up @@ -719,53 +759,50 @@ private bool IsFeedReachable(string feed, int timeoutMilliSeconds, int tryCount,
}

/// <summary>
/// Checks that we can connect to all NuGet feeds that are explicitly configured in configuration files
/// as well as any private package registry feeds that are configured.
/// Retrieves a list of excluded NuGet feeds from the corresponding environment variable.
/// </summary>
/// <param name="explicitFeeds">Outputs the set of explicit feeds.</param>
/// <param name="allFeeds">Outputs the set of all feeds (explicit and inherited).</param>
/// <returns>True if all feeds are reachable or false otherwise.</returns>
private bool CheckFeeds(out HashSet<string> explicitFeeds, out HashSet<string> allFeeds)
private HashSet<string> GetExcludedFeeds()
{
(explicitFeeds, allFeeds) = GetAllFeeds();
HashSet<string> feedsToCheck = explicitFeeds;

// If private package registries are configured for C#, then check those
// in addition to the ones that are configured in `nuget.config` files.
this.dependabotProxy?.RegistryURLs.ForEach(url => feedsToCheck.Add(url));

var allFeedsReachable = this.CheckSpecifiedFeeds(feedsToCheck);
var excludedFeeds = EnvironmentVariables.GetURLs(EnvironmentVariableNames.ExcludedNugetFeedsFromResponsivenessCheck)
.ToHashSet();

var inheritedFeeds = allFeeds.Except(explicitFeeds).ToHashSet();
if (inheritedFeeds.Count > 0)
if (excludedFeeds.Count > 0)
{
logger.LogInfo($"Inherited NuGet feeds (not checked for reachability): {string.Join(", ", inheritedFeeds.OrderBy(f => f))}");
compilationInfoContainer.CompilationInfos.Add(("Inherited NuGet feed count", inheritedFeeds.Count.ToString()));
logger.LogInfo($"Excluded NuGet feeds from responsiveness check: {string.Join(", ", excludedFeeds.OrderBy(f => f))}");
}

return allFeedsReachable;
return excludedFeeds;
}

/// <summary>
/// Checks that we can connect to the specified NuGet feeds.
/// </summary>
/// <param name="feeds">The set of package feeds to check.</param>
/// <returns>True if all feeds are reachable or false otherwise.</returns>
private bool CheckSpecifiedFeeds(HashSet<string> feeds)
/// <returns>
/// True if all feeds are reachable or false otherwise.
/// Also returns the list of reachable feeds.
/// </returns>
private (bool, List<string>) CheckSpecifiedFeeds(HashSet<string> feeds)
{
logger.LogInfo("Checking that NuGet feeds are reachable...");
// Exclude any feeds that are configured by the corresponding environment variable.
var excludedFeeds = GetExcludedFeeds();
Copy link
Copy Markdown
Contributor

@michaelnebel michaelnebel Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that the original intention with this list returned here is that they should be included in the feeds be used no matter whether the feed reachability check fails - this serves as an override mechanism.


var excludedFeeds = EnvironmentVariables.GetURLs(EnvironmentVariableNames.ExcludedNugetFeedsFromResponsivenessCheck)
.ToHashSet();
var feedsToCheck = feeds.Where(feed => !excludedFeeds.Contains(feed)).ToHashSet();
var reachableFeeds = this.GetReachableNuGetFeeds(feedsToCheck, isFallback: false);
var allFeedsReachable = reachableFeeds.Count == feedsToCheck.Count;

if (excludedFeeds.Count > 0)
{
logger.LogInfo($"Excluded NuGet feeds from responsiveness check: {string.Join(", ", excludedFeeds.OrderBy(f => f))}");
}
this.EmitUnreachableFeedsDiagnostics(allFeedsReachable);

var (initialTimeout, tryCount) = GetFeedRequestSettings(isFallback: false);
return (allFeedsReachable, reachableFeeds);
Copy link
Copy Markdown
Contributor

@michaelnebel michaelnebel Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reachableFeeds should also include the feeds that was execluded on l. 790.

}

var allFeedsReachable = feeds.All(feed => excludedFeeds.Contains(feed) || IsFeedReachable(feed, initialTimeout, tryCount));
/// <summary>
/// If <paramref name="allFeedsReachable"/> is `false`, logs this and emits a diagnostic.
/// Adds a `CompilationInfos` entry either way.
/// </summary>
/// <param name="allFeedsReachable">Whether all feeds were reachable or not.</param>
private void EmitUnreachableFeedsDiagnostics(bool allFeedsReachable)
{
if (!allFeedsReachable)
{
logger.LogWarning("Found unreachable NuGet feed in C# analysis with build-mode 'none'. This may cause missing dependencies in the analysis.");
Expand All @@ -779,8 +816,6 @@ private bool CheckSpecifiedFeeds(HashSet<string> feeds)
));
}
compilationInfoContainer.CompilationInfos.Add(("All NuGet feeds reachable", allFeedsReachable ? "1" : "0"));

return allFeedsReachable;
}

private IEnumerable<string> GetFeeds(Func<IList<string>> getNugetFeeds)
Expand Down