Configure GeneratePackageVersions to support a "cooldown" on dependencies (#8371)

## Summary of changes

> default cooldown period is 2 days

Adds a configurable cooldown period to the `GeneratePackageVersions` to
support remediation efforts for follow up after #incident-51602.

To use supply the optional parameter `--PackageVersionCooldownDays X`
where `X` is some number of days. The current period is at the moment is
going to be 2 days and is the default now when the overall
`GeneratePackageVersion` target is ran. Additionally, this is overridden
to 0 if `--IncludePackages` is supplied (this is commonly used when
working on a singular package locally).

After running the tool a "cooldown" report is generated, this file will
contain packages that we see have a newer version, but will not
incorporate into the test, the fallback version that falls within the
cooldown period is provided. This file content will show up in the
output of future test package version bump PRs.

Note that this is for _automated_ updates, so if it sees something
already updated it will honor it.

Here's an example output ran locally with 14 days set:

```
## Package Version Cooldown Report

The following versions were published less than **14 days** ago and have been overridden.
These require manual review before inclusion.

| Package | Integration | Overridden Version | Published | Age (days) | Using Instead |
|---------|-------------|--------------------|-----------|------------|---------------|
| AWSSDK.Core | AwsSdk | 4.0.3.22 | 2026-03-25 | 0 | 4.0.3.21 |
| AWSSDK.S3 | AwsS3 | 4.0.19.2 | 2026-03-25 | 0 | 4.0.19.1 |
| StackExchange.Redis | StackExchangeRedis | 2.12.8 | 2026-03-25 | 0 | 2.12.4 |
```

## Reason for change

In #8364 and
#incident-51602 all automated dependency updaters to be disabled
temporarily, to re-enable we need to supply a 2 day "cooldown" to any
version that we update to (in other words the version of the NuGet must
be published for at least 2 days before we can update to it).

## Implementation details

I made Claude do this 🤖 

- NuGetPackageHelper now captures the Published date from
IPackageSearchMetadata via a new VersionWithDate record (previously
discarded)
  - NuGetVersionCache stores the new {Version, Published} format
- PackageVersionGenerator.ApplyCooldown filters selected versions after
LatestMajors/LatestMinors/LatestSpecific selection:
    - Versions outside the cooldown window pass through unchanged
- Versions at or below the baseline (derived from
supported_versions.json MaxVersionTestedInclusive) are kept even if
  within cooldown -- no downgrades
- Versions above the baseline and within cooldown are overridden to the
best available fallback
- CooldownReport collects overridden versions and renders a markdown
table saved to tracer/build/cooldown_report.md
- The GitHub Actions workflow reads the report and appends it to the
auto-bump PR body
- Honeypot IntegrationGroups.cs fixes: MSTest.TestFramework now maps to
itself, Hangfire.Core maps to Hangfire.Core (was Hangfire), OpenFeature
mapping moved to Datadog.FeatureFlags.OpenFeature

Passing `--IncludePackages` will override the cooldown to 0

## Test coverage

I ran `GeneratePackageVersions --PackageVersionCooldownDays 14` locally
seems good enough IMO (also ran without, with different days etc)

```
[WRN] GeneratePackageVersi: 3 package version(s) were excluded due to the 14-day cooldown period
[WRN] GeneratePackageVersi:   AWSSDK.Core 4.0.3.22 overridden (published 2026-03-25, using: 4.0.3.21)
[WRN] GeneratePackageVersi:   AWSSDK.S3 4.0.19.2 overridden (published 2026-03-25, using: 4.0.19.1)
[WRN] GeneratePackageVersi:   StackExchange.Redis 2.12.8 overridden (published 2026-03-25, using: 2.12.4)
```

## Other details
<!-- Fixes #{issue} -->

The workflow file (`auto_bump_test_package_versions.yml`) will be
re-enabled with this PR

<!--  ⚠️ Note:

Where possible, please obtain 2 approvals prior to merging. Unless
CODEOWNERS specifies otherwise, for external teams it is typically best
to have one review from a team member, and one review from apm-dotnet.
Trivial changes do not require 2 reviews.

MergeQueue is NOT enabled in this repository. If you have write access
to the repo, the PR has 1-2 approvals (see above), and all of the
required checks have passed, you can use the Squash and Merge button to
merge the PR. If you don't have write access, or you need help, reach
out in the #apm-dotnet channel in Slack.
-->

Author: bouwkast

…_bump_test_package_versions.yml.disabled …lows/auto_bump_test_package_versions.yml.github/workflows/auto_bump_test_package_versions.yml.disabled renamed to .github/workflows/auto_bump_test_package_versions.yml .github/workflows/auto_bump_test_package_versions.yml.disabled renamed to .github/workflows/auto_bump_test_package_versions.yml

@@ -38,7 +38,21 @@ jobs:
           global-json-file: global.json
 
       - name: "Regenerating package versions"
-        run: .\tracer\build.ps1 GeneratePackageVersions
+        run: .\tracer\build.ps1 GeneratePackageVersions --PackageVersionCooldownDays 2
+
+      - name: Read cooldown report
+        id: cooldown
+        if: always()
+        shell: pwsh
+        run: |
+          $report = ""
+          $reportPath = ".nuke/temp/cooldown_report.md"
+          if (Test-Path $reportPath) {
+            $report = Get-Content $reportPath -Raw
+          }
+          "report<<EOF" >> $env:GITHUB_OUTPUT
+          $report >> $env:GITHUB_OUTPUT
+          "EOF" >> $env:GITHUB_OUTPUT
 
       - name: Create Pull Request
         id: pr
@@ -55,6 +69,8 @@ jobs:
           body: |
             Updates the package versions for integration tests.
 
+            ${{ steps.cooldown.outputs.report }}
+
       - name: Send Slack notification about generating failure
         if: failure()
         uses: slackapi/slack-github-action@91efab103c0de0a537f72a35f6b8cda0ee76bf0a # v2.1.1

tracer/build/PackageVersionsLatestMinors.g.props

@@ -10507,7 +10507,7 @@ NOTE:  This code was generated by the GeneratePackageVersions tool. To safely
       <SampleName>Samples.GoogleProtobuf</SampleName>
     </PackageVersionSample>
     <PackageVersionSample Include="test\test-applications\integrations\Samples.GoogleProtobuf\Samples.GoogleProtobuf.csproj">
-      <Properties>ApiVersion=3.25.8;RestoreRecursive=false;BuildProjectReferences=false</Properties>
+      <Properties>ApiVersion=3.25.9;RestoreRecursive=false;BuildProjectReferences=false</Properties>
       <TargetFramework>net48</TargetFramework>
       <RequiresDockerDependency>None</RequiresDockerDependency>
       <SampleName>Samples.GoogleProtobuf</SampleName>
@@ -10717,7 +10717,7 @@ NOTE:  This code was generated by the GeneratePackageVersions tool. To safely
       <SampleName>Samples.GoogleProtobuf</SampleName>
     </PackageVersionSample>
     <PackageVersionSample Include="test\test-applications\integrations\Samples.GoogleProtobuf\Samples.GoogleProtobuf.csproj">
-      <Properties>ApiVersion=3.25.8;RestoreRecursive=false;BuildProjectReferences=false</Properties>
+      <Properties>ApiVersion=3.25.9;RestoreRecursive=false;BuildProjectReferences=false</Properties>
       <TargetFramework>netcoreapp2.1</TargetFramework>
       <RequiresDockerDependency>None</RequiresDockerDependency>
       <SampleName>Samples.GoogleProtobuf</SampleName>
@@ -10927,7 +10927,7 @@ NOTE:  This code was generated by the GeneratePackageVersions tool. To safely
       <SampleName>Samples.GoogleProtobuf</SampleName>
     </PackageVersionSample>
     <PackageVersionSample Include="test\test-applications\integrations\Samples.GoogleProtobuf\Samples.GoogleProtobuf.csproj">
-      <Properties>ApiVersion=3.25.8;RestoreRecursive=false;BuildProjectReferences=false</Properties>
+      <Properties>ApiVersion=3.25.9;RestoreRecursive=false;BuildProjectReferences=false</Properties>
       <TargetFramework>netcoreapp3.0</TargetFramework>
       <RequiresDockerDependency>None</RequiresDockerDependency>
       <SampleName>Samples.GoogleProtobuf</SampleName>
@@ -11137,7 +11137,7 @@ NOTE:  This code was generated by the GeneratePackageVersions tool. To safely
       <SampleName>Samples.GoogleProtobuf</SampleName>
     </PackageVersionSample>
     <PackageVersionSample Include="test\test-applications\integrations\Samples.GoogleProtobuf\Samples.GoogleProtobuf.csproj">
-      <Properties>ApiVersion=3.25.8;RestoreRecursive=false;BuildProjectReferences=false</Properties>
+      <Properties>ApiVersion=3.25.9;RestoreRecursive=false;BuildProjectReferences=false</Properties>
       <TargetFramework>netcoreapp3.1</TargetFramework>
       <RequiresDockerDependency>None</RequiresDockerDependency>
       <SampleName>Samples.GoogleProtobuf</SampleName>
@@ -11347,7 +11347,7 @@ NOTE:  This code was generated by the GeneratePackageVersions tool. To safely
       <SampleName>Samples.GoogleProtobuf</SampleName>
     </PackageVersionSample>
     <PackageVersionSample Include="test\test-applications\integrations\Samples.GoogleProtobuf\Samples.GoogleProtobuf.csproj">
-      <Properties>ApiVersion=3.25.8;RestoreRecursive=false;BuildProjectReferences=false</Properties>
+      <Properties>ApiVersion=3.25.9;RestoreRecursive=false;BuildProjectReferences=false</Properties>
       <TargetFramework>net5.0</TargetFramework>
       <RequiresDockerDependency>None</RequiresDockerDependency>
       <SampleName>Samples.GoogleProtobuf</SampleName>
@@ -11557,7 +11557,7 @@ NOTE:  This code was generated by the GeneratePackageVersions tool. To safely
       <SampleName>Samples.GoogleProtobuf</SampleName>
     </PackageVersionSample>
     <PackageVersionSample Include="test\test-applications\integrations\Samples.GoogleProtobuf\Samples.GoogleProtobuf.csproj">
-      <Properties>ApiVersion=3.25.8;RestoreRecursive=false;BuildProjectReferences=false</Properties>
+      <Properties>ApiVersion=3.25.9;RestoreRecursive=false;BuildProjectReferences=false</Properties>
       <TargetFramework>net6.0</TargetFramework>
       <RequiresDockerDependency>None</RequiresDockerDependency>
       <SampleName>Samples.GoogleProtobuf</SampleName>
@@ -11767,7 +11767,7 @@ NOTE:  This code was generated by the GeneratePackageVersions tool. To safely
       <SampleName>Samples.GoogleProtobuf</SampleName>
     </PackageVersionSample>
     <PackageVersionSample Include="test\test-applications\integrations\Samples.GoogleProtobuf\Samples.GoogleProtobuf.csproj">
-      <Properties>ApiVersion=3.25.8;RestoreRecursive=false;BuildProjectReferences=false</Properties>
+      <Properties>ApiVersion=3.25.9;RestoreRecursive=false;BuildProjectReferences=false</Properties>
       <TargetFramework>net7.0</TargetFramework>
       <RequiresDockerDependency>None</RequiresDockerDependency>
       <SampleName>Samples.GoogleProtobuf</SampleName>
@@ -11977,7 +11977,7 @@ NOTE:  This code was generated by the GeneratePackageVersions tool. To safely
       <SampleName>Samples.GoogleProtobuf</SampleName>
     </PackageVersionSample>
     <PackageVersionSample Include="test\test-applications\integrations\Samples.GoogleProtobuf\Samples.GoogleProtobuf.csproj">
-      <Properties>ApiVersion=3.25.8;RestoreRecursive=false;BuildProjectReferences=false</Properties>
+      <Properties>ApiVersion=3.25.9;RestoreRecursive=false;BuildProjectReferences=false</Properties>
       <TargetFramework>net8.0</TargetFramework>
       <RequiresDockerDependency>None</RequiresDockerDependency>
       <SampleName>Samples.GoogleProtobuf</SampleName>
@@ -12187,7 +12187,7 @@ NOTE:  This code was generated by the GeneratePackageVersions tool. To safely
       <SampleName>Samples.GoogleProtobuf</SampleName>
     </PackageVersionSample>
     <PackageVersionSample Include="test\test-applications\integrations\Samples.GoogleProtobuf\Samples.GoogleProtobuf.csproj">
-      <Properties>ApiVersion=3.25.8;RestoreRecursive=false;BuildProjectReferences=false</Properties>
+      <Properties>ApiVersion=3.25.9;RestoreRecursive=false;BuildProjectReferences=false</Properties>
       <TargetFramework>net9.0</TargetFramework>
       <RequiresDockerDependency>None</RequiresDockerDependency>
       <SampleName>Samples.GoogleProtobuf</SampleName>
@@ -12397,7 +12397,7 @@ NOTE:  This code was generated by the GeneratePackageVersions tool. To safely
       <SampleName>Samples.GoogleProtobuf</SampleName>
     </PackageVersionSample>
     <PackageVersionSample Include="test\test-applications\integrations\Samples.GoogleProtobuf\Samples.GoogleProtobuf.csproj">
-      <Properties>ApiVersion=3.25.8;RestoreRecursive=false;BuildProjectReferences=false</Properties>
+      <Properties>ApiVersion=3.25.9;RestoreRecursive=false;BuildProjectReferences=false</Properties>
       <TargetFramework>net10.0</TargetFramework>
       <RequiresDockerDependency>None</RequiresDockerDependency>
       <SampleName>Samples.GoogleProtobuf</SampleName>

tracer/build/PackageVersionsLatestSpecific.g.props

@@ -7051,13 +7051,13 @@ NOTE:  This code was generated by the GeneratePackageVersions tool. To safely
       <SampleName>Samples.CosmosDb.Vnext</SampleName>
     </PackageVersionSample>
     <PackageVersionSample Include="test\test-applications\integrations\LogsInjection.Serilog\LogsInjection.Serilog.csproj">
-      <Properties>ApiVersion=1.5.14;RestoreRecursive=false;BuildProjectReferences=false</Properties>
+      <Properties>ApiVersion=1.4.214;RestoreRecursive=false;BuildProjectReferences=false</Properties>
       <TargetFramework>net48</TargetFramework>
       <RequiresDockerDependency>None</RequiresDockerDependency>
       <SampleName>LogsInjection.Serilog</SampleName>
     </PackageVersionSample>
     <PackageVersionSample Include="test\test-applications\integrations\LogsInjection.Serilog\LogsInjection.Serilog.csproj">
-      <Properties>ApiVersion=1.4.214;RestoreRecursive=false;BuildProjectReferences=false</Properties>
+      <Properties>ApiVersion=1.5.14;RestoreRecursive=false;BuildProjectReferences=false</Properties>
       <TargetFramework>net48</TargetFramework>
       <RequiresDockerDependency>None</RequiresDockerDependency>
       <SampleName>LogsInjection.Serilog</SampleName>

tracer/build/_build/Build.Utilities.cs

@@ -59,6 +59,9 @@ partial class Build
     [Parameter("Only update package versions for packages with the following names")]
     readonly string[] IncludePackages;
 
+    [Parameter("Minimum age in days a NuGet package version must have been published before auto-including. Defaults to 2 days, or 0 when --IncludePackages is set")]
+    readonly int? PackageVersionCooldownDays;
+
     [LazyLocalExecutable(@"C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.8 Tools\gacutil.exe")]
     readonly Lazy<Tool> GacUtil;
     [LazyLocalExecutable(@"C:\Program Files\IIS Express\iisexpress.exe")]
@@ -247,11 +250,95 @@ partial class Build
            var previousSupportedVersions = await GenerateSupportMatrix.LoadPreviousVersions(supportedVersionsPath);
            Logger.Information("Loaded previous supported versions with {Count} entries", previousSupportedVersions.Count);
 
+           // Derive baseline from supported_versions.json: the max tested version per package
+           // acts as a floor to prevent cooldown filtering from downgrading previously accepted versions.
+           // We collect all max tested versions per package (not just the global max) so that
+           // split-range packages (e.g., GraphQL 4.x-6.x and 7.x-9.x) get a per-range baseline.
+           var baseline = previousSupportedVersions
+               .Where(kvp => kvp.Value.MaxVersionTestedInclusive is not null)
+               .GroupBy(kvp => kvp.Key.PackageName)
+               .ToDictionary(
+                   g => g.Key,
+                   g => g.Select(kvp => new Version(kvp.Value.MaxVersionTestedInclusive!)).ToList());
+           Logger.Information("Derived version baseline with {Count} entries from supported_versions.json", baseline.Count);
+
+           // Resolve effective cooldown:
+           //  - Explicit --PackageVersionCooldownDays wins
+           //  - --IncludePackages without explicit cooldown defaults to 0
+           var effectiveCooldownDays = PackageVersionCooldownDays ?? (IncludePackages is not null ? 0 : 2);
+
            // Pipeline A: generate .g.props/.g.cs files
-           var versionGenerator = new PackageVersionGenerator(TracerDirectory, testDir, shouldUpdatePackage, previousVersionCache);
+           Logger.Information("Using package version cooldown of {Days} days", effectiveCooldownDays);
+           var versionGenerator = new PackageVersionGenerator(TracerDirectory, testDir, shouldUpdatePackage, previousVersionCache, effectiveCooldownDays, baseline);
            var testedVersions = await versionGenerator.GenerateVersions(Solution);
            await NuGetVersionCache.Save(cacheFilePath, versionGenerator.VersionCache);
 
+           // Log version changes: bumps, unchanged, and overridden
+           var versionCache = versionGenerator.VersionCache;
+           var bumped = 0;
+           var unchanged = 0;
+           foreach (var tested in testedVersions)
+           {
+               var packageName = tested.NugetPackageSearchName;
+               baseline.TryGetValue(packageName, out var previousMaxVersions);
+               var previousMax = previousMaxVersions?
+                   .Where(v => v >= tested.MinVersion && v <= tested.MaxVersion)
+                   .OrderByDescending(v => v)
+                   .FirstOrDefault();
+
+               if (previousMax is null || tested.MaxVersion > previousMax)
+               {
+                   bumped++;
+                   var publishedDate = "(unknown)";
+                   if (versionCache.TryGetValue(packageName, out var cachedVersions))
+                   {
+                       var match = cachedVersions.FirstOrDefault(v => v.Version == tested.MaxVersion.ToString());
+                       if (match?.Published is not null)
+                       {
+                           publishedDate = match.Published.Value.ToString("yyyy-MM-dd");
+                       }
+                   }
+
+                   Logger.Information(
+                       "  {Package} {Previous} -> {Current} (published {Date}, https://www.nuget.org/packages/{Package}/{Current})",
+                       packageName,
+                       previousMax?.ToString() ?? "(new)",
+                       tested.MaxVersion,
+                       publishedDate,
+                       packageName,
+                       tested.MaxVersion);
+               }
+               else
+               {
+                   unchanged++;
+               }
+           }
+
+           Logger.Information("{Bumped} package(s) bumped, {Unchanged} unchanged", bumped, unchanged);
+
+           if (versionGenerator.CooldownReport.HasEntries)
+           {
+               Logger.Warning(
+                   "{Count} package version(s) were excluded due to the {Days}-day cooldown period",
+                   versionGenerator.CooldownReport.Entries.Count,
+                   effectiveCooldownDays);
+
+               foreach (var entry in versionGenerator.CooldownReport.Entries)
+               {
+                   var resolvedText = entry.ResolvedVersion is not null ? $"using: {entry.ResolvedVersion}" : "skipped";
+                   Logger.Warning(
+                       "  {Package} {Version} overridden (published {Date}, {Resolved})",
+                       entry.PackageName,
+                       entry.OverriddenVersion,
+                       entry.PublishedDate?.ToString("yyyy-MM-dd") ?? "unknown",
+                       resolvedText);
+               }
+
+               var reportPath = TemporaryDirectory / "cooldown_report.md";
+               await versionGenerator.CooldownReport.SaveToFile(reportPath);
+               Logger.Information("Cooldown report saved to {Path}", reportPath);
+           }
+
            var assemblies = MonitoringHomeDirectory
                            .GlobFiles("**/Datadog.Trace.dll")
                            .Select(x => x.ToString())
@@ -260,6 +347,8 @@ partial class Build
            var integrations = GenerateIntegrationDefinitions.GetAllIntegrations(assemblies, definitionsFile);
 
            // Pipeline B: generate dependabot files + supported_versions.json
+           // TestedVersions are cooldown-filtered but the baseline prevents downgrades,
+           // so they accurately reflect what we're testing.
            var distinctIntegrations = await DependabotFileManager.BuildDistinctIntegrationMaps(
                integrations, testedVersions, shouldUpdatePackage, previousSupportedVersions);
 

tracer/build/_build/GeneratePackageVersions/CooldownReport.cs

@@ -0,0 +1,89 @@
+// <copyright file="CooldownReport.cs" company="Datadog">
+// Unless explicitly stated otherwise all files in this repository are licensed under the Apache 2 License.
+// This product includes software developed at Datadog (https://www.datadoghq.com/). Copyright 2017 Datadog, Inc.
+// </copyright>
+
+using System;
+using System.Collections.Generic;
+using System.IO;
+using System.Text;
+using System.Threading.Tasks;
+
+namespace GeneratePackageVersions;
+
+/// <summary>
+/// Collects package versions that were excluded by the cooldown filter
+/// and renders them as a markdown report for inclusion in the PR body.
+/// </summary>
+public class CooldownReport
+{
+    private readonly int _cooldownDays;
+    private readonly List<CooldownEntry> _entries = new();
+    private readonly HashSet<(string PackageName, string Version)> _seen = new();
+
+    public CooldownReport(int cooldownDays)
+    {
+        _cooldownDays = cooldownDays;
+    }
+
+    public bool HasEntries => _entries.Count > 0;
+
+    public IReadOnlyList<CooldownEntry> Entries => _entries;
+
+    public void Add(CooldownEntry entry)
+    {
+        // Deduplicate: the same version gets flagged once per framework and per selection group,
+        // but we only need to report it once.
+        if (_seen.Add((entry.PackageName, entry.OverriddenVersion)))
+        {
+            _entries.Add(entry);
+        }
+    }
+
+    public string ToMarkdown()
+    {
+        if (_entries.Count == 0)
+        {
+            return string.Empty;
+        }
+
+        var sb = new StringBuilder();
+        sb.AppendLine($"## Package Version Cooldown Report");
+        sb.AppendLine();
+        sb.AppendLine($"The following versions were published less than **{_cooldownDays} days** ago and have been overridden.");
+        sb.AppendLine("These require manual review before inclusion.");
+        sb.AppendLine();
+        sb.AppendLine("| Package | Integration | Overridden Version | Published | Age (days) | Using Instead |");
+        sb.AppendLine("|---------|-------------|--------------------|-----------|------------|---------------|");
+
+        foreach (var entry in _entries)
+        {
+            var published = entry.PublishedDate?.ToString("yyyy-MM-dd") ?? "unknown";
+            var age = entry.PublishedDate.HasValue
+                ? ((int)(DateTimeOffset.UtcNow - entry.PublishedDate.Value).TotalDays).ToString()
+                : "?";
+            var usingInstead = entry.ResolvedVersion ?? "(skipped)";
+
+            sb.AppendLine($"| {entry.PackageName} | {entry.IntegrationName} | {entry.OverriddenVersion} | {published} | {age} | {usingInstead} |");
+        }
+
+        return sb.ToString();
+    }
+
+    public async Task SaveToFile(string path)
+    {
+        var markdown = ToMarkdown();
+        if (!string.IsNullOrEmpty(markdown))
+        {
+            // necessary to save to a file so that we can then output it in the PR description
+            await File.WriteAllTextAsync(path, markdown);
+        }
+    }
+
+    public record CooldownEntry(
+        string PackageName,
+        string IntegrationName,
+        string OverriddenVersion,
+        DateTimeOffset? PublishedDate,
+        string ResolvedVersion);
+}