Skip to content

Build and publish NuGet.Client.EndToEnd.TestData if needed #7459

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
martinrrm wants to merge 1 commit into
base: dev
Choose a base branch
from
+98 −3
Open

Build and publish NuGet.Client.EndToEnd.TestData if needed #7459

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Directory.Packages.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@
<PackageVersion Include="MSTest.TestAdapter" Version="$(MSTestPackageVersion)" />
<PackageVersion Include="MSTest.TestFramework" Version="$(MSTestPackageVersion)" />
<PackageVersion Include="Newtonsoft.Json" Version="$(NewtonsoftJsonPackageVersion)" />
<PackageVersion Include="NuGet.Client.EndToEnd.TestData" Version="1.0.0" />
<PackageVersion Include="NuGet.Client.EndToEnd.TestData" Version="1.0.3" />
<PackageVersion Include="NuGet.Core" Version="2.14.0-rtm-832" />
<PackageVersion Include="NuGetValidator" version="2.1.1" />
<PackageVersion Include="SharpZipLib" Version="1.4.2" />
Expand Down
5 changes: 5 additions & 0 deletions eng/pipelines/official.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ parameters:
values:
- Real
- Test
- name: PublishEndToEndTestData
displayName: Publish EndToEnd TestData Package
type: boolean
default: false

resources:
repositories:
Expand Down Expand Up @@ -56,3 +60,4 @@ extends:
RunBuildForPublishing: ${{parameters.RunBuildForPublishing}}
SigningType: ${{ parameters.SigningType }}
TryRunOneLocBuild: true
PublishEndToEndTestData: ${{ parameters.PublishEndToEndTestData }}
47 changes: 47 additions & 0 deletions eng/pipelines/templates/Build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ parameters:
- name: SigningType
displayName: Type of signing to use
type: string
- name: PublishEndToEndTestData
type: boolean
default: false

steps:
- task: PowerShell@1
Expand Down Expand Up @@ -131,6 +134,50 @@ steps:
configuration: "$(BuildConfiguration)"
msbuildArguments: "/restore:false /target:Pack /property:BuildRTM=$(BuildRTM) /property:ExcludeTestProjects=$(BuildRTM) /property:BuildNumber=$(BuildNumber) /binarylogger:$(Build.StagingDirectory)\\binlog\\11.Pack.binlog /property:MicroBuild_SigningEnabled=false"

- ${{ if and(eq(parameters.PublishEndToEndTestData, true), not(parameters.BuildRTM)) }}:
# The EndToEnd test data package bundles nested .nupkg fixtures as content. It is generated into a
# staging directory that is deliberately OUTSIDE the repo 'artifacts' tree so it is never signed
# (SignTool would recurse into and corrupt the nested fixtures), never picked up by BAR publishing,
# and never scanned by the product SBOM. It is published unsigned to the dnceng nuget-build feed
# via the '- output: nuget' block in pipeline.yml.
Comment on lines +138 to +142

@zivkan zivkan Jun 6, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, the current design is manually queue an official build, setting the PublishEndToEndTestData parameter to true, then wait for the official build to complete, then run the release pipeline.

It seems to me we could have a single pipeline that both builds and publishes the package, so we don't need to queue two pipelines to perform one outcome.

@martinrrm martinrrm Jun 8, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that would be the easiest way to develop and publish the package. But TBH I don't see us publishing a new package in years, last version is from 2024 and we weren't using it. 1.0.0 is from 2020.

I just don't want to create a new pipeline for something that we might never use, I was thinking maybe Official can do the publishing too, but Official feels more like a build only pipeline.

We were doing something similar when we wanted to publish CommandLine packages, so I was following that.

I think we should choose the way that is easier for the team to maintain and do compliance on the pipeline, either a new one, keep it as two, or maybe even do everything as a new step in release pipeline, maybe we can just build this project and publish.

- task: PowerShell@1
displayName: "Prepare EndToEnd TestData output directory"
condition: "succeeded()"
inputs:
scriptType: "inlineScript"
inlineScript: |
$dir = "$(Build.StagingDirectory)\e2etestdata"
if (Test-Path $dir) { Remove-Item -Path $dir -Recurse -Force }
New-Item -ItemType Directory -Path $dir -Force | Out-Null

- task: MSBuild@1
displayName: "Build GenerateTestPackages"
condition: "succeeded()"
inputs:
solution: "test\\TestExtensions\\GenerateTestPackages\\GenerateTestPackages.csproj"
configuration: "$(BuildConfiguration)"
msbuildArguments: "/restore:true /property:BuildNumber=$(BuildNumber) /binarylogger:$(Build.StagingDirectory)\\binlog\\GenerateTestPackages.binlog /property:MicroBuild_SigningEnabled=false"

- task: PowerShell@1
displayName: "Generate EndToEnd TestData Package"
condition: "succeeded()"
inputs:
scriptName: "$(Build.Repository.LocalPath)\\scripts\\e2etests\\CreateTestDataPackage.ps1"
arguments: "-configuration '$(BuildConfiguration)' -outputDirectoryPath '$(Build.StagingDirectory)\\e2etestdata'"

- task: PowerShell@1
displayName: "Verify EndToEnd TestData Package exists"
condition: "succeeded()"
inputs:
scriptType: "inlineScript"
inlineScript: |
$packages = @(Get-ChildItem -Path "$(Build.StagingDirectory)\e2etestdata\NuGet.Client.EndToEnd.TestData.*.nupkg" -ErrorAction SilentlyContinue)
if ($packages.Count -ne 1) {
Write-Host "##vso[task.LogIssue type=error;]Expected exactly one NuGet.Client.EndToEnd.TestData package, but found $($packages.Count)."
exit 1
}
Write-Host "Found EndToEnd TestData package: $($packages[0].Name)"

- task: PowerShell@1
displayName: "Check expected packages exist for publishing"
name: "EnsureAllPackagesExist"
Expand Down
18 changes: 18 additions & 0 deletions eng/pipelines/templates/pipeline.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ parameters:
values:
- Real
- Test
- name: PublishEndToEndTestData
displayName: Publish EndToEnd TestData Package
type: boolean
default: false

stages:
- stage: Initialize
Expand Down Expand Up @@ -183,11 +187,24 @@ stages:
targetPath: "$(Build.StagingDirectory)/sbom"
sbomEnabled: false

# Publishes the generated EndToEnd test data package (and its SBOM) as a pipeline artifact. The
# NuGet.Client-Release pipeline downloads this artifact and pushes it to the public dnceng nuget-build
# feed. The package is generated into $(Build.StagingDirectory)\e2etestdata by Build.yml (outside the
# signed/BAR/product-SBOM artifacts tree).
- ${{ if eq(parameters.PublishEndToEndTestData, true) }}:
- output: pipelineArtifact
displayName: 'Publish EndToEnd TestData package'
condition: "and(succeeded(), eq(variables['BuildRTM'], 'false'), eq(variables['IsOfficialBuild'], 'true'))"

@jebriede jebriede Jun 9, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The runtime condition on the pipelineArtifact output duplicates the BuildRTM/IsOfficialBuild logic that also lives in Build.yml. If these conditions drift, the package could be generated but not published (or vice versa). Consider centralizing the "should we publish E2E test data" predicate so the generate step and the artifact output can't get out of sync.

targetPath: '$(Build.StagingDirectory)\e2etestdata'
artifactName: 'EndToEndTestData'
sbomBuildDropPath: '$(Build.StagingDirectory)\e2etestdata'

steps:
- template: /eng/pipelines/templates/Build.yml@self
parameters:
BuildRTM: false
SigningType: ${{ parameters.SigningType }}
PublishEndToEndTestData: ${{ parameters.PublishEndToEndTestData }}

- stage: Build_For_Publishing
displayName: Build NuGet published to nuget.org
Expand Down Expand Up @@ -250,6 +267,7 @@ stages:
parameters:
BuildRTM: true
SigningType: ${{ parameters.SigningType }}
PublishEndToEndTestData: ${{ parameters.PublishEndToEndTestData }}

- ${{ if eq(parameters.isOfficialBuild, true) }}:
- template: /eng/common/templates-official/post-build/post-build.yml
Expand Down
14 changes: 13 additions & 1 deletion scripts/e2etests/CreateTestDataPackage.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,19 @@ Function Get-File([string[]] $pathParts)

Function Get-GenerateTestPackagesFile()
{
Return Get-File($repositoryRootDirectoryPath, 'artifacts', 'GenerateTestPackages', "$toolsetVersion.0", 'bin', $configuration, 'net472', 'GenerateTestPackages.exe')
$filePathWithToolset = [System.IO.Path]::Combine($repositoryRootDirectoryPath, 'artifacts', 'GenerateTestPackages', "$toolsetVersion.0", 'bin', $configuration, 'net472', 'GenerateTestPackages.exe')
if (Test-Path $filePathWithToolset)
{
Return [System.IO.FileInfo]::new($filePathWithToolset)
}

$filePathWithoutToolset = [System.IO.Path]::Combine($repositoryRootDirectoryPath, 'artifacts', 'GenerateTestPackages', 'bin', $configuration, 'net472', 'GenerateTestPackages.exe')
if (Test-Path $filePathWithoutToolset)
{
Return [System.IO.FileInfo]::new($filePathWithoutToolset)
}

throw [System.IO.FileNotFoundException]::new("Could not find GenerateTestPackages.exe. Please build first. Probed:`n $filePathWithToolset`n $filePathWithoutToolset", $filePathWithoutToolset)
}

Function Get-NuGetFile()
Expand Down
2 changes: 1 addition & 1 deletion test/EndToEnd/NuGet.Client.EndToEnd.TestData.nuspec
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<package xmlns="http://schemas.microsoft.com/packaging/2013/05/nuspec.xsd">
<metadata>
<id>NuGet.Client.EndToEnd.TestData</id>
<version>1.0.0</version>
<version>1.0.3</version>

@jebriede jebriede Jun 9, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying to understand the publish/consume ordering and could use some help. This PR bumps the central NuGet.Client.EndToEnd.TestData pin to 1.0.3, but as I understand it 1.0.3 only lands in the dnceng nuget-build feed after someone manually queues an official build with PublishEndToEndTestData=true and then runs the companion release pipeline. Meanwhile CreateEndToEndTestPackage.proj consumes this package via (central package management) and restores it from nuget-build. So, my question is: how do we avoid a window where this is merged but 1.0.3 isn't on the feed yet. Wouldn't restore of that project fail with NU1102 until the publish happens? Is 1.0.3 already published, or does the pin bump need to wait for that? (I noticed the PR is currently mergeable_state: blocked, is that related?)

<title>Test data for NuGet.Client EndToEnd tests</title>
<authors>Microsoft</authors>
<owners>Microsoft</owners>
Expand Down
13 changes: 13 additions & 0 deletions test/EndToEnd/Packages/InstallPackagesConfigLocal/InstallPackagesConfigLocalGraph.dgml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<?xml version="1.0" encoding="utf-8"?>
<DirectedGraph xmlns="http://schemas.microsoft.com/vs/2009/dgml">
<Nodes>
<Node Id="A:1.0.0" Label="A:1.0.0" />
<Node Id="B:1.0.0" Label="B:1.0.0" />
</Nodes>
<Links>
<Link Source="A:1.0.0" Target="B:1.0.0" />
</Links>
<Properties>
<Property Id="Label" Label="Label" Description="Displayable label of an Annotatable object" DataType="System.String" />
</Properties>
</DirectedGraph>