Skip to content

Commit 8842350

Browse files
paulmedynskipaulmedynski
committed
Migrate SqlClient packaging to dotnet pack
Replace nuget.exe invocation in build.proj with dotnet pack. Add PrepareSqlClientPackNuspec target in the SqlClient csproj to materialize dependency version tokens into an intermediate nuspec before pack runs, working around SDK token-substitution limitations in dependency version fields (dotnet/sdk#15482). Pack design notes and known SDK behavior documented in src/Microsoft.Data.SqlClient/src/sqlclient-dotnet-pack.md.
1 parent 1129387 commit 8842350

3 files changed

Lines changed: 156 additions & 32 deletions

File tree

‎build.proj‎

Lines changed: 35 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -69,16 +69,6 @@
6969
-->
7070
<DotnetPath Condition="'$(DotnetPath)' == ''" />
7171

72-
<!--
73-
NugetPath
74-
Applies to: PackSqlClient
75-
Description: Path to NuGet executable. Override if `nuget` is not included in the PATH
76-
variable.
77-
Default value: nuget
78-
Example: C:\nuget\nuget.exe
79-
-->
80-
<NugetPath Condition="'$(NugetPath)' == ''">nuget</NugetPath>
81-
8272
<!--
8373
PackBuild
8474
Applies to: Pack*
@@ -97,10 +87,8 @@
9787
--no-build
9888
</PackBuildArgument>
9989
<!--
100-
@TODO: This is kinda hacky, ideally we can pack via dotnet pack, which will handle building,
101-
and can thus be switched off by passing -no-build. However, we're not there because we need
102-
to build multiple projects before packaging, and thus need to use nuget. So, until then we
103-
will switch off dependency building dynamically building the target dependencies
90+
SqlClient packaging uses dotnet pack and keeps this dependency gate so signed-binary workflows
91+
can still set PackBuild=false and package pre-built artifacts.
10492
-->
10593
<PackSqlClientDependsOn Condition="'$(PackBuild.ToLower())' == 'true'">BuildSqlClient</PackSqlClientDependsOn>
10694

@@ -394,7 +382,6 @@
394382
<SqlClientUnitTestProjectPath>$(SqlClientSrcRoot)tests/UnitTests/Microsoft.Data.SqlClient.UnitTests.csproj</SqlClientUnitTestProjectPath>
395383

396384
<!-- Tools/Specs Paths -->
397-
<SqlClientNuspecPath>$(RepoRoot)tools/specs/Microsoft.Data.SqlClient.nuspec</SqlClientNuspecPath>
398385
<GenApiPath>$(RepoRoot)tools/GenAPI/Microsoft.DotNet.GenAPI/</GenApiPath>
399386
<GenApiProjectPath>$(GenApiPath)Microsoft.DotNet.GenAPI.csproj</GenApiProjectPath>
400387
</PropertyGroup>
@@ -531,10 +518,9 @@
531518

532519
<!-- Pack SqlClient Target ===================================================== -->
533520
<!--
534-
PackSqlClient: Packages SqlClient binaries into a NuGet package.
535-
Because the SqlClient package depends on the "not supported" and ref projects, we can't use dotnet to
536-
automatically build before packaging. Instead, we rely on target dependency to handle building
537-
the same binaries that will be packaged.
521+
PackSqlClient: Packages SqlClient binaries with dotnet pack.
522+
The package layout is defined by a nuspec file because we package the artifacts from multiple
523+
projects into a single .nupkg file.
538524
-->
539525
<Target Name="PackSqlClient" DependsOnTargets="$(PackSqlClientDependsOn)">
540526
<PropertyGroup>
@@ -549,10 +535,10 @@
549535
<Output TaskParameter="ConsoleOutput" PropertyName="CommitId" />
550536
</Exec>
551537

552-
<!-- This is a bit hacky, and can be (@TODO:) removed once packaging is done w/o a nuspec from dotnet -->
538+
<!-- Evaluate package version used to stamp the nuspec-driven pack output. -->
553539
<PropertyGroup>
554540
<GetSqlClientPackageVersionCommand>
555-
"$(DotnetPath)dotnet" msbuild "$(SqlClientProjectPath)"
541+
"$(DotnetPath)dotnet" build "$(SqlClientProjectPath)"
556542
-nologo
557543
-verbosity:quiet
558544
-getProperty:SqlClientPackageVersion
@@ -571,23 +557,40 @@
571557
</Exec>
572558
<Error Text="Failed to evaluate SqlClientPackageVersion for PackSqlClient."
573559
Condition="'$(_EvaluatedSqlClientPackageVersion)' == ''" />
560+
<Error Text="PackSqlClient requires PackageVersionAbstractions. Specify -p:PackageVersionAbstractions=&lt;version&gt;."
561+
Condition="'$(PackageVersionAbstractions)' == ''" />
562+
<Error Text="PackSqlClient requires PackageVersionLogging. Specify -p:PackageVersionLogging=&lt;version&gt;."
563+
Condition="'$(PackageVersionLogging)' == ''" />
574564

575565
<PropertyGroup>
576566
<_EvaluatedSqlClientPackageVersion>$([System.Text.RegularExpressions.Regex]::Replace($(_EvaluatedSqlClientPackageVersion), "\s", ""))</_EvaluatedSqlClientPackageVersion>
577567
<CommitId>$([System.Text.RegularExpressions.Regex]::Replace($(CommitId), "\s", ""))</CommitId>
578-
<NuGetCommand>
579-
"$(NugetPath)" pack "$(SqlClientNuspecPath)"
580-
-Symbols
581-
-SymbolPackageFormat snupkg
582-
-Version "$(_EvaluatedSqlClientPackageVersion)"
583-
-OutputDirectory "$(SqlClientArtifactRoot)/$(ReferenceType)-$(Configuration)"
584-
-properties "COMMITID=$(CommitId);Configuration=$(Configuration);ReferenceType=$(ReferenceType);AbstractionsPackageVersion=$(PackageVersionAbstractions);LoggingPackageVersion=$(PackageVersionLogging)"
585-
</NuGetCommand>
568+
<DotnetCommand>
569+
"$(DotnetPath)dotnet" pack "$(SqlClientProjectPath)"
570+
-p:Configuration=$(Configuration)
571+
$(PackBuildArgument)
572+
$(SigningKeyPathArgument)
573+
574+
<!-- Versioning arguments -->
575+
$(BuildNumberArgument)
576+
$(BuildSuffixArgument)
577+
$(PackageVersionSqlClientArgument)
578+
579+
<!-- Reference Type Arguments -->
580+
$(ReferenceTypeArgument)
581+
$(PackageVersionAbstractionsArgument)
582+
$(PackageVersionLoggingArgument)
583+
584+
<!-- Pack settings are defined in Microsoft.Data.SqlClient.csproj -->
585+
-p:CommitId="$(CommitId)"
586+
-p:NuspecVersion="$(_EvaluatedSqlClientPackageVersion)"
587+
-p:PackageOutputPath="$(SqlClientArtifactRoot)/$(ReferenceType)-$(Configuration)"
588+
</DotnetCommand>
586589
<!-- Convert more than one whitespace character into one space -->
587-
<NuGetCommand>$([System.Text.RegularExpressions.Regex]::Replace($(NuGetCommand), "\s+", " "))</NuGetCommand>
590+
<DotnetCommand>$([System.Text.RegularExpressions.Regex]::Replace($(DotnetCommand), "\s+", " "))</DotnetCommand>
588591
</PropertyGroup>
589-
<Message Text=">>> Packing SqlClient nuget via command: $(NuGetCommand)" />
590-
<Exec ConsoleToMsBuild="true" Command="$(NuGetCommand)" />
592+
<Message Text=">>> Packing SqlClient via command: $(DotnetCommand)" />
593+
<Exec ConsoleToMsBuild="true" Command="$(DotnetCommand)" />
591594
</Target>
592595

593596
<!-- Test SqlClient Targets ==================================================== -->

‎src/Microsoft.Data.SqlClient/src/Microsoft.Data.SqlClient.csproj‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,41 @@
7676
<OutputPath>$(ArtifactPath)$(AssemblyName)/$(ReferenceType)-$(Configuration)/$(NormalizedTargetOs)/</OutputPath>
7777
</PropertyGroup>
7878

79+
<!-- NuGet Package Config ============================================ -->
80+
<!-- Pack design notes: ./sqlclient-dotnet-pack.md -->
81+
<PropertyGroup>
82+
<SqlClientPackNuspecTemplatePath Condition="'$(SqlClientPackNuspecTemplatePath)' == ''">$(RepoRoot)tools/specs/Microsoft.Data.SqlClient.nuspec</SqlClientPackNuspecTemplatePath>
83+
<SqlClientPackNuspecGeneratedPath Condition="'$(SqlClientPackNuspecGeneratedPath)' == ''">$(BaseIntermediateOutputPath)Microsoft.Data.SqlClient.pack.nuspec</SqlClientPackNuspecGeneratedPath>
84+
85+
<NuspecFile Condition="'$(NuspecFile)' == ''">$(RepoRoot)tools/specs/Microsoft.Data.SqlClient.nuspec</NuspecFile>
86+
<NuspecBasePath Condition="'$(NuspecBasePath)' == ''">$([System.IO.Path]::GetDirectoryName('$(SqlClientPackNuspecTemplatePath)'))</NuspecBasePath>
87+
<IncludeSymbols Condition="'$(IncludeSymbols)' == ''">true</IncludeSymbols>
88+
<SymbolPackageFormat Condition="'$(SymbolPackageFormat)' == ''">snupkg</SymbolPackageFormat>
89+
<SuppressDependenciesWhenPacking Condition="'$(SuppressDependenciesWhenPacking)' == ''">true</SuppressDependenciesWhenPacking>
90+
91+
<NuspecProperties Condition="'$(NuspecProperties)' == ''">COMMITID=$(CommitId);Configuration=$(Configuration);ReferenceType=$(ReferenceType)</NuspecProperties>
92+
</PropertyGroup>
93+
94+
<!--
95+
SDK pack does not reliably substitute dependency version tokens in this nuspec flow.
96+
We materialize those dependency versions into an intermediate nuspec before PackTask runs.
97+
-->
98+
<Target Name="PrepareSqlClientPackNuspec"
99+
BeforeTargets="GenerateNuspec"
100+
Condition="'$(NuspecFile)' == '$(SqlClientPackNuspecTemplatePath)'">
101+
<PropertyGroup>
102+
<_SqlClientPackNuspecExpandedText>$([System.IO.File]::ReadAllText('$(SqlClientPackNuspecTemplatePath)').Replace('$AbstractionsPackageVersion$','$(AbstractionsPackageVersion)').Replace('$LoggingPackageVersion$','$(LoggingPackageVersion)').Replace('<version>1.0.0</version>','<version>$(NuspecVersion)</version>'))</_SqlClientPackNuspecExpandedText>
103+
</PropertyGroup>
104+
105+
<WriteLinesToFile File="$(SqlClientPackNuspecGeneratedPath)"
106+
Lines="$(_SqlClientPackNuspecExpandedText)"
107+
Overwrite="true" />
108+
109+
<PropertyGroup>
110+
<NuspecFile>$(SqlClientPackNuspecGeneratedPath)</NuspecFile>
111+
</PropertyGroup>
112+
</Target>
113+
79114
<!-- Embedded resources ============================================== -->
80115
<ItemGroup>
81116
<!-- Linker directives to replace UseManagedNetworking with a constant if consumer specifies -->

‎src/Microsoft.Data.SqlClient/src/sqlclient-dotnet-pack.md‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
# SqlClient Dotnet Pack Flow
2+
3+
This document describes how the SqlClient package is produced with dotnet pack in this branch.
4+
5+
## Scope
6+
7+
- Package: `Microsoft.Data.SqlClient`
8+
- Entry target: `PackSqlClient` in `build.proj`
9+
- Pack engine: `dotnet pack` (no `nuget.exe`)
10+
11+
## How to run
12+
13+
Run from repo root:
14+
15+
```bash
16+
dotnet build build.proj -t:PackSqlClient -p:Configuration=Release -p:ReferenceType=Project -p:PackBuild=false -p:PackageVersionAbstractions=<version> -p:PackageVersionLogging=<version>
17+
```
18+
19+
Required pack inputs:
20+
21+
- `PackageVersionAbstractions`
22+
- `PackageVersionLogging`
23+
24+
These are required because dependency versions are injected into the SqlClient nuspec at pack time.
25+
26+
## Why a generated nuspec is used
27+
28+
SqlClient packaging still relies on `tools/specs/Microsoft.Data.SqlClient.nuspec` for file mapping and dependency groups (`lib/*`, `ref/*`, `runtimes/*`, resources, metadata).
29+
30+
In this flow, SDK pack token substitution is reliable for general nuspec properties, but dependency version token replacement in the nuspec dependency section can fail. To avoid that, the project generates an intermediate nuspec before `GenerateNuspec`:
31+
32+
- Template nuspec: `tools/specs/Microsoft.Data.SqlClient.nuspec`
33+
- Generated nuspec: `obj/Microsoft.Data.SqlClient.pack.nuspec`
34+
- Replacements applied:
35+
- `$AbstractionsPackageVersion$` -> `$(AbstractionsPackageVersion)`
36+
- `$LoggingPackageVersion$` -> `$(LoggingPackageVersion)`
37+
- `<version>1.0.0</version>` -> `<version>$(NuspecVersion)</version>`
38+
39+
This keeps layout parity with the existing nuspec while using `dotnet pack` end-to-end.
40+
41+
## Known SDK behavior and repro
42+
43+
Passing all tokens through `NuspecProperties` looks correct, but this command fails on SDK `10.0.107`:
44+
45+
```bash
46+
dotnet pack src/Microsoft.Data.SqlClient/src/Microsoft.Data.SqlClient.csproj --no-build -p:Configuration=Debug -p:ReferenceType=Project -p:NuspecFile=<repo>/tools/specs/Microsoft.Data.SqlClient.nuspec -p:NuspecBasePath=<repo>/tools/specs -p:NuspecProperties="COMMITID=abc;Configuration=Debug;ReferenceType=Project;AbstractionsPackageVersion=1.0.0-dev;LoggingPackageVersion=1.0.0-dev" -p:NuspecVersion=7.1.0-preview1-dev -p:PackageOutputPath=<repo>/artifacts/tmp -p:IncludeSymbols=true -p:SymbolPackageFormat=snupkg
47+
```
48+
49+
Observed error:
50+
51+
- `An error occured while trying to parse the value '' of property 'dependencies' in the manifest file.`
52+
- `'' is not a valid version string.`
53+
54+
The same flow succeeds when dependency tokens are pre-materialized into the intermediate nuspec by `PrepareSqlClientPackNuspec`.
55+
56+
## Related upstream issues
57+
58+
The behavior appears related to long-standing SDK/NuGet pack substitution issues when using nuspec files with `dotnet pack`:
59+
60+
- [dotnet/sdk#15482](https://github.com/dotnet/sdk/issues/15482) (open): multiple `NuspecProperties` values, only first substituted and others become empty.
61+
- [dotnet/sdk#29661](https://github.com/dotnet/sdk/issues/29661) (closed as duplicate): same symptom on SDK 6.0.404.
62+
- [dotnet/sdk#10516](https://github.com/dotnet/sdk/issues/10516) (closed): `dotnet pack` with nuspec not filling tokens like `$version$`.
63+
- [dotnet/sdk#15407](https://github.com/dotnet/sdk/issues/15407) (closed): `$configuration$` empty when packing with nuspec.
64+
- [dotnet/sdk#16816](https://github.com/dotnet/sdk/issues/16816) (closed): nuspec token/path behavior inconsistencies with `dotnet pack`.
65+
66+
These links do not prove this exact dependency-token timing path is identical, but they strongly indicate related substitution behavior in the same pack surface area.
67+
68+
## Where pack properties are defined
69+
70+
SqlClient-specific pack defaults are set in:
71+
72+
- `src/Microsoft.Data.SqlClient/src/Microsoft.Data.SqlClient.csproj`
73+
74+
The `build.proj` target passes dynamic values only:
75+
76+
- `CommitId`
77+
- `NuspecVersion`
78+
- `PackageOutputPath`
79+
- plus standard version/reference-type arguments
80+
81+
## Outputs
82+
83+
Expected artifacts:
84+
85+
- `artifacts/Microsoft.Data.SqlClient/<ReferenceType>-<Configuration>/Microsoft.Data.SqlClient.<version>.nupkg`
86+
- `artifacts/Microsoft.Data.SqlClient/<ReferenceType>-<Configuration>/Microsoft.Data.SqlClient.<version>.snupkg`

0 commit comments

Comments
 (0)