-
Notifications
You must be signed in to change notification settings - Fork 272
Add design for packed project references #14875
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: dev
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,137 @@ | ||
| # Pack opted-in project reference outputs | ||
|
|
||
| - Author Name: [Saibamen](https://github.com/Saibamen) | ||
| - GitHub Issue: [#3891](https://github.com/NuGet/Home/issues/3891) | ||
| - Implementation PR: [NuGet.Client#7325](https://github.com/NuGet/NuGet.Client/pull/7325) | ||
|
|
||
| ## Summary | ||
|
|
||
| Add an explicit opt-in for SDK-style pack to include selected `ProjectReference` build outputs in the parent `.nupkg`, instead of always representing project references as package dependencies. The feature is intended for projects that deliberately ship several implementation assemblies as one NuGet package, while preserving the existing default behavior for all project references. | ||
|
|
||
| ## Motivation | ||
|
|
||
| `dotnet pack` and `msbuild /t:pack` currently treat project references as package dependencies. This is the right default for projects that are independently packaged and versioned, but it is painful for packages that intentionally expose one package while their implementation is split across multiple projects. | ||
|
|
||
| Common scenarios from [NuGet/Home#3891](https://github.com/NuGet/Home/issues/3891) include: | ||
|
|
||
| - A public client package that has one or more private implementation projects. The implementation assemblies are required at runtime, but the project owners do not want to publish, version, and support each implementation project as an independent package. | ||
| - A package that contains a tightly versioned set of assemblies, such as a facade/client assembly plus contracts or shared implementation that must always ship in the same package version. | ||
| - Migration from `nuget.exe pack -IncludeReferencedProjects`, nuspec-based packaging, or custom `TargetsForTfmSpecificBuildOutput` workarounds to SDK-style pack. | ||
| - Packages that need referenced project outputs in non-default package folders, such as analyzer or build-task packages, where the package author must choose the final package layout explicitly. | ||
|
|
||
| There are also important anti-scenarios. This feature should not make it easy to accidentally redistribute arbitrary assemblies, hide dependencies that should be modeled as packages, or bundle the same implementation assembly into several packages that can be installed together. The default behavior therefore remains unchanged, and package authors must opt in per reference. | ||
|
|
||
| ## Explanation | ||
|
|
||
| ### Functional explanation | ||
|
|
||
| By default, a `ProjectReference` continues to be represented as a package dependency when packing an SDK-style project: | ||
|
|
||
| ```xml | ||
| <ProjectReference Include="..\Implementation\Implementation.csproj" /> | ||
| ``` | ||
|
|
||
| To bundle the referenced project's output assemblies into the parent package, the package author sets `Pack="true"` on the project reference: | ||
|
|
||
| ```xml | ||
| <ProjectReference Include="..\Implementation\Implementation.csproj" Pack="true" /> | ||
| ``` | ||
|
|
||
| For this opted-in reference: | ||
|
|
||
| - The referenced project's build outputs are added to the parent package. | ||
| - The referenced project is not emitted as a nuspec dependency. | ||
| - Package dependencies needed by the bundled project are still emitted as package dependencies of the parent package, so consumers can restore the external packages needed at runtime. | ||
| - Symbols are included consistently with existing `IncludeSymbols` behavior. | ||
| - The default package location follows the existing pack convention for build output, for example `lib/<tfm>/Implementation.dll`. | ||
|
|
||
| If a bundled project references another project, the default behavior should be to include the project-reference closure under the opted-in project. This is required for runtime correctness in common "private implementation assembly" scenarios. The package author opts in at the first redistributed edge; NuGet then follows the restored project graph from that edge so the package contains the implementation assemblies needed by that bundled project. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Edge case: the child project has a ProjectReference to a project that the parent project also has a ProjectReference to, but the parent doesn't set I guess possible options are:
I don't know which is best. |
||
|
|
||
| The package author may override the destination with `PackagePath`: | ||
|
|
||
| ```xml | ||
| <ProjectReference | ||
| Include="..\Analyzer\Analyzer.csproj" | ||
| Pack="true" | ||
| PackagePath="analyzers/dotnet/cs" /> | ||
| ``` | ||
|
|
||
| When `PackagePath` is specified, NuGet places the referenced project's outputs under that path instead of under `BuildOutputTargetFolder` and the target framework folder. The package author is responsible for choosing a path that is meaningful for the package type. For normal library assemblies, omitting `PackagePath` is preferred so outputs remain under the target framework's `lib` folder. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm wondering if this could be problematic for multi-targeting projects. Say I have project Contoso with a ProjectReference to Contoso.Analyzers, which I want to put in If both Contoso and Contoso.Analyzers multi-target, we can't have two files with the same name with different contents. If NuGet's instructed to put both copies in the same location, it needs to fail with a duplicate file error. I did a search, but wasn't able to quickly find what the NU error code is for this. Only if Contoso.Analyzers is single targeting (probably netstandard2.0), then it can make sense to allow it. It becomes an implementation detail to de-duplicate, as long as the source path is the same for duplicate target paths. I don't believe NuGet does this at the moment. edit: this appears to be called out already in unresolved questions. |
||
|
|
||
| For multi-targeting projects, pack applies the behavior per target framework. A project reference that only applies to a subset of target frameworks is bundled only for those frameworks. The feature is also controlled by existing pack switches: if `IncludeBuildOutput=false`, bundled project build outputs are not added to the package. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this should be rephrased, and possibly an example given for clarity. I think I know what you're trying to say, but I don't know if I'm just guessing based on my own knowledge. For example, something like:
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should there be a warning when On one hand, I don't like it when mutually exclusive features are configured at the same time, and there's no feedback about this probable configuration mistake. If I'm new to the technology, I don't yet know enough to debug why it's silently failing. On the other hand, the scenario should be simple enough to debug, since the project's own assembly won't be making it into the package. Also trying to consider every permutation of incompatible features and emit warnings could be noisy, slow down the product from an explosion of validation checks, make docs more confusing with an overabundance of warning messages. |
||
|
|
||
| ### Technical explanation | ||
|
|
||
| The implementation should keep restore as the source of truth for the project graph. Pack should not rediscover an independent project graph or infer project reference metadata from only the entry project. | ||
|
|
||
| At restore time, NuGet should preserve pack-specific project reference metadata in the restore graph and `project.assets.json`: | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thinking out loud: restore is NuGet's primary use case. Far more people restore packages and never pack, than people that pack. Let alone people who will want to pack and use this spec's feature. Adding more content to the assets file means that reading the file has more work to do, and therefore will be slower, even in scenarios (restore) where the additional metadata isn't used. As long as the assets file is effectively unmodified in projects that don't use this spec's feature, then it should be fine, no measurable perf degradation. But within Microsoft we have some repos using 1000+ projects per restore, so even a few additional memory allocations can add up to additional garbage collections and slower restores. If we can't avoid perf regressions for restore by adding this feature, while using the assets file, then I think it'll be better to make pack slower by needing to run additional msbuild targets, or whatever is needed, to collect the additional project information it needs. |
||
|
|
||
| - `Pack` | ||
| - `PackagePath` | ||
|
Comment on lines
+67
to
+70
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please provide a json snippet of the part of the assets file that will change, and what it looks like. I highly recommend using a diff code block, so github can highlight the changes, making it easier for people reading the spec. |
||
|
|
||
| This metadata must flow through the existing restore entry points: | ||
|
|
||
| - MSBuild restore | ||
| - Static graph restore | ||
| - Visual Studio nomination | ||
| - `project.assets.json` read/write round-tripping | ||
|
|
||
| At pack time, NuGet reads the assets file to identify the project references that opted in to bundling. It then collects build outputs through the existing pack output group targets, using the target framework from the restore graph and avoiding a rebuild of project references during output collection. | ||
|
|
||
| Dependency generation changes only for opted-in project references. The bundled project itself is suppressed as a nuspec dependency. Package dependencies reachable through the bundled project are promoted into the parent package dependency group so package consumers still restore required external packages. Project dependencies reachable through the bundled project are traversed so their package dependencies are promoted too. | ||
|
|
||
| The initial implementation should be conservative about the packaging surface: | ||
|
|
||
| - Include build outputs that existing pack output groups already understand. | ||
| - Include debug symbols through the existing debug symbol output groups when symbols are requested. | ||
| - Do not include arbitrary content, source, native assets, or runtime-specific assets from referenced projects unless a follow-up design explicitly defines those behaviors. | ||
| - Do not change nuspec pack. | ||
| - Do not change `nuget.exe pack -IncludeReferencedProjects`. | ||
| - Do not change the default handling of project references. | ||
|
|
||
| The current implementation shape uses two assets-file reads during pack: one to collect all project references for the existing package-version lookup, and one to collect the subset of project references whose outputs should be packed. The packed project-reference items carry `TargetFramework`, `BuildProjectReferences=false`, and, when specified, the requested package path. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. are you referring to the pull request you already opened, or are you referring to how pack is already implemented in the .NET SDK that ships today? In either case, it sounds like an implementation detail not really relevant for a feature spec. |
||
|
|
||
| ## Drawbacks | ||
|
|
||
| Bundling referenced assemblies can create packages that are harder for consumers to reason about. Assemblies inside a package are not truly private in .NET's normal load context; consumers can still reference them, observe type identities, or encounter binding/version conflicts. | ||
|
|
||
| The feature can encourage package authors to avoid creating proper packages for independently useful components. If two packages bundle different versions of the same implementation assembly, a consuming application can see duplicate assembly conflicts that NuGet cannot resolve as package version conflicts. | ||
|
|
||
| The feature also makes pack more complex because restore metadata, assets-file serialization, dependency generation, and output collection all need to agree on the same project-reference metadata. | ||
|
|
||
| Promoting dependencies from bundled projects can also change the apparent dependency graph of the parent package. This is desirable when it preserves runtime dependencies, but it may surprise authors if a private project has package references they did not expect to expose to package consumers. | ||
|
|
||
| ## Rationale and alternatives | ||
|
|
||
| The design uses per-reference opt-in because the current default is safer and well established. Automatically bundling all non-packable projects, or all projects without `PackageId`, would be more convenient for some repositories but could redistribute assemblies without a clear author decision. | ||
|
|
||
| `Pack="true"` is reused because pack already uses `Pack` metadata on other MSBuild items to mean "include this item in the package." Reusing the existing term is more discoverable than adding a new metadata name such as `IncludeOutputDll`, and it leaves room for package layout to be controlled by `PackagePath`. | ||
|
|
||
| `PackagePath` is reused because it already controls where packed items land in the package. A new metadata name such as `PackageRoot` would be narrower, but it would create a separate concept for project references without clear benefit. For project-reference outputs, `PackagePath` is intentionally treated as a single destination for the referenced project's output files rather than as the multi-destination content-file syntax. | ||
|
|
||
| An alternative is to require users to keep using `TargetsForTfmSpecificBuildOutput` and custom MSBuild targets. That avoids a product change, but it leaves users to manually copy assemblies, handle symbols, and replicate package dependencies from referenced projects. The long-running issue shows that these workarounds are common and easy to get wrong. | ||
|
|
||
| Another alternative is to revive a single switch equivalent to `IncludeReferencedProjects`. That is simpler to explain, but it is too broad for SDK-style pack because it does not force authors to identify which references are intended to be redistributed. | ||
|
|
||
| Another alternative is to require `Pack="true"` on every project-reference edge that should be bundled. That gives the author more control over redistribution, but it is easier to produce broken packages because the first bundled project may depend on another project whose assembly is required at runtime. Following the restored project-reference closure from an explicitly opted-in edge better matches the runtime shape of the bundled implementation. | ||
|
|
||
| ## Prior Art | ||
|
|
||
| - `nuget.exe pack -IncludeReferencedProjects` supports a broad version of this behavior for legacy pack workflows. | ||
| - SDK-style pack already supports adding additional files through `TargetsForTfmSpecificBuildOutput` and `TfmSpecificPackageFile`, but users must hand-author the graph and dependencies. | ||
| - NuGetizer has prior art for distinguishing project references that should become dependencies from projects whose outputs should be bundled. | ||
| - The original MSBuild pack design states that project-to-project references are treated as NuGet package references by default. | ||
|
|
||
| ## Unresolved Questions | ||
|
|
||
| - Should NuGet warn when an opted-in project reference points to a project that is itself packable or has a `PackageId`, since that may indicate the project should remain a package dependency? | ||
| - Should transitive bundled project outputs inherit the root opted-in reference's `PackagePath`, or should only the directly opted-in project's outputs use that override? Inheriting the path is simple, but may be wrong for mixed asset types. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There's also the question of diamond dependencies. If I'm packing ProjectA, which have references to ProjectB and ProjectC, both of which depend on ProjectD, if I use a different PackagePath for ProjectB and ProjectC, where does ProjectD's output go? |
||
| - Should package dependencies promoted from bundled projects preserve all include/exclude/private asset metadata, or should pack define a narrower dependency projection for bundled projects? The current implementation promotes package dependencies with include/exclude information from the lock file and does not preserve a private-assets suppress-parent value. | ||
| - Should pack emit a warning when two bundled project outputs resolve to the same package path, beyond the existing duplicate file warning? | ||
| - Should this feature require any new package authoring guidance about licensing and redistribution of referenced project outputs? | ||
|
|
||
| ## Future Possibilities | ||
|
|
||
| Future designs could extend this to content files, runtime-specific assets, native assets, analyzers, or source files from bundled project references. Those should be designed separately because each asset class has different package layout and consumer behavior. | ||
|
|
||
| NuGet could also add warnings or analyzers that detect likely anti-patterns, such as the same assembly being bundled into multiple packages in a repository, or a project being both bundled and published as a package dependency. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that describing it as painful is unhelpful opinionated commentary.
In the list of common scenarios below, the first two are personal preferences. So, I don't believe it's fair to say it's painful. Both scenarios would work just fine from a technical perspective with multiple nupkgs.
The third talks about needing a migration, but doesn't explain why it's difficult or painful. If the projects are simple classlibs, the only migration cost of
nuget.exe pack -IncludeReferencedProjectstomsbuild -t:packis migrating from packages.config to PackageReference. But if the developer is willing to accept their single package becoming multiple packages, there is no additional migration cost. On the other hand, if they want to onboard onto the feature this spec is proposing, then there is additional migration cost.The example in the list of common scenarios is a good scenario, and if we can make this easier for package authors, it will be a great win. I don't have experience creating packages with roslyn analyzers, so I don't know how painful it is, but if it's achievable with a
<None Include="../../My.Analyzer/bin/$(Configuration)/netstandard2.0/My.Analyzer.dll" Pack="true" PackagePath="analyzers/dotnet/cs" />, then that doesn't feel very painful to me.From a package consumer point of view, there's pretty much no difference between a package that has multiple assemblies, or a package with one assembly and one or more dependencies. NuGet will download all the packages and hook them into the build either way.