[🚀 Feature]: Provide a Single Source of truth for .NET dependency versions

[titusfortner]

Rather than doing this piecemeal by PR (see #17563), this issue serves to explain the root problem, with recommended solution and discussion of alternatives.

Description

Note that this plan is worth doing on its own, but this approach aligns with Simon's rules_dotnet proposal to replace Paket entirely with NuGet. If that is implemented we just delete the pieces from this plan that would no longer apply, we wouldn't need to re-architect anything.

Problem

.NET package versions live in four places in this repo, everything is hard coded to explicit versions and updates must be made by hand (not currently possible to automate updates with Bazel or Renovate).

File	Role
dotnet/paket.dependencies	What Bazel uses for build/test
dotnet/src/*/.csproj <PackageReference Version=...>	IDE-only, ignored by bazel
third_party/dotnet/devtools/src/generator/DevToolsGenerator.csproj <PackageReference Version=...>	IDE-only, ignored by bazel
dotnet/src/webdriver/Selenium.WebDriver.nuspec <dependency version=...>	Floor versions required for users in nupkg

So our build process ignores the csproj, and doesn't have any contract to validate that what is in nuspec or csproj is what we're actually testing or building.

Renovate doesn't work with Paket, only csproj files, but since bazel ignores them we can't use Renovate to update things.

These versions have already drifted between sources. Three packages have already diverged between paket.dependencies and the devtools generator csproj: CommandLineParser (2.8.0 vs 2.9.1), Humanizer.Core (2.8.26 vs 2.14.1), Microsoft.Extensions.DependencyInjection (3.1.9 vs 3.1.32).

Also because how .NET dependency version management works, we want to test with the oldest supported user-facing versions and the latest dev-only versions

What this proposal would do

• dotnet/Directory.Packages.props Create this file in NuGet's Central Package Management format (mainstream since 2021 with Renovate support) as the single source of truth for .NET versions and derive other sources from it.
• csproj files remove versions and have the IDE get required versions from CPM natively (no additional support required from Selenium)
• The published nupkg's <dependencies> block is rendered from CPM at build time by the existing nuspec template substitution mechanism in nuget_pack (which already substitutes $packageid$ and $version$, so this is just an additional parsing with existing approach).
• paket.dependencies gets automatically generated based on CPM values. This is the heaviest lift, but we'll keep the file version controlled for visibility.
• paket.lock and paket.nuget.bzl will pin transitive versions based on what is in paket.dependencies.
• renovate.json will differentiate user-facing versions to only update the floor version for security issues, and allow the dev-dependencies to update patch or minor versions.

Potential concerns

• There are CPM features that wouldn't be recognized by Bazel
  • Nothing in csproj is currently recognized by Bazel so this is no different from status quo.
  • We can add additional checks if this is considered a blocker.
• Added obfuscation from build process
  • We can add comments to each relevant file pointing at the source-of-truth explaining usage and restrictions
  • paket.dependencies stays version-controlled and human-readable (not buried in a generator's stdout).
  • The primary conversion code is a small, single-purpose script that extends the existing paket2bazel chain rather than introducing a parallel mechanism.
• Need to make sure we have the right behavior for transitive dependencies, and we may need to add some to CPM to pin them if needed.

Alternatives

• Make paket.dependencies the single source of truth (see [dotnet] [build] Use paket as package manager in sdk projects #17577)
  • Requires adding additional custom reference files and imports to each csproj file versus using the CPM standard
  • Would need to be completely reverted if we convert rules_dotnet to use Nuget instead of Paket
  • Updates must still be done by hand
  • Remains disconnected from nuspec
• Update the nuspec template dynamically the same way we update paket.dependencies.
  • Pro: we keep visibility
  • Con: it's a different approach from what we currently do

Does this apply to specific language bindings?

C#/.NET

What part(s) of Selenium does this relate to?

Build (CI or Bazel)