Using Source Generator, instead of GenAPI to produce NotSupported library

[tetolv]

Description

This is an attempt to replace GenAPI-based generation of the NotSupported version of the MDS library with Source Generator based one.

Sorry for breaking the rule of not surprising dev team with big unannounced PRs, but at the beginning I wasn't even expected, that something will come out of it. I just saw a comment

    alternatively:
    @TODO: Replace the GenAPI project with a C# Source Generator

in \src\Microsoft.Data.SqlClient\notsupported\Microsoft.Data.SqlClient.csproj and wondered if source generators would really simplify that matter.

Current solution

Current solution is temporary, as it combines both GenAPI-based and SourceGen-based approaches for NotSupported library generation, so that source produced by both tools could be compared to validate, that they are compatible, and nothing is added, lost, or modified, if we switch from GenAPI to Source Generators.

At the heart of SourceGen-based solution is tools/GenAPI/Microsoft.Data.SqlClient.SourceGenerator/Microsoft.Data.SqlClient.SourceGenerator.csproj project which contains RefToNotSupportedGenerator class, that performs source code rewriting by substituting bodies of all public, or internal methods, constructors, or properties with PlatformNotSupportedException throwing. This source generator is referenced only in src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj via:

  <PropertyGroup>
    ...
    <EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>
    <CompilerGeneratedFilesOutputPath>../notsupported</CompilerGeneratedFilesOutputPath>
  </PropertyGroup>

    <ProjectReference Include="../../../tools/GenAPI/Microsoft.Data.SqlClient.SourceGenerator/Microsoft.Data.SqlClient.SourceGenerator.csproj">
      <ReferenceOutputAssembly>false</ReferenceOutputAssembly>
      <OutputItemType>Analyzer</OutputItemType>
    </ProjectReference>

build.proj is extended with a new target BuildSqlClientNotSupportedSG, which builds Not Supported library using Source Generators. It does it as following:

• Target BuildSqlClientNotSupportedSG depends on BuildSqlClientRef, which compiles ref library and also performs source rewriting, adding PlatformNotSupportedException throwing to all methods/constructors/property accessors.
• Rewritten source is stored to the \src\Microsoft.Data.SqlClient\notsupported (see, CompilerGeneratedFilesOutputPath parameter above). This is the same project, which uses GenAPI to build unsupported library. As a result with will contain both sources generated by GenAPI and by Source Generator. To prevent duplicate type errors compiling it, a new property NotSupportedSG is introduced, which controls, which sources are taken for build:

  <PropertyGroup>
    <NotSupportedSourceFile Condition="'$(NotSupportedSG)' != 'true'">$(AssemblyName).$(TargetFramework).notsupported.cs</NotSupportedSourceFile>
    <NotSupportedSourceFile Condition="'$(NotSupportedSG)' == 'true'">Microsoft.Data.SqlClient.SourceGenerator/Microsoft.Data.SqlClient.SourceGenerator.RefToNotSupportedGenerator/$(AssemblyName).$(TargetFramework).notsupported.g.cs</NotSupportedSourceFile>
  </PropertyGroup>

• Target BuildSqlClientNotSupportedSG builds notsupported project passing -p:NotSupportedSG=true, which takes source generated sources for compilation.

In addition I have added separated target to the \tools\targets\CompareMdsRefAssemblies.targets, which builds side-by-side both GenAPI-based and SourceGen-based version of notsupported library and compares them using dotnet apicompat to verify API compatibility.

Current results

If we are looking at the differences between .cs files generated by GenAPI and Source Generator, then there appears to be many, but most of them are purely syntactical and does not represent differences in actual APIs:

Area	GenAPI	Source Generator
Class declaration	Has partial	No partial
Class declaration	Implemented interface list often includes inherited interfaces	Interface list is the same as in ref project
Constructors	Some throw, others are empty	All throw PlatformNotSupportedException
Class with no default constructor	Generates throwing constructor	No default constructor
Referenced types	Sometimes are generated with long name, which includes namespace	Usually without namespace (are in ref project)
Class members	Are ordered according to C# guidelines	Are ordered the same as in ref project
Dispose() method	Is always generated with empty body	Is throwing like the rest
default keyword	contains type, like default(System.Threading.CancellationToken)	just default
default keyword	In some constructors like public SqlRowUpdatedEventArgs(System.Data.DataRow row, System.Data.IDbCommand command, System.Data.StatementType statementType, System.Data.Common.DataTableMapping tableMapping) : base (default(System.Data.DataRow), default(System.Data.IDbCommand), default(System.Data.StatementType), default(System.Data.Common.DataTableMapping)) { throw ... }, when there were no default in ref source	default is not used
Enums	Last item always has comma	As in ref library
Attributes	Attribute name usually contains Attribute suffix	Attribute name is the same as in ref library, ussualy without Attribute suffix

Here I attach both sources produced by GenAPI and Source Generator, so that you could compare them in a diff tool:
GenAPI-produced.zip
SourceGen-produced.zip

When binary comparison is performed using dotnet apicompat, following result is returned (results for all targets net8.0, net9.0, net462, netstandard2.0 are the same):

API compatibility errors between 'D:/Projects/SqlClient/artifacts/Microsoft.Data.SqlClient.notsupported/Project-Debug/net9.0/Microsoft.Data.SqlClient.dll' (left) and 'D:/Projects/SqlClient/artifacts/Microsoft.Data.SqlClient.notsupported/Project-Debug-SG/net9.0/Microsoft.Data.SqlClient.dll' (right):
CP0021: Cannot add constraint '!:' on type parameter 'T' of 'Microsoft.Data.SqlClient.SqlDataReader.GetSqlVector<T>(int)'.
CP0021: Cannot remove constraint '!:System.ValueType' on type parameter 'T' of 'Microsoft.Data.SqlClient.SqlDataReader.GetSqlVector<T>(int)'.
CP0021: Cannot add constraint '!:' on type parameter 'T' of 'Microsoft.Data.SqlTypes.SqlVector<T>'.
CP0021: Cannot remove constraint '!:System.ValueType' on type parameter 'T' of 'Microsoft.Data.SqlTypes.SqlVector<T>'.
API breaking changes found. If those are intentional, the APICompat suppression file can be updated by specifying the '--generate-suppression-file' parameter.

, which correspond to the following method (difference is highlighted):

In ref project	GenAPI	Source Generator
public virtual Microsoft.Data.SqlTypes.SqlVector GetSqlVector(int i) where T : unmanaged { throw null; }	public virtual Microsoft.Data.SqlTypes.SqlVector GetSqlVector(int i) where T : struct { throw new System.PlatformNotSupportedException("Microsoft.Data.SqlClient is not supported on this platform."); }	public virtual Microsoft.Data.SqlTypes.SqlVector GetSqlVector(int i) where T : unmanaged { throw new System.PlatformNotSupportedException("Microsoft.Data.SqlClient is not supported on this platform.");}

In general I think, that Source Generators solution produces compatible result in more straightforward and modern way. Also some of the complexity of the current solution is attributed to coexistence with GenAPI, as all the #if GENAPI_COMPAT ... #endif blocks in [RefToNotsupportedTypeRewriter](https://github.com/dotnet/SqlClient/compare/main...tetolv:SqlClient:not-supported-lib-sg?expand=1#diff-f3fda4d678f3a1e56e9521baf31cfc37412f3a2fab035d1e63261344d95a91a7), only exists to produce code more similar to what GenAPI produces, to make diff reading a little easier. Eventually those hacks could be removed.

Other notes

• Current source generator only handles language features currently used in ref project. In case more advanced language features will be used, source generator should be tuned up to properly handle them as well.
• This branch is in a proof-of-concept stage, so some things are implemented in a simplified way. For instance msbuild target for comparing GenAPI and SourceGen generated assemblies is just added to \tools\targets\CompareMdsRefAssemblies.targets, because it already conveniently had other targets I was needed, although it is meant for different specific purpose. Microsoft.Data.SqlClient.SourceGenerator project itself is added to the \tools\GenAPI\ directory, although it might not be the best place for it.
• I'm wondering if ref project itself could be source generated from the base source... Are there only signatures with empty bodies of the base types, or there is more to it? Automatically generating ref project could noticeable reduce manual work.

Will be happy to hear any feedback.

[tetolv]

@dotnet-policy-service agree

[cheenamalhotra]

Hi @tetolv Thanks for the contribution, but we'll be removing the NotSupported libraries in future iterations, as they are no longer needed - so we won't be making more changes to it's build system.