OpenAPI document generation fails for JsonNumberHandling.WriteAsString and a ValidationAttribute

[CaringDev]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

A vanilla WebAPI app fails to generate the OpenAPI document if JsonNumberHandling.WriteAsString is set and validation attributes are present.

Yes, I'm aware that the behavior of JsonNumberHandling.WriteAsString is not defined by the spec. However, I'd like safely to roundtrip longs (so I'm also setting JsonNumberHandling.AllowReadingFromString) and my client-side stack can handle the schema and data.

Expected Behavior

OpenAPI document is generated:

// ...
"summary": {
  "maxLength": 100,
  "minLength": 0,
  "type": [
    "null",
    "string"
  ]
}
// ...

Steps To Reproduce

dotnet new webapi -n Repro
cd Repro
dotnet add package Microsoft.Extensions.ApiDescription.Server --version 10.0.5
dotnet build

• ./obj/Repro.json is generated correctly
• change the following in Program.cs

builder.Services.ConfigureHttpJsonOptions(o =>
    o.SerializerOptions.NumberHandling = JsonNumberHandling.WriteAsString);

// ...

record WeatherForecast(DateOnly Date, int TemperatureC, [property: StringLength(100)] string? Summary)

// ...

Exceptions (if any)

dotnet build fails with an exception, also the OpenAPI document at /openapi/v1.json cannot be generated for the same reason.

System.AggregateException: One or more errors occurred. (The JSON value could not be converted to OpenApiJsonSchema. Path: $ | LineNumber: 0 | BytePositionInLine: 74.)
 ---> System.Text.Json.JsonException: The JSON value could not be converted to OpenApiJsonSchema. Path: $ | LineNumber: 0 | BytePositionInLine: 74.
 ---> System.InvalidOperationException: Cannot get the value of a token type 'String' as a number.
   at System.Text.Json.ThrowHelper.ThrowInvalidOperationException_ExpectedNumber(JsonTokenType tokenType)
   at System.Text.Json.Utf8JsonReader.TryGetInt32(Int32& value)
   at System.Text.Json.Utf8JsonReader.GetInt32()
   at OpenApiJsonSchema.ReadProperty(Utf8JsonReader& reader, String propertyName, OpenApiSchema schema, JsonSerializerOptions options)
   at OpenApiJsonSchema.JsonConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, T& value, JsonSerializerOptions options, ReadStack& state)
   --- End of inner exception stack trace ---
   at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& state, Utf8JsonReader& reader, Exception ex)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, T& value, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsObject(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadAsObject(Utf8JsonReader& reader, JsonTypeInfo jsonTypeInfo)
   at OpenApiJsonSchema.ReadDictionary[T](Utf8JsonReader& reader)
   at OpenApiJsonSchema.ReadProperty(Utf8JsonReader& reader, String propertyName, OpenApiSchema schema, JsonSerializerOptions options)
   at OpenApiJsonSchema.JsonConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)
   at OpenApiJsonSchema.ReadProperty(Utf8JsonReader& reader, String propertyName, OpenApiSchema schema, JsonSerializerOptions options)
   at OpenApiJsonSchema.JsonConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, T& value, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo`1 jsonTypeInfo, Nullable`1 actualByteCount)
   at System.Text.Json.JsonSerializer.ReadFromNode[TValue](JsonNode node, JsonTypeInfo`1 jsonTypeInfo)
   at Microsoft.AspNetCore.OpenApi.OpenApiSchemaService.GetOrCreateUnresolvedSchemaAsync(OpenApiDocument document, Type type, IServiceProvider scopedServiceProvider, IOpenApiSchemaTransformer[] schemaTransformers, ApiParameterDescription parameterDescription, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.OpenApi.OpenApiSchemaService.GetOrCreateSchemaAsync(OpenApiDocument document, Type type, IServiceProvider scopedServiceProvider, IOpenApiSchemaTransformer[] schemaTransformers, ApiParameterDescription parameterDescription, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.OpenApi.OpenApiDocumentService.GetResponseAsync(OpenApiDocument document, ApiDescription apiDescription, Int32 statusCode, ApiResponseType apiResponseType, IServiceProvider scopedServiceProvider, IOpenApiSchemaTransformer[] schemaTransformers, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.OpenApi.OpenApiDocumentService.GetResponsesAsync(OpenApiDocument document, ApiDescription description, IServiceProvider scopedServiceProvider, IOpenApiSchemaTransformer[] schemaTransformers, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.OpenApi.OpenApiDocumentService.GetOperationAsync(ApiDescription description, OpenApiDocument document, IServiceProvider scopedServiceProvider, IOpenApiSchemaTransformer[] schemaTransformers, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.OpenApi.OpenApiDocumentService.GetOperationsAsync(IGrouping`2 descriptions, OpenApiDocument document, IServiceProvider scopedServiceProvider, IOpenApiOperationTransformer[] operationTransformers, IOpenApiSchemaTransformer[] schemaTransformers, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.OpenApi.OpenApiDocumentService.GetOpenApiPathsAsync(OpenApiDocument document, IServiceProvider scopedServiceProvider, IOpenApiOperationTransformer[] operationTransformers, IOpenApiSchemaTransformer[] schemaTransformers, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.OpenApi.OpenApiDocumentService.GetOpenApiDocumentAsync(IServiceProvider scopedServiceProvider, HttpRequest httpRequest, CancellationToken cancellationToken)
   at Microsoft.Extensions.ApiDescriptions.OpenApiDocumentProvider.GenerateAsync(String documentName, TextWriter writer, OpenApiSpecVersion openApiSpecVersion)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait(TimeSpan timeout, CancellationToken cancellationToken)
   at Microsoft.Extensions.ApiDescription.Tool.Commands.GetDocumentCommandWorker.GetDocument(String documentName, String projectName, String outputDirectory, MethodInfo generateMethod, Object service, MethodInfo generateWithVersionMethod, String fileName)
   at Microsoft.Extensions.ApiDescription.Tool.Commands.GetDocumentCommandWorker.GetDocuments(IServiceProvider services)
   at Microsoft.Extensions.ApiDescription.Tool.Commands.GetDocumentCommandWorker.Process()

where each line starts with path\to\microsoft.extensions.apidescription.server\10.0.5\build\Microsoft.Extensions.ApiDescription.Server.targets(67,5): error

.NET Version

10.0.201

Anything else?

CLI, all packages 10.0.5

[BrennanConroy]

The problem is that the custom converter directly calls Get*() methods on Utf8JsonReader, and that has no knowledge of the serializer options.

You can see an example below of how to fix it, by wrapping the Utf8JsonReader in another JsonSerializer.Deserialize() call.

var opt = new JsonSerializerOptions()
{
    NumberHandling = JsonNumberHandling.AllowReadingFromString | JsonNumberHandling.WriteAsString,
};
opt.Converters.Add(new Converter());
var o = JsonSerializer.Deserialize<MyObj>("{\"Number\":\"1234\"}", opt);

Console.WriteLine(o?.Number);

public class MyObj
{
    public MyObj(){}
    
    public int Number {get;set;}
}

public class Converter : JsonConverter<MyObj>
{
    public override MyObj? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        reader.Read(); // begin object '{'
        reader.GetString(); // property name
        reader.Read(); // move to 'Number'
        var i = JsonSerializer.Deserialize<int>(ref reader, options);
        reader.Read(); // end object '}'

        //var i = reader.GetInt32(); // What the converter does today. Doesn't use options so fails to deserialize a string that can be interpreted as a number

        return new MyObj() { Number = i };
    }

    public override void Write(Utf8JsonWriter writer, MyObj value, JsonSerializerOptions options)
    {
        throw new NotImplementedException();
    }
}

Somewhat similar to dotnet/runtime#69718.
Wonder if @eiriktsarpalis has thoughts here. Should all custom converters (that care about supporting user options) be wrapping Utf8JsonReader in JsonSerializer.Deserialize() calls to allow better handling of these cases?

[eiriktsarpalis]

What is OpenApiJsonSchema.JsonConverter? If it's something we're shipping then it should try to honor JsonSerializerOptions.NumberHandling by optionaly checking string token types.

[BrennanConroy]

It's our type

aspnetcore/src/OpenApi/src/Schemas/OpenApiJsonSchema.Helpers.cs

Line 240 in 3d9044c

var minLength = reader.GetInt32();

So you recommend we explicitly check JsonSerializerOptions.NumberHandling and then check the token type for string or number to decide on what reader method to call?

I get that we're using the low level APIs so need to do some manual work, but I'm not a huge fan of that because now we need to be aware of all Json options (both current and new) and what they apply to. For example, GetBoolean could in the future be able to handle 1 and 0, or "1" and "0" based on an option so now we'd need to be aware/update to handle that.

It also feels a bit inconsistent, AllowTrailingCommas is automatically passed to the Utf8JsonReader. (I get that it's about how the structure of Json is handled and not about specific values)

[eiriktsarpalis]

Unless you've rerouted the serialization of int to this custom converter, you can always do JsonSerializer.Deserialize<int>(ref reader, options.GetTypeInfo<int>()) and let the built-in int converter do its thing.

It also feels a bit inconsistent, AllowTrailingCommas is automatically passed to the Utf8JsonReader. (I get that it's about how the structure of Json is handled and not about specific values)

I would say that AllowTrailingCommas is a JSON tokenization concern. Whether a string token is interpreted as an integer is a deserialization concern and therefore should be the purview of a converter. Utf8JsonReader was designed with maximizing performance in mind, as such it doesn't attempt to do any kind of content sniffing and reinterpretation (as is the case with, say, Json.NET).

[BrennanConroy]

you can always do JsonSerializer.Deserialize<int>(ref reader, options.GetTypeInfo<int>()) and let the built-in int converter do its thing.

Right, this is my question, should we in general be preferring that over calling the explicit methods on Utf8JsonReader?

[eiriktsarpalis]

If you're looking to honor all the behaviour of the serializer, then yeah that's the right call.