Support writing only (some?) longs as strings (opt-in) for Javascript compatibility

[rdesgroppes]

Although Jackson exposes a WRITE_NUMBERS_AS_STRINGS feature that forces all regular number values to be written as JSON Strings rather than JSON Numbers, it doesn't currently provide a way to write only 64-bit integers using their canonical representation, which poses a problem for strict downstream parsers given the limitation of the maximum value to 2^53 - 1 (alt) in JavaScript, whereas this value is for instance 2^63 - 1 in Java.

Related specs:

• https://www.rfc-editor.org/rfc/rfc7159#section-6,
• https://262.ecma-international.org/5.1/#sec-8.5.

Related issues:

• Add support to get canonical JSON output (what is still missing as of 3.0?) #1037,
• Add serialization feature: strict numbers (FPs that fit long, Javascript compatibility) jackson-databind#911,
• Adding a configuration in the ProtobufJacksonConfig to serialize long… HubSpot/jackson-datatype-protobuf#77,
• Support writing longs as strings (opt-in) HubSpot/jackson-datatype-protobuf#98.

[pjfanning]

This would need to be an optional feature to not change behaviour for existing users. See StreamWriteFeature.

@rdesgroppes any interest in submitting a PR yourself (targeted at 2.16 branch)?

[pjfanning]

This would need to be an optional feature to not change behaviour for existing users. See StreamWriteFeature.

@rdesgroppes any interest in submitting a PR yourself (targeted at 2.16 branch)?

In the end of the day, users wanting max interoperability should treat all numbers as JSON strings or use double types.

[cowtowncoder]

@pjfanning This would definitely be disabled by default. Whether this should be StreamWriteFeature or JsonWriteFeature is up for debate -- I can see it being useful for some but definitely not all or most formats. Beyond JSON, maybe just YAML.
There may be practical considerations as well, making one choice or another easier to support (in 3.0 JsonWriteFeatures are fully separated from generic StreamWriteFeatures, but in 2.x there's some coupling, for example).

Note, too, that one could consider alternative approach of implementation at jackson-databind level as well. I don't have strong preference; suffice it to say there are various trade-offs wrt this choice as well.

This is something that has been requested before, fwtw, so will mark as highly-requested.

[cowtowncoder]

@rdesgroppes Quick question: just to make sure, would this only affect values that are beyond double range (beyond 2^53 and its negative counterpart) or anything written as long? Jackson does provide separate writeNumber() methods so this is possible to implement either way. Just Stringifying all long/Long/BigInteger values would be simpler but in some ways less consistent; checking for actual range is bit more work but might be more likely what is desired.

I changed title slightly but based on specific request feel free to further improve it.

[pjfanning]

@cowtowncoder is it possible that this feature should affect writing BigInteger and BigDecimal too? Feels like it should. Those number types could represent numbers that make big longs look puny.

[pjfanning]

I'm not sure about hardcoding Javascript magic numbers into Jackson, to be honest. Would we better off just sticking with WRITE_NUMBERS_AS_STRINGS and leaving it at that?

Also, users can write their own custom serializers in Jackson.

[rdesgroppes]

Well, I have to admit I'm a little embarrassed because the need I initially expressed (HubSpot/jackson-datatype-protobuf#98) applied to serializing protocol buffer messages (typically to pass them through a JSON proxy) and, according to the JSON Mapping of the proto3 specification, int64, fixed64 and uint64 are serialized as strings.

[cowtowncoder]

@pjfanning I think I'll disagree a bit here: given that JSON use is so heterogenous (client/server running on different platforms), we have long since given up on only supporting Java-to-Java case. So while I am not a big fan of language-specific support per se, this particular thing seems to be common enough request.

Still, this is just a request and we can and should carefully consider what to do here.

[cowtowncoder]

@cowtowncoder is it possible that this feature should affect writing BigInteger and BigDecimal too? Feels like it should. Those number types could represent numbers that make big longs look puny.

BigInteger for sure. BigDecimal I am less sure about as I suspect it would be difficult to implement proper checks (unlike with integral numbers).

[cowtowncoder]

Thinking about this bit more, I wonder if instead of adding a Feature (on/off) of some kind, it'd make more sense to allow configuring a predicate that determines if given number ought to be stringified? Along with perhaps a standard implementation for this use case.

The idea being there'd be multiple methods called, passing typed number -- first, maybe just one for integrals, so int, long, BigInteger -- and return true if output should use String instead of number.

This would allow supporting wider range of conventions; starting with "all integer numbers" and "JS-range compatible" but allowing others, if any found.

[digulla]

I think a better approach would be to offer a bouquet of number serializers, each dealing with a different quirk (stripping trailing zeros from BigDecimal, quoting numbers that JS can't handle properly).

The API should be simple and just take a number and return either String or Number. This would then allow to write a wrapper which accepts Number and either returns them unmodified or converts them to String.

The code in Jackson should look at the return type and write Number as before and when it's a String, it should delegate to writing a string (which would then add quotes).

That way, users can quickly build a custom module picking and choosing whatever they need without having to litter the code base with if (feature).

[cowtowncoder]

While availability of alternate serializers can be useful, there are some challengers there: for example, serialization of JsonNode values will not use JsonSerializers. So there are various trade-offs regarding wider applicability (when done at low-level by jackson-core) vs. more flexibility (when done at databind).

For what it is worth, most Number deserializers will (by default) accept JSON String, although this permissibility can be changed with CoercionConfig to not allow.

I agree that perhaps another approach would be a new "standard" value type of JavascriptNumber which would enclose a double and handle (de)serialization aspects appropriately. It would not (need) to be a Number, but wrapper type.
If such a type was to be added, it could form a core of something like speculative jackson-datatype-javascript?
I am guessing there would be other useful (wrapper) types that would help with Javascript interoperability.