fix(js/ts): Fix Decimal.GetBits returning incorrect mantissa after round arithmetic result#4561
Open
NaserAnjum21 wants to merge 1 commit intofable-compiler:mainfrom
Conversation
Author
|
Alternate solution was to skip string/char operations if granular performance is a potential concern. It would be something like below. const trailingZeros = Math.max(0, d.e - (d.c.length - 1));
const decDigits = trailingZeros > 0
? Uint8Array.from([...d.c, ...new Array(trailingZeros).fill(0)])
: Uint8Array.from(d.c);
....Benefit of the above would be no string alloc and no charCode conversion, it's pure numeric array ops. However, that would depend more on big.js internals and can break with change in normalization rules. I'd be happy to discuss the trade-offs. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Decimal.GetBitsreturned incorrect bits for any decimal value produced by arithmetic that yields a round result (e.g. 1M * 10M, 10M + 10M). Reconstructing a decimal from these bits viaDecimal(GetBits(d))would give a value off by some power of 10.Root cause: big.js normalizes its internal representation by stripping trailing zeros from
d.cand compensating viad.e. The old implementation readd.cdirectly as the mantissa digit array, so for 10 (stored asc=[1], e=1), it would extract mantissa 1 instead of 10.Repro
Following code snippet can be used for repro.
Also, Repl
Fix
Derive the mantissa from
d.toString()instead ofd.c. Strip the sign and decimal point from the string representation to get the mantissa digits, and compute scale from the dot position. This approach should be less prone to big.js internal representation changes.Notes
(fromParts → big.js)seems to be unaffected; only reading back via getBits was brokend.cpart of big.js representation where it could be prone to the same normalization issue, did not find any.