Skip to content

fix(js/ts): Fix Decimal.GetBits returning incorrect mantissa after round arithmetic result#4561

Open
NaserAnjum21 wants to merge 1 commit intofable-compiler:mainfrom
NaserAnjum21:fix/getbits-wrong-mantissa-after-round-arithmetic-result
Open

fix(js/ts): Fix Decimal.GetBits returning incorrect mantissa after round arithmetic result#4561
NaserAnjum21 wants to merge 1 commit intofable-compiler:mainfrom
NaserAnjum21:fix/getbits-wrong-mantissa-after-round-arithmetic-result

Conversation

@NaserAnjum21
Copy link
Copy Markdown

Problem

Decimal.GetBits returned incorrect bits for any decimal value produced by arithmetic that yields a round result (e.g. 1M * 10M, 10M + 10M). Reconstructing a decimal from these bits via Decimal(GetBits(d)) would give a value off by some power of 10.

Root cause: big.js normalizes its internal representation by stripping trailing zeros from d.c and compensating via d.e. The old implementation read d.c directly as the mantissa digit array, so for 10 (stored as c=[1], e=1), it would extract mantissa 1 instead of 10.

Repro

Following code snippet can be used for repro.

open System

let check (d: decimal) =
    let bits = Decimal.GetBits(d)
    let d2 = Decimal(bits)
    printfn "original=%A reconstructed=%A match=%A" d d2 (d = d2)

check (10M + 10M) // Expected 20, gets 2
check (100.5M - 0.5M) // Expected 100 gets 1
check (-0.1M * 100M) // Expected -10 gets -1 

Also, Repl

Fix

Derive the mantissa from d.toString() instead of d.c. Strip the sign and decimal point from the string representation to get the mantissa digits, and compute scale from the dot position. This approach should be less prone to big.js internal representation changes.

Notes

  • The inverse direction (fromParts → big.js) seems to be unaffected; only reading back via getBits was broken
  • Searched for other use cases of d.c part of big.js representation where it could be prone to the same normalization issue, did not find any.

@NaserAnjum21
Copy link
Copy Markdown
Author

Alternate solution was to skip string/char operations if granular performance is a potential concern. It would be something like below.

const trailingZeros = Math.max(0, d.e - (d.c.length - 1));
const decDigits = trailingZeros > 0
  ? Uint8Array.from([...d.c, ...new Array(trailingZeros).fill(0)])
  : Uint8Array.from(d.c);
  
....

Benefit of the above would be no string alloc and no charCode conversion, it's pure numeric array ops. However, that would depend more on big.js internals and can break with change in normalization rules. I'd be happy to discuss the trade-offs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant