issue 59: Fix decade boundary detection in find_exponent and mantissa for long double#60
Open
MitchellThompkins wants to merge 2 commits into
Open
Conversation
slightly larger than the corresponding long double values 0.1L and 0.001L (which was prevoiusly used). This caused find_exponent to return the wrong exponent for those exact boundary values, e.g. log(0.1L) returned 2*ln(0.1). This actually effected all calls to gcem::log; it just only manifests when log lands exactly on 0.1 or 0.001 in long double. add test case for failing log case add 8 epsilon of error add tests for find_exponent make epsilon ULP choice configurable and add mantissa.hpp tests no reason to use T(8) wording updates fix fall-through for 10.0L and 100.0L
be6732e to
7f52241
Compare
This was referenced Apr 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This corrects #59 (and additionally #44), which discusses in detail the bug corrected by this PR.
There are 4 fixes here:
gcem::find_exponentto correctly calculate the exponent forlong doublesfor values that lie at the boundary used to find the exponent.gcem::mantissathat does the same as (2).gcem::find_exponentandgcem::mantissato correctly handle exact decade boundaries (10.0L,100.0L).I also updated 3 tests:
gcem::find_exponent.gcem::mantissa.gcem::logto cover edge cases.Changes
The fix here is two-fold. First, values are scaled up so that the comparison threshold becomes an exact integer T(1), rather than the imprecise T(1e-03) literal. This is the x * T(10) and x * T(1000) portions. Because T(1) is exactly represnteable, the rounding issue of T(1e-03) goes away. Second, they are compared against 1 - machine epsilon, so that values whose scaled representation falls just below 1 due to rounding are still guaranteed to fall into the correct decade. This is tabulated below:
This gives a "snap zone" of 1 ULP below each scaled boundary. Real numbers that fall within this zone cannot be distinguished from the boundary at long double precision, so they snap "up" to T(1).
A similar technique is implemented in
gcem::mantissa, although sans the scaling because that function on checks for bounding cases between 1 and 10.After a lot of thought and testing, I'm pretty convinced that
gcem::find_exponentoriginally returned incorrect values for every odd negative of power 10 (i.e. 10^-1, 10^-3, 10^-5, etc...) forlong double. I think the reason for the alternating pattern is that the value cascades through both broken thresholds (T(1e-03) and T(1e-01)), scaling through each in turn until the final product lands just below 10.0, where the > T(10) branch that would otherwise fix the exponent cannot be taken (b/c of the rounding). So even powers of 10 are rounded but in a way that they take the correct branch.I also fixed a separate simpler issue: both
gcem::find_exponentandgcem::mantissaused a strict x > T(10) comparison for the downscaling branch. Since10.0Land100.0Lare exactly representable, they fell through to the base case, returning an exponent one too low. Changing > to >= fixes this. Unlike the odd-power bug, this did not produce an incorrect log output.