Commit 3b1d078
[SPARK-56089][SQL] Align asinh/acosh with fdlibm algorithm for cross-engine compatibility
### What changes were proposed in this pull request?
Replace the naive `log(x + sqrt(x^2 ± 1))` formula in `Asinh` and `Acosh` with the full fdlibm algorithm (`s_asinh.c` / `e_acosh.c`), as implemented in OpenJDK `FdLibm.java`.
For `Acosh` (5 branches):
- `x < 1`: NaN
- `x >= 2^28`: `log(x) + log(2)`
- `x == 1`: `0.0`
- `2 < x < 2^28`: `log(2x - 1/(x + sqrt(x*x - 1)))`
- `1 < x < 2`: `log1p(t + sqrt(2*t + t*t))`
For `Asinh` (4 branches):
- `|x| < 2^-28`: `x` (identity)
- `|x| > 2^28`: `sign(x) * (log(|x|) + log(2))`
- `|x| > 2`: `sign(x) * log(2|x| + 1/(|x| + sqrt(x*x + 1)))`
- otherwise: `sign(x) * log1p(|x| + x*x/(1 + sqrt(1 + x*x)))`
### Why are the changes needed?
The fdlibm algorithm (`s_asinh.c` / `e_acosh.c`) is the de facto standard used by OpenJDK, glibc, musl, Go, Python, PostgreSQL, and DuckDB. Spark's previous naive formula `log(x + sqrt(x^2 ± 1))` produces results that differ by 1-2 ULP from these platforms for certain input ranges, causing cross-engine inconsistencies.
The fdlibm algorithm also avoids catastrophic cancellation near x=1 (acosh) and x=0 (asinh) by using range-specific formulas (log1p, identity).
This also changes the large-value threshold from `sqrt(Double.MaxValue)` to `2^28`, matching the fdlibm standard.
### Does this PR introduce _any_ user-facing change?
Results may differ by 1-2 ULP for inputs in the `(1, 2]` range (acosh) and `[-2, 2]` range (asinh) due to the more precise formulas. The new results are closer to the mathematically exact values and consistent with other platforms.
### How was this patch tested?
Updated tests in `MathExpressionsSuite` with fdlibm-aligned reference functions, plus new tests covering all branches with hardcoded reference values cross-verified against C libm.
### Was this patch authored or co-authored using generative AI tooling?
Yes, co-authored with Kiro.
Closes #54912 from xiaoxuandev/fix-56089.
Authored-by: Xiaoxuan Li <xioxuan@amazon.com>
Signed-off-by: Peter Toth <peter.toth@gmail.com>1 parent dce5047 commit 3b1d078
4 files changed
Lines changed: 142 additions & 32 deletions
File tree
- sql
- catalyst/src
- main/scala/org/apache/spark/sql/catalyst/expressions
- test/scala/org/apache/spark/sql/catalyst/expressions
- core/src/test
- resources/sql-tests/results/postgreSQL
- scala/org/apache/spark/sql
Lines changed: 58 additions & 24 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
418 | 418 | | |
419 | 419 | | |
420 | 420 | | |
421 | | - | |
422 | | - | |
423 | | - | |
424 | | - | |
425 | | - | |
426 | | - | |
427 | | - | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
428 | 424 | | |
429 | | - | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
| 430 | + | |
| 431 | + | |
| 432 | + | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
430 | 436 | | |
431 | 437 | | |
432 | 438 | | |
433 | 439 | | |
434 | | - | |
435 | | - | |
436 | | - | |
| 440 | + | |
437 | 441 | | |
| 442 | + | |
| 443 | + | |
| 444 | + | |
| 445 | + | |
| 446 | + | |
| 447 | + | |
438 | 448 | | |
439 | | - | |
| 449 | + | |
| 450 | + | |
440 | 451 | | |
441 | 452 | | |
442 | 453 | | |
| |||
865 | 876 | | |
866 | 877 | | |
867 | 878 | | |
868 | | - | |
869 | | - | |
870 | | - | |
871 | | - | |
872 | | - | |
873 | | - | |
874 | | - | |
875 | | - | |
| 879 | + | |
| 880 | + | |
| 881 | + | |
| 882 | + | |
| 883 | + | |
| 884 | + | |
| 885 | + | |
| 886 | + | |
| 887 | + | |
| 888 | + | |
| 889 | + | |
| 890 | + | |
| 891 | + | |
| 892 | + | |
| 893 | + | |
| 894 | + | |
| 895 | + | |
876 | 896 | | |
877 | | - | |
| 897 | + | |
878 | 898 | | |
879 | | - | |
880 | | - | |
881 | | - | |
| 899 | + | |
| 900 | + | |
| 901 | + | |
| 902 | + | |
| 903 | + | |
| 904 | + | |
| 905 | + | |
| 906 | + | |
| 907 | + | |
| 908 | + | |
| 909 | + | |
| 910 | + | |
| 911 | + | |
| 912 | + | |
| 913 | + | |
| 914 | + | |
| 915 | + | |
882 | 916 | | |
883 | 917 | | |
884 | 918 | | |
| |||
Lines changed: 67 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
238 | 238 | | |
239 | 239 | | |
240 | 240 | | |
241 | | - | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
242 | 258 | | |
243 | 259 | | |
244 | 260 | | |
| |||
280 | 296 | | |
281 | 297 | | |
282 | 298 | | |
283 | | - | |
284 | | - | |
285 | | - | |
286 | | - | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
287 | 318 | | |
288 | 319 | | |
289 | 320 | | |
| |||
1025 | 1056 | | |
1026 | 1057 | | |
1027 | 1058 | | |
| 1059 | + | |
| 1060 | + | |
| 1061 | + | |
| 1062 | + | |
| 1063 | + | |
| 1064 | + | |
| 1065 | + | |
| 1066 | + | |
| 1067 | + | |
| 1068 | + | |
| 1069 | + | |
| 1070 | + | |
| 1071 | + | |
| 1072 | + | |
| 1073 | + | |
| 1074 | + | |
| 1075 | + | |
| 1076 | + | |
| 1077 | + | |
| 1078 | + | |
| 1079 | + | |
| 1080 | + | |
| 1081 | + | |
| 1082 | + | |
| 1083 | + | |
| 1084 | + | |
| 1085 | + | |
| 1086 | + | |
| 1087 | + | |
| 1088 | + | |
| 1089 | + | |
1028 | 1090 | | |
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
577 | 577 | | |
578 | 578 | | |
579 | 579 | | |
580 | | - | |
| 580 | + | |
581 | 581 | | |
582 | 582 | | |
583 | 583 | | |
| |||
Lines changed: 16 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
145 | 145 | | |
146 | 146 | | |
147 | 147 | | |
148 | | - | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
149 | 157 | | |
150 | 158 | | |
151 | 159 | | |
| |||
167 | 175 | | |
168 | 176 | | |
169 | 177 | | |
170 | | - | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
171 | 185 | | |
172 | 186 | | |
173 | 187 | | |
| |||
0 commit comments