Skip to content

fix: Avoid precision loss for atan2 with integer args#22516

Open
neilconway wants to merge 1 commit into
apache:mainfrom
neilconway:neilc/fix-atan2-integer-precision
Open

fix: Avoid precision loss for atan2 with integer args#22516
neilconway wants to merge 1 commit into
apache:mainfrom
neilconway:neilc/fix-atan2-integer-precision

Conversation

@neilconway
Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

atan2 defined two input signatures: (Float32, Float32) and (Float64, Float64) (in that order). That meant that integer inputs were coerced into Float32 values, which lead to surprising results: atan2(1, 1000000) resulted in less precision than atan2(1.0, 1000000.0); the results for the former were also inconsistent with the behavior of atan2 in Postgres and DuckDB.

We could fix this by preferring Float64 inputs, but it seems simpler to just remove Float32 entirely. This matches the approach taken by most SQL implementations (Postgres, DuckDB, Snowflake, Spark SQL, etc.)

What changes are included in this PR?

  • Simplify make_math_binary_udf to assume Float64 inputs and outputs (atan2 is the only caller)
  • Remove Float32 case from atan2 benchmarks
  • Update SLT, add new SLT test

Are these changes tested?

Yes, new test added.

Are there any user-facing changes?

Yes; atan2 will be computed in double precision in all cases now and will never return Float32.

@github-actions github-actions Bot added documentation Improvements or additions to documentation sqllogictest SQL Logic Tests (.slt) functions Changes to functions implementation labels May 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation functions Changes to functions implementation sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

atan2 precision loss with integer arguments

1 participant