Skip to content

Commit 9e13442

Browse files
committed
[SPARK-56909][SQL] Simplify Cast to int/long codegen under ANSI mode
### What changes were proposed in this pull request? In `Cast.scala`, the ANSI codegen for narrowing casts to `int` / `long` previously emitted a 5-line inline body per call site (bounds check + cast + throw). After this PR it emits a single static call into the existing `LongExactNumeric` / `FloatExactNumeric` / `DoubleExactNumeric` objects in `numerics.scala`, which already implement the same overflow check + `castingCauseOverflowError` throw that this codegen needs. The rewrite uses the same `getClass.getCanonicalName.stripSuffix("$")` pattern as the adjacent `MathUtils` / `IntervalMathUtils` calls. The Scala compiler emits `public static` forwarders on the companion class of top-level objects, so generated Java code can call e.g. `org.apache.spark.sql.types.LongExactNumeric.toInt(v)` directly. Touched `Cast.scala` helpers: * `castIntegralTypeToIntegralTypeExactCode`: the `int` target branch now emits `LongExactNumeric.toInt($c)` (byte/short narrowing stays inline; refactored in SPARK-56910). * `castFractionToIntegralTypeCode`: the `int` / `long` target branches now emit `FloatExactNumeric` / `DoubleExactNumeric` `toInt` / `toLong` (byte/short narrowing stays inline; refactored in SPARK-56910). Primitive widening branches and the non-ANSI paths are untouched. ### Why are the changes needed? Part of SPARK-56908 (umbrella). The narrow-cast ANSI branches in `Cast.doGenCode` are some of the longer inline bodies still emitted per call site. Multiplied across the many cast paths in a TPC-DS plan, they contribute meaningfully to the generated source size and Janino compile time, and push whole-stage methods closer to the 64KB JVM method limit. Compared to v1 of this PR (which added a new `CastUtils.java` with `longToIntExact` / `floatToIntExact` / etc.), this version calls the existing `LongExactNumeric.toInt` / `FloatExactNumeric.toInt` / `toLong` / `DoubleExactNumeric.toInt` / `toLong` directly. Those are public static forwarders on top-level Scala objects that already implement the same `castingCauseOverflowError(v, FROM, TO)` throw — no new helper class needed. (Applying the same lesson cloud-fan called out on #55938.) ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? ``` build/sbt "catalyst/testOnly *CastSuite *CastWithAnsiOnSuite \ *CastWithAnsiOffSuite *AnsiCastSuite *TryCastSuite *ExpressionClassIdentitySuite" ``` 307/307 pass. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Cursor 1.x Closes #55934 from gengliangwang/SPARK-56909-cast-int-long. Authored-by: Gengliang Wang <gengliang@apache.org> Signed-off-by: Gengliang Wang <gengliang@apache.org>
1 parent 12ab2c5 commit 9e13442

1 file changed

Lines changed: 54 additions & 28 deletions

File tree

  • sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala

Lines changed: 54 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ import java.time.{ZoneId, ZoneOffset}
2121
import java.util.Locale
2222
import java.util.concurrent.TimeUnit._
2323

24-
import org.apache.spark.{QueryContext, SparkArithmeticException, SparkIllegalArgumentException}
24+
import org.apache.spark.{QueryContext, SparkArithmeticException, SparkException, SparkIllegalArgumentException}
2525
import org.apache.spark.sql.catalyst.InternalRow
2626
import org.apache.spark.sql.catalyst.analysis.TypeCheckResult
2727
import org.apache.spark.sql.catalyst.analysis.TypeCheckResult.DataTypeMismatch
@@ -1988,16 +1988,28 @@ case class Cast(
19881988
from: DataType,
19891989
to: DataType): CastFunction = {
19901990
assert(ansiEnabled)
1991-
val fromDt = ctx.addReferenceObj("from", from, from.getClass.getName)
1992-
val toDt = ctx.addReferenceObj("to", to, to.getClass.getName)
1993-
(c, evPrim, _) =>
1994-
code"""
1995-
if ($c == ($integralType) $c) {
1996-
$evPrim = ($integralType) $c;
1997-
} else {
1998-
throw QueryExecutionErrors.castingCauseOverflowError($c, $fromDt, $toDt);
1999-
}
2000-
"""
1991+
if (integralType == "int") {
1992+
// Integral -> Int: call the existing *ExactNumeric.toInt directly. It already does the
1993+
// bounds check and throws castingCauseOverflowError -- same as the inline body.
1994+
// Only LongType reaches this branch today (`castToIntCode` gates on `case LongType`).
1995+
val numericObj = (from match {
1996+
case LongType => LongExactNumeric
1997+
case _ => throw SparkException.internalError(
1998+
s"Unexpected source type $from for castIntegralTypeToIntegralTypeExactCode int branch")
1999+
}).getClass.getCanonicalName.stripSuffix("$")
2000+
(c, evPrim, _) => code"$evPrim = $numericObj.toInt($c);"
2001+
} else {
2002+
val fromDt = ctx.addReferenceObj("from", from, from.getClass.getName)
2003+
val toDt = ctx.addReferenceObj("to", to, to.getClass.getName)
2004+
(c, evPrim, _) =>
2005+
code"""
2006+
if ($c == ($integralType) $c) {
2007+
$evPrim = ($integralType) $c;
2008+
} else {
2009+
throw QueryExecutionErrors.castingCauseOverflowError($c, $fromDt, $toDt);
2010+
}
2011+
"""
2012+
}
20012013
}
20022014

20032015

@@ -2017,23 +2029,37 @@ case class Cast(
20172029
from: DataType,
20182030
to: DataType): CastFunction = {
20192031
assert(ansiEnabled)
2020-
val (min, max) = lowerAndUpperBound(integralType)
2021-
val mathClass = classOf[Math].getName
2022-
val fromDt = ctx.addReferenceObj("from", from, from.getClass.getName)
2023-
val toDt = ctx.addReferenceObj("to", to, to.getClass.getName)
2024-
// When casting floating values to integral types, Spark uses the method `Numeric.toInt`
2025-
// Or `Numeric.toLong` directly. For positive floating values, it is equivalent to `Math.floor`;
2026-
// for negative floating values, it is equivalent to `Math.ceil`.
2027-
// So, we can use the condition `Math.floor(x) <= upperBound && Math.ceil(x) >= lowerBound`
2028-
// to check if the floating value x is in the range of an integral type after rounding.
2029-
(c, evPrim, _) =>
2030-
code"""
2031-
if ($mathClass.floor($c) <= $max && $mathClass.ceil($c) >= $min) {
2032-
$evPrim = ($integralType) $c;
2033-
} else {
2034-
throw QueryExecutionErrors.castingCauseOverflowError($c, $fromDt, $toDt);
2035-
}
2036-
"""
2032+
if (integralType == "int" || integralType == "long") {
2033+
// Float/Double -> Int/Long: call FloatExactNumeric/DoubleExactNumeric.toInt/toLong
2034+
// directly. Each already does the floor/ceil bounds check and throws
2035+
// castingCauseOverflowError -- same as the inline body.
2036+
val numericObj = (from match {
2037+
case FloatType => FloatExactNumeric
2038+
case DoubleType => DoubleExactNumeric
2039+
case _ => throw SparkException.internalError(
2040+
s"Unexpected source type $from for castFractionToIntegralTypeCode")
2041+
}).getClass.getCanonicalName.stripSuffix("$")
2042+
val method = s"to${integralType.capitalize}"
2043+
(c, evPrim, _) => code"$evPrim = $numericObj.$method($c);"
2044+
} else {
2045+
val (min, max) = lowerAndUpperBound(integralType)
2046+
val mathClass = classOf[Math].getName
2047+
val fromDt = ctx.addReferenceObj("from", from, from.getClass.getName)
2048+
val toDt = ctx.addReferenceObj("to", to, to.getClass.getName)
2049+
// When casting floating values to integral types, Spark uses the method `Numeric.toInt`
2050+
// Or `Numeric.toLong` directly. For positive floating values, it is equivalent to
2051+
// `Math.floor`; for negative floating values, it is equivalent to `Math.ceil`.
2052+
// So, we can use the condition `Math.floor(x) <= upperBound && Math.ceil(x) >= lowerBound`
2053+
// to check if the floating value x is in the range of an integral type after rounding.
2054+
(c, evPrim, _) =>
2055+
code"""
2056+
if ($mathClass.floor($c) <= $max && $mathClass.ceil($c) >= $min) {
2057+
$evPrim = ($integralType) $c;
2058+
} else {
2059+
throw QueryExecutionErrors.castingCauseOverflowError($c, $fromDt, $toDt);
2060+
}
2061+
"""
2062+
}
20372063
}
20382064

20392065
private[this] def castToByteCode(from: DataType, ctx: CodegenContext): CastFunction = from match {

0 commit comments

Comments
 (0)