Skip to content

Commit 9789272

Browse files
committed
fix: use JDK java.time field names in make_timestamp_ansi expect_error
CI failed on Spark 3.5.8 because the executor-thrown SparkDateTimeException's getMessage() does NOT preserve the driver-formatted '[DATETIME_FIELD_OUT_OF_BOUNDS]' error-class prefix; only the inner JDK message ('Invalid value for MonthOfYear ...', 'Invalid date FEBRUARY 30', 'Invalid value for HourOfDay ...') survives the 'Job aborted ... Lost task ... SparkDateTimeException: <inner>' wrapping that shows up in the test's caught exception. Switching to the JDK java.time field-name substrings (MonthOfYear, Invalid date, HourOfDay) makes the assertions stable across Spark 3.4, 3.5.x, and 4.x without needing a MinSparkVersion gate, so the make_timestamp_ansi_spark34.sql variant becomes redundant and is deleted in the same commit. Verified locally: passes under -Pspark-3.4 (3.4.3) and -Pspark-3.5 (3.5.8).
1 parent 1b952e7 commit 9789272

2 files changed

Lines changed: 10 additions & 58 deletions

File tree

spark/src/test/resources/sql-tests/expressions/datetime/make_timestamp_ansi.sql

Lines changed: 10 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -17,29 +17,27 @@
1717

1818
-- ANSI mode: make_timestamp throws on out-of-range argument values. With the codegen
1919
-- dispatcher enabled, Spark's own MakeTimestamp.doGenCode produces the throw site, so
20-
-- Comet's kernel raises the same exception as Spark. The expect_error substring matches
21-
-- the DATETIME_FIELD_OUT_OF_BOUNDS error class that Spark 3.5+ wraps all three cases in
22-
-- (the inner JDK message text varies: "Invalid value for MonthOfYear", "Invalid date
23-
-- 'FEBRUARY 30'", "Invalid value for HourOfDay"); the error class is the only stable
24-
-- common substring.
20+
-- Comet's kernel raises the same exception as Spark. The expect_error substrings target
21+
-- the inner JDK java.time.DateTimeException message text (which is wrapped in a
22+
-- SparkDateTimeException whose getMessage() preserves it). The driver-formatted error
23+
-- class string `DATETIME_FIELD_OUT_OF_BOUNDS` is NOT preserved when the exception is
24+
-- thrown from a task on the executor side (only the wrapped `Job aborted ... Lost task
25+
-- ... SparkDateTimeException: <inner message>` form is preserved), so we match the JDK
26+
-- field names which are stable from Spark 3.4 through 4.x.
2527
-- Config: spark.sql.session.timeZone=UTC
2628
-- Config: spark.sql.ansi.enabled=true
2729
-- Config: spark.comet.exec.scalaUDF.codegen.enabled=true
28-
-- The DATETIME_FIELD_OUT_OF_BOUNDS error class was standardized in Spark 3.5; earlier
29-
-- versions wrap the JDK DateTimeException with a generic _LEGACY_ERROR_TEMP_ code whose
30-
-- message does not contain that substring.
31-
-- MinSparkVersion: 3.5
3230

3331
-- month out of range
34-
query expect_error(DATETIME_FIELD_OUT_OF_BOUNDS)
32+
query expect_error(MonthOfYear)
3533
SELECT make_timestamp(2024, 13, 1, 0, 0, 0)
3634

3735
-- day out of range
38-
query expect_error(DATETIME_FIELD_OUT_OF_BOUNDS)
36+
query expect_error(Invalid date)
3937
SELECT make_timestamp(2024, 2, 30, 0, 0, 0)
4038

4139
-- hour out of range
42-
query expect_error(DATETIME_FIELD_OUT_OF_BOUNDS)
40+
query expect_error(HourOfDay)
4341
SELECT make_timestamp(2024, 6, 15, 25, 0, 0)
4442

4543
-- Sentinel: a valid input must still execute on the Comet codegen path. If the dispatcher

spark/src/test/resources/sql-tests/expressions/datetime/make_timestamp_ansi_spark34.sql

Lines changed: 0 additions & 46 deletions
This file was deleted.

0 commit comments

Comments
 (0)