Skip to content

Commit 574a1e6

Browse files
xfocus3Ahmed El amraouiyine
andauthored
fix: preserve Spark next_day whitespace validation (#22720)
## Which issue does this PR close? - Closes #22717. ## Rationale for this change Spark does not trim `dayOfWeek` before matching it in `next_day`, but `datafusion-spark` currently does. That makes values like `' MO '` succeed in DataFusion even though Spark treats them as invalid. ## What changes are included in this PR? - remove the `.trim()` call from `spark_next_day` - add a regression test proving whitespace-padded day names are rejected ## Are these changes tested? - `cargo test -p datafusion-spark next_day_rejects_whitespace_padded_day_names -- --nocapture` - `cargo test -p datafusion-spark` - `cargo fmt --all --check` - `cargo clippy -p datafusion-spark --all-targets --all-features --no-deps -- -D warnings` Note: the broader package clippy invocation still reports an existing unused import warning in untouched `datafusion/core/src/execution/session_state.rs` on current main. ## Are there any user-facing changes? Behavior now matches Spark for whitespace-padded `dayOfWeek` inputs in `next_day`. --------- Signed-off-by: xfocus3 <xfocus3@users.noreply.github.com> Co-authored-by: xfocus3 <xfocus3@users.noreply.github.com> Co-authored-by: Ahmed El amraouiyine <ahmed.elamraouiyine@vilavi.fr>
1 parent e5f7af1 commit 574a1e6

2 files changed

Lines changed: 13 additions & 1 deletion

File tree

datafusion/spark/src/function/datetime/next_day.rs

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -210,7 +210,7 @@ where
210210
fn spark_next_day(days: i32, day_of_week: &str) -> Option<i32> {
211211
let date = Date32Type::to_naive_date_opt(days)?;
212212

213-
let day_of_week = day_of_week.trim().to_uppercase();
213+
let day_of_week = day_of_week.to_uppercase();
214214
let day_of_week = match day_of_week.as_str() {
215215
"MO" | "MON" | "MONDAY" => Some("MONDAY"),
216216
"TU" | "TUE" | "TUESDAY" => Some("TUESDAY"),
@@ -279,4 +279,10 @@ mod tests {
279279
assert_eq!(field.data_type(), &DataType::Date32);
280280
assert!(field.is_nullable());
281281
}
282+
283+
#[test]
284+
fn next_day_rejects_whitespace_padded_day_names() {
285+
let monday = 19723; // 2024-01-01
286+
assert_eq!(spark_next_day(monday, " MO "), None);
287+
}
282288
}

datafusion/sqllogictest/test_files/spark/datetime/next_day.slt

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,12 @@ SELECT next_day('2015-07-27'::DATE, 'Sat'::string);
3636
----
3737
2015-08-01
3838

39+
# Whitespace-padded day names should be rejected (return NULL) per Spark behavior
40+
query D
41+
SELECT next_day('2015-01-14'::DATE, ' MO '::string);
42+
----
43+
NULL
44+
3945
query error Failed to coerce arguments to satisfy a call to 'next_day' function
4046
SELECT next_day('2015-07-27'::DATE);
4147

0 commit comments

Comments
 (0)