Skip to content

Commit f5de55e

Browse files
committed
[SPARK-57774][CORE] DriverLogger should not propagate log4j internal exception to caller
### What changes were proposed in this pull request? Flip the `setIgnoreExceptions` from false to true in `DriverLogger` ### Why are the changes needed? https://logging.apache.org/log4j/2.x/plugin-reference.html > ignoreExceptions - If "true" (default) exceptions encountered when appending events are logged; otherwise they are propagated to the caller. The recent log4j 2.25.0 ~ 2.25.3 and 2.26.0 have [known internal bugs](#51719 (comment)) and may fail the Spark caller, this has been observed from the GHA several times. One example https://github.com/apache/spark/actions/runs/27894816608/attempts/1 ``` [info] - provider reports error after FS leaves safe mode *** FAILED *** (3 seconds, 25 milliseconds) [info] org.apache.logging.log4j.core.appender.AppenderLoggingException: An exception occurred processing Appender _DriverLogAppender [info] at org.apache.logging.log4j.core.appender.DefaultErrorHandler.error(DefaultErrorHandler.java:95) [info] at org.apache.logging.log4j.core.config.AppenderControl.handleAppenderError(AppenderControl.java:169) [info] at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:162) [info] at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:133) [info] at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:124) [info] at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:88) [info] at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:711) [info] at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:669) [info] at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:645) [info] at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:589) [info] at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:92) [info] at org.apache.logging.log4j.core.Logger.log(Logger.java:187) [info] at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2970) [info] at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2922) [info] at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2904) [info] at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2648) [info] at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:2587) [info] at org.apache.logging.slf4j.Log4jLogger.warn(Log4jLogger.java:253) [info] at org.apache.spark.internal.Logging.logWarning(Logging.scala:299) [info] at org.apache.spark.internal.Logging.logWarning$(Logging.scala:298) [info] at org.apache.spark.SparkFunSuite.logWarning(SparkFunSuite.scala:33) ... ``` ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Review and monitor CI. ### Was this patch authored or co-authored using generative AI tooling? GLM 5.2 is used for issue analysis. Closes #56890 from pan3793/SPARK-driverlog-ignoreexceptions. Authored-by: Cheng Pan <pan3793@gmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org>
1 parent 4296b3c commit f5de55e

1 file changed

Lines changed: 3 additions & 1 deletion

File tree

core/src/main/scala/org/apache/spark/util/logging/DriverLogger.scala

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,9 @@ private[spark] class DriverLogger(conf: SparkConf) extends Logging {
7171
builder.setBufferedIo(false)
7272
builder.setConfiguration(config)
7373
builder.withFileName(localLogFile)
74-
builder.setIgnoreExceptions(false)
74+
// SPARK-57774: Set to true to avoid propagating log4j internal exception
75+
// (e.g., due to log4j bugs) to the caller.
76+
builder.setIgnoreExceptions(true)
7577
builder.setLayout(layout)
7678
builder.setName(DriverLogger.APPENDER_NAME)
7779
builder.build()

0 commit comments

Comments
 (0)