Skip to content

Commit 9c521a6

Browse files
committed
[SPARK-57874] Exclude Hadoop MapReduce resources from shadow jar
### What changes were proposed in this pull request? This PR excludes the remaining Hadoop MapReduce resource files from the operator shadow jar. - `mapred-default.xml` - `META-INF/services/org.apache.hadoop.mapreduce.protocol.ClientProtocolProvider` ### Why are the changes needed? SPARK-57871 excluded the Hadoop MapReduce classes, but these two resource files remain. They are dead resources because the classes that use them (`JobConf`, `YarnClientProtocolProvider`, `LocalClientProtocolProvider`) are already excluded. This shrinks the shadow jar by ~21KB. - #732 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Fable 5 Closes #733 from dongjoon-hyun/SPARK-57874. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
1 parent 7d73a2d commit 9c521a6

1 file changed

Lines changed: 2 additions & 0 deletions

File tree

spark-operator/build.gradle

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,6 +104,8 @@ shadowJar {
104104
// Hadoop MapReduce classes - not used for K8s deployment
105105
exclude("org/apache/hadoop/mapred/**")
106106
exclude("org/apache/hadoop/mapreduce/**")
107+
exclude("mapred-default.xml")
108+
exclude("META-INF/services/org.apache.hadoop.mapreduce.protocol.ClientProtocolProvider")
107109
// Exclude 3rd party metadata to avoid conflicts
108110
exclude("META-INF/maven/**")
109111
// Netty QUIC native libraries - not needed for K8s API server communication (HTTP/1.1 & HTTP/2)

0 commit comments

Comments
 (0)