Skip to content

Commit 8d52490

Browse files
committed
[SPARK-57875] Exclude Hadoop filecache classes from shadow jar
### What changes were proposed in this pull request? This PR excludes the Hadoop `filecache` package (`org/apache/hadoop/filecache/**`) from the operator shadow jar by adding a class-path filter to the `shadowJar` block in `spark-operator/build.gradle`. ### Why are the changes needed? The operator never uses Hadoop `DistributedCache`. While the `minimize` step already drops the unreferenced `DistributedCache` class, the package directory entry and `package-info.class` (from `hadoop-client-api`) still remain in the shadow jar. This change removes those leftover entries explicitly, consistent with the existing MapReduce exclusions. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Fable 5 Closes #734 from dongjoon-hyun/SPARK-57875. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
1 parent 9c521a6 commit 8d52490

1 file changed

Lines changed: 1 addition & 0 deletions

File tree

spark-operator/build.gradle

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,7 @@ shadowJar {
102102
// Hadoop YARN classes - not used for K8s deployment
103103
exclude("org/apache/hadoop/yarn/**")
104104
// Hadoop MapReduce classes - not used for K8s deployment
105+
exclude("org/apache/hadoop/filecache/**")
105106
exclude("org/apache/hadoop/mapred/**")
106107
exclude("org/apache/hadoop/mapreduce/**")
107108
exclude("mapred-default.xml")

0 commit comments

Comments
 (0)