Commit aa702a7
committed
[SPARK-57798] Exclude
### What changes were proposed in this pull request?
This PR excludes the transitive `net.sf.py4j:py4j` dependency from the operator shadow jar.
### Why are the changes needed?
`py4j` is PySpark's Python-to-JVM gateway, pulled in transitively by `spark-core`. It only runs inside the Spark driver pod, never in the operator JVM, yet all 71 `py4j/` classes were bundled into the operator fat jar as dead weight.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 4.8
Closes #724 from dongjoon-hyun/SPARK-57798.
Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>net.sf.py4j dependency1 parent 63fd088 commit aa702a7
1 file changed
Lines changed: 1 addition & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
| 33 | + | |
33 | 34 | | |
34 | 35 | | |
35 | 36 | | |
| |||
0 commit comments