Skip to content

Commit a434e94

Browse files
committed
[SPARK-57789] Update and bundle LICENSE-binary and licenses-binary to describe all bundled libraries
### What changes were proposed in this pull request? This PR updates `LICENSE-binary` to describe all third-party libraries bundled in the operator shadow jar, adds the `licenses-binary/` directory with the license texts of the non-Apache components, and bundles them into the jar like Apache Hadoop's `hadoop-client-runtime`. - `LICENSE-binary`: rewritten in Apache Spark's `LICENSE-binary` format, listing the 170 bundled artifacts grouped by license. The list was verified against the actual shadow jar contents. - `licenses-binary/`: adds the 11 license texts for the non-Apache-licensed components. - `spark-operator/build.gradle`: the shadow jar now ships `LICENSE-binary`/`NOTICE-binary` as `META-INF/LICENSE`/`META-INF/NOTICE` plus `META-INF/licenses-binary/`, instead of the source-release `LICENSE`/`NOTICE`. - `.github/.licenserc.yaml`: skips the header check for `licenses-binary/**`. ### Why are the changes needed? `LICENSE-binary` was a copy of `LICENSE` and did not enumerate any bundled dependency. Per the [ASF licensing policy](https://www.apache.org/legal/resolved.html), a convenience binary must account for all bundled licenses and include the non-Apache license texts. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass the CIs. Manually verified that `META-INF/LICENSE` in the shadow jar is identical to `LICENSE-binary`, and that every listed artifact actually has classes in the shadow jar (relocation-aware comparison against the original jars from Maven Central). ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Fable 5 Closes #721 from dongjoon-hyun/SPARK-57789. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
1 parent 8d52490 commit a434e94

14 files changed

Lines changed: 2237 additions & 9 deletions

.github/.licenserc.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ header:
1313
- 'dev/**'
1414
- 'LICENSE'
1515
- 'LICENSE-binary'
16+
- 'licenses-binary/**'
1617
- 'NOTICE'
1718
- 'NOTICE-binary'
1819
- '.asf.yaml'

LICENSE-binary

Lines changed: 204 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -200,12 +200,207 @@
200200
See the License for the specific language governing permissions and
201201
limitations under the License.
202202

203-
--------------------------------------------------------------------------------
204-
205-
This product includes a gradle wrapper.
206-
207-
* gradlew and gradle/wrapper/gradle-wrapper.properties
208-
209-
Copyright: 2015-2021 Gradle Authors.
210-
Home page: https://github.com/gradle/gradle
211-
License: https://www.apache.org/licenses/LICENSE-2.0
203+
------------------------------------------------------------------------------------
204+
This project bundles some components that are also licensed under the Apache
205+
License Version 2.0:
206+
207+
at.yawk.lz4:lz4-java
208+
com.clearspring.analytics:stream
209+
com.fasterxml.jackson.core:jackson-annotations
210+
com.fasterxml.jackson.core:jackson-core
211+
com.fasterxml.jackson.core:jackson-databind
212+
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml
213+
com.fasterxml.jackson.datatype:jackson-datatype-jsr310
214+
com.fasterxml.jackson.jaxrs:jackson-jaxrs-base
215+
com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider
216+
com.fasterxml.jackson.module:jackson-module-jaxb-annotations
217+
com.fasterxml.jackson.module:jackson-module-scala_2.13
218+
com.fasterxml.woodstox:woodstox-core
219+
com.google.code.findbugs:jsr305
220+
com.google.code.gson:gson
221+
com.google.crypto.tink:tink
222+
com.google.errorprone:error_prone_annotations
223+
com.google.guava:failureaccess
224+
com.google.guava:guava
225+
com.google.j2objc:j2objc-annotations
226+
com.nimbusds:nimbus-jose-jwt
227+
com.squareup:javapoet
228+
com.twitter:chill-java
229+
com.twitter:chill_2.13
230+
commons-cli:commons-cli
231+
commons-codec:commons-codec
232+
commons-io:commons-io
233+
commons-net:commons-net
234+
io.dropwizard.metrics:metrics-core
235+
io.dropwizard.metrics:metrics-graphite
236+
io.dropwizard.metrics:metrics-jmx
237+
io.dropwizard.metrics:metrics-json
238+
io.dropwizard.metrics:metrics-jvm
239+
io.fabric8:kubernetes-client
240+
io.fabric8:kubernetes-client-api
241+
io.fabric8:kubernetes-httpclient-vertx
242+
io.fabric8:kubernetes-model-admissionregistration
243+
io.fabric8:kubernetes-model-apiextensions
244+
io.fabric8:kubernetes-model-apps
245+
io.fabric8:kubernetes-model-autoscaling
246+
io.fabric8:kubernetes-model-batch
247+
io.fabric8:kubernetes-model-certificates
248+
io.fabric8:kubernetes-model-common
249+
io.fabric8:kubernetes-model-coordination
250+
io.fabric8:kubernetes-model-core
251+
io.fabric8:kubernetes-model-discovery
252+
io.fabric8:kubernetes-model-events
253+
io.fabric8:kubernetes-model-extensions
254+
io.fabric8:kubernetes-model-flowcontrol
255+
io.fabric8:kubernetes-model-gatewayapi
256+
io.fabric8:kubernetes-model-metrics
257+
io.fabric8:kubernetes-model-networking
258+
io.fabric8:kubernetes-model-node
259+
io.fabric8:kubernetes-model-policy
260+
io.fabric8:kubernetes-model-rbac
261+
io.fabric8:kubernetes-model-resource
262+
io.fabric8:kubernetes-model-scheduling
263+
io.fabric8:kubernetes-model-storageclass
264+
io.fabric8:zjsonpatch
265+
io.github.java-diff-utils:java-diff-utils
266+
io.javaoperatorsdk:operator-framework
267+
io.javaoperatorsdk:operator-framework-core
268+
io.netty:netty-buffer
269+
io.netty:netty-codec-base
270+
io.netty:netty-codec-compression
271+
io.netty:netty-codec-dns
272+
io.netty:netty-codec-http
273+
io.netty:netty-codec-http2
274+
io.netty:netty-codec-socks
275+
io.netty:netty-common
276+
io.netty:netty-handler
277+
io.netty:netty-handler-proxy
278+
io.netty:netty-resolver
279+
io.netty:netty-resolver-dns
280+
io.netty:netty-transport
281+
io.netty:netty-transport-classes-epoll
282+
io.netty:netty-transport-classes-kqueue
283+
io.netty:netty-transport-native-epoll
284+
io.netty:netty-transport-native-kqueue
285+
io.netty:netty-transport-native-unix-common
286+
io.vertx:vertx-auth-common
287+
io.vertx:vertx-core
288+
io.vertx:vertx-uri-template
289+
io.vertx:vertx-web-client
290+
io.vertx:vertx-web-common
291+
jakarta.inject:jakarta.inject-api
292+
jakarta.validation:jakarta.validation-api
293+
org.apache.avro:avro
294+
org.apache.commons:commons-collections4
295+
org.apache.commons:commons-compress
296+
org.apache.commons:commons-configuration2
297+
org.apache.commons:commons-crypto
298+
org.apache.commons:commons-lang3
299+
org.apache.commons:commons-math3
300+
org.apache.commons:commons-text
301+
org.apache.curator:curator-client
302+
org.apache.curator:curator-framework
303+
org.apache.curator:curator-recipes
304+
org.apache.hadoop.thirdparty:hadoop-shaded-guava
305+
org.apache.hadoop.thirdparty:hadoop-shaded-protobuf_3_25
306+
org.apache.hadoop:hadoop-annotations
307+
org.apache.hadoop:hadoop-auth
308+
org.apache.hadoop:hadoop-client-api
309+
org.apache.hadoop:hadoop-client-runtime
310+
org.apache.hadoop:hadoop-common
311+
org.apache.hadoop:hadoop-hdfs-client
312+
org.apache.httpcomponents:httpclient
313+
org.apache.httpcomponents:httpcore
314+
org.apache.ivy:ivy
315+
org.apache.kerby:kerb-core
316+
org.apache.kerby:kerb-util
317+
org.apache.kerby:kerby-asn1
318+
org.apache.logging.log4j:log4j-api
319+
org.apache.logging.log4j:log4j-core
320+
org.apache.logging.log4j:log4j-layout-template-json
321+
org.apache.logging.log4j:log4j-slf4j2-impl
322+
org.apache.spark:spark-common-utils-java_2.13
323+
org.apache.spark:spark-common-utils_2.13
324+
org.apache.spark:spark-core_2.13
325+
org.apache.spark:spark-kubernetes_2.13
326+
org.apache.spark:spark-kvstore_2.13
327+
org.apache.spark:spark-launcher_2.13
328+
org.apache.spark:spark-network-common_2.13
329+
org.apache.spark:spark-network-shuffle_2.13
330+
org.apache.spark:spark-tags_2.13
331+
org.apache.spark:spark-unsafe_2.13
332+
org.apache.xbean:xbean-asm9-shaded
333+
org.apache.yetus:audience-annotations
334+
org.apache.zookeeper:zookeeper
335+
org.apache.zookeeper:zookeeper-jute
336+
org.eclipse.jetty.compression:jetty-compression-common
337+
org.eclipse.jetty.compression:jetty-compression-server
338+
org.eclipse.jetty.ee10:jetty-ee10-proxy
339+
org.eclipse.jetty.ee10:jetty-ee10-servlet
340+
org.eclipse.jetty:jetty-client
341+
org.eclipse.jetty:jetty-http
342+
org.eclipse.jetty:jetty-io
343+
org.eclipse.jetty:jetty-security
344+
org.eclipse.jetty:jetty-server
345+
org.eclipse.jetty:jetty-session
346+
org.eclipse.jetty:jetty-util
347+
org.glassfish.jersey.containers:jersey-container-servlet
348+
org.glassfish.jersey.containers:jersey-container-servlet-core
349+
org.glassfish.jersey.core:jersey-client
350+
org.glassfish.jersey.core:jersey-common
351+
org.glassfish.jersey.core:jersey-server
352+
org.jctools:jctools-core
353+
org.json4s:json4s-ast_2.13
354+
org.json4s:json4s-core_2.13
355+
org.json4s:json4s-jackson-core_2.13
356+
org.json4s:json4s-jackson_2.13
357+
org.json4s:json4s-scalap_2.13
358+
org.objenesis:objenesis
359+
org.scala-lang.modules:scala-parallel-collections_2.13
360+
org.scala-lang.modules:scala-xml_2.13
361+
org.scala-lang:scala-library
362+
org.snakeyaml:snakeyaml-engine
363+
org.yaml:snakeyaml
364+
oro:oro
365+
366+
367+
------------------------------------------------------------------------------------
368+
This product bundles various third-party components under other open source licenses.
369+
This section summarizes those components and their licenses. See licenses-binary/
370+
for text of these licenses.
371+
372+
373+
BSD 2-Clause
374+
------------
375+
org.codehaus.woodstox:stax2-api
376+
377+
378+
BSD 3-Clause
379+
------------
380+
com.google.re2j:re2j
381+
com.thoughtworks.paranamer:paranamer
382+
dnsjava:dnsjava
383+
384+
385+
MIT License
386+
-----------
387+
org.slf4j:slf4j-api
388+
389+
390+
Common Development and Distribution License (CDDL) 1.1
391+
------------------------------------------------------
392+
javax.servlet:javax.servlet-api https://oss.oracle.com/licenses/CDDL+GPL-1.1
393+
394+
395+
Eclipse Distribution License (EDL) 1.0
396+
--------------------------------------
397+
jakarta.xml.bind:jakarta.xml.bind-api
398+
399+
400+
Eclipse Public License (EPL) 2.0
401+
--------------------------------
402+
jakarta.annotation:jakarta.annotation-api https://projects.eclipse.org/projects/ee4j.ca
403+
jakarta.servlet:jakarta.servlet-api https://projects.eclipse.org/projects/ee4j.servlet
404+
jakarta.ws.rs:jakarta.ws.rs-api https://github.com/eclipse-ee4j/jaxrs-api
405+
org.glassfish.hk2:hk2-api
406+
org.glassfish.hk2:osgi-resource-locator
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
Copyright (c) 1998-2011, Brian Wellington.
2+
All rights reserved.
3+
4+
Redistribution and use in source and binary forms, with or without
5+
modification, are permitted provided that the following conditions are met:
6+
7+
* Redistributions of source code must retain the above copyright notice,
8+
this list of conditions and the following disclaimer.
9+
10+
* Redistributions in binary form must reproduce the above copyright notice,
11+
this list of conditions and the following disclaimer in the documentation
12+
and/or other materials provided with the distribution.
13+
14+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
15+
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
16+
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
17+
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
18+
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
19+
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
20+
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
21+
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
22+
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
23+
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
24+
POSSIBILITY OF SUCH DAMAGE.

0 commit comments

Comments
 (0)