You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs: update user guide with instructions for publishing local snapshots and inspecting Spark dependency trees
- Added steps for publishing snapshot artifacts to Maven local using Gradle tasks.
- Documented usage of `dependencies` and `dependencyInsight` tasks for analyzing Spark dependency configurations.
Copy file name to clipboardExpand all lines: doc/user-guide.adoc
+60Lines changed: 60 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -73,6 +73,36 @@ The `extensions` module keeps Hadoop, Kafka, Avro, Schema Registry, and serializ
73
73
74
74
If an application needs to force its own runtime versions, use normal Gradle dependency constraints, platforms, or exclusions in the consuming build. Future extension modules can split these integrations further by feature so users can depend only on the extension families they need.
75
75
76
+
== Publishing Local Snapshots
77
+
78
+
Publish all snapshot artifacts to Maven local when another local project needs to consume the current checkout:
The consuming build must include `mavenLocal()` before remote repositories if it should prefer the local snapshot:
95
+
96
+
[source,kotlin]
97
+
----
98
+
repositories {
99
+
mavenLocal()
100
+
mavenCentral()
101
+
}
102
+
----
103
+
104
+
Use the current project version from `gradle.properties` or the root build. For example, if this checkout is `0.1.1-SNAPSHOT`, consume `org.openprojectx.bigdata.test:core:0.1.1-SNAPSHOT`, `org.openprojectx.bigdata.test:junit5:0.1.1-SNAPSHOT`, or `org.openprojectx.bigdata.test:extensions:0.1.1-SNAPSHOT`.
105
+
76
106
== Programmatic Usage
77
107
78
108
Use `BigDataTestKit.builder()` when you want direct lifecycle control.
@@ -1104,6 +1134,36 @@ The Kerberos axis currently enables the shared KDC and Kafka Kerberos path used
1104
1134
1105
1135
`sparkClouderaDepsApacheHmsTest` and `sparkClouderaDepsApacheHmsKerberosTest` are compatibility task names only; both use Cloudera HMS.
1106
1136
1137
+
To inspect the resolved dependency tree for each Spark dependency line, use Gradle's `dependencies` task with the resolvable classpath configurations from `example/spark/build.gradle.kts`:
`testRuntimeClasspath` is not the right configuration for comparing the matrix lines because the Spark matrix overrides each test task's runtime classpath. Use `apacheSparkRuntimeClasspath` for the Apache Spark/Hadoop line and `clouderaSparkRuntimeClasspath` for the Cloudera line.
0 commit comments