Skip to content

Commit 4c9aaab

Browse files
shivam2680shivam2680
authored
[PECOBLR-747] Thin jar (#987)
## Description Customer reported that the Databricks thin JAR "specifies no dependencies at all" in its POM, preventing dependency introspection tools from working. Users cannot: - View dependency trees with mvn dependency:tree or Clojure's -Stree - Pin, override, or exclude specific dependency versions - Resolve dependency conflicts using standard build tools Solution: Deploy the existing thin JAR as a separate artifact (databricks-jdbc-thin) with a POM that declares all runtime dependencies: 1. Created thin_public_pom.xml: Declares runtime dependencies (Arrow, BouncyCastle, Jackson, Thrift, etc.) with explicit versions. exclude provide and test scope deps. 2. Updated release workflow: Deploy thin JAR using mvn deploy-file with dependency metadata 3. Updated README.md: Added thin JAR usage instructions - Enables introspection: clj -Sdeps '{:deps {com.databricks/databricks-jdbc-thin {:mvn/version "1.0.9-oss"}}}' -Stree now works - Separate artifact: com.databricks:databricks-jdbc-thin vs com.databricks:databricks-jdbc - Standard dependency management: Users can exclude/override versions using Maven/Gradle - No build changes: Uses existing thin JAR (2.5MB), just adds deployment metadata ## Testing - mvn validate -f thin_public_pom.xml - POM structure is valid - Existing thin JAR builds correctly: target/databricks-jdbc-*-thin.jar (2.5MB) ## Additional Notes to the Reviewer <!-- Share any additional context or insights that may help the reviewer understand the changes better. This could include challenges faced, limitations, or compromises made during the development process. Also, mention any areas of the code that you would like the reviewer to focus on specifically. -->
1 parent fcfa625 commit 4c9aaab

5 files changed

Lines changed: 253 additions & 5 deletions

File tree

‎.github/workflows/release.yml‎

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,25 @@ jobs:
3232
gpg-connect-agent reloadagent /bye
3333
3434
- name: Publish to the Maven Central Repository
35-
run: mvn -Prelease --batch-mode deploy -Dnvd.api.key=${{ secrets.NVD_API_KEY }}
35+
run: |
36+
# Deploy main artifacts (uber JAR, sources, javadoc)
37+
mvn -Prelease --batch-mode deploy -Dnvd.api.key=${{ secrets.NVD_API_KEY }}
38+
39+
- name: Publish Thin JAR as Separate Artifact
40+
run: |
41+
# Sign and deploy thin JAR with full dependency metadata
42+
# Uses GPG signing to ensure artifact integrity
43+
VERSION=$(grep -m1 '<version>' pom.xml | sed 's/.*<version>\(.*\)<\/version>.*/\1/')
44+
mvn -Prelease gpg:sign-and-deploy-file \
45+
-Dfile="target/databricks-jdbc-${VERSION}-thin.jar" \
46+
-DpomFile=thin_public_pom.xml \
47+
-DrepositoryId=central \
48+
-DgroupId=com.databricks \
49+
-DartifactId=databricks-jdbc-thin \
50+
-Dversion="${VERSION}" \
51+
-Dpackaging=jar \
52+
-DgeneratePom=false \
53+
-Dgpg.passphrase="${GPG_PASSPHRASE}"
3654
env:
3755
GPG_PASSPHRASE: ${{ secrets.GPG_PASSPHRASE }}
3856
MAVEN_CENTRAL_USERNAME: ${{ secrets.MAVEN_CENTRAL_USERNAME }}

‎NEXT_CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
- **Configurable SQL validation in isValid()**: Added `EnableSQLValidationForIsValid` connection property to control whether `isValid()` method executes an actual SQL query for server-side validation. Default value is 0.
1212
- Implement multi-row INSERT batching optimization for prepared statements to improve performance when executing large batches of INSERT operations.
1313
- Implement lazy/incremental fetching for columnar results when using Databricks JDBC in Thrift mode without Arrow support. The change modifies the behavior from buffering entire result sets in memory to maintaining only a limited number of rows at a time, reducing peak heap memory usage and preventing OutOfMemory errors.
14+
- Added new artifact `databricks-jdbc-thin` for thin jar with runtime dependency metadata
1415

1516
### Updated
1617
- Databricks SDK dependency upgraded to latest version 0.60.0

‎README.md‎

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,16 +24,39 @@ Add the following dependency to your `pom.xml`:
2424
</dependency>
2525
```
2626

27+
#### Thin JAR
28+
29+
For applications requiring explicit dependency management, use the thin JAR variant:
30+
31+
```xml
32+
<!-- Note: Available from version 1.0.10-oss onwards -->
33+
<dependency>
34+
<groupId>com.databricks</groupId>
35+
<artifactId>databricks-jdbc-thin</artifactId>
36+
<version>1.0.10-oss</version>
37+
</dependency>
38+
```
39+
40+
The thin JAR contains only the driver code and declares all dependencies in its POM, enabling dependency introspection and version management.
41+
2742
### Build from Source
2843

2944
1. Clone the repository
3045
2. Run the following command:
3146
```bash
3247
mvn clean package
3348
```
34-
3. The jar file is generated as `target/databricks-jdbc-<version>.jar`
49+
3. The following JAR files are generated:
50+
- `target/databricks-jdbc-<version>.jar` (standard JAR with bundled dependencies)
51+
- `target/databricks-jdbc-<version>-thin.jar` (thin JAR without dependencies)
3552
4. The test coverage report is generated in `target/site/jacoco/index.html`
3653

54+
To install the thin JAR locally with dependency metadata:
55+
```bash
56+
VERSION=$(grep -m1 '<version>' pom.xml | sed 's/.*<version>\(.*\)<\/version>.*/\1/')
57+
mvn install:install-file -Dfile="target/databricks-jdbc-${VERSION}-thin.jar" -DpomFile=thin_public_pom.xml
58+
```
59+
3760
## Usage
3861

3962
### Connection String

‎pom.xml‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -671,9 +671,6 @@
671671
<pomFile>${project.basedir}/uber-minimal-pom.xml</pomFile>
672672
<sources>${project.build.directory}/${project.build.finalName}-sources.jar</sources>
673673
<javadoc>${project.build.directory}/${project.build.finalName}-javadoc.jar</javadoc>
674-
<files>${project.build.directory}/${project.build.finalName}-thin.jar</files>
675-
<types>jar</types>
676-
<classifiers>thin</classifiers>
677674
</configuration>
678675
<executions>
679676
<execution>

‎thin_public_pom.xml‎

Lines changed: 209 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,209 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
3+
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
4+
<modelVersion>4.0.0</modelVersion>
5+
<groupId>com.databricks</groupId>
6+
<artifactId>databricks-jdbc-thin</artifactId>
7+
<version>1.0.9-oss</version>
8+
<packaging>jar</packaging>
9+
<name>Databricks JDBC Driver Thin</name>
10+
<description>Databricks JDBC Driver Thin JAR - requires external dependencies.</description>
11+
<url>https://github.com/databricks/databricks-jdbc</url>
12+
13+
<licenses>
14+
<license>
15+
<name>Apache License, Version 2.0</name>
16+
<url>https://github.com/databricks/databricks-jdbc/blob/main/LICENSE</url>
17+
</license>
18+
</licenses>
19+
20+
<developers>
21+
<developer>
22+
<name>Databricks JDBC Team</name>
23+
<email>eng-oss-sql-driver@databricks.com</email>
24+
<organization>Databricks</organization>
25+
<organizationUrl>https://www.databricks.com</organizationUrl>
26+
</developer>
27+
</developers>
28+
29+
<scm>
30+
<connection>scm:git:https://github.com/databricks/databricks-jdbc.git</connection>
31+
<developerConnection>scm:git:https://github.com/databricks/databricks-jdbc.git</developerConnection>
32+
<url>https://github.com/databricks/databricks-jdbc</url>
33+
</scm>
34+
35+
<issueManagement>
36+
<system>GitHub Issues</system>
37+
<url>https://github.com/databricks/databricks-jdbc/issues</url>
38+
</issueManagement>
39+
40+
<dependencies>
41+
<!-- Databricks SDK -->
42+
<dependency>
43+
<groupId>com.databricks</groupId>
44+
<artifactId>databricks-sdk-java</artifactId>
45+
<version>0.60.0</version>
46+
</dependency>
47+
48+
<!-- Apache Commons -->
49+
<dependency>
50+
<groupId>org.apache.commons</groupId>
51+
<artifactId>commons-lang3</artifactId>
52+
<version>3.18.0</version>
53+
</dependency>
54+
<dependency>
55+
<groupId>org.apache.commons</groupId>
56+
<artifactId>commons-configuration2</artifactId>
57+
<version>2.10.1</version>
58+
</dependency>
59+
<dependency>
60+
<groupId>commons-io</groupId>
61+
<artifactId>commons-io</artifactId>
62+
<version>2.14.0</version>
63+
</dependency>
64+
65+
<!-- Apache Arrow for columnar data -->
66+
<dependency>
67+
<groupId>org.apache.arrow</groupId>
68+
<artifactId>arrow-memory-core</artifactId>
69+
<version>17.0.0</version>
70+
</dependency>
71+
<dependency>
72+
<groupId>org.apache.arrow</groupId>
73+
<artifactId>arrow-memory-unsafe</artifactId>
74+
<version>17.0.0</version>
75+
</dependency>
76+
<dependency>
77+
<groupId>org.apache.arrow</groupId>
78+
<artifactId>arrow-vector</artifactId>
79+
<version>17.0.0</version>
80+
</dependency>
81+
<dependency>
82+
<groupId>org.apache.arrow</groupId>
83+
<artifactId>arrow-memory-netty</artifactId>
84+
<version>17.0.0</version>
85+
</dependency>
86+
87+
<!-- HTTP Client -->
88+
<dependency>
89+
<groupId>org.apache.httpcomponents</groupId>
90+
<artifactId>httpclient</artifactId>
91+
<version>4.5.14</version>
92+
</dependency>
93+
<dependency>
94+
<groupId>org.apache.httpcomponents.client5</groupId>
95+
<artifactId>httpclient5</artifactId>
96+
<version>5.3.1</version>
97+
</dependency>
98+
<dependency>
99+
<groupId>org.apache.httpcomponents.core5</groupId>
100+
<artifactId>httpcore5</artifactId>
101+
<version>5.3.1</version>
102+
</dependency>
103+
104+
<!-- Thrift -->
105+
<dependency>
106+
<groupId>org.apache.thrift</groupId>
107+
<artifactId>libthrift</artifactId>
108+
<version>0.19.0</version>
109+
</dependency>
110+
111+
<!-- Logging -->
112+
<dependency>
113+
<groupId>org.slf4j</groupId>
114+
<artifactId>slf4j-api</artifactId>
115+
<version>2.0.13</version>
116+
</dependency>
117+
118+
<!-- Google libraries -->
119+
<dependency>
120+
<groupId>com.google.code.findbugs</groupId>
121+
<artifactId>annotations</artifactId>
122+
<version>3.0.1</version>
123+
</dependency>
124+
<dependency>
125+
<groupId>com.google.guava</groupId>
126+
<artifactId>guava</artifactId>
127+
<version>33.0.0-jre</version>
128+
</dependency>
129+
130+
<!-- Security libraries -->
131+
<dependency>
132+
<groupId>com.nimbusds</groupId>
133+
<artifactId>nimbus-jose-jwt</artifactId>
134+
<version>10.0.2</version>
135+
</dependency>
136+
<dependency>
137+
<groupId>org.bouncycastle</groupId>
138+
<artifactId>bcprov-jdk18on</artifactId>
139+
<version>1.78.1</version>
140+
</dependency>
141+
<dependency>
142+
<groupId>org.bouncycastle</groupId>
143+
<artifactId>bcpkix-jdk18on</artifactId>
144+
<version>1.78.1</version>
145+
</dependency>
146+
147+
<!-- Jackson JSON processing -->
148+
<dependency>
149+
<groupId>com.fasterxml.jackson.core</groupId>
150+
<artifactId>jackson-databind</artifactId>
151+
<version>2.18.3</version>
152+
</dependency>
153+
<dependency>
154+
<groupId>com.fasterxml.jackson.core</groupId>
155+
<artifactId>jackson-annotations</artifactId>
156+
<version>2.18.3</version>
157+
</dependency>
158+
<dependency>
159+
<groupId>com.fasterxml.jackson.core</groupId>
160+
<artifactId>jackson-core</artifactId>
161+
<version>2.18.3</version>
162+
</dependency>
163+
164+
<!-- Compression -->
165+
<dependency>
166+
<groupId>org.lz4</groupId>
167+
<artifactId>lz4-java</artifactId>
168+
<version>1.8.0</version>
169+
</dependency>
170+
171+
<!-- gRPC Context -->
172+
<dependency>
173+
<groupId>io.grpc</groupId>
174+
<artifactId>grpc-context</artifactId>
175+
<version>1.71.0</version>
176+
</dependency>
177+
178+
<!-- Netty -->
179+
<dependency>
180+
<groupId>io.netty</groupId>
181+
<artifactId>netty-common</artifactId>
182+
<version>4.2.0.Final</version>
183+
</dependency>
184+
<dependency>
185+
<groupId>io.netty</groupId>
186+
<artifactId>netty-buffer</artifactId>
187+
<version>4.2.0.Final</version>
188+
</dependency>
189+
190+
<!-- Jakarta Annotations -->
191+
<dependency>
192+
<groupId>jakarta.annotation</groupId>
193+
<artifactId>jakarta.annotation-api</artifactId>
194+
<version>1.3.5</version>
195+
</dependency>
196+
197+
<!-- Resilience4j circuit breaker -->
198+
<dependency>
199+
<groupId>io.github.resilience4j</groupId>
200+
<artifactId>resilience4j-circuitbreaker</artifactId>
201+
<version>1.7.0</version>
202+
</dependency>
203+
<dependency>
204+
<groupId>io.github.resilience4j</groupId>
205+
<artifactId>resilience4j-core</artifactId>
206+
<version>1.7.0</version>
207+
</dependency>
208+
</dependencies>
209+
</project>

0 commit comments

Comments
 (0)