Skip to content

[PECOBLR-747] Thin jar #987

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
shivam2680 merged 10 commits into from
Sep 11, 2025
Merged

[PECOBLR-747] Thin jar #987

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
63a3aea
add thin pom
shivam2680 Sep 9, 2025
d8645ea
fix script for thin-pom
shivam2680 Sep 9, 2025
42985fe
correct approach
shivam2680 Sep 9, 2025
c31ddd8
Merge branch 'main' into thin-jar
shivam2680 Sep 9, 2025
96e6807
update release process
shivam2680 Sep 9, 2025
cb01d78
add version to thin pom
shivam2680 Sep 10, 2025
5172cb8
add next_changelog
shivam2680 Sep 10, 2025
f69e903
don't deploy thin jar artifacts in databricks-jdbc
shivam2680 Sep 11, 2025
6cb8960
delete wrong sed command
shivam2680 Sep 11, 2025
b482e87
Merge branch 'main' into thin-jar
shivam2680 Sep 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 19 additions & 1 deletion .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,25 @@ jobs:
gpg-connect-agent reloadagent /bye

- name: Publish to the Maven Central Repository
run: mvn -Prelease --batch-mode deploy -Dnvd.api.key=${{ secrets.NVD_API_KEY }}
run: |
# Deploy main artifacts (uber JAR, sources, javadoc)
mvn -Prelease --batch-mode deploy -Dnvd.api.key=${{ secrets.NVD_API_KEY }}

- name: Publish Thin JAR as Separate Artifact
run: |
# Sign and deploy thin JAR with full dependency metadata
# Uses GPG signing to ensure artifact integrity
VERSION=$(grep -m1 '<version>' pom.xml | sed 's/.*<version>\(.*\)<\/version>.*/\1/')
mvn -Prelease gpg:sign-and-deploy-file \
-Dfile="target/databricks-jdbc-${VERSION}-thin.jar" \
-DpomFile=thin_public_pom.xml \
-DrepositoryId=central \
-DgroupId=com.databricks \
-DartifactId=databricks-jdbc-thin \
-Dversion="${VERSION}" \
-Dpackaging=jar \
-DgeneratePom=false \
-Dgpg.passphrase="${GPG_PASSPHRASE}"
env:
Comment on lines +39 to 54
Copy link
Copy Markdown
Collaborator

@samikshya-db samikshya-db Sep 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not required, this is already taken care in the below patch where we say :

 files: target/*.jar

Copy link
Copy Markdown
Collaborator Author

@shivam2680 shivam2680 Sep 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is for github release artifacts and not maven release artifacts

GPG_PASSPHRASE: ${{ secrets.GPG_PASSPHRASE }}
MAVEN_CENTRAL_USERNAME: ${{ secrets.MAVEN_CENTRAL_USERNAME }}
Expand Down
1 change: 1 addition & 0 deletions NEXT_CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
- **Configurable SQL validation in isValid()**: Added `EnableSQLValidationForIsValid` connection property to control whether `isValid()` method executes an actual SQL query for server-side validation. Default value is 0.
- Implement multi-row INSERT batching optimization for prepared statements to improve performance when executing large batches of INSERT operations.
- Implement lazy/incremental fetching for columnar results when using Databricks JDBC in Thrift mode without Arrow support. The change modifies the behavior from buffering entire result sets in memory to maintaining only a limited number of rows at a time, reducing peak heap memory usage and preventing OutOfMemory errors.
- Added new artifact `databricks-jdbc-thin` for thin jar with runtime dependency metadata

### Updated
- Databricks SDK dependency upgraded to latest version 0.60.0
Expand Down
25 changes: 24 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,16 +24,39 @@ Add the following dependency to your `pom.xml`:
</dependency>
```

#### Thin JAR

For applications requiring explicit dependency management, use the thin JAR variant:

```xml
<!-- Note: Available from version 1.0.10-oss onwards -->
<dependency>
<groupId>com.databricks</groupId>
<artifactId>databricks-jdbc-thin</artifactId>
<version>1.0.10-oss</version>
</dependency>
```

The thin JAR contains only the driver code and declares all dependencies in its POM, enabling dependency introspection and version management.

### Build from Source

1. Clone the repository
2. Run the following command:
```bash
mvn clean package
```
3. The jar file is generated as `target/databricks-jdbc-<version>.jar`
3. The following JAR files are generated:
- `target/databricks-jdbc-<version>.jar` (standard JAR with bundled dependencies)
- `target/databricks-jdbc-<version>-thin.jar` (thin JAR without dependencies)
4. The test coverage report is generated in `target/site/jacoco/index.html`

To install the thin JAR locally with dependency metadata:
```bash
VERSION=$(grep -m1 '<version>' pom.xml | sed 's/.*<version>\(.*\)<\/version>.*/\1/')
mvn install:install-file -Dfile="target/databricks-jdbc-${VERSION}-thin.jar" -DpomFile=thin_public_pom.xml
```

## Usage

### Connection String
Expand Down
3 changes: 0 additions & 3 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -671,9 +671,6 @@
<pomFile>${project.basedir}/uber-minimal-pom.xml</pomFile>
<sources>${project.build.directory}/${project.build.finalName}-sources.jar</sources>
<javadoc>${project.build.directory}/${project.build.finalName}-javadoc.jar</javadoc>
<files>${project.build.directory}/${project.build.finalName}-thin.jar</files>
<types>jar</types>
<classifiers>thin</classifiers>
</configuration>
<executions>
<execution>
Expand Down
209 changes: 209 additions & 0 deletions thin_public_pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,209 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.databricks</groupId>
<artifactId>databricks-jdbc-thin</artifactId>
<version>1.0.9-oss</version>
<packaging>jar</packaging>
<name>Databricks JDBC Driver Thin</name>
<description>Databricks JDBC Driver Thin JAR - requires external dependencies.</description>
<url>https://github.com/databricks/databricks-jdbc</url>

<licenses>
<license>
<name>Apache License, Version 2.0</name>
<url>https://github.com/databricks/databricks-jdbc/blob/main/LICENSE</url>
</license>
</licenses>

<developers>
<developer>
<name>Databricks JDBC Team</name>
<email>eng-oss-sql-driver@databricks.com</email>
<organization>Databricks</organization>
<organizationUrl>https://www.databricks.com</organizationUrl>
</developer>
</developers>

<scm>
<connection>scm:git:https://github.com/databricks/databricks-jdbc.git</connection>
<developerConnection>scm:git:https://github.com/databricks/databricks-jdbc.git</developerConnection>
<url>https://github.com/databricks/databricks-jdbc</url>
</scm>

<issueManagement>
<system>GitHub Issues</system>
<url>https://github.com/databricks/databricks-jdbc/issues</url>
</issueManagement>

<dependencies>
<!-- Databricks SDK -->
<dependency>
<groupId>com.databricks</groupId>
<artifactId>databricks-sdk-java</artifactId>
<version>0.60.0</version>
</dependency>

<!-- Apache Commons -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.18.0</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-configuration2</artifactId>
<version>2.10.1</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.14.0</version>
</dependency>

<!-- Apache Arrow for columnar data -->
<dependency>
<groupId>org.apache.arrow</groupId>
<artifactId>arrow-memory-core</artifactId>
<version>17.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.arrow</groupId>
<artifactId>arrow-memory-unsafe</artifactId>
<version>17.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.arrow</groupId>
<artifactId>arrow-vector</artifactId>
<version>17.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.arrow</groupId>
<artifactId>arrow-memory-netty</artifactId>
<version>17.0.0</version>
</dependency>

<!-- HTTP Client -->
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.14</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents.client5</groupId>
<artifactId>httpclient5</artifactId>
<version>5.3.1</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents.core5</groupId>
<artifactId>httpcore5</artifactId>
<version>5.3.1</version>
</dependency>

<!-- Thrift -->
<dependency>
<groupId>org.apache.thrift</groupId>
<artifactId>libthrift</artifactId>
<version>0.19.0</version>
</dependency>

<!-- Logging -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>2.0.13</version>
</dependency>

<!-- Google libraries -->
<dependency>
<groupId>com.google.code.findbugs</groupId>
<artifactId>annotations</artifactId>
<version>3.0.1</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>33.0.0-jre</version>
</dependency>

<!-- Security libraries -->
<dependency>
<groupId>com.nimbusds</groupId>
<artifactId>nimbus-jose-jwt</artifactId>
<version>10.0.2</version>
</dependency>
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-jdk18on</artifactId>
<version>1.78.1</version>
</dependency>
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcpkix-jdk18on</artifactId>
<version>1.78.1</version>
</dependency>

<!-- Jackson JSON processing -->
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.18.3</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
<version>2.18.3</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
<version>2.18.3</version>
</dependency>

<!-- Compression -->
<dependency>
<groupId>org.lz4</groupId>
<artifactId>lz4-java</artifactId>
<version>1.8.0</version>
</dependency>

<!-- gRPC Context -->
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-context</artifactId>
<version>1.71.0</version>
</dependency>

<!-- Netty -->
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-common</artifactId>
<version>4.2.0.Final</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-buffer</artifactId>
<version>4.2.0.Final</version>
</dependency>

<!-- Jakarta Annotations -->
<dependency>
<groupId>jakarta.annotation</groupId>
<artifactId>jakarta.annotation-api</artifactId>
<version>1.3.5</version>
</dependency>

<!-- Resilience4j circuit breaker -->
<dependency>
<groupId>io.github.resilience4j</groupId>
<artifactId>resilience4j-circuitbreaker</artifactId>
<version>1.7.0</version>
</dependency>
<dependency>
<groupId>io.github.resilience4j</groupId>
<artifactId>resilience4j-core</artifactId>
<version>1.7.0</version>
</dependency>
</dependencies>
</project>
Loading