Skip to content

Latest commit

 

History

History
65 lines (43 loc) · 1.78 KB

File metadata and controls

65 lines (43 loc) · 1.78 KB

Connecting Java and Apache Spark with ADBC

Instructions

Tip

If you already have a Spark instance running, skip the steps to set up and clean up Spark.

Prerequisites

  1. Install Maven

  2. Install dbc

Set up Spark

  1. Install Docker

  2. Start a Spark instance:

    docker run -d --rm --name spark-connect -p 15002:15002 apache/spark:4.1.2 bash -c "/opt/spark/sbin/start-connect-server.sh && tail -f /dev/null"

Connect to Spark

  1. Install the Spark ADBC driver:

    dbc install spark --pre
  2. Customize the main method in Example.java

    • Change the connection arguments in the params.put() calls
    • If you changed which database you're connecting to, also change the SQL SELECT statement in stmt.setSqlQuery()
  3. Run the Java program:

    mvn compile exec:exec

Clean up

Stop the Docker container running Spark:

docker stop spark-connect