databricks

Connecting C++ and Databricks with ADBC

Instructions

Prerequisites

1. Install Pixi

2. Install dbc

3. Create a Databricks account or be able to log in to an existing one.

Set up Databricks

1. Log into Databricks and create or locate an existing SQL warehouse.

2. Open the "Connection details" tab and record the server hostname and HTTP path. See the Databricks documentation describing how to get these connection details.

Connect to Databricks

1. Install the Databricks ADBC driver:

   dbc install --level user databricks

2. Customize the C++ program main.cpp:

   • Change the connection arguments in the AdbcDatabaseSetOption() calls:
     • uri is the URI for your Databricks instance. The program includes several authentication options. See the Databricks ADBC driver documentation for details.
   • Change the SQL SELECT statement in AdbcStatementSetSqlQuery(), or keep it as is.
     • Specify the catalog and schema by fully qualifying the table name as catalog.schema.table.

3. Build and run the C++ program:

   Using Make:

   pixi run make
   ./databricks_demo

   Or using CMake:

   pixi run cmake -B build
   pixi run cmake --build build
   ./build/databricks_demo

Clean up

1. Clean build artifacts:

   Using Make:

   pixi run make clean

   Using CMake:

   rm -rf build