53 lines (35 loc) · 1.9 KB

Connecting Python and Databricks with ADBC

Instructions

Prerequisites

Install uv
Install dbc
Create a Databricks account or be able to log in to an existing one.

Set up Databricks

Log into Databricks and create or locate an existing SQL warehouse.
Open the "Connection details" tab and record the server hostname and HTTP path. See the Databricks documentation describing how to get these connection details.

Connect to Databricks

Install the Databricks ADBC driver:
```
dbc install databricks
```
Customize the Python script main.py:
- Change the connection arguments in db_kwargs:
  - uri is the URI for your Databricks instance. The script includes several authentication options. See the Databricks ADBC driver documentation for details.
- Change the SQL SELECT statement in cursor.execute(), or keep it as is.
  - Specify the catalog and schema by fully qualifying the table name as catalog.schema.table.
Run the Python script:
```
uv run main.py
```