diff --git a/assets/ai_builder/integrations/databricks_integration_1.webp b/assets/ai_builder/integrations/databricks_integration_1.webp new file mode 100644 index 000000000..1359568a9 Binary files /dev/null and b/assets/ai_builder/integrations/databricks_integration_1.webp differ diff --git a/assets/ai_builder/integrations/databricks_integration_2.webp b/assets/ai_builder/integrations/databricks_integration_2.webp new file mode 100644 index 000000000..8c22eafaa Binary files /dev/null and b/assets/ai_builder/integrations/databricks_integration_2.webp differ diff --git a/assets/ai_builder/integrations/databricks_integration_3.webp b/assets/ai_builder/integrations/databricks_integration_3.webp new file mode 100644 index 000000000..8259d526a Binary files /dev/null and b/assets/ai_builder/integrations/databricks_integration_3.webp differ diff --git a/assets/ai_builder/integrations/databricks_integration_4.webp b/assets/ai_builder/integrations/databricks_integration_4.webp new file mode 100644 index 000000000..0a5a0e6f9 Binary files /dev/null and b/assets/ai_builder/integrations/databricks_integration_4.webp differ diff --git a/docs/ai_builder/integrations/databricks.md b/docs/ai_builder/integrations/databricks.md index c92858a27..6705a3ea4 100644 --- a/docs/ai_builder/integrations/databricks.md +++ b/docs/ai_builder/integrations/databricks.md @@ -3,23 +3,80 @@ tags: Data Infrastructure description: Connect with Databricks to run data pipelines and advanced analytics seamlessly. --- + # Databricks Integration -The **Databricks Integration** allows your app to connect to a Databricks workspace, query data from warehouses, and use catalogs and schemas directly in your app. This integration supports secure authentication via tokens and can be configured per environment. +The **Databricks Integration** allows your app to connect to [Databricks](https://www.databricks.com/) for secure data access, querying, and analytics. Once connected, you can run SQL queries, retrieve results, and power data-driven workflows directly from your app. + +## What You Can Do + +With Databricks, your app can: +- Connect securely to your Databricks workspace. +- Run **SQL queries** or fetch data programmatically. +- Build **dashboards and data visualizations** on top of your Databricks tables. +- Automate workflows triggered by new or updated data. +- Combine Databricks with AI models for advanced analytics. + ```python exec import reflex as rx from reflex_image_zoom import image_zoom ``` + +## Step 1: Get Your Databricks Credentials + +1 - Log in to your [Databricks Workspace](https://databricks.com/). + +2 - Get your **DATABRICKS_HOST** and **DATABRICKS_WAREHOUSE_ID**: + - Go to `SQL Warehouses` from the sidebar. + - Select your desired warehouse. + - Click `Connection details`. + - Copy the Server hostname (this is your **DATABRICKS_HOST**). + - Copy the HTTP path removing the `/sql/1.0/warehouses/` prefix (this is your **DATABRICKS_WAREHOUSE_ID**). + + +```python eval +rx.el.div( + image_zoom( + rx.image( + src="/ai_builder/integrations/databricks_integration_1.webp", + class_name="p-2 rounded-md h-auto", + border=f"0.81px solid {rx.color('slate', 5)}", + ), + class_name="rounded-md overflow-hidden", + ), + class_name="w-full flex flex-col rounded-md cursor-pointer", +) +``` + +3 - Generate a **Personal Access Token** (**DATABRICKS_TOKEN**): + - Click on your profile icon → **Settings**. + - Click **Developer**. + - Click **Manage** in Access Tokens. + - Click **Generate New Token**, provide a name and expiration, then copy it (**DATABRICKS_TOKEN**). + +```python eval +rx.el.div( + image_zoom( + rx.image( + src="/ai_builder/integrations/databricks_integration_2.webp", + class_name="p-2 rounded-md h-auto", + border=f"0.81px solid {rx.color('slate', 5)}", + ), + class_name="rounded-md overflow-hidden", + ), + class_name="w-full flex flex-col rounded-md cursor-pointer", +) +``` + + + ```python eval rx.el.div( image_zoom( rx.image( - src=rx.color_mode_cond( - "/ai_builder/integrations/databricks_light.webp", - "/ai_builder/integrations/databricks_dark.webp", - ), + src="/ai_builder/integrations/databricks_integration_3.webp", class_name="p-2 rounded-md h-auto", border=f"0.81px solid {rx.color('slate', 5)}", ), @@ -29,30 +86,44 @@ rx.el.div( ) ``` -## Step 1: Gather Your Credentials +4 - Get your **DATABRICKS_CATALOG** and **DATABRICKS_SCHEMA**: + - Click the SQL Editor from the sidebar. + - Choose the **DATABRICKS_CATALOG** and **DATABRICKS_SCHEMA** from the dropdowns as shown below. + + +```python eval +rx.el.div( + image_zoom( + rx.image( + src="/ai_builder/integrations/databricks_integration_4.webp", + class_name="p-2 rounded-md h-auto", + border=f"0.81px solid {rx.color('slate', 5)}", + ), + class_name="rounded-md overflow-hidden", + ), + class_name="w-full flex flex-col rounded-md cursor-pointer", +) +``` -Before connecting, make sure you have the following information: -- **Hostname**: `https://adb-1234567890123456.7.azuredatabricks.net` - *The URL of your Databricks workspace* -- **Personal Access Token**: `dapi1234567890abcdef1234567890abcd` - *Used for secure authentication* +## Step 2: Configure the Integration in Your App -- **Warehouse / SQL Endpoint ID**: `wh-9876543210` - *The warehouse ID you want to query* +1. In your app, go to **Integrations** and **Add Databricks**. +2. Paste your + 1. **DATABRICKS_TOKEN** + 2. **DATABRICKS_HOST** + 3. **DATABRICKS_WAREHOUSE_ID** + 4. **DATABRICKS_CATALOG** + 5. **DATABRICKS_SCHEMA** +3. Click **Connect** to validate and save your integration. -- **Catalog**: `databricks_catalog_demo` - *The catalog containing your databases* +Once connected, the AI Builder can execute queries directly against your Databricks environment. -- **Schema**: `demo_schema` - *The schema inside the catalog to query* -> **Note:** Use environment-specific credentials for development, staging, and production. +## Step 3: Notes -## Step 2: Configure the Integration +* **Secure your token:** Never expose tokens in public code. +* **Permissions:** Ensure your token or service account has the required workspace and table permissions. +* **Combine with AI:** Use query outputs to power models, summaries, or alerts in real time. -1. Go to the **Integrations** section in your app settings by clicking **`@`** and then clicking the **Integrations** tab at the top. -2. Click **Add Databricks Integration**. -3. Fill in the fields using your credentials -4. Save the integration. Your app can now access Databricks data securely.