diff --git a/docs/snippets/cloud/integrations/databricks.mdx b/docs/snippets/cloud/integrations/databricks.mdx index 5252f24b1..9514f122f 100644 --- a/docs/snippets/cloud/integrations/databricks.mdx +++ b/docs/snippets/cloud/integrations/databricks.mdx @@ -13,12 +13,38 @@ This guide contains the necessary steps to connect a Databricks environment to y In the Elementary platform, go to Environments in the left menu, and click on the "Create Environment" button. Choose a name for your environment, and then choose Databricks as your data warehouse type. -Provide the following fields in the form: +Provide the following common fields in the form: - **Server Host**: The hostname of your Databricks account to connect to. - **Http path**: The path to the Databricks cluster or SQL warehouse. -- **Access token**: The token you generated for the Elementary service principal (see step 7 under "Create service principal" above) - **Catalog (optional)**: The name of the Databricks Catalog. - **Elementary schema**: The name of your Elementary schema. Usually `[your dbt target schema]_elementary`. +Then, select your authentication method: + +#### OAuth (M2M) — Recommended + +Authenticate with M2M OAuth + +- **Client ID**: The Application (client) ID of the service principal (the "Application ID" you copied in [step 5](#create-service-principal)). +- **Client secret**: The OAuth secret you generated for the service principal (see [step 7](#create-service-principal)). + + + OAuth machine-to-machine (M2M) authentication is the recommended method for connecting to Databricks. + It uses short-lived tokens that are automatically refreshed, providing better security compared to + long-lived personal access tokens. + + +#### Access token (legacy) + +Authenticate with Access Token + +- **Access token**: A personal access token generated for the Elementary service principal. + diff --git a/docs/snippets/dwh/databricks/create_service_principal.mdx b/docs/snippets/dwh/databricks/create_service_principal.mdx index b04eaa7cd..c0742d6eb 100644 --- a/docs/snippets/dwh/databricks/create_service_principal.mdx +++ b/docs/snippets/dwh/databricks/create_service_principal.mdx @@ -32,15 +32,38 @@ to access your Databricks instance. alt="Add databricks SQL access" /> -6. Next, you may also need to allow Token Usage for this service principal (if it is not allowed for all users). To do so, under the settings menu choose Advanced -> Personal Access Tokens -> Permission Settings. -Then add the service principal there. +6. Next, generate credentials for your service principal. Choose one of the following methods: -Add databricks SQL access + **Option A: Generate an OAuth secret (Recommended)** + + On the service principal page, go to the *Secrets* tab and click *Generate secret*. Copy the **Client ID** (this is the same as the "Application ID" from step 5) and the generated **Client secret** — you will need both when configuring the Elementary environment. + + {/* TODO: Add screenshot of Databricks service principal Secrets tab with "Generate secret" button */} + Generate OAuth M2M Secret (1) + Generate OAuth M2M Secret (2) + + + OAuth secrets are the recommended authentication method. They enable short-lived token generation + with automatic refresh, providing better security than long-lived personal access tokens. + + + **Option B: Create a personal access token (legacy)** + + In order to generate a personal access token for your service principal, you may first need to allow Token Usage for it. + To do so, go to the settings menu and choose Advanced -> Personal Access Tokens -> Permission Settings, then make sure the service principal is in the list. + + Allow token usage for service principal -7. Create a personal access token for your service principal. For more details, please click [here](https://docs.databricks.com/aws/en/dev-tools/auth/pat#databricks-personal-access-tokens-for-service-principals) + Then, create a personal access token for your service principal. For more details, please click [here](https://docs.databricks.com/aws/en/dev-tools/auth/pat#databricks-personal-access-tokens-for-service-principals). -8. Finally, in order to enable Elementary's automated monitors feature, please ensure [predictive optimization](https://docs.databricks.com/aws/en/optimizations/predictive-optimization#enable-or-disable-predictive-optimization-for-your-account) is enabled in your account. +7. Finally, in order to enable Elementary's automated monitors feature, please ensure [predictive optimization](https://docs.databricks.com/aws/en/optimizations/predictive-optimization#enable-or-disable-predictive-optimization-for-your-account) is enabled in your account. This is required for table statistics to be updated (Elementary relies on this to obtain up-to-date row counts)