docs: add OAuth M2M authentication as recommended option for Databricks (#2169)

* docs: add OAuth M2M authentication as recommended option for Databricks

Co-Authored-By: Itamar Hartstein <haritamar@gmail.com>

* databricks docs - oauth m2m screenshots + fixes

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Itamar Hartstein <haritamar@gmail.com>

Author: devin-ai-integration[bot]

docs/snippets/cloud/integrations/databricks.mdx

@@ -13,12 +13,38 @@ This guide contains the necessary steps to connect a Databricks environment to y
 In the Elementary platform, go to Environments in the left menu, and click on the "Create Environment" button.
 Choose a name for your environment, and then choose Databricks as your data warehouse type.
 
-Provide the following fields in the form:
+Provide the following common fields in the form:
 
 - **Server Host**: The hostname of your Databricks account to connect to.
 - **Http path**: The path to the Databricks cluster or SQL warehouse.
-- **Access token**: The token you generated for the Elementary service principal (see step 7 under "Create service principal" above)
 - **Catalog (optional)**: The name of the Databricks Catalog.
 - **Elementary schema**: The name of your Elementary schema. Usually `[your dbt target schema]_elementary`.
 
+Then, select your authentication method:
+
+#### OAuth (M2M) — Recommended
+
+<img
+  src="https://res.cloudinary.com/dgpojk42n/image/upload/v1774307890/databricks_form_oauth_m2m_v0jhp9.png"
+  alt="Authenticate with M2M OAuth"
+/>
+
+- **Client ID**: The Application (client) ID of the service principal (the "Application ID" you copied in [step 5](#create-service-principal)).
+- **Client secret**: The OAuth secret you generated for the service principal (see [step 7](#create-service-principal)).
+
+<Info>
+  OAuth machine-to-machine (M2M) authentication is the recommended method for connecting to Databricks.
+  It uses short-lived tokens that are automatically refreshed, providing better security compared to
+  long-lived personal access tokens.
+</Info>
+
+#### Access token (legacy)
+
+<img
+  src="https://res.cloudinary.com/dgpojk42n/image/upload/v1774307890/databricks_form_token_dluwcr.png"
+  alt="Authenticate with Access Token"
+/>
+
+- **Access token**: A personal access token generated for the Elementary service principal.
+
 <IpAllowlist />

docs/snippets/dwh/databricks/create_service_principal.mdx

@@ -32,15 +32,38 @@ to access your Databricks instance.
   alt="Add databricks SQL access"
 />
 
-6. Next, you may also need to allow Token Usage for this service principal (if it is not allowed for all users). To do so, under the settings menu choose Advanced -> Personal Access Tokens -> Permission Settings.
-Then add the service principal there.
+6. Next, generate credentials for your service principal. Choose one of the following methods:
 
-<img
-  src="https://res.cloudinary.com/dgpojk42n/image/upload/v1763316575/databricks_06_token_usage_eufjwv.png"
-  alt="Add databricks SQL access"
-/>
+   **Option A: Generate an OAuth secret (Recommended)**
+
+   On the service principal page, go to the *Secrets* tab and click *Generate secret*. Copy the **Client ID** (this is the same as the "Application ID" from step 5) and the generated **Client secret** — you will need both when configuring the Elementary environment.
+
+   {/* TODO: Add screenshot of Databricks service principal Secrets tab with "Generate secret" button */}
+   <img
+     src="https://res.cloudinary.com/dgpojk42n/image/upload/v1774307890/databricks_m2m_oauth_1_lzix16.png"
+     alt="Generate OAuth M2M Secret (1)"
+   />
+   <img
+     src="https://res.cloudinary.com/dgpojk42n/image/upload/v1774307890/databrick_m2m_oauth_2_yhwws8.png"
+     alt="Generate OAuth M2M Secret (2)"
+   />
+
+   <Info>
+     OAuth secrets are the recommended authentication method. They enable short-lived token generation
+     with automatic refresh, providing better security than long-lived personal access tokens.
+   </Info>
+
+   **Option B: Create a personal access token (legacy)**
+
+   In order to generate a personal access token for your service principal, you may first need to allow Token Usage for it.
+   To do so, go to the settings menu and choose Advanced -> Personal Access Tokens -> Permission Settings, then make sure the service principal is in the list.
+
+   <img
+     src="https://res.cloudinary.com/dgpojk42n/image/upload/v1763316575/databricks_06_token_usage_eufjwv.png"
+     alt="Allow token usage for service principal"
+   />
 
-7. Create a personal access token for your service principal. For more details, please click [here](https://docs.databricks.com/aws/en/dev-tools/auth/pat#databricks-personal-access-tokens-for-service-principals)
+   Then, create a personal access token for your service principal. For more details, please click [here](https://docs.databricks.com/aws/en/dev-tools/auth/pat#databricks-personal-access-tokens-for-service-principals).
 
-8. Finally, in order to enable Elementary's automated monitors feature, please ensure [predictive optimization](https://docs.databricks.com/aws/en/optimizations/predictive-optimization#enable-or-disable-predictive-optimization-for-your-account) is enabled in your account.
+7. Finally, in order to enable Elementary's automated monitors feature, please ensure [predictive optimization](https://docs.databricks.com/aws/en/optimizations/predictive-optimization#enable-or-disable-predictive-optimization-for-your-account) is enabled in your account.
 This is required for table statistics to be updated (Elementary relies on this to obtain up-to-date row counts)