From 398e39db17093633cfab1affc26b47b44880af3f Mon Sep 17 00:00:00 2001 From: Chris Stephens Date: Thu, 20 Nov 2025 15:31:19 +0000 Subject: [PATCH 1/6] Document all valid auth_type values in README MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Explicitly list all 18 valid auth_type values in the "Additional authentication configuration options" section to improve developer experience and reduce confusion about which authentication types are supported. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 82cb34e78..2e031974f 100644 --- a/README.md +++ b/README.md @@ -228,7 +228,7 @@ For all authentication methods, you can override the default behavior in client | Argument | Description | Environment variable | |-------------------------|-------------|------------------------| -| `auth_type` | _(String)_ When multiple auth attributes are available in the environment, use the auth type specified by this argument. This argument also holds the currently selected auth. | `DATABRICKS_AUTH_TYPE` | +| `auth_type` | _(String)_ When multiple auth attributes are available in the environment, use the auth type specified by this argument. This argument also holds the currently selected auth. Valid values are: `pat` (Personal Access Token), `basic` (username/password), `oauth-m2m` (OAuth machine-to-machine), `external-browser` (External browser OAuth), `databricks-cli` (Databricks CLI), `azure-client-secret` (Azure service principal), `azure-cli` (Azure CLI), `github-oidc` (GitHub OIDC), `github-oidc-azure` (GitHub OIDC for Azure), `azure-devops-oidc` (Azure DevOps OIDC), `google-credentials` (Google Cloud credentials), `google-id` (Google Cloud ID), `metadata-service` (Databricks metadata service), `runtime` (Databricks runtime native auth), `runtime-oauth` (Databricks runtime OAuth), `model-serving` (Model serving environment), `env-oidc` (Environment variable OIDC), and `file-oidc` (File-based OIDC). | `DATABRICKS_AUTH_TYPE` | | `http_timeout_seconds` | _(Integer)_ Number of seconds for HTTP timeout. Default is _60_. | _(None)_ | | `retry_timeout_seconds` | _(Integer)_ Number of seconds to keep retrying HTTP requests. Default is _300 (5 minutes)_. | _(None)_ | | `debug_truncate_bytes` | _(Integer)_ Truncate JSON fields in debug logs above this limit. Default is 96. | `DATABRICKS_DEBUG_TRUNCATE_BYTES` | From 5d94067479dd0908b0ef93d9131e3ed02d8b9851 Mon Sep 17 00:00:00 2001 From: Chris Stephens Date: Thu, 20 Nov 2025 15:54:42 +0000 Subject: [PATCH 2/6] Also update auth_type documentation in docs/authentication.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Added the same comprehensive list of valid auth_type values to the docs/authentication.md file to ensure consistency across all documentation. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- docs/authentication.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/authentication.md b/docs/authentication.md index 4307ac3e6..f83cf14e1 100644 --- a/docs/authentication.md +++ b/docs/authentication.md @@ -130,7 +130,7 @@ For all authentication methods, you can override the default behavior in client | Argument | Description | Environment variable | |-------------------------|-------------|------------------------| -| `auth_type` | _(String)_ When multiple auth attributes are available in the environment, use the auth type specified by this argument. This argument also holds the currently selected auth. | `DATABRICKS_AUTH_TYPE` | +| `auth_type` | _(String)_ When multiple auth attributes are available in the environment, use the auth type specified by this argument. This argument also holds the currently selected auth. Valid values are: `pat` (Personal Access Token), `basic` (username/password), `oauth-m2m` (OAuth machine-to-machine), `external-browser` (External browser OAuth), `databricks-cli` (Databricks CLI), `azure-client-secret` (Azure service principal), `azure-cli` (Azure CLI), `github-oidc` (GitHub OIDC), `github-oidc-azure` (GitHub OIDC for Azure), `azure-devops-oidc` (Azure DevOps OIDC), `google-credentials` (Google Cloud credentials), `google-id` (Google Cloud ID), `metadata-service` (Databricks metadata service), `runtime` (Databricks runtime native auth), `runtime-oauth` (Databricks runtime OAuth), `model-serving` (Model serving environment), `env-oidc` (Environment variable OIDC), and `file-oidc` (File-based OIDC). | `DATABRICKS_AUTH_TYPE` | | `http_timeout_seconds` | _(Integer)_ Number of seconds for HTTP timeout. Default is _60_. | _(None)_ | | `retry_timeout_seconds` | _(Integer)_ Number of seconds to keep retrying HTTP requests. Default is _300 (5 minutes)_. | _(None)_ | | `debug_truncate_bytes` | _(Integer)_ Truncate JSON fields in debug logs above this limit. Default is 96. | `DATABRICKS_DEBUG_TRUNCATE_BYTES` | From 8e221f937c28f800fc4807bfe283514631de1e21 Mon Sep 17 00:00:00 2001 From: Chris Stephens Date: Thu, 20 Nov 2025 16:35:13 +0000 Subject: [PATCH 3/6] Add comprehensive authentication types reference guide MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Created a new detailed reference document (docs/auth-types-reference.md) that provides: - Complete table of all 18 auth types with descriptions - Required and optional parameters for each auth type - Environment variables for all parameters - Usage examples for common auth types - Authentication priority order - Cross-references and notes Also added links from README.md and docs/authentication.md to the new reference guide for easy discoverability. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- README.md | 2 +- docs/auth-types-reference.md | 162 +++++++++++++++++++++++++++++++++++ docs/authentication.md | 2 +- 3 files changed, 164 insertions(+), 2 deletions(-) create mode 100644 docs/auth-types-reference.md diff --git a/README.md b/README.md index 2e031974f..e8c3b210c 100644 --- a/README.md +++ b/README.md @@ -228,7 +228,7 @@ For all authentication methods, you can override the default behavior in client | Argument | Description | Environment variable | |-------------------------|-------------|------------------------| -| `auth_type` | _(String)_ When multiple auth attributes are available in the environment, use the auth type specified by this argument. This argument also holds the currently selected auth. Valid values are: `pat` (Personal Access Token), `basic` (username/password), `oauth-m2m` (OAuth machine-to-machine), `external-browser` (External browser OAuth), `databricks-cli` (Databricks CLI), `azure-client-secret` (Azure service principal), `azure-cli` (Azure CLI), `github-oidc` (GitHub OIDC), `github-oidc-azure` (GitHub OIDC for Azure), `azure-devops-oidc` (Azure DevOps OIDC), `google-credentials` (Google Cloud credentials), `google-id` (Google Cloud ID), `metadata-service` (Databricks metadata service), `runtime` (Databricks runtime native auth), `runtime-oauth` (Databricks runtime OAuth), `model-serving` (Model serving environment), `env-oidc` (Environment variable OIDC), and `file-oidc` (File-based OIDC). | `DATABRICKS_AUTH_TYPE` | +| `auth_type` | _(String)_ When multiple auth attributes are available in the environment, use the auth type specified by this argument. This argument also holds the currently selected auth. Valid values are: `pat` (Personal Access Token), `basic` (username/password), `oauth-m2m` (OAuth machine-to-machine), `external-browser` (External browser OAuth), `databricks-cli` (Databricks CLI), `azure-client-secret` (Azure service principal), `azure-cli` (Azure CLI), `github-oidc` (GitHub OIDC), `github-oidc-azure` (GitHub OIDC for Azure), `azure-devops-oidc` (Azure DevOps OIDC), `google-credentials` (Google Cloud credentials), `google-id` (Google Cloud ID), `metadata-service` (Databricks metadata service), `runtime` (Databricks runtime native auth), `runtime-oauth` (Databricks runtime OAuth), `model-serving` (Model serving environment), `env-oidc` (Environment variable OIDC), and `file-oidc` (File-based OIDC). See the [Authentication Types Reference](./docs/auth-types-reference.md) for detailed information about each auth type and its required parameters. | `DATABRICKS_AUTH_TYPE` | | `http_timeout_seconds` | _(Integer)_ Number of seconds for HTTP timeout. Default is _60_. | _(None)_ | | `retry_timeout_seconds` | _(Integer)_ Number of seconds to keep retrying HTTP requests. Default is _300 (5 minutes)_. | _(None)_ | | `debug_truncate_bytes` | _(Integer)_ Truncate JSON fields in debug logs above this limit. Default is 96. | `DATABRICKS_DEBUG_TRUNCATE_BYTES` | diff --git a/docs/auth-types-reference.md b/docs/auth-types-reference.md new file mode 100644 index 000000000..b628c110c --- /dev/null +++ b/docs/auth-types-reference.md @@ -0,0 +1,162 @@ +# Authentication Types Reference + +This document provides a comprehensive reference for all authentication types (`auth_type`) supported by the Databricks SDK for Python. + +## Authentication Types Table + +| Auth Type | Description | Required Parameters | Optional Parameters | Environment Variables | +|-----------|-------------|---------------------|---------------------|----------------------| +| `pat` | Personal Access Token authentication - the most common method for programmatic access | `host`, `token` | - | `DATABRICKS_HOST`, `DATABRICKS_TOKEN` | +| `basic` | Basic HTTP authentication using username and password (primarily for AWS) | `host`, `username`, `password` | `account_id` (for account-level operations) | `DATABRICKS_HOST`, `DATABRICKS_USERNAME`, `DATABRICKS_PASSWORD`, `DATABRICKS_ACCOUNT_ID` | +| `oauth-m2m` | OAuth 2.0 Machine-to-Machine (service principal) authentication | `host`, `client_id`, `client_secret` | `scopes`, `authorization_details` | `DATABRICKS_HOST`, `DATABRICKS_CLIENT_ID`, `DATABRICKS_CLIENT_SECRET` | +| `external-browser` | OAuth 2.0 authentication flow using local browser for user login | `host`, `auth_type='external-browser'` | `client_id`, `client_secret` | `DATABRICKS_HOST`, `DATABRICKS_AUTH_TYPE`, `DATABRICKS_CLIENT_ID` | +| `databricks-cli` | Uses tokens from the Databricks CLI (`databricks auth login`) | `host` | `account_id` (for account-level), `databricks_cli_path` | `DATABRICKS_HOST`, `DATABRICKS_ACCOUNT_ID`, `DATABRICKS_CLI_PATH` | +| `azure-client-secret` | Azure Active Directory (AAD) Service Principal authentication | `azure_client_id`, `azure_client_secret`, `azure_tenant_id` | `host`, `azure_workspace_resource_id`, `azure_environment` | `ARM_CLIENT_ID`, `ARM_CLIENT_SECRET`, `ARM_TENANT_ID`, `DATABRICKS_HOST`, `DATABRICKS_AZURE_RESOURCE_ID`, `ARM_ENVIRONMENT` | +| `azure-cli` | Uses credentials from Azure CLI (`az login`) | `host` (or `azure_workspace_resource_id`) | `azure_tenant_id` | `DATABRICKS_HOST`, `DATABRICKS_AZURE_RESOURCE_ID`, `ARM_TENANT_ID` | +| `github-oidc` | GitHub Actions OIDC authentication (workload identity federation) | `host`, `client_id` | `token_audience`, `account_id` | `DATABRICKS_HOST`, `DATABRICKS_CLIENT_ID`, `DATABRICKS_TOKEN_AUDIENCE`, `DATABRICKS_ACCOUNT_ID` | +| `github-oidc-azure` | GitHub Actions OIDC for Azure Databricks workspaces | `host`, `azure_client_id` | `azure_tenant_id` | `DATABRICKS_HOST`, `ARM_CLIENT_ID`, `ARM_TENANT_ID` | +| `azure-devops-oidc` | Azure DevOps Pipelines OIDC authentication | `host`, `client_id` | `token_audience`, `account_id` | `DATABRICKS_HOST`, `DATABRICKS_CLIENT_ID`, `SYSTEM_ACCESSTOKEN` | +| `google-credentials` | Google Cloud service account authentication using credentials JSON | `host`, `google_credentials` | - | `DATABRICKS_HOST`, `GOOGLE_CREDENTIALS` | +| `google-id` | Google Cloud authentication using service account impersonation | `host`, `google_service_account` | - | `DATABRICKS_HOST`, `DATABRICKS_GOOGLE_SERVICE_ACCOUNT` | +| `metadata-service` | Authentication using Databricks-hosted metadata service | `host`, `metadata_service_url` | - | `DATABRICKS_HOST`, `DATABRICKS_METADATA_SERVICE_URL` | +| `runtime` | Auto-detected authentication when running in Databricks Runtime (notebooks, jobs) | _(auto-detected)_ | - | `DATABRICKS_RUNTIME_VERSION` (auto-set) | +| `runtime-oauth` | OAuth authentication for Databricks Runtime with fine-grained permissions | `scopes` | `authorization_details` | `DATABRICKS_RUNTIME_VERSION` (auto-set) | +| `model-serving` | Auto-detected authentication when running in Databricks Model Serving environment | _(auto-detected)_ | - | `IS_IN_DB_MODEL_SERVING_ENV` or `IS_IN_DATABRICKS_MODEL_SERVING_ENV` (auto-set) | +| `env-oidc` | OIDC token from environment variable | `host` | `oidc_token_env`, `client_id` | `DATABRICKS_HOST`, `DATABRICKS_OIDC_TOKEN`, `DATABRICKS_OIDC_TOKEN_ENV`, `DATABRICKS_CLIENT_ID` | +| `file-oidc` | OIDC token from file path | `host`, `oidc_token_filepath` | `client_id` | `DATABRICKS_HOST`, `DATABRICKS_OIDC_TOKEN_FILE`, `DATABRICKS_CLIENT_ID` | + +## Common Parameters Across All Auth Types + +These parameters can be used with any authentication type: + +| Parameter | Description | Environment Variable | +|-----------|-------------|---------------------| +| `http_timeout_seconds` | HTTP request timeout (default: 60 seconds) | - | +| `retry_timeout_seconds` | Total retry timeout (default: 300 seconds / 5 minutes) | - | +| `debug_truncate_bytes` | Truncate debug logs above this size (default: 96 bytes) | `DATABRICKS_DEBUG_TRUNCATE_BYTES` | +| `debug_headers` | Enable debug logging of HTTP headers (default: false) | `DATABRICKS_DEBUG_HEADERS` | +| `rate_limit` | Maximum requests per second to Databricks API | `DATABRICKS_RATE_LIMIT` | +| `skip_verify` | Skip SSL certificate verification (not recommended) | - | + +## Usage Examples + +### Personal Access Token (PAT) +```python +from databricks.sdk import WorkspaceClient + +w = WorkspaceClient( + host="https://your-workspace.cloud.databricks.com", + token="dapi1234567890abcdef" +) +``` + +### OAuth Machine-to-Machine (Service Principal) +```python +from databricks.sdk import WorkspaceClient + +w = WorkspaceClient( + host="https://your-workspace.cloud.databricks.com", + client_id="your-client-id", + client_secret="your-client-secret" +) +``` + +### External Browser (OAuth for Users) +```python +from databricks.sdk import WorkspaceClient + +w = WorkspaceClient( + host="https://your-workspace.cloud.databricks.com", + auth_type="external-browser" +) +``` + +### Azure Service Principal +```python +from databricks.sdk import WorkspaceClient + +w = WorkspaceClient( + host="https://adb-1234567890.azuredatabricks.net", + azure_client_id="your-azure-client-id", + azure_client_secret="your-azure-client-secret", + azure_tenant_id="your-azure-tenant-id" +) +``` + +### Databricks CLI +```python +from databricks.sdk import WorkspaceClient + +# Assumes you've run: databricks auth login --host https://your-workspace.cloud.databricks.com +w = WorkspaceClient( + host="https://your-workspace.cloud.databricks.com" +) +``` + +### GitHub Actions OIDC +```python +from databricks.sdk import WorkspaceClient + +# In GitHub Actions with OIDC configured +w = WorkspaceClient( + host="https://your-workspace.cloud.databricks.com", + client_id="your-databricks-oauth-client-id" +) +``` + +### Google Cloud Credentials +```python +from databricks.sdk import WorkspaceClient + +w = WorkspaceClient( + host="https://your-workspace.gcp.databricks.com", + google_credentials="/path/to/service-account-key.json" +) +``` + +### Runtime (in Databricks Notebooks) +```python +from databricks.sdk import WorkspaceClient + +# No credentials needed when running in Databricks Runtime +w = WorkspaceClient() +``` + +## Authentication Priority Order + +When no `auth_type` is explicitly specified, the SDK attempts authentication methods in this order: + +1. `pat` - Personal Access Token +2. `basic` - Username/Password +3. `metadata-service` - Metadata Service (if URL provided) +4. `oauth-m2m` - OAuth Service Principal +5. `env-oidc` - Environment OIDC token +6. `file-oidc` - File-based OIDC token +7. `github-oidc` - GitHub OIDC +8. `azure-client-secret` - Azure Service Principal +9. `github-oidc-azure` - GitHub OIDC for Azure +10. `azure-cli` - Azure CLI +11. `azure-devops-oidc` - Azure DevOps OIDC +12. `external-browser` - Browser-based OAuth +13. `databricks-cli` - Databricks CLI +14. `runtime-oauth` - Databricks Runtime OAuth +15. `runtime` - Databricks Runtime native +16. `google-credentials` - Google Cloud credentials +17. `google-id` - Google Cloud ID +18. `model-serving` - Model Serving environment + +You can override this order by explicitly setting the `auth_type` parameter. + +## Notes + +- **Auto-detected auth types** (`runtime`, `runtime-oauth`, `model-serving`): These are automatically detected based on environment variables and don't require explicit configuration. +- **Azure authentication**: When using Azure-specific auth types, if `host` is not provided but `azure_workspace_resource_id` is, the SDK will automatically resolve the workspace URL. +- **OIDC authentication**: OIDC-based methods (`github-oidc`, `azure-devops-oidc`, `env-oidc`, `file-oidc`) use token exchange to obtain Databricks tokens from external identity providers. +- **Scopes**: OAuth-based methods support the `scopes` parameter for fine-grained access control (e.g., `scopes="clusters sql"`). + +## See Also + +- [Main README Authentication Section](../README.md#authentication) +- [OAuth Documentation](./oauth.md) +- [Azure AD Authentication](./azure-ad.md) +- [Databricks Authentication Documentation](https://docs.databricks.com/dev-tools/auth.html) diff --git a/docs/authentication.md b/docs/authentication.md index f83cf14e1..738378b53 100644 --- a/docs/authentication.md +++ b/docs/authentication.md @@ -130,7 +130,7 @@ For all authentication methods, you can override the default behavior in client | Argument | Description | Environment variable | |-------------------------|-------------|------------------------| -| `auth_type` | _(String)_ When multiple auth attributes are available in the environment, use the auth type specified by this argument. This argument also holds the currently selected auth. Valid values are: `pat` (Personal Access Token), `basic` (username/password), `oauth-m2m` (OAuth machine-to-machine), `external-browser` (External browser OAuth), `databricks-cli` (Databricks CLI), `azure-client-secret` (Azure service principal), `azure-cli` (Azure CLI), `github-oidc` (GitHub OIDC), `github-oidc-azure` (GitHub OIDC for Azure), `azure-devops-oidc` (Azure DevOps OIDC), `google-credentials` (Google Cloud credentials), `google-id` (Google Cloud ID), `metadata-service` (Databricks metadata service), `runtime` (Databricks runtime native auth), `runtime-oauth` (Databricks runtime OAuth), `model-serving` (Model serving environment), `env-oidc` (Environment variable OIDC), and `file-oidc` (File-based OIDC). | `DATABRICKS_AUTH_TYPE` | +| `auth_type` | _(String)_ When multiple auth attributes are available in the environment, use the auth type specified by this argument. This argument also holds the currently selected auth. Valid values are: `pat` (Personal Access Token), `basic` (username/password), `oauth-m2m` (OAuth machine-to-machine), `external-browser` (External browser OAuth), `databricks-cli` (Databricks CLI), `azure-client-secret` (Azure service principal), `azure-cli` (Azure CLI), `github-oidc` (GitHub OIDC), `github-oidc-azure` (GitHub OIDC for Azure), `azure-devops-oidc` (Azure DevOps OIDC), `google-credentials` (Google Cloud credentials), `google-id` (Google Cloud ID), `metadata-service` (Databricks metadata service), `runtime` (Databricks runtime native auth), `runtime-oauth` (Databricks runtime OAuth), `model-serving` (Model serving environment), `env-oidc` (Environment variable OIDC), and `file-oidc` (File-based OIDC). See the [Authentication Types Reference](./auth-types-reference.md) for detailed information about each auth type and its required parameters. | `DATABRICKS_AUTH_TYPE` | | `http_timeout_seconds` | _(Integer)_ Number of seconds for HTTP timeout. Default is _60_. | _(None)_ | | `retry_timeout_seconds` | _(Integer)_ Number of seconds to keep retrying HTTP requests. Default is _300 (5 minutes)_. | _(None)_ | | `debug_truncate_bytes` | _(Integer)_ Truncate JSON fields in debug logs above this limit. Default is 96. | `DATABRICKS_DEBUG_TRUNCATE_BYTES` | From 359dc2cb1f7c94bedf83fbe7f533b6c709f3e98d Mon Sep 17 00:00:00 2001 From: Chris Stephens Date: Fri, 12 Dec 2025 16:01:02 +0000 Subject: [PATCH 4/6] Update auth documentation to link to new auth-types-reference - Add auth_type parameter to all examples in auth-types-reference.md - Clarify that explicit auth_type short-circuits automatic detection - Update README.md and authentication.md to link to auth-types-reference - Add auth-types-reference to docs/index.rst toctree for readthedocs - Fix broken azure-ad.md link in README.md - Add new examples: basic, azure-cli, github-oidc-azure, azure-devops-oidc, google-id, metadata-service, runtime-oauth, env-oidc, file-oidc, model-serving --- README.md | 6 +- docs/auth-types-reference.md | 146 +++++++++++++++++++++++++++++++---- docs/authentication.md | 4 +- docs/index.rst | 1 + 4 files changed, 141 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index e8c3b210c..ac7f31051 100644 --- a/README.md +++ b/README.md @@ -178,7 +178,7 @@ w = WorkspaceClient(host=input('Databricks Workspace URL: '), azure_client_secret=input('AAD Client Secret: ')) ``` -Please see more examples in [this document](./docs/azure-ad.md). +For more Azure authentication examples, see the [Authentication Types Reference](./docs/auth-types-reference.md#azure-service-principal). ### Google Cloud Platform native authentication @@ -228,13 +228,15 @@ For all authentication methods, you can override the default behavior in client | Argument | Description | Environment variable | |-------------------------|-------------|------------------------| -| `auth_type` | _(String)_ When multiple auth attributes are available in the environment, use the auth type specified by this argument. This argument also holds the currently selected auth. Valid values are: `pat` (Personal Access Token), `basic` (username/password), `oauth-m2m` (OAuth machine-to-machine), `external-browser` (External browser OAuth), `databricks-cli` (Databricks CLI), `azure-client-secret` (Azure service principal), `azure-cli` (Azure CLI), `github-oidc` (GitHub OIDC), `github-oidc-azure` (GitHub OIDC for Azure), `azure-devops-oidc` (Azure DevOps OIDC), `google-credentials` (Google Cloud credentials), `google-id` (Google Cloud ID), `metadata-service` (Databricks metadata service), `runtime` (Databricks runtime native auth), `runtime-oauth` (Databricks runtime OAuth), `model-serving` (Model serving environment), `env-oidc` (Environment variable OIDC), and `file-oidc` (File-based OIDC). See the [Authentication Types Reference](./docs/auth-types-reference.md) for detailed information about each auth type and its required parameters. | `DATABRICKS_AUTH_TYPE` | +| `auth_type` | _(String)_ When multiple auth attributes are available in the environment, use the auth type specified by this argument. This argument also holds the currently selected auth. When set explicitly, the SDK will **only** attempt that specific authentication method, skipping automatic detection of others. See the **[Authentication Types Reference](./docs/auth-types-reference.md)** for all valid values, required parameters, and usage examples. | `DATABRICKS_AUTH_TYPE` | | `http_timeout_seconds` | _(Integer)_ Number of seconds for HTTP timeout. Default is _60_. | _(None)_ | | `retry_timeout_seconds` | _(Integer)_ Number of seconds to keep retrying HTTP requests. Default is _300 (5 minutes)_. | _(None)_ | | `debug_truncate_bytes` | _(Integer)_ Truncate JSON fields in debug logs above this limit. Default is 96. | `DATABRICKS_DEBUG_TRUNCATE_BYTES` | | `debug_headers` | _(Boolean)_ `true` to debug HTTP headers of requests made by the application. Default is `false`, as headers contain sensitive data, such as access tokens. | `DATABRICKS_DEBUG_HEADERS` | | `rate_limit` | _(Integer)_ Maximum number of requests per second made to Databricks REST API. | `DATABRICKS_RATE_LIMIT` | +For a complete reference of all authentication types including required parameters, environment variables, and code examples, see the **[Authentication Types Reference](./docs/auth-types-reference.md)**. + For example, here's how you can update the overall retry timeout: ```python diff --git a/docs/auth-types-reference.md b/docs/auth-types-reference.md index b628c110c..ca8dc5568 100644 --- a/docs/auth-types-reference.md +++ b/docs/auth-types-reference.md @@ -40,13 +40,28 @@ These parameters can be used with any authentication type: ## Usage Examples +When you explicitly set `auth_type`, the SDK will **only** attempt that specific authentication method, skipping the automatic detection of other methods. This is useful when you want to ensure a specific authentication method is used, or when you have multiple credentials configured but want to use a specific one. + ### Personal Access Token (PAT) ```python from databricks.sdk import WorkspaceClient w = WorkspaceClient( host="https://your-workspace.cloud.databricks.com", - token="dapi1234567890abcdef" + token="dapi1234567890abcdef", + auth_type="pat" +) +``` + +### Basic Authentication (Username/Password) +```python +from databricks.sdk import WorkspaceClient + +w = WorkspaceClient( + host="https://your-workspace.cloud.databricks.com", + username="your-username", + password="your-password", + auth_type="basic" ) ``` @@ -57,7 +72,8 @@ from databricks.sdk import WorkspaceClient w = WorkspaceClient( host="https://your-workspace.cloud.databricks.com", client_id="your-client-id", - client_secret="your-client-secret" + client_secret="your-client-secret", + auth_type="oauth-m2m" ) ``` @@ -71,6 +87,17 @@ w = WorkspaceClient( ) ``` +### Databricks CLI +```python +from databricks.sdk import WorkspaceClient + +# Assumes you've run: databricks auth login --host https://your-workspace.cloud.databricks.com +w = WorkspaceClient( + host="https://your-workspace.cloud.databricks.com", + auth_type="databricks-cli" +) +``` + ### Azure Service Principal ```python from databricks.sdk import WorkspaceClient @@ -79,17 +106,19 @@ w = WorkspaceClient( host="https://adb-1234567890.azuredatabricks.net", azure_client_id="your-azure-client-id", azure_client_secret="your-azure-client-secret", - azure_tenant_id="your-azure-tenant-id" + azure_tenant_id="your-azure-tenant-id", + auth_type="azure-client-secret" ) ``` -### Databricks CLI +### Azure CLI ```python from databricks.sdk import WorkspaceClient -# Assumes you've run: databricks auth login --host https://your-workspace.cloud.databricks.com +# Assumes you've run: az login w = WorkspaceClient( - host="https://your-workspace.cloud.databricks.com" + host="https://adb-1234567890.azuredatabricks.net", + auth_type="azure-cli" ) ``` @@ -100,7 +129,33 @@ from databricks.sdk import WorkspaceClient # In GitHub Actions with OIDC configured w = WorkspaceClient( host="https://your-workspace.cloud.databricks.com", - client_id="your-databricks-oauth-client-id" + client_id="your-databricks-oauth-client-id", + auth_type="github-oidc" +) +``` + +### GitHub Actions OIDC for Azure +```python +from databricks.sdk import WorkspaceClient + +# In GitHub Actions with Azure OIDC configured +w = WorkspaceClient( + host="https://adb-1234567890.azuredatabricks.net", + azure_client_id="your-azure-client-id", + auth_type="github-oidc-azure" +) +``` + +### Azure DevOps OIDC +```python +from databricks.sdk import WorkspaceClient + +# In Azure DevOps with OIDC configured +# Note: SYSTEM_ACCESSTOKEN must be exposed as an environment variable +w = WorkspaceClient( + host="https://your-workspace.cloud.databricks.com", + client_id="your-databricks-oauth-client-id", + auth_type="azure-devops-oidc" ) ``` @@ -110,7 +165,30 @@ from databricks.sdk import WorkspaceClient w = WorkspaceClient( host="https://your-workspace.gcp.databricks.com", - google_credentials="/path/to/service-account-key.json" + google_credentials="/path/to/service-account-key.json", + auth_type="google-credentials" +) +``` + +### Google Cloud ID (Service Account Impersonation) +```python +from databricks.sdk import WorkspaceClient + +w = WorkspaceClient( + host="https://your-workspace.gcp.databricks.com", + google_service_account="your-service-account@project.iam.gserviceaccount.com", + auth_type="google-id" +) +``` + +### Metadata Service +```python +from databricks.sdk import WorkspaceClient + +w = WorkspaceClient( + host="https://your-workspace.cloud.databricks.com", + metadata_service_url="http://localhost:8080/metadata", + auth_type="metadata-service" ) ``` @@ -119,7 +197,50 @@ w = WorkspaceClient( from databricks.sdk import WorkspaceClient # No credentials needed when running in Databricks Runtime -w = WorkspaceClient() +# The runtime auth type is auto-detected +w = WorkspaceClient(auth_type="runtime") +``` + +### Runtime OAuth (in Databricks Notebooks with scoped access) +```python +from databricks.sdk import WorkspaceClient + +# For fine-grained access control in notebooks +w = WorkspaceClient( + scopes="clusters sql", + auth_type="runtime-oauth" +) +``` + +### Environment Variable OIDC +```python +from databricks.sdk import WorkspaceClient + +# OIDC token from DATABRICKS_OIDC_TOKEN environment variable +w = WorkspaceClient( + host="https://your-workspace.cloud.databricks.com", + auth_type="env-oidc" +) +``` + +### File-based OIDC +```python +from databricks.sdk import WorkspaceClient + +# OIDC token from a file +w = WorkspaceClient( + host="https://your-workspace.cloud.databricks.com", + oidc_token_filepath="/path/to/oidc-token", + auth_type="file-oidc" +) +``` + +### Model Serving Environment +```python +from databricks.sdk import WorkspaceClient + +# Auto-detected when running in Databricks Model Serving +w = WorkspaceClient(auth_type="model-serving") ``` ## Authentication Priority Order @@ -156,7 +277,6 @@ You can override this order by explicitly setting the `auth_type` parameter. ## See Also -- [Main README Authentication Section](../README.md#authentication) -- [OAuth Documentation](./oauth.md) -- [Azure AD Authentication](./azure-ad.md) -- [Databricks Authentication Documentation](https://docs.databricks.com/dev-tools/auth.html) +- [Authentication Overview](./authentication.md) - Default authentication flow and configuration +- [OAuth Documentation](./oauth.md) - OAuth-based authentication details +- [Databricks Authentication Documentation](https://docs.databricks.com/dev-tools/auth.html) - Official Databricks authentication docs diff --git a/docs/authentication.md b/docs/authentication.md index 738378b53..0ebe03027 100644 --- a/docs/authentication.md +++ b/docs/authentication.md @@ -130,13 +130,15 @@ For all authentication methods, you can override the default behavior in client | Argument | Description | Environment variable | |-------------------------|-------------|------------------------| -| `auth_type` | _(String)_ When multiple auth attributes are available in the environment, use the auth type specified by this argument. This argument also holds the currently selected auth. Valid values are: `pat` (Personal Access Token), `basic` (username/password), `oauth-m2m` (OAuth machine-to-machine), `external-browser` (External browser OAuth), `databricks-cli` (Databricks CLI), `azure-client-secret` (Azure service principal), `azure-cli` (Azure CLI), `github-oidc` (GitHub OIDC), `github-oidc-azure` (GitHub OIDC for Azure), `azure-devops-oidc` (Azure DevOps OIDC), `google-credentials` (Google Cloud credentials), `google-id` (Google Cloud ID), `metadata-service` (Databricks metadata service), `runtime` (Databricks runtime native auth), `runtime-oauth` (Databricks runtime OAuth), `model-serving` (Model serving environment), `env-oidc` (Environment variable OIDC), and `file-oidc` (File-based OIDC). See the [Authentication Types Reference](./auth-types-reference.md) for detailed information about each auth type and its required parameters. | `DATABRICKS_AUTH_TYPE` | +| `auth_type` | _(String)_ When multiple auth attributes are available in the environment, use the auth type specified by this argument. This argument also holds the currently selected auth. When set explicitly, the SDK will **only** attempt that specific authentication method, skipping automatic detection of others. See the **[Authentication Types Reference](./auth-types-reference.md)** for all valid values, required parameters, and usage examples. | `DATABRICKS_AUTH_TYPE` | | `http_timeout_seconds` | _(Integer)_ Number of seconds for HTTP timeout. Default is _60_. | _(None)_ | | `retry_timeout_seconds` | _(Integer)_ Number of seconds to keep retrying HTTP requests. Default is _300 (5 minutes)_. | _(None)_ | | `debug_truncate_bytes` | _(Integer)_ Truncate JSON fields in debug logs above this limit. Default is 96. | `DATABRICKS_DEBUG_TRUNCATE_BYTES` | | `debug_headers` | _(Boolean)_ `true` to debug HTTP headers of requests made by the application. Default is `false`, as headers contain sensitive data, such as access tokens. | `DATABRICKS_DEBUG_HEADERS` | | `rate_limit` | _(Integer)_ Maximum number of requests per second made to Databricks REST API. | `DATABRICKS_RATE_LIMIT` | +For a complete reference of all authentication types including required parameters, environment variables, and code examples, see the **[Authentication Types Reference](./auth-types-reference.md)**. + For example, to turn on debug HTTP headers: ```python diff --git a/docs/index.rst b/docs/index.rst index 3d3a5dfc5..ccf5e1629 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -12,6 +12,7 @@ We are keen to hear feedback from you on these SDKs. Please `file GitHub issues getting-started authentication + auth-types-reference oauth wait pagination From fadfb420051eea8e95edd8e8afe1cb24af757a4b Mon Sep 17 00:00:00 2001 From: Chris Stephens Date: Fri, 12 Jun 2026 15:47:57 -0700 Subject: [PATCH 5/6] Correct azure_tenant_id and file-oidc env var details in auth reference Co-authored-by: Isaac --- docs/auth-types-reference.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/auth-types-reference.md b/docs/auth-types-reference.md index ca8dc5568..d07239b83 100644 --- a/docs/auth-types-reference.md +++ b/docs/auth-types-reference.md @@ -11,7 +11,7 @@ This document provides a comprehensive reference for all authentication types (` | `oauth-m2m` | OAuth 2.0 Machine-to-Machine (service principal) authentication | `host`, `client_id`, `client_secret` | `scopes`, `authorization_details` | `DATABRICKS_HOST`, `DATABRICKS_CLIENT_ID`, `DATABRICKS_CLIENT_SECRET` | | `external-browser` | OAuth 2.0 authentication flow using local browser for user login | `host`, `auth_type='external-browser'` | `client_id`, `client_secret` | `DATABRICKS_HOST`, `DATABRICKS_AUTH_TYPE`, `DATABRICKS_CLIENT_ID` | | `databricks-cli` | Uses tokens from the Databricks CLI (`databricks auth login`) | `host` | `account_id` (for account-level), `databricks_cli_path` | `DATABRICKS_HOST`, `DATABRICKS_ACCOUNT_ID`, `DATABRICKS_CLI_PATH` | -| `azure-client-secret` | Azure Active Directory (AAD) Service Principal authentication | `azure_client_id`, `azure_client_secret`, `azure_tenant_id` | `host`, `azure_workspace_resource_id`, `azure_environment` | `ARM_CLIENT_ID`, `ARM_CLIENT_SECRET`, `ARM_TENANT_ID`, `DATABRICKS_HOST`, `DATABRICKS_AZURE_RESOURCE_ID`, `ARM_ENVIRONMENT` | +| `azure-client-secret` | Azure Active Directory (AAD) Service Principal authentication | `azure_client_id`, `azure_client_secret` | `azure_tenant_id` (auto-detected from `host` if not set), `host`, `azure_workspace_resource_id`, `azure_environment` | `ARM_CLIENT_ID`, `ARM_CLIENT_SECRET`, `ARM_TENANT_ID`, `DATABRICKS_HOST`, `DATABRICKS_AZURE_RESOURCE_ID`, `ARM_ENVIRONMENT` | | `azure-cli` | Uses credentials from Azure CLI (`az login`) | `host` (or `azure_workspace_resource_id`) | `azure_tenant_id` | `DATABRICKS_HOST`, `DATABRICKS_AZURE_RESOURCE_ID`, `ARM_TENANT_ID` | | `github-oidc` | GitHub Actions OIDC authentication (workload identity federation) | `host`, `client_id` | `token_audience`, `account_id` | `DATABRICKS_HOST`, `DATABRICKS_CLIENT_ID`, `DATABRICKS_TOKEN_AUDIENCE`, `DATABRICKS_ACCOUNT_ID` | | `github-oidc-azure` | GitHub Actions OIDC for Azure Databricks workspaces | `host`, `azure_client_id` | `azure_tenant_id` | `DATABRICKS_HOST`, `ARM_CLIENT_ID`, `ARM_TENANT_ID` | @@ -23,7 +23,7 @@ This document provides a comprehensive reference for all authentication types (` | `runtime-oauth` | OAuth authentication for Databricks Runtime with fine-grained permissions | `scopes` | `authorization_details` | `DATABRICKS_RUNTIME_VERSION` (auto-set) | | `model-serving` | Auto-detected authentication when running in Databricks Model Serving environment | _(auto-detected)_ | - | `IS_IN_DB_MODEL_SERVING_ENV` or `IS_IN_DATABRICKS_MODEL_SERVING_ENV` (auto-set) | | `env-oidc` | OIDC token from environment variable | `host` | `oidc_token_env`, `client_id` | `DATABRICKS_HOST`, `DATABRICKS_OIDC_TOKEN`, `DATABRICKS_OIDC_TOKEN_ENV`, `DATABRICKS_CLIENT_ID` | -| `file-oidc` | OIDC token from file path | `host`, `oidc_token_filepath` | `client_id` | `DATABRICKS_HOST`, `DATABRICKS_OIDC_TOKEN_FILE`, `DATABRICKS_CLIENT_ID` | +| `file-oidc` | OIDC token from file path | `host`, `oidc_token_filepath` | `client_id` | `DATABRICKS_HOST`, `DATABRICKS_OIDC_TOKEN_FILEPATH` (alias: `DATABRICKS_OIDC_TOKEN_FILE`), `DATABRICKS_CLIENT_ID` | ## Common Parameters Across All Auth Types From ab3f4dd2b7b467e2e5ca005a15878f4ad1f2d369 Mon Sep 17 00:00:00 2001 From: Chris Stephens Date: Fri, 12 Jun 2026 15:49:24 -0700 Subject: [PATCH 6/6] Remove duplicated cross-references and common-parameters table from auth docs Co-authored-by: Isaac --- README.md | 4 +--- docs/auth-types-reference.md | 17 +++-------------- docs/authentication.md | 4 +--- 3 files changed, 5 insertions(+), 20 deletions(-) diff --git a/README.md b/README.md index e808b02c7..2ffab543a 100644 --- a/README.md +++ b/README.md @@ -276,15 +276,13 @@ For all authentication methods, you can override the default behavior in client | Argument | Description | Environment variable | |-------------------------|-------------|------------------------| -| `auth_type` | _(String)_ When multiple auth attributes are available in the environment, use the auth type specified by this argument. This argument also holds the currently selected auth. When set explicitly, the SDK will **only** attempt that specific authentication method, skipping automatic detection of others. See the **[Authentication Types Reference](./docs/auth-types-reference.md)** for all valid values, required parameters, and usage examples. | `DATABRICKS_AUTH_TYPE` | +| `auth_type` | _(String)_ When multiple auth attributes are available in the environment, use the auth type specified by this argument. This argument also holds the currently selected auth. When set explicitly, the SDK only attempts that authentication method, skipping automatic detection of others. See the [Authentication Types Reference](./docs/auth-types-reference.md) for all valid values, required parameters, and usage examples. | `DATABRICKS_AUTH_TYPE` | | `http_timeout_seconds` | _(Integer)_ Number of seconds for HTTP timeout. Default is _60_. | _(None)_ | | `retry_timeout_seconds` | _(Integer)_ Number of seconds to keep retrying HTTP requests. Default is _300 (5 minutes)_. | _(None)_ | | `debug_truncate_bytes` | _(Integer)_ Truncate JSON fields in debug logs above this limit. Default is 96. | `DATABRICKS_DEBUG_TRUNCATE_BYTES` | | `debug_headers` | _(Boolean)_ `true` to debug HTTP headers of requests made by the application. Default is `false`, as headers contain sensitive data, such as access tokens. | `DATABRICKS_DEBUG_HEADERS` | | `rate_limit` | _(Integer)_ Maximum number of requests per second made to Databricks REST API. | `DATABRICKS_RATE_LIMIT` | -For a complete reference of all authentication types including required parameters, environment variables, and code examples, see the **[Authentication Types Reference](./docs/auth-types-reference.md)**. - For example, here's how you can update the overall retry timeout: ```python diff --git a/docs/auth-types-reference.md b/docs/auth-types-reference.md index d07239b83..47837cc1c 100644 --- a/docs/auth-types-reference.md +++ b/docs/auth-types-reference.md @@ -1,6 +1,6 @@ # Authentication Types Reference -This document provides a comprehensive reference for all authentication types (`auth_type`) supported by the Databricks SDK for Python. +This document lists all authentication types (`auth_type`) supported by the Databricks SDK for Python. ## Authentication Types Table @@ -25,22 +25,11 @@ This document provides a comprehensive reference for all authentication types (` | `env-oidc` | OIDC token from environment variable | `host` | `oidc_token_env`, `client_id` | `DATABRICKS_HOST`, `DATABRICKS_OIDC_TOKEN`, `DATABRICKS_OIDC_TOKEN_ENV`, `DATABRICKS_CLIENT_ID` | | `file-oidc` | OIDC token from file path | `host`, `oidc_token_filepath` | `client_id` | `DATABRICKS_HOST`, `DATABRICKS_OIDC_TOKEN_FILEPATH` (alias: `DATABRICKS_OIDC_TOKEN_FILE`), `DATABRICKS_CLIENT_ID` | -## Common Parameters Across All Auth Types - -These parameters can be used with any authentication type: - -| Parameter | Description | Environment Variable | -|-----------|-------------|---------------------| -| `http_timeout_seconds` | HTTP request timeout (default: 60 seconds) | - | -| `retry_timeout_seconds` | Total retry timeout (default: 300 seconds / 5 minutes) | - | -| `debug_truncate_bytes` | Truncate debug logs above this size (default: 96 bytes) | `DATABRICKS_DEBUG_TRUNCATE_BYTES` | -| `debug_headers` | Enable debug logging of HTTP headers (default: false) | `DATABRICKS_DEBUG_HEADERS` | -| `rate_limit` | Maximum requests per second to Databricks API | `DATABRICKS_RATE_LIMIT` | -| `skip_verify` | Skip SSL certificate verification (not recommended) | - | +For configuration options that apply to all authentication types (timeouts, debug settings, rate limits), see [Authentication](./authentication.md#additional-configuration-options). ## Usage Examples -When you explicitly set `auth_type`, the SDK will **only** attempt that specific authentication method, skipping the automatic detection of other methods. This is useful when you want to ensure a specific authentication method is used, or when you have multiple credentials configured but want to use a specific one. +When you explicitly set `auth_type`, the SDK only attempts that authentication method, skipping the automatic detection of other methods. This is useful when you have multiple credentials configured but want to use a specific one. ### Personal Access Token (PAT) ```python diff --git a/docs/authentication.md b/docs/authentication.md index 0ebe03027..52d375ca4 100644 --- a/docs/authentication.md +++ b/docs/authentication.md @@ -130,15 +130,13 @@ For all authentication methods, you can override the default behavior in client | Argument | Description | Environment variable | |-------------------------|-------------|------------------------| -| `auth_type` | _(String)_ When multiple auth attributes are available in the environment, use the auth type specified by this argument. This argument also holds the currently selected auth. When set explicitly, the SDK will **only** attempt that specific authentication method, skipping automatic detection of others. See the **[Authentication Types Reference](./auth-types-reference.md)** for all valid values, required parameters, and usage examples. | `DATABRICKS_AUTH_TYPE` | +| `auth_type` | _(String)_ When multiple auth attributes are available in the environment, use the auth type specified by this argument. This argument also holds the currently selected auth. When set explicitly, the SDK only attempts that authentication method, skipping automatic detection of others. See the [Authentication Types Reference](./auth-types-reference.md) for all valid values, required parameters, and usage examples. | `DATABRICKS_AUTH_TYPE` | | `http_timeout_seconds` | _(Integer)_ Number of seconds for HTTP timeout. Default is _60_. | _(None)_ | | `retry_timeout_seconds` | _(Integer)_ Number of seconds to keep retrying HTTP requests. Default is _300 (5 minutes)_. | _(None)_ | | `debug_truncate_bytes` | _(Integer)_ Truncate JSON fields in debug logs above this limit. Default is 96. | `DATABRICKS_DEBUG_TRUNCATE_BYTES` | | `debug_headers` | _(Boolean)_ `true` to debug HTTP headers of requests made by the application. Default is `false`, as headers contain sensitive data, such as access tokens. | `DATABRICKS_DEBUG_HEADERS` | | `rate_limit` | _(Integer)_ Maximum number of requests per second made to Databricks REST API. | `DATABRICKS_RATE_LIMIT` | -For a complete reference of all authentication types including required parameters, environment variables, and code examples, see the **[Authentication Types Reference](./auth-types-reference.md)**. - For example, to turn on debug HTTP headers: ```python