|
| 1 | +# ES-1769884: SDK Recipient Activation API Investigation |
| 2 | + |
| 3 | +## Ticket Summary |
| 4 | + |
| 5 | +A customer reports that the Databricks Python SDK's `RecipientActivationAPI` for Delta Sharing does not work. The REST API (via `curl`) and browser-based download both work correctly, but the SDK implementation fails to download credential files. |
| 6 | + |
| 7 | +## Background: How Delta Sharing Activation Works |
| 8 | + |
| 9 | +Delta Sharing allows a data provider to share data with external recipients who may not have a Databricks workspace. |
| 10 | + |
| 11 | +### The Activation Flow |
| 12 | + |
| 13 | +1. **Provider** creates a recipient using `w.recipients.create(name=..., authentication_type=AuthenticationType.TOKEN)`. |
| 14 | +2. The server generates an `activation_url` and returns it as part of the `RecipientInfo` response. The field is described as: *"Full activation url to retrieve the access token."* |
| 15 | +3. **Provider** sends this `activation_url` to the recipient (e.g., via email). |
| 16 | +4. **Recipient** calls the activation URL (a simple unauthenticated GET request) to download a credential containing a `bearer_token` and `endpoint`. |
| 17 | +5. The **recipient** uses this credential with a Delta Sharing client (e.g., the `delta-sharing` Python library) to access the shared data. This token is **not** a Databricks OAuth/PAT token — it only works with the Delta Sharing protocol. |
| 18 | + |
| 19 | +**Important:** The activation URL is one-time use. Once redeemed, it cannot be used again. |
| 20 | + |
| 21 | +## Bugs Found |
| 22 | + |
| 23 | +### Bug 1: Authentication is always sent on a public endpoint |
| 24 | + |
| 25 | +The `retrieve_token` method (`sharing.py:2755`) is documented as *"a public API without any authentication"*, but the SDK **always injects auth headers**. |
| 26 | + |
| 27 | +**Root cause chain:** |
| 28 | + |
| 29 | +- `_base_client.py:86` — the HTTP session is initialized with `self._session.auth = self._authenticate`, which adds credentials to every request. |
| 30 | +- `retrieve_token` calls `self._api.do("GET", path, headers=headers)` without passing an `auth` override. |
| 31 | +- In `_base_client.py:286`, `self._session.request(..., auth=auth)` is called with `auth=None`. |
| 32 | +- Python's `requests` library treats `auth=None` as "use the session default" — so **auth headers are always added**. |
| 33 | + |
| 34 | +To make an unauthenticated request, the code would need to pass a no-op auth callable (e.g., `auth=lambda r: r`) instead of `None`. There is no mechanism for this currently. |
| 35 | + |
| 36 | +### Bug 2: The activation URL is incorrectly used as a path segment |
| 37 | + |
| 38 | +The `retrieve_token` method at `sharing.py:2773` constructs: |
| 39 | + |
| 40 | +```python |
| 41 | +f"/api/2.1/unity-catalog/public/data_sharing_activation/{activation_url}" |
| 42 | +``` |
| 43 | + |
| 44 | +Then in `core.py:86`, the configured workspace host is prepended: |
| 45 | + |
| 46 | +```python |
| 47 | +url = f"{self._cfg.host}{path}" |
| 48 | +``` |
| 49 | + |
| 50 | +The `activation_url` from `RecipientInfo` is documented as a **"full activation URL"** — a complete URL pointing to the provider's workspace. Embedding a full URL inside another API path produces a nonsensical URL. Even if the user passes just the token portion, the request is sent to the configured workspace host (the recipient's or whoever created the `WorkspaceClient`), not to the provider's host where the activation endpoint lives. |
| 51 | + |
| 52 | +### Bug 3: `get_activation_url_info` silently discards its response |
| 53 | + |
| 54 | +At `sharing.py:2751`, the return value of `self._api.do(...)` is never returned — the caller always gets `None`. |
| 55 | + |
| 56 | +## Fundamental Design Issue |
| 57 | + |
| 58 | +Beyond the implementation bugs, there is a deeper architectural problem: `RecipientActivationAPI` is attached to `WorkspaceClient`, which requires a Databricks host and credentials to instantiate. This creates a paradox: |
| 59 | + |
| 60 | +- The **provider** has no reason to call `retrieve_token` — they created the share and already have access to the data. |
| 61 | +- The **recipient** cannot use it — they would need a `WorkspaceClient` with credentials for the provider's workspace, which they don't have (and may not even be a Databricks customer). |
| 62 | + |
| 63 | +Neither persona can meaningfully use this API through the SDK as currently implemented. |
| 64 | + |
| 65 | +### Why `curl` / browser works but the SDK doesn't |
| 66 | + |
| 67 | +With `curl` or a browser, the customer: |
| 68 | +- Hits the provider's host directly (the URL is self-contained) |
| 69 | +- Sends no auth headers (it's a public endpoint) |
| 70 | + |
| 71 | +With the SDK: |
| 72 | +- The request goes to the wrong host (the configured workspace, not the provider's) |
| 73 | +- Auth headers are added that the provider's public endpoint doesn't expect |
| 74 | +- If the full activation URL is passed, it gets mangled into the path |
| 75 | + |
| 76 | +## Recommended Fix |
| 77 | + |
| 78 | +The `RecipientActivationAPI` should be a **standalone utility** that does not require a `WorkspaceClient`. The implementation should: |
| 79 | + |
| 80 | +1. Accept the full activation URL as-is (as returned by `RecipientInfo.activation_url`). |
| 81 | +2. Make a direct, unauthenticated GET request to that URL. |
| 82 | +3. Return the typed `RetrieveTokenResponse`. |
| 83 | + |
| 84 | +Example of the desired API: |
| 85 | + |
| 86 | +```python |
| 87 | +from databricks.sdk.service.sharing import RecipientActivation |
| 88 | + |
| 89 | +# No WorkspaceClient needed |
| 90 | +response = RecipientActivation.retrieve_token("https://provider.databricks.com/api/2.1/...") |
| 91 | +# response.bearer_token — Delta Sharing bearer token |
| 92 | +# response.endpoint — Provider's sharing endpoint |
| 93 | +``` |
| 94 | + |
| 95 | +Similarly, `get_activation_url_info` should follow the same pattern and should also return its response instead of discarding it. |
| 96 | + |
| 97 | +## References |
| 98 | + |
| 99 | +- Ticket: https://databricks.atlassian.net/browse/ES-1769884 |
| 100 | +- SDK docs: https://databricks-sdk-py.readthedocs.io/en/latest/workspace/sharing/recipient_activation.html |
| 101 | +- Relevant source files: |
| 102 | + - `databricks/sdk/service/sharing.py` — `RecipientActivationAPI` (line 2722), `RecipientInfo` (line 1404), `RetrieveTokenResponse` (line 1742) |
| 103 | + - `databricks/sdk/core.py` — `ApiClient.do()` (line 68) |
| 104 | + - `databricks/sdk/_base_client.py` — session auth setup (line 86), `_perform()` (line 274) |
0 commit comments