Skip to content

[ISSUE] Token Lifetime Exceeded Error when Using Unity Catalog Connections #1710

Description

@benswatling

Description
When using Unity Catalog connections, along with the API functionality to connect to SharePoint, we occasionally get this error:

databricks.sdk.errors.platform.Unauthenticated: unable to parse response. This is likely a bug in the Databricks SDK for Python or the underlying API. Please report this issue with the following debugging information to the SDK issue tracker at https://github.com/databricks/databricks-sdk-go/issues. Request log:```POST /api/2.0/external-function
<   "error": {
<     "code": "InvalidAuthenticationToken",
<     "innerError": {
<       "client-request-id": "2256907e-4df5-4b04-b6e3-481c768ac42e",
<       "date": "2026-06-05T19:15:19",
<       "request-id": "2256907e-4df5-4b04-b6e3-481c768ac42e"
<     },
<     "message": "Lifetime validation failed, the token is expired."
<   }

Reproduction
It seems to happen when I call these functions over the course of a few hours. It seems like the UC Catalog connection isn't refreshing the token.

Expected behavior
No errors to happen.

Is it a regression?
I do not believe so

Debug Logs
Stack Trace below. Context: custom functions are used to call the API.

databricks.sdk.errors.platform.Unauthenticated: Traceback (most recent call last):
  File "/Workspace/Users/c9633514-a365-47b6-a0a2-81b60f63c58d/benswatling/.bundle/data-dap-bronze-emr-bundle/dev/files/src/raw/MV/meta-sharepoint/sharepoint_file_metadata", cell 3, line 34, in sharepoint_file_metadata
    content = download_sharepoint_file(get_sharepoint_file_download_url(file_path, site_name))
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-940343c1-cb32-4425-8af0-7f35b58443c3/lib/python3.12/site-packages/ih_ea_python/ih_sharepoint.py", line 206, in get_sharepoint_file_download_url
    file_results = get_sharepoint_file(file_path, site_name, sharepoint_drive_name)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-940343c1-cb32-4425-8af0-7f35b58443c3/lib/python3.12/site-packages/ih_ea_python/ih_sharepoint.py", line 171, in get_sharepoint_file
    drive_id = get_sharepoint_drive_id(site_id, sharepoint_drive_name)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-940343c1-cb32-4425-8af0-7f35b58443c3/lib/python3.12/site-packages/ih_ea_python/ih_sharepoint.py", line 144, in get_sharepoint_drive_id
    return get_default_sharepoint_drive_id(site_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-940343c1-cb32-4425-8af0-7f35b58443c3/lib/python3.12/site-packages/ih_ea_python/ih_sharepoint.py", line 124, in get_default_sharepoint_drive_id
    r = WorkspaceClient().serving_endpoints.http_request(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-940343c1-cb32-4425-8af0-7f35b58443c3/lib/python3.12/site-packages/databricks/sdk/mixins/open_ai_client.py", line 169, in http_request
    res = self._api.do(
          ^^^^^^^^^^^^^

  File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-940343c1-cb32-4425-8af0-7f35b58443c3/lib/python3.12/site-packages/databricks/sdk/core.py", line 92, in do
    return self._api_client.do(
           ^^^^^^^^^^^^^^^^^^^^

  File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-940343c1-cb32-4425-8af0-7f35b58443c3/lib/python3.12/site-packages/databricks/sdk/_base_client.py", line 197, in do
    response = call(
               ^^^^^

  File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-940343c1-cb32-4425-8af0-7f35b58443c3/lib/python3.12/site-packages/databricks/sdk/retries.py", line 59, in wrapper
    raise err

  File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-940343c1-cb32-4425-8af0-7f35b58443c3/lib/python3.12/site-packages/databricks/sdk/retries.py", line 38, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^

  File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-940343c1-cb32-4425-8af0-7f35b58443c3/lib/python3.12/site-packages/databricks/sdk/_base_client.py", line 299, in _perform
    raise error from None

Other Information
Context of error: Serverless Declarative Pipeline, notebook tasks with the most recent databricks SDK.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions