Skip to content

Commit ad5a861

Browse files
Divyansh-dbDivyansh-db
authored
Merge branch 'main' into fix/runtime-spark-connect-import
Signed-off-by: Divyansh Vijayvergia <171924202+Divyansh-db@users.noreply.github.com>
2 parents 655a577 + c05cc6f commit ad5a861

16 files changed

Lines changed: 208 additions & 82 deletions

‎.codegen/_openapi_sha‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
805d88b51faa37301c51d1848da2db5a2e6ed921
1+
55d22d7e7879e6be815cfe8e3e4cf7130f822c94

‎.release_metadata.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
{
2-
"timestamp": "2026-06-08 09:40:21+0000"
2+
"timestamp": "2026-06-10 06:11:05+0000"
33
}

‎CHANGELOG.md‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,42 @@
11
# Version changelog
22

3+
## Release v0.116.0 (2026-06-10)
4+
5+
### API Changes
6+
* Add `databricks.sdk.service.aisearch` and `databricks.sdk.service.bundledeployments` packages.
7+
* Add [w.ai_search](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/aisearch/ai_search.html) workspace-level service.
8+
* Add [w.bundle_deployments](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/bundledeployments/bundle_deployments.html) workspace-level service.
9+
* Add `running_instances` field for `databricks.sdk.service.apps.ApplicationStatus`.
10+
* Add `custom_max_retention_hours` field for `databricks.sdk.service.catalog.CatalogInfo`.
11+
* Add `environment_settings` field for `databricks.sdk.service.catalog.ConnectionInfo`.
12+
* Add `custom_max_retention_hours` field for `databricks.sdk.service.catalog.CreateCatalog`.
13+
* Add `environment_settings` field for `databricks.sdk.service.catalog.CreateConnection`.
14+
* Add `custom_max_retention_hours` field for `databricks.sdk.service.catalog.CreateSchema`.
15+
* Add `custom_max_retention_hours` field for `databricks.sdk.service.catalog.SchemaInfo`.
16+
* Add `custom_max_retention_hours` field for `databricks.sdk.service.catalog.UpdateCatalog`.
17+
* Add `environment_settings` field for `databricks.sdk.service.catalog.UpdateConnection`.
18+
* Add `custom_max_retention_hours` field for `databricks.sdk.service.catalog.UpdateSchema`.
19+
* Add `stream_source` field for `databricks.sdk.service.ml.DataSource`.
20+
* Add `ingestion_config` field for `databricks.sdk.service.ml.KafkaConfig`.
21+
* Add `clustering_columns`, `enable_auto_clustering` and `table_properties` fields for `databricks.sdk.service.pipelines.TableSpecificConfig`.
22+
* Add `branch_id` field for `databricks.sdk.service.postgres.Branch`.
23+
* Add `catalog_id` field for `databricks.sdk.service.postgres.Catalog`.
24+
* Add `database_id` field for `databricks.sdk.service.postgres.Database`.
25+
* Add `endpoint_id` field for `databricks.sdk.service.postgres.Endpoint`.
26+
* Add `project_id` field for `databricks.sdk.service.postgres.Project`.
27+
* Add `role_id` field for `databricks.sdk.service.postgres.Role`.
28+
* Add `synced_table_id` field for `databricks.sdk.service.postgres.SyncedTable`.
29+
* Add `allowed_databricks_destinations` field for `databricks.sdk.service.settings.EgressNetworkPolicyNetworkAccessPolicy`.
30+
* Add `facets`, `query_columns` and `sort_columns` fields for `databricks.sdk.service.vectorsearch.QueryVectorIndexRequest`.
31+
* Add `facet_result` field for `databricks.sdk.service.vectorsearch.QueryVectorIndexResponse`.
32+
* Add `facet_column_count` and `facet_columns` fields for `databricks.sdk.service.vectorsearch.ResultManifest`.
33+
* Add `dangerously_force_discard_all` field for `databricks.sdk.service.workspace.UpdateRepoRequest`.
34+
* [Breaking] Remove `databricks.sdk.service.bundle` package.
35+
* [Breaking] Remove [w.bundle](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/bundle/bundle.html) workspace-level service.
36+
* Add `accelerated_sync` field for `databricks.sdk.service.database.SyncedTableSpec`.
37+
* Add `accelerated_sync` field for `databricks.sdk.service.postgres.SyncedTableSyncedTableSpec`.
38+
39+
340
## Release v0.115.0 (2026-06-08)
441

542
### New Features and Improvements

‎NEXT_CHANGELOG.md‎

Lines changed: 2 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# NEXT CHANGELOG
22

3-
## Release v0.116.0
3+
## Release v0.117.0
44

55
### New Features and Improvements
66

@@ -9,6 +9,7 @@
99
### Bug Fixes
1010

1111
* Fall back to the remote runtime implementation when the legacy user namespace cannot be materialized. On Spark Connect runtimes (e.g. shared-access-mode clusters), importing `databricks.sdk.runtime` — which happens when constructing a `WorkspaceClient` on such a cluster — tried to build a legacy `SparkContext` and raised `CONTEXT_UNAVAILABLE_FOR_REMOTE_CLIENT` at import time. It now logs a warning and falls back to the Spark Connect-compatible remote implementation instead of crashing.
12+
* Cache tokens minted by `DatabricksOidcTokenSource` (Workload Identity Federation / account-wide token federation). Previously a fresh `/oidc/v1/token` exchange was performed on every authenticated API call, adding latency, amplifying transient federation-policy errors, and hitting OIDC token-endpoint rate limits. The token source now reuses the cached token until it is stale or expired, fetching a fresh ID token on each refresh to support rotation.
1213

1314
### Documentation
1415

@@ -17,33 +18,3 @@
1718
### Internal Changes
1819

1920
### API Changes
20-
* Add `databricks.sdk.service.aisearch` and `databricks.sdk.service.bundledeployments` packages.
21-
* Add [w.ai_search](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/aisearch/ai_search.html) workspace-level service.
22-
* Add [w.bundle_deployments](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/bundledeployments/bundle_deployments.html) workspace-level service.
23-
* Add `running_instances` field for `databricks.sdk.service.apps.ApplicationStatus`.
24-
* Add `custom_max_retention_hours` field for `databricks.sdk.service.catalog.CatalogInfo`.
25-
* Add `environment_settings` field for `databricks.sdk.service.catalog.ConnectionInfo`.
26-
* Add `custom_max_retention_hours` field for `databricks.sdk.service.catalog.CreateCatalog`.
27-
* Add `environment_settings` field for `databricks.sdk.service.catalog.CreateConnection`.
28-
* Add `custom_max_retention_hours` field for `databricks.sdk.service.catalog.CreateSchema`.
29-
* Add `custom_max_retention_hours` field for `databricks.sdk.service.catalog.SchemaInfo`.
30-
* Add `custom_max_retention_hours` field for `databricks.sdk.service.catalog.UpdateCatalog`.
31-
* Add `environment_settings` field for `databricks.sdk.service.catalog.UpdateConnection`.
32-
* Add `custom_max_retention_hours` field for `databricks.sdk.service.catalog.UpdateSchema`.
33-
* Add `stream_source` field for `databricks.sdk.service.ml.DataSource`.
34-
* Add `ingestion_config` field for `databricks.sdk.service.ml.KafkaConfig`.
35-
* Add `clustering_columns`, `enable_auto_clustering` and `table_properties` fields for `databricks.sdk.service.pipelines.TableSpecificConfig`.
36-
* Add `branch_id` field for `databricks.sdk.service.postgres.Branch`.
37-
* Add `catalog_id` field for `databricks.sdk.service.postgres.Catalog`.
38-
* Add `database_id` field for `databricks.sdk.service.postgres.Database`.
39-
* Add `endpoint_id` field for `databricks.sdk.service.postgres.Endpoint`.
40-
* Add `project_id` field for `databricks.sdk.service.postgres.Project`.
41-
* Add `role_id` field for `databricks.sdk.service.postgres.Role`.
42-
* Add `synced_table_id` field for `databricks.sdk.service.postgres.SyncedTable`.
43-
* Add `allowed_databricks_destinations` field for `databricks.sdk.service.settings.EgressNetworkPolicyNetworkAccessPolicy`.
44-
* Add `facets`, `query_columns` and `sort_columns` fields for `databricks.sdk.service.vectorsearch.QueryVectorIndexRequest`.
45-
* Add `facet_result` field for `databricks.sdk.service.vectorsearch.QueryVectorIndexResponse`.
46-
* Add `facet_column_count` and `facet_columns` fields for `databricks.sdk.service.vectorsearch.ResultManifest`.
47-
* Add `dangerously_force_discard_all` field for `databricks.sdk.service.workspace.UpdateRepoRequest`.
48-
* [Breaking] Remove `databricks.sdk.service.bundle` package.
49-
* [Breaking] Remove [w.bundle](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/bundle/bundle.html) workspace-level service.

‎databricks/sdk/oidc.py‎

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -123,9 +123,15 @@ def id_token(self) -> IdToken:
123123
return IdToken(jwt=token)
124124

125125

126-
class DatabricksOidcTokenSource(oauth.TokenSource):
126+
class DatabricksOidcTokenSource(oauth.Refreshable):
127127
"""A TokenSource which exchanges a token using Workload Identity Federation.
128128
129+
The exchanged token is cached and reused across calls. It is only refreshed
130+
when it is stale or expired, mirroring the M2M/PAT auth flows. This avoids
131+
minting a fresh token on every authenticated API call. Each refresh fetches a
132+
fresh ID token from the ``id_token_source``, so short-lived (rotating) ID
133+
tokens are handled transparently.
134+
129135
Parameters
130136
----------
131137
host : str
@@ -164,15 +170,20 @@ def __init__(
164170
self._client_id = client_id
165171
self._account_id = account_id
166172
self._audience = audience
167-
self._disable_async = disable_async
168173
self._scopes = scopes
174+
# Refreshable.__init__ stores disable_async as self._disable_async, which
175+
# _exchange_id_token reads — no need to duplicate it here.
176+
super().__init__(disable_async=disable_async)
177+
178+
def refresh(self) -> oauth.Token:
179+
"""Mint a fresh token by exchanging the ID token.
169180
170-
def token(self) -> oauth.Token:
171-
"""Get a token by exchanging the ID token.
181+
Called by the base :class:`oauth.Refreshable` only when the cached token
182+
is missing, stale, or expired. The result is cached by the base class.
172183
173184
Returns
174185
-------
175-
dict
186+
oauth.Token
176187
The exchanged token.
177188
178189
Raises

‎databricks/sdk/service/database.py‎

Lines changed: 9 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎databricks/sdk/service/postgres.py‎

Lines changed: 10 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎databricks/sdk/version.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
__version__ = "0.115.0"
1+
__version__ = "0.116.0"

‎docs/workspace/catalog/catalogs.rst‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,10 +24,10 @@
2424
2525
w = WorkspaceClient()
2626
27-
new_catalog = w.catalogs.create(name=f"sdk-{time.time_ns()}")
27+
created_catalog = w.catalogs.create(name=f"sdk-{time.time_ns()}")
2828
2929
# cleanup
30-
w.catalogs.delete(name=new_catalog.name, force=True)
30+
w.catalogs.delete(name=created_catalog.name, force=True)
3131
3232
Creates a new catalog instance in the parent metastore if the caller is a metastore admin or has the
3333
**CREATE_CATALOG** privilege.

‎docs/workspace/catalog/external_locations.rst‎

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -115,20 +115,20 @@
115115
116116
credential = w.storage_credentials.create(
117117
name=f"sdk-{time.time_ns()}",
118-
aws_iam_role=catalog.AwsIamRole(role_arn=os.environ["TEST_METASTORE_DATA_ACCESS_ARN"]),
118+
aws_iam_role=catalog.AwsIamRoleRequest(role_arn=os.environ["TEST_METASTORE_DATA_ACCESS_ARN"]),
119119
)
120120
121121
created = w.external_locations.create(
122122
name=f"sdk-{time.time_ns()}",
123123
credential_name=credential.name,
124-
url=f's3://{os.environ["TEST_BUCKET"]}/sdk-{time.time_ns()}',
124+
url="s3://%s/%s" % (os.environ["TEST_BUCKET"], f"sdk-{time.time_ns()}"),
125125
)
126126
127-
_ = w.external_locations.get(get=created.name)
127+
_ = w.external_locations.get(name=created.name)
128128
129129
# cleanup
130-
w.storage_credentials.delete(delete=credential.name)
131-
w.external_locations.delete(delete=created.name)
130+
w.storage_credentials.delete(name=credential.name)
131+
w.external_locations.delete(name=created.name)
132132
133133
Gets an external location from the metastore. The caller must be either a metastore admin, the owner
134134
of the external location, or a user that has some privilege on the external location.
@@ -200,24 +200,24 @@
200200
201201
credential = w.storage_credentials.create(
202202
name=f"sdk-{time.time_ns()}",
203-
aws_iam_role=catalog.AwsIamRole(role_arn=os.environ["TEST_METASTORE_DATA_ACCESS_ARN"]),
203+
aws_iam_role=catalog.AwsIamRoleRequest(role_arn=os.environ["TEST_METASTORE_DATA_ACCESS_ARN"]),
204204
)
205205
206206
created = w.external_locations.create(
207207
name=f"sdk-{time.time_ns()}",
208208
credential_name=credential.name,
209-
url=f's3://{os.environ["TEST_BUCKET"]}/sdk-{time.time_ns()}',
209+
url="s3://%s/%s" % (os.environ["TEST_BUCKET"], f"sdk-{time.time_ns()}"),
210210
)
211211
212212
_ = w.external_locations.update(
213213
name=created.name,
214214
credential_name=credential.name,
215-
url=f's3://{os.environ["TEST_BUCKET"]}/sdk-{time.time_ns()}',
215+
url="s3://%s/%s" % (os.environ["TEST_BUCKET"], f"sdk-{time.time_ns()}"),
216216
)
217217
218218
# cleanup
219-
w.storage_credentials.delete(delete=credential.name)
220-
w.external_locations.delete(delete=created.name)
219+
w.storage_credentials.delete(name=credential.name)
220+
w.external_locations.delete(name=created.name)
221221
222222
Updates an external location in the metastore. The caller must be the owner of the external location,
223223
or be a metastore admin. In the second case, the admin can only update the name of the external

0 commit comments

Comments
 (0)