Migrate from databricks-vectorsearch to databricks-ai-search (#437)

## Summary

`databricks-vectorsearch` is deprecated and has been renamed to `databricks-ai-search`. Follow-up to #435.

When tests run, `databricks-vectorsearch` now emits:

> DeprecationWarning: databricks-vectorsearch is deprecated and has been renamed to databricks-ai-search. Imports under 'databricks.vector_search.*' will continue to work as a thin re-export of 'databricks.ai_search.*', but new code should switch to 'pip install databricks-ai-search' and 'from databricks.ai_search.* import ...'.

This PR follows that recommendation literally.

## Changes

- **`pyproject.toml` × 4** (root, `integrations/openai`, `integrations/langchain`, `integrations/llamaindex`): replace `databricks-vectorsearch>=*` with `databricks-ai-search>=0.73`.
- **12 Python files** under `src/`, `integrations/*/src/`, `integrations/*/tests/`, and `tests/`: every `from databricks.vector_search.* import ...` → `from databricks.ai_search.* import ...`. Includes `mock.patch` target strings and the `ImportError` fallback message in `databricks_langchain/vectorstores.py`.
- Symbol names are unchanged. `databricks-ai-search==0.73` exposes back-compat aliases for the names the bridge uses:
  - `databricks/ai_search/client.py:1090`: `VectorSearchClient = AISearchClient`
  - `databricks/ai_search/index.py:682`: `VectorSearchIndex = AISearchIndex`
  - `databricks/ai_search/exceptions.py:47`: `VectorSearchException = AISearchException`
  - `Reranker`, `DatabricksReranker`, `CredentialStrategy` exist under the same submodule paths.

Net diff: **+54 / −54 across 16 files** (pure rename).

## Validation

- **Package landscape**: in a fresh venv with the migrated branch installed editable, only `databricks-ai-search==0.73` is present — `databricks-vectorsearch` is no longer pulled at all.
- **Smoke**: `from databricks_openai.vector_search_retriever_tool import VectorSearchRetrieverTool` resolves `VectorSearchIndex` to `databricks.ai_search.index` even with the `databricks.vector_search` package completely absent.
- **Tests**:
  - 53/53 openai unit tests pass.
  - 258/258 langchain unit tests pass (4 pre-existing skips).
  - `ruff check` + `ruff format --check` clean.

## Companion notes

- This complements #435 (the `VectorSearchIndex` canonical-path fix). With both changes shipped, the bridge no longer touches the `databricks.vector_search.*` namespace at all and stops triggering the deprecation warning.
- The langchain pyproject's `databricks-openai = { path = "../openai", editable = true }` source pin (added in #435) is preserved — still needed until the next `databricks-openai` release containing the canonical-path fix ships to PyPI; can be removed in a follow-up cleanup after that release.
- Out of scope: the cross-version matrix (`openai_test (3.10, v0.x)` / `langchain_test (3.10, v0.x)`) still tests *historical* tags whose frozen source imports `databricks.vector_search.client.VectorSearchIndex`. Those tags cannot be retroactively migrated — this PR doesn't try to. Tracked separately for Dhruv.

## Test plan

- [ ] CI green on this branch
- [ ] After merge + release: any consumers of `databricks_ai_bridge`, `databricks_langchain`, `databricks_openai`, or `databricks_llamaindex` no longer see the upstream `DeprecationWarning` about `databricks-vectorsearch`

Author: jennsun

integrations/dspy/src/databricks_dspy/retrievers/databricks_rm.py

@@ -39,7 +39,7 @@ class DatabricksRM(dspy.Retrieve):
 
         .. code-block:: python
 
-            from databricks.vector_search.client import VectorSearchClient
+            from databricks.ai_search.client import VectorSearchClient
             from databricks.sdk import WorkspaceClient
 
             # Create a Databricks workspace client

integrations/langchain/pyproject.toml

@@ -10,7 +10,7 @@ license = { text="Apache-2.0" }
 requires-python = ">=3.10"
 dependencies = [
     "langchain>=1.0.0",
-    "databricks-vectorsearch>=0.50",
+    "databricks-ai-search>=0.73",
     "databricks-ai-bridge>=0.19.0",
     "mlflow>=3.0.0",
     "pydantic>2.10.0",

integrations/langchain/src/databricks_langchain/vector_search_retriever_tool.py

@@ -1,7 +1,7 @@
 import json
 from typing import Type
 
-from databricks.vector_search.reranker import DatabricksReranker
+from databricks.ai_search.reranker import DatabricksReranker
 from databricks_ai_bridge.utils.vector_search import IndexDetails
 from databricks_ai_bridge.vector_search_retriever_tool import (
     FilterItem,
@@ -24,7 +24,7 @@ class VectorSearchRetrieverTool(BaseTool, VectorSearchRetrieverToolMixin):
     for building a retriever tool for agents.
 
     **Note**: Any additional keyword arguments passed to the constructor will be passed along to
-    `databricks.vector_search.client.VectorSearchIndex.similarity_search` when executing the tool. `See
+    `databricks.ai_search.index.VectorSearchIndex.similarity_search` when executing the tool. `See
     documentation <https://api-docs.databricks.com/python/vector-search/databricks.vector_search.html#databricks.vector_search.index.VectorSearchIndex.similarity_search>`_
     to see the full set of supported keyword arguments,
     e.g. `score_threshold`. Also, see documentation for

integrations/langchain/src/databricks_langchain/vectorstores.py

@@ -17,8 +17,8 @@
 )
 
 import numpy as np
+from databricks.ai_search.reranker import DatabricksReranker, Reranker
 from databricks.sdk import WorkspaceClient
-from databricks.vector_search.reranker import DatabricksReranker, Reranker
 from databricks_ai_bridge.utils.vector_search import (
     IndexDetails,
     RetrieverSchema,
@@ -67,7 +67,7 @@ class DatabricksVectorSearch(VectorStore):
                     and ``service_principal_client_secret`` to allow for
                     service principal authentication instead of personal access token authentication.
         reranker: Optional reranker to apply on the top results. Pass an instance of
-                    ``databricks.vector_search.reranker.DatabricksReranker`` with
+                    ``databricks.ai_search.reranker.DatabricksReranker`` with
                     ``columns_to_rerank=[...]``. The reranker reorders the initial results using
                     the specified text columns.
 
@@ -107,7 +107,7 @@ class DatabricksVectorSearch(VectorStore):
 
         .. code-block:: python
 
-            from databricks.vector_search.reranker import DatabricksReranker
+            from databricks.ai_search.reranker import DatabricksReranker
 
             vector_store = DatabricksVectorSearch(
                 index_name="<your-index-name>",
@@ -249,12 +249,12 @@ def __init__(
             )
 
         try:
-            from databricks.vector_search.client import VectorSearchClient
-            from databricks.vector_search.utils import CredentialStrategy
+            from databricks.ai_search.client import VectorSearchClient
+            from databricks.ai_search.utils import CredentialStrategy
         except ImportError as e:
             raise ImportError(
-                "Could not import databricks-vectorsearch python package. "
-                "Please install it with `pip install databricks-vectorsearch`."
+                "Could not import databricks-ai-search python package. "
+                "Please install it with `pip install databricks-ai-search`."
             ) from e
 
         if endpoint is not None:
@@ -422,7 +422,7 @@ def similarity_search(
             filter: Filters to apply to the query. Defaults to None.
             query_type: The type of this query. Supported values are "ANN" and "HYBRID".
             reranker: Allows reranking the results. Defaults to None.
-            kwargs: Additional keyword arguments to pass to `databricks.vector_search.client.VectorSearchIndex.similarity_search`. `See
+            kwargs: Additional keyword arguments to pass to `databricks.ai_search.index.VectorSearchIndex.similarity_search`. `See
                     documentation <https://api-docs.databricks.com/python/vector-search/databricks.vector_search.html#databricks.vector_search.index.VectorSearchIndex.similarity_search>`_
                     to see the full set of supported keyword arguments
 
@@ -464,7 +464,7 @@ def similarity_search_with_score(
             filter: Filters to apply to the query. Defaults to None.
             query_type: The type of this query. Supported values are "ANN" and "HYBRID".
             reranker: Allows reranking the results. Defaults to None.
-            kwargs: Additional keyword arguments to pass to `databricks.vector_search.client.VectorSearchIndex.similarity_search`. `See
+            kwargs: Additional keyword arguments to pass to `databricks.ai_search.index.VectorSearchIndex.similarity_search`. `See
                     documentation <https://api-docs.databricks.com/python/vector-search/databricks.vector_search.html#databricks.vector_search.index.VectorSearchIndex.similarity_search>`_
                     to see the full set of supported keyword arguments
 
@@ -538,7 +538,7 @@ def similarity_search_by_vector(
             filter: Filters to apply to the query. Defaults to None.
             query_type: The type of this query. Supported values are "ANN" and "HYBRID".
             reranker: Allows reranking the results. Defaults to None.
-            kwargs: Additional keyword arguments to pass to `databricks.vector_search.client.VectorSearchIndex.similarity_search`. `See
+            kwargs: Additional keyword arguments to pass to `databricks.ai_search.index.VectorSearchIndex.similarity_search`. `See
                     documentation <https://api-docs.databricks.com/python/vector-search/databricks.vector_search.html#databricks.vector_search.index.VectorSearchIndex.similarity_search>`_
                     to see the full set of supported keyword arguments
 
@@ -591,7 +591,7 @@ def similarity_search_by_vector_with_score(
             filter: Filters to apply to the query. Defaults to None.
             query_type: The type of this query. Supported values are "ANN" and "HYBRID".
             reranker: Allows reranking the results. Defaults to None.
-            kwargs: Additional keyword arguments to pass to `databricks.vector_search.client.VectorSearchIndex.similarity_search`. `See
+            kwargs: Additional keyword arguments to pass to `databricks.ai_search.index.VectorSearchIndex.similarity_search`. `See
                     documentation <https://api-docs.databricks.com/python/vector-search/databricks.vector_search.html#databricks.vector_search.index.VectorSearchIndex.similarity_search>`_
                     to see the full set of supported keyword arguments
 

integrations/langchain/tests/unit_tests/test_vector_search_retriever_tool.py

@@ -6,9 +6,9 @@
 
 import mlflow
 import pytest
+from databricks.ai_search.utils import CredentialStrategy
 from databricks.sdk import WorkspaceClient
 from databricks.sdk.credentials_provider import ModelServingUserCredentials
-from databricks.vector_search.utils import CredentialStrategy
 from databricks_ai_bridge.test_utils.vector_search import (  # noqa: F401
     ALL_INDEX_NAMES,
     DELTA_SYNC_INDEX,
@@ -301,7 +301,7 @@ def test_vector_search_client_model_serving_environment():
             host="testDogfod.com", credentials_strategy=ModelServingUserCredentials()
         )
 
-        with patch("databricks.vector_search.client.VectorSearchClient") as mockVSClient:
+        with patch("databricks.ai_search.client.VectorSearchClient") as mockVSClient:
             mock_instance = mockVSClient.return_value
             mock_instance.get_index.side_effect = _get_index
             with patch("databricks.sdk.service.serving.ServingEndpointsAPI.get", return_value=None):
@@ -317,7 +317,7 @@ def test_vector_search_client_model_serving_environment():
 
 
 def test_vector_search_client_non_model_serving_environment():
-    with patch("databricks.vector_search.client.VectorSearchClient") as mockVSClient:
+    with patch("databricks.ai_search.client.VectorSearchClient") as mockVSClient:
         mock_instance = mockVSClient.return_value
         mock_instance.get_index.side_effect = _get_index
         vsTool = VectorSearchRetrieverTool(
@@ -332,7 +332,7 @@ def test_vector_search_client_with_pat_workspace_client():
     w.config.auth_type = "pat"
     w.config.host = "https://testDogfod.com"
     w.config.token = "fakeToken"
-    with patch("databricks.vector_search.client.VectorSearchClient") as mockVSClient:
+    with patch("databricks.ai_search.client.VectorSearchClient") as mockVSClient:
         with patch("databricks.sdk.service.serving.ServingEndpointsAPI.get", return_value=None):
             mock_instance = mockVSClient.return_value
             mock_instance.get_index.side_effect = _get_index
@@ -356,7 +356,7 @@ def test_vector_search_client_with_sp_workspace_client():
     w.config.client_id = "fakeClientId"
     w.config.client_secret = "fakeClientSecret"
 
-    with patch("databricks.vector_search.client.VectorSearchClient") as mockVSClient:
+    with patch("databricks.ai_search.client.VectorSearchClient") as mockVSClient:
         with patch("databricks.sdk.service.serving.ServingEndpointsAPI.get", return_value=None):
             mock_instance = mockVSClient.return_value
             mock_instance.get_index.side_effect = _get_index

integrations/langchain/tests/unit_tests/test_vectorstores.py

@@ -4,8 +4,8 @@
 from unittest.mock import MagicMock, patch
 
 import pytest
-from databricks.vector_search.index import VectorSearchIndex
-from databricks.vector_search.reranker import DatabricksReranker, Reranker
+from databricks.ai_search.index import VectorSearchIndex
+from databricks.ai_search.reranker import DatabricksReranker, Reranker
 from databricks_ai_bridge.test_utils.vector_search import (
     ALL_INDEX_NAMES,
     DELTA_SYNC_INDEX,

integrations/llamaindex/pyproject.toml

@@ -9,7 +9,7 @@ readme = "README.md"
 license = { text="Apache-2.0" }
 requires-python = ">=3.10"
 dependencies = [
-    "databricks-vectorsearch>=0.40",
+    "databricks-ai-search>=0.73",
     "databricks-ai-bridge>=0.1.0",
     "llama-index>=0.12.0",
     "unitycatalog-llamaindex[databricks]>=0.2.0",

integrations/llamaindex/src/databricks_llamaindex/vector_search_retriever_tool.py

@@ -47,8 +47,8 @@ def __init__(self, **data):
         VectorSearchRetrieverToolMixin.__init__(self, **data)
 
         # Initialize private attributes
-        from databricks.vector_search.client import VectorSearchClient
-        from databricks.vector_search.utils import CredentialStrategy
+        from databricks.ai_search.client import VectorSearchClient
+        from databricks.ai_search.utils import CredentialStrategy
 
         credential_strategy = None
         if (

integrations/llamaindex/tests/unit_tests/test_vector_search_retriever_tool.py

@@ -4,11 +4,11 @@
 from unittest.mock import MagicMock, create_autospec, patch
 
 import pytest
+from databricks.ai_search.index import VectorSearchIndex
+from databricks.ai_search.reranker import DatabricksReranker, Reranker
+from databricks.ai_search.utils import CredentialStrategy
 from databricks.sdk import WorkspaceClient
 from databricks.sdk.credentials_provider import ModelServingUserCredentials
-from databricks.vector_search.index import VectorSearchIndex
-from databricks.vector_search.reranker import DatabricksReranker, Reranker
-from databricks.vector_search.utils import CredentialStrategy
 from databricks_ai_bridge.test_utils.vector_search import (  # noqa: F401
     ALL_INDEX_NAMES,
     DEFAULT_VECTOR_DIMENSION,
@@ -156,7 +156,7 @@ def test_vector_search_client_model_serving_environment():
             host="testDogfod.com", credentials_strategy=ModelServingUserCredentials()
         )
 
-        with patch("databricks.vector_search.client.VectorSearchClient") as mockVSClient:
+        with patch("databricks.ai_search.client.VectorSearchClient") as mockVSClient:
             with patch("databricks.sdk.service.serving.ServingEndpointsAPI.get", return_value=None):
                 vsTool = VectorSearchRetrieverTool(
                     index_name="catalog.schema.my_index_name",
@@ -172,7 +172,7 @@ def test_vector_search_client_model_serving_environment():
 
 
 def test_vector_search_client_non_model_serving_environment():
-    with patch("databricks.vector_search.client.VectorSearchClient") as mockVSClient:
+    with patch("databricks.ai_search.client.VectorSearchClient") as mockVSClient:
         vsTool = VectorSearchRetrieverTool(
             index_name="catalog.schema.my_index_name",
             text_column="abc",
@@ -182,7 +182,7 @@ def test_vector_search_client_non_model_serving_environment():
         mockVSClient.assert_called_once_with(disable_notice=True, credential_strategy=None)
 
     w = WorkspaceClient(host="testDogfod.com", token="fakeToken")
-    with patch("databricks.vector_search.client.VectorSearchClient") as mockVSClient:
+    with patch("databricks.ai_search.client.VectorSearchClient") as mockVSClient:
         with patch("databricks.sdk.service.serving.ServingEndpointsAPI.get", return_value=None):
             vsTool = VectorSearchRetrieverTool(
                 index_name="catalog.schema.my_index_name",

integrations/openai/pyproject.toml

@@ -9,7 +9,7 @@ readme = "README.md"
 license = { text="Apache-2.0" }
 requires-python = ">=3.10"
 dependencies = [
-    "databricks-vectorsearch>=0.50",
+    "databricks-ai-search>=0.73",
     "databricks-ai-bridge>=0.19.0",
     "openai>=1.99.9",
     "pydantic>2.10.0",