Skip to content

Vector Search follow-up#5367

Open
janniklasrose wants to merge 9 commits into
mainfrom
janniklasrose/vector-search-followup
Open

Vector Search follow-up#5367
janniklasrose wants to merge 9 commits into
mainfrom
janniklasrose/vector-search-followup

Conversation

@janniklasrose

@janniklasrose janniklasrose commented May 28, 2026

Copy link
Copy Markdown
Contributor

Changes

  • Use the typed VectorSearchEndpointPermission (backed by vectorsearch.VectorSearchEndpointPermissionLevel) for vector search endpoint permissions, now that the level enum exists upstream, instead of the generic Permission type. This gives the endpoint's permissions its own distinctly-named JSON schema entry.
  • Add an acceptance test covering vector search endpoint recreation on rename (recreate/name).
  • Have the testserver echo columns_to_sync/columns_to_index on VS index reads, and surface the index response's frozen endpoint_id so the fake mirrors the real API surface.
  • Refresh the Terraform provider SHA in the createIndexTimeout comment link.

Note on endpoint UUID handling

An earlier commit switched the index resource to read endpoint_id off the VectorIndex response instead of looking the endpoint up by name. That was reverted: endpoint_id is frozen at index-creation time, so it can't detect a same-name endpoint being deleted and recreated out of band. The CLI keeps the live GetEndpointByEndpointName lookup feeding the state-tracked endpoint_uuid, which is what drives orphan/recreate drift detection (see #5123). endpoint_id remains a REMOTE-only informational field.

Why

Follow-ups to #5123.

Tests

Unit tests in bundle/direct/dresources, plus acceptance tests under acceptance/bundle/resources/vector_search_* (recreate-on-rename, recreate-with-endpoint drift, refschema, schema). All pass locally.

This pull request and its description were written by Isaac, an AI coding agent.

Drop the separate VectorSearchEndpoints.GetEndpointByEndpointName call
from DoRead/DoCreate and read endpoint_id directly off the VectorIndex
response. This removes the race noted in #5123 where the index's
endpoint and the lookup-by-name could disagree if the endpoint was
deleted/recreated between the two calls.

Won't compile until the SDK gains EndpointId on VectorIndex.
The SDK exposes a dedicated permission level type for vector search
endpoints; switch VectorSearchEndpoint.Permissions to use it via the
PermissionT[L] generic, matching the typing already in place for jobs,
apps, model serving, etc. Regenerated jsonschema.json and added the
annotations placeholder for the new VectorSearchEndpointPermission.
Production RemapState already round-trips both fields from the GET
response, but the testserver was dropping them. Bring the fake in line
with the real backend so acceptance tests see the same shape on read.
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented May 28, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 8625837

Run: 28587645748

Env 🟨​KNOWN 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
🟨​ aws linux 7 3 13 230 1042 11:30
🟨​ aws windows 7 3 13 232 1040 7:33
💚​ aws-ucws linux 10 13 314 960 4:43
💚​ aws-ucws windows 10 13 316 958 4:14
💚​ azure linux 4 15 230 1041 4:12
💚​ azure windows 4 15 232 1039 3:34
💚​ azure-ucws linux 4 15 316 957 4:44
💚​ azure-ucws windows 4 15 318 955 3:39
💚​ gcp linux 4 15 229 1043 3:47
💚​ gcp windows 4 15 231 1041 3:28
23 interesting tests: 13 SKIP, 7 KNOWN, 3 RECOVERED
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🙈​ TestAccept/bundle/resources/postgres_branches/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/replace_existing 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/update_protected 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/without_branch_id 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_projects/update_display_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestFetchRepositoryInfoAPI_FromRepo 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestFetchRepositoryInfoAPI_FromRepo/root 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
💚​ TestFetchRepositoryInfoAPI_FromRepo/subdir 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 6 slowest tests (at least 2 minutes):
duration env testname
8:08 aws linux TestSecretsPutSecretStringValue
3:02 aws-ucws windows TestAccept
2:44 gcp windows TestAccept
2:41 azure windows TestAccept
2:37 azure-ucws windows TestAccept
2:31 aws windows TestSecretsPutSecretStringValue

@janniklasrose janniklasrose changed the title Vector Search follow-up [Blocked] Vector Search follow-up Jun 4, 2026
@janniklasrose janniklasrose changed the title [Blocked] Vector Search follow-up Vector Search follow-up Jun 18, 2026
…search-followup

# Conflicts:
#	bundle/internal/schema/annotations.yml
#	bundle/schema/jsonschema.json
#	bundle/schema/jsonschema_for_docs.json
@janniklasrose janniklasrose marked this pull request as ready for review July 2, 2026 11:50
@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Approval status: pending

/acceptance/bundle/ - needs approval

8 files changed
Suggested: @pietern
Also eligible: @andrewnester, @denik, @shreyas-goenka, @anton-107, @lennartkats-db

/bundle/ - needs approval

7 files changed
Suggested: @pietern
Also eligible: @andrewnester, @denik, @shreyas-goenka, @anton-107, @lennartkats-db

General files (require maintainer)

Files: libs/testserver/fake_workspace.go, libs/testserver/vector_search_indexes.go
Based on git history:

  • @pietern -- recent work in libs/testserver/, bundle/schema/, bundle/internal/validation/generated/

Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db) can approve all areas.
See OWNERS for ownership rules.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants