-
Notifications
You must be signed in to change notification settings - Fork 191
Fix missing git info for bundles deployed from the new type of in-workspace Git folder #5709
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
ilyakuz-db
wants to merge
11
commits into
main
Choose a base branch
from
fix-git-info-new-git-folders
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
b614a66
Fix missing git info for bundles deployed from in-workspace Git folders
ilyakuz-db 804d517
Add PR link to changelog
ilyakuz-db 188cd6d
Merge remote-tracking branch 'origin/main' into pr5709-merge
ilyakuz-db 8c2b25a
acc: Add git-info-from-git-folder acceptance test
ilyakuz-db 0788d20
Cover all three in-workspace git folder types in info_test
ilyakuz-db a3721ef
Log Repos API fallback failure as a warning
ilyakuz-db 0c18e29
Merge remote-tracking branch 'origin/main' into pr5709-merge
ilyakuz-db ca49023
Clarify changelog: only the new workspace Git folder type was affected
ilyakuz-db fc58540
acc: Finalize git-info test for both local and DBR execution
ilyakuz-db ad956ca
acc: drop Ignore entry already inherited from bundle/test.toml
ilyakuz-db 3eabf97
acc: drop git-info acceptance test
ilyakuz-db File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,179 @@ | ||
| package git | ||
|
|
||
| import ( | ||
| "context" | ||
| "sync/atomic" | ||
| "testing" | ||
|
|
||
| "github.com/databricks/cli/libs/dbr" | ||
| "github.com/databricks/cli/libs/testserver" | ||
| "github.com/databricks/databricks-sdk-go" | ||
| "github.com/databricks/databricks-sdk-go/service/workspace" | ||
| "github.com/stretchr/testify/assert" | ||
| "github.com/stretchr/testify/require" | ||
| ) | ||
|
|
||
| const ( | ||
| // Bundle root passed to FetchRepositoryInfo: a subdirectory of the git folder. | ||
| testBundleRoot = "/Workspace/Users/test/bundle-examples/dabs_in_ws_bundle" | ||
| // Git folder path as get-status returns it (without the /Workspace prefix). | ||
| testGitFolderRaw = "/Users/test/bundle-examples" | ||
| // Expected worktree root after ensureWorkspacePrefix is applied. | ||
| testWorktreeRoot = "/Workspace/Users/test/bundle-examples" | ||
| testRepoID = int64(2884540697170475) | ||
| testOriginURL = "https://github.com/databricks/bundle-examples.git" | ||
| ) | ||
|
|
||
| func newTestWorkspaceClient(t *testing.T, server *testserver.Server) *databricks.WorkspaceClient { | ||
| t.Helper() | ||
| w, err := databricks.NewWorkspaceClient(&databricks.Config{ | ||
| Host: server.URL, | ||
| Token: "testtoken", | ||
| }) | ||
| require.NoError(t, err) | ||
| return w | ||
| } | ||
|
|
||
| // runtimeContext forces the in-workspace API branch of FetchRepositoryInfo | ||
| // without needing a real /databricks directory on the test host. | ||
| func runtimeContext(t *testing.T) context.Context { | ||
| return dbr.MockRuntime(t.Context(), dbr.Environment{IsDbr: true, Version: "15.4"}) | ||
| } | ||
|
|
||
| // FetchRepositoryInfo reads git provenance from the workspace get-status API when | ||
| // running on DBR. What get-status returns depends on the kind of in-workspace Git | ||
| // folder the bundle lives in (see the backend tree-node contract): | ||
| // | ||
| // - Classic Repos (/Repos/...) and workspace Git folders (/Workspace/...) are the | ||
| // same NodeType.Project backend and return full git info inline, so the Repos API | ||
| // is not consulted. | ||
| // - New in-workspace Git folders (git-in-dataplane) are a plain folder promoted to a | ||
| // repo by a .git child; get-status returns only id+path, so the missing | ||
| // branch/commit/url are recovered from the Repos API by id. | ||
| func TestFetchRepositoryInfoAPI(t *testing.T) { | ||
| fullGitInfo := map[string]any{ | ||
| "id": testRepoID, | ||
| "path": testGitFolderRaw, | ||
| "branch": "main", | ||
| "head_commit_id": "abc123", | ||
| "url": testOriginURL, | ||
| } | ||
| idOnlyGitInfo := map[string]any{ | ||
| "id": testRepoID, | ||
| "path": testGitFolderRaw, | ||
| } | ||
|
|
||
| tests := []struct { | ||
| name string | ||
| // gitInfo is the git_info object returned by get-status; nil means it is omitted. | ||
| gitInfo map[string]any | ||
| // repoResponse, when set, is returned by GET /api/2.0/repos/{repo_id}. | ||
| repoResponse *workspace.GetRepoResponse | ||
| // repoStatusCode, when set and repoResponse is nil, is the Repos API error status. | ||
| repoStatusCode int | ||
|
|
||
| wantReposCalled bool | ||
| wantBranch string | ||
| wantCommit string | ||
| wantOriginURL string | ||
| wantWorktreeRoot string | ||
| }{ | ||
| { | ||
| name: "classic repo returns full git info inline", | ||
| gitInfo: fullGitInfo, | ||
| wantReposCalled: false, | ||
| wantBranch: "main", | ||
| wantCommit: "abc123", | ||
| wantOriginURL: testOriginURL, | ||
| wantWorktreeRoot: testWorktreeRoot, | ||
| }, | ||
| { | ||
| // Same NodeType.Project backend as a classic Repo, just relocated under | ||
| // /Workspace; get-status returns the same full git info inline. | ||
| name: "workspace git folder returns full git info inline", | ||
| gitInfo: fullGitInfo, | ||
| wantReposCalled: false, | ||
| wantBranch: "main", | ||
| wantCommit: "abc123", | ||
| wantOriginURL: testOriginURL, | ||
| wantWorktreeRoot: testWorktreeRoot, | ||
| }, | ||
| { | ||
| name: "git-in-dataplane folder falls back to Repos API", | ||
| gitInfo: idOnlyGitInfo, | ||
| repoResponse: &workspace.GetRepoResponse{ | ||
| Id: testRepoID, | ||
| Branch: "main", | ||
| HeadCommitId: "d53214abc", | ||
| Url: testOriginURL, | ||
| Provider: "gitHub", | ||
| Path: testGitFolderRaw, | ||
| }, | ||
| wantReposCalled: true, | ||
| wantBranch: "main", | ||
| wantCommit: "d53214abc", | ||
| wantOriginURL: testOriginURL, | ||
| wantWorktreeRoot: testWorktreeRoot, | ||
| }, | ||
| { | ||
| // A remoteless dataplane folder has no origin URL even from the Repos API, | ||
| // which back-fills branch/commit live but leaves url empty. | ||
| name: "remoteless dataplane folder recovers branch and commit but not url", | ||
| gitInfo: idOnlyGitInfo, | ||
| repoResponse: &workspace.GetRepoResponse{ | ||
| Id: testRepoID, | ||
| Branch: "main", | ||
| HeadCommitId: "d53214abc", | ||
| Url: "", | ||
| Path: testGitFolderRaw, | ||
| }, | ||
| wantReposCalled: true, | ||
| wantBranch: "main", | ||
| wantCommit: "d53214abc", | ||
| wantOriginURL: "", | ||
| wantWorktreeRoot: testWorktreeRoot, | ||
| }, | ||
| { | ||
| // A failed Repos lookup must not fail the deploy: the worktree root stays | ||
| // set and provenance stays empty, with no error. | ||
| name: "Repos lookup failure degrades gracefully", | ||
| gitInfo: idOnlyGitInfo, | ||
| repoStatusCode: 404, | ||
| wantReposCalled: true, | ||
| wantBranch: "", | ||
| wantCommit: "", | ||
| wantOriginURL: "", | ||
| wantWorktreeRoot: testWorktreeRoot, | ||
| }, | ||
| } | ||
|
|
||
| for _, tt := range tests { | ||
| t.Run(tt.name, func(t *testing.T) { | ||
| server := testserver.New(t) | ||
| var reposCalled atomic.Bool | ||
|
|
||
| server.Handle("GET", "/api/2.0/workspace/get-status", func(_ testserver.Request) any { | ||
| body := map[string]any{} | ||
| if tt.gitInfo != nil { | ||
| body["git_info"] = tt.gitInfo | ||
| } | ||
| return testserver.Response{Body: body} | ||
| }) | ||
| server.Handle("GET", "/api/2.0/repos/{repo_id}", func(_ testserver.Request) any { | ||
| reposCalled.Store(true) | ||
| if tt.repoResponse != nil { | ||
| return testserver.Response{Body: *tt.repoResponse} | ||
| } | ||
| return testserver.Response{StatusCode: tt.repoStatusCode, Body: map[string]string{"message": "not found"}} | ||
| }) | ||
|
|
||
| info, err := FetchRepositoryInfo(runtimeContext(t), testBundleRoot, newTestWorkspaceClient(t, server)) | ||
| require.NoError(t, err) | ||
| assert.Equal(t, tt.wantReposCalled, reposCalled.Load()) | ||
| assert.Equal(t, tt.wantBranch, info.CurrentBranch) | ||
| assert.Equal(t, tt.wantCommit, info.LatestCommit) | ||
| assert.Equal(t, tt.wantOriginURL, info.OriginURL) | ||
| assert.Equal(t, tt.wantWorktreeRoot, info.WorktreeRoot) | ||
| }) | ||
| } | ||
| } |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what happens in the new Git folders case? Does the CLI successfully read GitInfo from the local
.git? Can we add some acceptance test coverage for all three cases (new git, classic repo, and vanilla). You can mock the API endpoints.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On DBR this code never read locally, see this guard
cli/libs/git/info.go
Line 56 in 804d517
That also make it harder to test with acceptance tests as we can't properly mock the environment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added some unit tests
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With new git folders API returns only gitInfo.ID
See example
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There must be a way to read the commit ID? From the
.gitfolders directly maybe as we do locally?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can be a followup to this PR - but capturing Git metadata from Git folders is an important usecase? Since DABs in the workspace is supported there?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's important — and this PR already covers it, so it doesn't need to be a followup. To clarify the scope:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll check if it's possible to add acceptance test using
RunOnDbr