Fix missing git info for bundles deployed from the new type of in-workspace Git folder#5709
Open
ilyakuz-db wants to merge 11 commits into
Open
Fix missing git info for bundles deployed from the new type of in-workspace Git folder#5709ilyakuz-db wants to merge 11 commits into
ilyakuz-db wants to merge 11 commits into
Conversation
Co-authored-by: Isaac
Contributor
Approval status: pending
|
Co-authored-by: Isaac
ilyakuz-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
ilyakuz-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
ilyakuz-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
ilyakuz-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
Collaborator
Integration test reportCommit: ad956ca
23 interesting tests: 13 SKIP, 7 KNOWN, 3 RECOVERED
Top 4 slowest tests (at least 2 minutes):
|
| result.CurrentBranch = gi.Branch | ||
| result.WorktreeRoot = fixedPath | ||
| } else { | ||
| if gi == nil { |
Contributor
There was a problem hiding this comment.
what happens in the new Git folders case? Does the CLI successfully read GitInfo from the local .git? Can we add some acceptance test coverage for all three cases (new git, classic repo, and vanilla). You can mock the API endpoints.
Contributor
Author
There was a problem hiding this comment.
On DBR this code never read locally, see this guard
Line 56 in 804d517
That also make it harder to test with acceptance tests as we can't properly mock the environment
Contributor
Author
There was a problem hiding this comment.
I added some unit tests
Contributor
Author
There was a problem hiding this comment.
With new git folders API returns only gitInfo.ID
See example
🔴 workspace/get-status doesn't return git provenance for git-in-data-plane folders
Same repo (github.com/databricks/bundle-examples), same path, two workspaces — different
result.
① AWS staging — git-in-DP folder (object_type: DIRECTORY)
Request:
GET /api/2.0/workspace/get-status
?path=/Workspace/Users/<user_name>/bundle-examples
&return_git_info=true
Response:
{
"object_type": "DIRECTORY",
"git_info": {
"id": 2884540697170475,
"path": "/Users/<user_name>/bundle-examples"
}
}
❌ missing branch, head_commit_id, url, provider
…yet the data does exist — same id, Repos API:
GET /api/2.0/repos/2884540697170475
{
"branch": "main",
"head_commit_id": "d53214e177cd372afa03bfc044be9bb94103ba9a",
"url": "https://github.com/databricks/bundle-examples.git",
"provider": "gitHub"
}
✅ full provenance
Contributor
There was a problem hiding this comment.
There must be a way to read the commit ID? From the .git folders directly maybe as we do locally?
Contributor
There was a problem hiding this comment.
This can be a followup to this PR - but capturing Git metadata from Git folders is an important usecase? Since DABs in the workspace is supported there?
Contributor
Author
There was a problem hiding this comment.
This can be a followup to this PR - but capturing Git metadata from Git folders is an important usecase? Since DABs in the workspace is supported there?
It's important — and this PR already covers it, so it doesn't need to be a followup. To clarify the scope:
- This only affects bundles deployed from the workspace. Local and CI/CD deploys are unaffected — they read .git directly
- We already supported two object types: classic Repos and Git folders. Both return the full git_info inline from Workspace get-status API, so they work correctly today
- The gap is a new Private Preview feature, Git in Dataplane. Objects created with that flag don't return a full git_info from get-status — only git_info.id (the repo id) — and there's no .git directory to read on the Dataplane either
- This PR handles that case: it takes the id and calls the Repos API (GET /api/2.0/repos/{id}) to recover branch/commit/url. So Git metadata ends up captured for all of them
Contributor
Author
There was a problem hiding this comment.
I'll check if it's possible to add acceptance test using RunOnDbr
# Conflicts: # NEXT_CHANGELOG.md
Verify bundle git provenance (origin_url, branch) is recorded when the bundle lives in a Git folder. The provenance is read through two paths in libs/git/info.go: locally from the on-disk .git directory, and on DBR (RunsOnDbr) from the workspace get-status API by running the bundle from a Git folder created with `repos create`. Both paths assert identical output; commit is excluded as non-deterministic.
Make the API-path unit test table-driven and enumerate the three folder types FetchRepositoryInfo can encounter: classic Repo and workspace Git folder (full git info inline, no Repos API call) and git-in-dataplane folder (id+path only, recovered via the Repos API), plus a remoteless dataplane folder (branch/commit recovered, url stays empty) and graceful degradation on a Repos lookup failure.
FetchRepositoryInfo contract says metadata-read errors are logged as warnings, and the .git path already does so; at debug level a deploy that records empty git provenance gives the user no clue why.
# Conflicts: # NEXT_CHANGELOG.md # internal/testarchive/ruff.go
Rename to bundle/git-info and assert environment-specific expectations with constant output: locally the bundle root is a real git repo and full provenance is recorded from .git; on DBR no git source can exist in the bundle root (git cannot operate on the workspace FUSE mount, and Repos-API-created Git folders do not materialize on it within the session), so the get-status API path must degrade to empty provenance without failing validate.
ilyakuz-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
ilyakuz-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
ilyakuz-db
changed the title
Neither environment can exercise the id-only Repos-API fallback end to end (Repos-API-created Git folders do not materialize on the serverless FUSE mount, git cannot run on it, and local runs never take the DBR API path), so the fallback is covered by unit tests in libs/git/info_test.go. The on-disk .git path is already covered by existing acceptance tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
The new type of in-workspace Git folder returns only
idandpathfromget-status?return_git_info=true(no url/branch/commit), so bundles deployed from one recorded empty git provenance. Classic Repos and existing workspace Git folders return full git info inline and were not affected.When the origin URL is missing but an
idis present, the CLI now fetches url/branch/commit fromRepos.GetByRepoId. Classic Repos skip the extra call. A failed lookup logs a warning and degrades to partial info instead of failing the deploy. (Reading on-disk.gitis not an option: workspace Git folders don't expose a usable.giton the compute.)Tests
libs/git/info_test.go: table-driven over all three folder types — classic Repo and workspace Git folder (inline info, Repos API not called), new git folder (id-only → recovered via Repos API), remoteless folder (branch/commit recovered, url stays empty), and Repos lookup failure (graceful degradation).No acceptance test: the new folder type cannot be provisioned from the test harness (Repos-API-created Git folders don't appear on the serverless FUSE mount and
git initcannot run on it), and local runs never take the DBR API path. The on-disk.gitpath is already covered by existing acceptance tests.This PR was written by Claude Code.