Skip to content

Include more details from SDK/API errors in the diagnostics #3379

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
denik merged 6 commits into from
Aug 18, 2025
Merged

Include more details from SDK/API errors in the diagnostics #3379

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
e02074a
Better error diagnostic for SDK errors
denik Aug 7, 2025
8f1b2cc
rewrite to return early
denik Aug 8, 2025
869a696
use errors.As
denik Aug 8, 2025
a8f0088
add unit test
denik Aug 8, 2025
7b9c3f6
update database_instance.go
denik Aug 18, 2025
3173862
update NEXT_CHANGELOG.md
denik Aug 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions NEXT_CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,6 @@

### Bundles
* Add support for Lakebase database instances in DABs ([#3283](https://github.com/databricks/cli/pull/3283))
* Improve error message for SDK/API errors ([#3379](https://github.com/databricks/cli/pull/3379))

### API Changes
7 changes: 6 additions & 1 deletion acceptance/bundle/resources/jobs/create-error/output.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,12 @@ Warning: required field "new_cluster" is not set

Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-bundle/default/files...
Deploying resources...
Error: deploying jobs.foo: creating: Method=Jobs.Create *retries.Err *apierr.APIError StatusCode=400 ErrorCode="INVALID_PARAMETER_VALUE" Message="Shared job cluster feature is only supported in multi-task jobs."
Error: deploying jobs.foo: creating: Shared job cluster feature is only supported in multi-task jobs. (400 INVALID_PARAMETER_VALUE)

Endpoint: POST [DATABRICKS_URL]/api/2.2/jobs/create
HTTP Status: 400 Bad Request
API error_code: INVALID_PARAMETER_VALUE
API message: Shared job cluster feature is only supported in multi-task jobs.

Updating deployment state...

Expand Down
4 changes: 2 additions & 2 deletions bundle/terranova/tnresources/app.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ func (r *ResourceApp) DoCreate(ctx context.Context) (string, error) {
}
waiter, err := r.client.Apps.Create(ctx, request)
if err != nil {
return "", SDKError{Method: "Apps.Create", Err: err}
return "", err
}

// TODO: Store waiter for Wait method
Expand All @@ -48,7 +48,7 @@ func (r *ResourceApp) DoUpdate(ctx context.Context, id string) error {
}
response, err := r.client.Apps.Update(ctx, request)
if err != nil {
return SDKError{Method: "Apps.Update", Err: err}
return err
}

if response.Name != id {
Expand Down
18 changes: 4 additions & 14 deletions bundle/terranova/tnresources/database_instance.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ func (d *ResourceDatabaseInstance) DoCreate(ctx context.Context) (string, error)
DatabaseInstance: d.config,
})
if err != nil {
return "", SDKError{Method: "Database.CreateDatabaseInstance", Err: err}
return "", err
}
d.waiter = waiter
return waiter.Response.Name, nil
Expand All @@ -38,21 +38,15 @@ func (d ResourceDatabaseInstance) DoUpdate(ctx context.Context, id string) error
request.DatabaseInstance.Uid = id

_, err := d.client.Database.UpdateDatabaseInstance(ctx, request)
if err != nil {
return SDKError{Method: "Database.UpdateDatabaseInstance", Err: err}
}
return nil
return err
}

func (d *ResourceDatabaseInstance) WaitAfterCreate(ctx context.Context) error {
if d.waiter == nil {
return nil
}
_, err := d.waiter.Get()
if err != nil {
return SDKError{Method: "WaitGetDatabaseInstanceDatabaseAvailable.Get", Err: err}
}
return nil
return err
}

func (d ResourceDatabaseInstance) WaitAfterUpdate(ctx context.Context) error {
Expand All @@ -68,14 +62,10 @@ func NewResourceDatabaseInstance(client *databricks.WorkspaceClient, resource *r
}

func DeleteDatabaseInstance(ctx context.Context, client *databricks.WorkspaceClient, name string) error {
err := client.Database.DeleteDatabaseInstance(ctx, database.DeleteDatabaseInstanceRequest{
return client.Database.DeleteDatabaseInstance(ctx, database.DeleteDatabaseInstanceRequest{
Name: name,
Purge: true,
Force: false,
ForceSendFields: nil,
})
if err != nil {
return SDKError{Method: "Database.DeleteDatabaseInstanceByName", Err: err}
}
return nil
}
14 changes: 3 additions & 11 deletions bundle/terranova/tnresources/job.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ func (r *ResourceJob) DoCreate(ctx context.Context) (string, error) {
}
response, err := r.client.Jobs.Create(ctx, request)
if err != nil {
return "", SDKError{Method: "Jobs.Create", Err: err}
return "", err
}
return strconv.FormatInt(response.JobId, 10), nil
}
Expand All @@ -43,23 +43,15 @@ func (r *ResourceJob) DoUpdate(ctx context.Context, id string) error {
if err != nil {
return err
}
err = r.client.Jobs.Reset(ctx, request)
if err != nil {
return SDKError{Method: "Jobs.Reset", Err: err}
}
return nil
return r.client.Jobs.Reset(ctx, request)
}

func DeleteJob(ctx context.Context, client *databricks.WorkspaceClient, id string) error {
idInt, err := strconv.ParseInt(id, 10, 64)
if err != nil {
return err
}
err = client.Jobs.DeleteByJobId(ctx, idInt)
if err != nil {
return SDKError{Method: "Jobs.DeleteByJobId", Err: err}
}
return nil
return client.Jobs.DeleteByJobId(ctx, idInt)
}

func (r *ResourceJob) WaitAfterCreate(ctx context.Context) error {
Expand Down
14 changes: 3 additions & 11 deletions bundle/terranova/tnresources/pipeline.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ func (r *ResourcePipeline) Config() any {
func (r *ResourcePipeline) DoCreate(ctx context.Context) (string, error) {
response, err := r.client.Pipelines.Create(ctx, r.config)
if err != nil {
return "", SDKError{Method: "Pipelines.Create", Err: err}
return "", err
}
return response.PipelineId, nil
}
Expand Down Expand Up @@ -68,19 +68,11 @@ func (r *ResourcePipeline) DoUpdate(ctx context.Context, id string) error {
ForceSendFields: filterFields[pipelines.EditPipeline](r.config.ForceSendFields),
}

err := r.client.Pipelines.Update(ctx, request)
if err != nil {
return SDKError{Method: "Pipelines.Update", Err: err}
}
return nil
return r.client.Pipelines.Update(ctx, request)
}

func DeletePipeline(ctx context.Context, client *databricks.WorkspaceClient, id string) error {
err := client.Pipelines.DeleteByPipelineId(ctx, id)
if err != nil {
return SDKError{Method: "Pipelines.DeleteByPipelineId", Err: err}
}
return nil
return client.Pipelines.DeleteByPipelineId(ctx, id)
}

func (r *ResourcePipeline) WaitAfterCreate(ctx context.Context) error {
Expand Down
10 changes: 3 additions & 7 deletions bundle/terranova/tnresources/schema.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ func (r *ResourceSchema) Config() any {
func (r *ResourceSchema) DoCreate(ctx context.Context) (string, error) {
response, err := r.client.Schemas.Create(ctx, r.config)
if err != nil {
return "", SDKError{Method: "Schemas.Create", Err: err}
return "", err
}
return response.FullName, nil
}
Expand All @@ -46,7 +46,7 @@ func (r *ResourceSchema) DoUpdate(ctx context.Context, id string) error {

response, err := r.client.Schemas.Update(ctx, updateRequest)
if err != nil {
return SDKError{Method: "Schemas.Update", Err: err}
return err
}

if response.FullName != id {
Expand All @@ -57,15 +57,11 @@ func (r *ResourceSchema) DoUpdate(ctx context.Context, id string) error {
}

func DeleteSchema(ctx context.Context, client *databricks.WorkspaceClient, id string) error {
err := client.Schemas.Delete(ctx, catalog.DeleteSchemaRequest{
return client.Schemas.Delete(ctx, catalog.DeleteSchemaRequest{
FullName: id,
Force: true,
ForceSendFields: nil,
})
if err != nil {
return SDKError{Method: "Schemas.Delete", Err: err}
}
return nil
}

func (r *ResourceSchema) WaitAfterCreate(ctx context.Context) error {
Expand Down
43 changes: 0 additions & 43 deletions bundle/terranova/tnresources/sdk_error.go
View file
Open in desktop

This file was deleted.

10 changes: 3 additions & 7 deletions bundle/terranova/tnresources/sql_warehouse.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ func (r *ResourceSqlWarehouse) Config() any {
func (r *ResourceSqlWarehouse) DoCreate(ctx context.Context) (string, error) {
waiter, err := r.client.Warehouses.Create(ctx, r.config)
if err != nil {
return "", SDKError{Method: "Warehouses.Create", Err: err}
return "", err
}

return waiter.Id, nil
Expand All @@ -55,7 +55,7 @@ func (r *ResourceSqlWarehouse) DoUpdate(ctx context.Context, id string) error {

waiter, err := r.client.Warehouses.Edit(ctx, request)
if err != nil {
return SDKError{Method: "Warehouses.Edit", Err: err}
return err
}

if waiter.Id != id {
Expand All @@ -76,9 +76,5 @@ func (r *ResourceSqlWarehouse) WaitAfterUpdate(ctx context.Context) error {
}

func DeleteSqlWarehouse(ctx context.Context, client *databricks.WorkspaceClient, oldID string) error {
err := client.Warehouses.DeleteById(ctx, oldID)
if err != nil {
return SDKError{Method: "Warehouses.DeleteById", Err: err}
}
return nil
return client.Warehouses.DeleteById(ctx, oldID)
}
8 changes: 4 additions & 4 deletions bundle/terranova/tnresources/volume.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ func (r *ResourceVolume) Config() any {
func (r *ResourceVolume) DoCreate(ctx context.Context) (string, error) {
response, err := r.client.Volumes.Create(ctx, r.config)
if err != nil {
return "", SDKError{Method: "Volumes.Create", Err: err}
return "", err
}
return response.FullName, nil
}
Expand All @@ -58,7 +58,7 @@ func (r *ResourceVolume) DoUpdate(ctx context.Context, id string) error {

response, err := r.client.Volumes.Update(ctx, updateRequest)
if err != nil {
return SDKError{Method: "Volumes.Update", Err: err}
return err
}

if id != response.FullName {
Expand Down Expand Up @@ -91,7 +91,7 @@ func (r *ResourceVolume) DoUpdateWithID(ctx context.Context, id string) (string,

response, err := r.client.Volumes.Update(ctx, updateRequest)
if err != nil {
return "", SDKError{Method: "Volumes.Update", Err: err}
return "", err
}

return response.FullName, nil
Expand All @@ -100,7 +100,7 @@ func (r *ResourceVolume) DoUpdateWithID(ctx context.Context, id string) (string,
func DeleteVolume(ctx context.Context, client *databricks.WorkspaceClient, id string) error {
err := client.Volumes.DeleteByName(ctx, id)
if err != nil {
return SDKError{Method: "Volumes.Delete", Err: err}
return err
}
return nil
}
Expand Down
3 changes: 2 additions & 1 deletion libs/diag/diagnostic.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,8 @@ func FromErr(err error) Diagnostics {
return []Diagnostic{
{
Severity: Error,
Summary: err.Error(),
Summary: FormatAPIErrorSummary(err),
Detail: FormatAPIErrorDetails(err),
},
}
}
Expand Down
44 changes: 44 additions & 0 deletions libs/diag/sdk_error.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
package diag

import (
"errors"
"fmt"
"strconv"
"strings"

"github.com/databricks/databricks-sdk-go/apierr"
)

func FormatAPIErrorSummary(e error) string {
var apiErr *apierr.APIError
if !errors.As(e, &apiErr) {
return e.Error()
}
extra := strings.TrimSpace(fmt.Sprintf("%d %s", apiErr.StatusCode, apiErr.ErrorCode))
return e.Error() + " (" + extra + ")"
}

func FormatAPIErrorDetails(e error) string {

@andrewnester andrewnester Aug 7, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to have a unit test for this to check the output when there is or no ResponseWrapper/Request/etc.

@denik denik Aug 8, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea, added here 5123233

var apiErr *apierr.APIError
if !errors.As(e, &apiErr) {
return ""
}

endpoint := "n/a"
httpStatus := ""
w := apiErr.ResponseWrapper
if w != nil {
resp := w.Response
if resp != nil {
httpStatus = resp.Status
req := resp.Request
if req != nil {
endpoint = fmt.Sprintf("%s %s", req.Method, req.URL)
}
}
}
if len(httpStatus) == 0 {
httpStatus = strconv.Itoa(apiErr.StatusCode)
}
return fmt.Sprintf("Endpoint: %s\nHTTP Status: %s\nAPI error_code: %s\nAPI message: %s", endpoint, httpStatus, apiErr.ErrorCode, apiErr.Message)

@andrewnester andrewnester Aug 7, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the API message always present or can it be empty?

@pietern pietern Aug 8, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

error_code -> error code?

They are not guaranteed to be set. I recall that the SDK does some work to populate them with HTTP error codes if there is no actual response, but double check...

@denik denik Aug 8, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, SDK tries very hard to populate those (e.g. it parses text and html) and falls back to setting them to UNKNOWN / full body.

So, if backend returns html or text, "API error_code" and "API message" labels are not strictly correct or rather they are represent SDK's view of error_code and message.

However, 99% of cases would get JSON error so "API error_code/message" does refer to error_code/message in the response. This is something that is mentioned in the docs, unlike "error code" which is vague.

BTW, this message is just a reasonably simple starting point. I think this message can be further improved in follow up PRs:

  • We can distinguish JSON vs html/text errors (could be useful for debugging).
  • We can mention useful headers (trace id?).
  • We can include full request/response if user asked for more verbosity.

}
Loading
Loading