Skip to content

Add new ErrHTMLContent value to represent errors due to HTML parsing. #1252

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
renaudhartert-db merged 3 commits into from
Jul 24, 2025
Merged

Add new ErrHTMLContent value to represent errors due to HTML parsing. #1252

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0563bef
Refactor error message for html answer
renaudhartert-db Jul 18, 2025
d3de920
Add changelog
renaudhartert-db Jul 22, 2025
e80221d
Merge branch 'main' into renaud-hartert_data/html
renaudhartert-db Jul 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions NEXT_CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

### New Features and Improvements

* Add new error value `ErrHTMLContent` to differentiate parsing errors caused by
HTML content from other parsing errors.
* Return more detailed error messages when OAuth endpoints cannot be resolved.
* Use a free port in `u2m` authentication flows rather than 8020.

Expand Down
122 changes: 41 additions & 81 deletions client/client_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,14 @@ import (
"io"
"net/http"
"net/url"
"runtime"
"strings"
"testing"

"github.com/databricks/databricks-sdk-go/apierr"
"github.com/databricks/databricks-sdk-go/config"
"github.com/databricks/databricks-sdk-go/httpclient"
"github.com/databricks/databricks-sdk-go/internal/env"
"github.com/databricks/databricks-sdk-go/useragent"
"github.com/databricks/databricks-sdk-go/version"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
Expand Down Expand Up @@ -353,67 +352,61 @@ func TestDoRemovesDoubleSlashesFromFilesAPI(t *testing.T) {
}

func TestNonJSONResponseIncludedInError(t *testing.T) {
cicdHeader := ""
if useragent.CiCdProvider() != "" {
cicdHeader = fmt.Sprintf(" cicd/%s", useragent.CiCdProvider())
}
goVersion := strings.TrimPrefix(runtime.Version(), "go")
type testCase struct {
statusCode int
status string
errorMessage string
}
cases := []testCase{
{
statusCode: 400,
status: "Bad Request",
errorMessage: `failed to unmarshal response body: invalid character '<' looking for beginning of value. This is likely a bug in the Databricks SDK for Go or the underlying REST API. Please report this issue with the following debugging information to the SDK issue tracker at https://github.com/databricks/databricks-sdk-go/issues. Request log:
` + "```" + `
GET /a
> * Host:
> * Accept: application/json
> * Authorization: REDACTED
> * User-Agent: unknown/0.0.0 databricks-sdk-go/` + version.Version + ` go/` + goVersion + ` os/` + runtime.GOOS + ` auth/pat` + cicdHeader + `
< HTTP/2.0 Bad Request
< * Content-Type: text/html
< <html><body>hello</body></html>
` + "```",
statusCode: 400,
status: "Bad Request",
errorMessage: httpclient.ErrHTMLContent.Error(),
},
{
statusCode: 500,
status: "Internal Server Error",
errorMessage: `failed to unmarshal response body: invalid character '<' looking for beginning of value. This is likely a bug in the Databricks SDK for Go or the underlying REST API. Please report this issue with the following debugging information to the SDK issue tracker at https://github.com/databricks/databricks-sdk-go/issues. Request log:
` + "```" + `
GET /a
> * Host:
> * Accept: application/json
> * Authorization: REDACTED
> * User-Agent: unknown/0.0.0 databricks-sdk-go/` + version.Version + ` go/` + goVersion + ` os/` + runtime.GOOS + ` auth/pat` + cicdHeader + `
< HTTP/2.0 Internal Server Error
< * Content-Type: text/html
< <html><body>hello</body></html>
` + "```",
statusCode: 500,
status: "Internal Server Error",
errorMessage: httpclient.ErrHTMLContent.Error(),
},
{
statusCode: 200,
status: "OK",
errorMessage: `failed to unmarshal response body: invalid character '<' looking for beginning of value. This is likely a bug in the Databricks SDK for Go or the underlying REST API. Please report this issue with the following debugging information to the SDK issue tracker at https://github.com/databricks/databricks-sdk-go/issues. Request log:
` + "```" + `
GET /a
> * Host:
> * Accept: application/json
> * Authorization: REDACTED
> * User-Agent: unknown/0.0.0 databricks-sdk-go/` + version.Version + ` go/` + goVersion + ` os/` + runtime.GOOS + ` auth/pat` + cicdHeader + `
< HTTP/2.0 OK
< * Content-Type: text/html
< <html><body>hello</body></html>
` + "```",
statusCode: 200,
status: "OK",
errorMessage: httpclient.ErrHTMLContent.Error(),
Comment on lines +372 to +374

@parthban-db parthban-db Jul 22, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my understanding, is this case even possible? Don't we have a single statuscode for this kind of error?

},
}

for _, tc := range cases {
tc := tc
t.Run(fmt.Sprintf("%d %s", tc.statusCode, tc.status), func(t *testing.T) {
testNonJSONResponseIncludedInError(t, tc.statusCode, tc.status, tc.errorMessage)
c, err := New(&config.Config{
Host: "some",
Token: "token",
ConfigFile: "/dev/null",
HTTPTransport: hc(func(r *http.Request) (*http.Response, error) {
r.Header.Del("traceparent") // clear nondeterministic traceparent header
return &http.Response{
Proto: "HTTP/2.0",
Status: tc.status,
Body: io.NopCloser(strings.NewReader(`<html><body>hello</body></html>`)),
Request: r,
Header: http.Header{
"Content-Type": []string{"text/html"},
},
}, nil
}),
})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}

var m map[string]string
gotErr := c.Do(context.Background(), "GET", "/a", nil, nil, nil, &m)

if gotErr == nil {
t.Fatalf("expected error, got nil")
}
if !strings.Contains(gotErr.Error(), tc.errorMessage) {
t.Fatalf("expected error to contain %q, got %q", tc.errorMessage, gotErr.Error())
}
})
}
}
Expand Down Expand Up @@ -534,39 +527,6 @@ func TestUserAgentForCiCd(t *testing.T) {

}

func testNonJSONResponseIncludedInError(t *testing.T, statusCode int, status, errorMessage string) {
c, err := New(&config.Config{
Host: "some",
Token: "token",
ConfigFile: "/dev/null",
HTTPTransport: hc(func(r *http.Request) (*http.Response, error) {
// Clear traceparent header which is nondeterministic.
r.Header.Del("traceparent")
return &http.Response{
Proto: "HTTP/2.0",
Status: status,
Body: io.NopCloser(strings.NewReader(`<html><body>hello</body></html>`)),
Request: r,
Header: http.Header{
"Content-Type": []string{"text/html"},
},
}, nil
}),
})
require.NoError(t, err)
var m map[string]string
err = c.Do(
context.Background(),
"GET",
"/a",
nil,
nil,
nil,
&m,
)
require.EqualError(t, err, errorMessage)
}

func TestRetryOn503(t *testing.T) {
var requested bool
c, err := New(&config.Config{
Expand Down
33 changes: 32 additions & 1 deletion httpclient/response.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
Expand All @@ -16,6 +17,29 @@ import (
"github.com/databricks/databricks-sdk-go/logger/httplog"
)

// ErrHTMLContent is returned when the response body is HTML instead of JSON.
//
// This almost always indicates an issue with your authentication configuration.
// If you encounter this error, please verify the following:
//
// - Databricks Host: Ensure your host is set correctly.
// - Permissions: Confirm that the authentication method has the required
// permissions for the API operation you are trying to perform.
// - Network/Proxy: If you are behind a corporate firewall, ensure it is not
// blocking or redirecting API traffic.
//
// A common cause of this error is Private Link redirecting the SDK to a login
// page, which the SDK cannot process. This usually happens when trying to
// access a Private Link-enabled workspace configured with no public internet
// access from a different network than the VPC endpoint belongs to.
//
// For more details, please refer to the [Unified Auth] documentation and
// [Private Link Authentication Troubleshooting].
//
// [Unified Auth]: https://docs.databricks.com/aws/en/dev-tools/auth/unified-auth
// [Private Link Authentication Troubleshooting]: https://learn.microsoft.com/en-us/azure/databricks/security/network/classic/private-link-standard#authentication-troubleshooting
var ErrHTMLContent = errors.New("received HTML response instead of JSON")

func WithResponseHeader(key string, value *string) DoOption {
return DoOption{
out: func(body *common.ResponseWrapper) error {
Expand Down Expand Up @@ -91,14 +115,21 @@ func WithResponseUnmarshal(response any) DoOption {
*bs = bodyBytes
return nil
}
if err = json.Unmarshal(bodyBytes, &response); err != nil {
if err := json.Unmarshal(bodyBytes, &response); err != nil {
if _, ok := err.(*json.SyntaxError); ok && isHTMLContent(bodyBytes) {
return ErrHTMLContent
}
return fmt.Errorf("failed to unmarshal response body: %w. %s", err, makeUnexpectedResponse(body.Response, body.RequestBody.DebugBytes, bodyBytes))
}
return nil
},
}
}

func isHTMLContent(bodyBytes []byte) bool {
return strings.HasPrefix(string(bodyBytes), "<")
}

func findContentsField(response any) (*reflect.Value, bool) {
value := reflect.ValueOf(response)
value = reflect.Indirect(value)
Expand Down
Loading