Pin cfg.Profile to the resolved name in resolveDefaultProfile (#5280)

## Why

`databricks auth login --profile DEFAULT --host ...` followed by a
no-flag `databricks auth describe` (or any other command that needs the
U2M token) fails when secure storage is in use:

```
Unable to authenticate: A new access token could not be retrieved because the refresh token is invalid.
```

`databricks auth describe --profile DEFAULT` works. Running the same
flow under `DATABRICKS_AUTH_STORAGE=plaintext` also works. So the bug is
specific to secure storage + the implicit DEFAULT fallback.

Root cause is a cache-key mismatch between login and read:

- `cmd/auth/login.go:222` hardcodes `profileName = "DEFAULT"` when no
`--profile` is given, so the OAuthArgument's cache key is the literal
string `"DEFAULT"`. The token lands in the keyring under account
`"DEFAULT"`.
- On the read path, `cfg.Profile` starts empty, `resolveDefaultProfile`
only consults `[__settings__].default_profile` (so it stays empty), and
the SDK's `configFileLoader.Configure` (`config_file.go:103-105`) loads
`[DEFAULT]`'s values but **deliberately leaves `cfg.Profile` empty**
when it falls back (`isFallback=true`). `CLICredentials.Configure` then
builds an OAuthArgument with `profile=""`, so `GetCacheKey()` falls back
to `GetHostCacheKey()` and the lookup goes to the host URL, not
`"DEFAULT"`. Miss.

plaintext mode masks the same mismatch with `DualWritingTokenCache`,
which mirrors every write under the host key — so reads via host URL
still find the token. secure mode does not dual-write, so the bug
surfaces.

This is a pre-existing bug independent of toggling secure-storage by
default, but doing so turns a corner case into the default experience.
The fix here is targeted enough to land standalone.

A defense-in-depth followup in `databricks-sdk-go` will drop the
SDK-side `if !isFallback` gate so all SDK consumers benefit from the
same self-consistency. The CLI fix lands first so secure-storage users
are unblocked without waiting on an SDK release cycle.

## Changes

- `cmd/root/auth.go`: `resolveDefaultProfile` swaps
`databrickscfg.ResolveDefaultProfile` (settings-only) for
`databrickscfg.GetDefaultProfile`, which already does the full 4-step
resolution: `[__settings__].default_profile` → the only profile in the
file → `[DEFAULT]` → empty. The SDK then sees a non-empty `cfg.Profile`,
takes the `isFallback=false` branch, and the name flows through to
`CLICredentials.Configure`. OAuthArgument's cache key now matches what
login wrote.
- `cmd/root/bundle.go` is intentionally NOT touched: bundles
deliberately limit their fallback to `[__settings__].default_profile` so
a hostless bundle does not get silently routed at a `[DEFAULT]` profile
pointing at the wrong workspace. That comment in `bundle.go:74-80` stays
load-bearing.
- `cmd/root/auth_test.go`:
-
`TestMustWorkspaceClientWithoutConfiguredDefaultFallsBackToDefaultSection`
now asserts `cfg.Profile == "DEFAULT"` (was `""`). The previous
assertion documented the bug; the new one documents the contract.
- New table-driven `TestResolveDefaultProfile` covers the full
resolution order: preset `cfg.Profile`, `DATABRICKS_CONFIG_PROFILE` env,
`[__settings__].default_profile`, single profile, `[DEFAULT]` section
among many, no fallback, missing file.
- `NEXT_CHANGELOG.md`: one-line entry describing the fix and the
mismatch it removes.

## Test plan

- [x] `task checks` clean
- [x] `task lint-q` clean
- [x] `go test ./cmd/root/... ./cmd/auth/... ./libs/databrickscfg/...`
passes
- [x] `go test ./acceptance -run 'TestAccept/cmd/auth'` passes
- [ ] Manual repro of Pieter's case (`auth login --profile DEFAULT
--host ...` then `auth describe` with no flag under secure storage)
succeeds after this PR; the same flow on `main` fails.
- [ ] Verify bundle resolution is unaffected: a bundle without
`workspace.host` and no `--profile` still uses
`[__settings__].default_profile` only (no silent DEFAULT routing).

This pull request and its description were written by Isaac.

Author: simonfaltum

NEXT_CHANGELOG.md

@@ -13,6 +13,7 @@
 
 * Added `databricks aitools` command group for installing Databricks skills into your coding agents (Claude Code, Cursor, Codex CLI, OpenCode, GitHub Copilot, Antigravity). Skills are fetched from [github.com/databricks/databricks-agent-skills](https://github.com/databricks/databricks-agent-skills) and either symlinked into each agent's skills directory or copied into the current project. Use `databricks aitools install` to set up, `update` to pull newer versions, `list` to see what's available, and `uninstall` to remove them. Pick where they go with `--scope=project|global` (`--scope=both` is accepted on `update` and `list`).
 * `[__settings__].default_profile` is now consulted as a fallback by `databricks api`, `databricks auth token`, and bundle commands when neither `--profile` nor `DATABRICKS_CONFIG_PROFILE` is set. `databricks auth token` continues to give precedence to `DATABRICKS_HOST` over `default_profile`. For bundle commands, `default_profile` only applies when the bundle does not pin its own `workspace.host`.
+* Fixed bug where auth commands did not load the DEFAULT profile properly during auth where type is `databricks-cli`.
 * `databricks workspace import-dir` now skips `.git`, `.databricks`, and `node_modules` directories during recursive imports. To import one of these directories deliberately, pass it as `SOURCE_PATH` ([#5118](https://github.com/databricks/cli/pull/5118)).
 * `databricks postgres create-role --help` now documents the `--json` body shape and rejects the common mistake of wrapping the body in `{"role": ...}` client-side with a hint pointing at the correct shape ([#5111](https://github.com/databricks/cli/pull/5111)).
 * `databricks aitools list` honors `--output json`, emitting a structured `{release, skills[...], summary{}}` document so coding agents and CI can consume the skill/version/installation matrix without scraping the tabular text output ([#5233](https://github.com/databricks/cli/pull/5233)).

cmd/root/auth_test.go

@@ -514,6 +514,7 @@ token = named-token
 
 	w := cmdctx.WorkspaceClient(cmd.Context())
 	require.NotNil(t, w)
-	assert.Equal(t, "", w.Config.Profile)
+	// Pinned so the OAuth cache key matches what `databricks auth login` writes.
+	assert.Equal(t, "DEFAULT", w.Config.Profile)
 	assert.Equal(t, "https://default.cloud.databricks.com", w.Config.Host)
 }

cmd/root/bundle.go

@@ -72,12 +72,14 @@ func configureProfile(cmd *cobra.Command, b *bundle.Bundle) {
 	profile := getProfile(cmd)
 
 	// Fall back to [__settings__].default_profile only when the bundle does
-	// not pin its own host. If the bundle declares workspace.host, applying
-	// default_profile here could route the user to a profile that points at
-	// a different host than the bundle expects — let the SDK resolve auth
-	// from the host instead.
+	// not pin its own host. The legacy [DEFAULT] section is intentionally
+	// NOT considered here: a hostless bundle silently routing to whatever
+	// [DEFAULT] points at could deploy to the wrong workspace and mask a
+	// missing workspace.host. Auth-only paths use the broader
+	// databrickscfg.ResolveDefaultProfile, which also accepts [DEFAULT].
 	if profile == "" && b.Config.Workspace.Host == "" && b.Config.Workspace.Profile == "" {
-		profile = databrickscfg.ResolveDefaultProfile(cmd.Context())
+		configFilePath := envlib.Get(cmd.Context(), "DATABRICKS_CONFIG_FILE")
+		profile, _ = databrickscfg.GetConfiguredDefaultProfile(cmd.Context(), configFilePath)
 	}
 
 	if profile == "" {

libs/databrickscfg/ops.go

@@ -38,26 +38,42 @@ func GetConfiguredDefaultProfile(ctx context.Context, configFilePath string) (st
 	return GetConfiguredDefaultProfileFrom(configFile), nil
 }
 
-// ResolveDefaultProfile returns the [__settings__].default_profile value from
-// the config file pointed to by DATABRICKS_CONFIG_FILE (or ~/.databrickscfg
-// when unset). Returns "" with no error when the file is missing, the setting
-// is absent, or parsing fails (a warning is logged on parse error).
+// ResolveDefaultProfile returns the default profile from the config file
+// pointed to by DATABRICKS_CONFIG_FILE (or ~/.databrickscfg when unset):
+// [__settings__].default_profile, else [DEFAULT] if it has a host key,
+// else "". Returns "" with no error when the file is missing or parsing
+// fails (a warning is logged on parse error).
 //
 // Callers must respect their own higher-priority sources (an explicit
 // --profile flag or DATABRICKS_CONFIG_PROFILE env var) before consulting
-// this helper. default_profile is a CLI-level fallback; the SDK does not
-// know about it.
+// this helper. default_profile and [DEFAULT] are CLI-level fallbacks; the
+// SDK loader silently falls back to [DEFAULT] but leaves cfg.Profile empty,
+// which breaks the per-profile OAuth cache key. Pinning the name here keeps
+// cfg.Profile in sync with what the SDK would read.
+//
+// Single-profile fallback (using "the only profile in the file" as the
+// default) is intentionally NOT applied: that is a prompt-seeding convenience
+// (see GetDefaultProfile), not an auth rule, and it would silently route a
+// single account-only profile through the workspace-client path.
 func ResolveDefaultProfile(ctx context.Context) string {
 	configFilePath := env.Get(ctx, "DATABRICKS_CONFIG_FILE")
-	resolved, err := GetConfiguredDefaultProfile(ctx, configFilePath)
+	configFile, err := loadConfigFile(ctx, configFilePath)
 	if err != nil {
 		log.Warnf(ctx, "Failed to load default profile: %v", err)
 		return ""
 	}
-	if resolved != "" {
-		log.Debugf(ctx, "profile %q resolved from [__settings__].default_profile", resolved)
+	if configFile == nil {
+		return ""
+	}
+	if profile := GetConfiguredDefaultProfileFrom(configFile); profile != "" {
+		log.Debugf(ctx, "profile %q resolved from [__settings__].default_profile", profile)
+		return profile
 	}
-	return resolved
+	if section := configFile.Section(ini.DefaultSection); section.HasKey("host") {
+		log.Debugf(ctx, "profile %q resolved from the [DEFAULT] section", ini.DefaultSection)
+		return ini.DefaultSection
+	}
+	return ""
 }
 
 // GetConfiguredDefaultProfileFrom returns the explicit default profile from

libs/databrickscfg/ops_test.go

@@ -327,7 +327,7 @@ func TestResolveDefaultProfile(t *testing.T) {
 			want:     "my-workspace",
 		},
 		{
-			name:     "settings without default_profile",
+			name:     "settings without default_profile, no DEFAULT section",
 			contents: "[__settings__]\nauth_storage = secure\n\n[my-workspace]\nhost = https://abc\n",
 			want:     "",
 		},
@@ -341,6 +341,21 @@ func TestResolveDefaultProfile(t *testing.T) {
 			contents: "[__settings__]\ndefault_profile = __settings__\n\n[profile1]\nhost = https://abc\n",
 			want:     "",
 		},
+		{
+			name:     "DEFAULT section with host is used when settings is empty",
+			contents: "[DEFAULT]\nhost = https://default.abc\n\n[profile1]\nhost = https://abc\n",
+			want:     "DEFAULT",
+		},
+		{
+			name:     "DEFAULT section without host is ignored",
+			contents: "[DEFAULT]\naccount_id = 1234\n\n[profile1]\nhost = https://abc\n",
+			want:     "",
+		},
+		{
+			name:     "settings takes precedence over DEFAULT section",
+			contents: "[__settings__]\ndefault_profile = override\n\n[DEFAULT]\nhost = https://default.abc\n\n[override]\nhost = https://override.abc\n",
+			want:     "override",
+		},
 	}
 
 	for _, tc := range cases {