api: route per-call against unified hosts (#5137)

## Why

`databricks api` always sent the workspace routing identifier
(`X-Databricks-Org-Id`) when the profile had one, even when the path was
an account API. On unified hosts (one host serving both workspace and
account APIs) this misrouted account calls. There was also no way to
explicitly route a call to the account API or override the identifier
per call.

## Changes

Before: routing was decided once from the profile and applied to every
call.
Now: routing is decided per call from the path being requested.

- Paths under `/accounts/{id}/` are auto-detected as account-scope; the
routing identifier is dropped.
- A small hand-written list in `cmd/api/paths.go` carves out
workspace-routed proxy APIs that happen to live under `/accounts/`, so
they keep the identifier.
- `--account` forces account-scope on a non-`/accounts/` path.
- `--workspace-id <id>` overrides the identifier per call. Mutually
exclusive with `--account`.
- `?o=<id>` on the path (the SPOG URL convention used by the Databricks
UI) is recognized as a per-call workspace override, so URLs pasted from
the browser route correctly.
- The CLI-only `workspace_id = none` sentinel is stripped before the
routing decision so the literal "none" never goes on the wire.

Routing logic lives in pure functions (`hasAccountSegment`,
`extractOrgIDFromQuery`, `resolveOrgID`, `normalizeWorkspaceID`,
`isWorkspaceProxyPath`) that take primitives. The cobra `RunE` is a thin
adapter that resolves config and calls them.

## Test plan

- [x] `go test ./cmd/api` covers the helpers with table-driven cases:
deny-list hits and misses, query/fragment edge cases, mutual-exclusion
errors, sentinel stripping, `?o=` extraction.
- [x] `go test ./acceptance -run TestAccept/cmd/api` exercises seven
variants end to end against terraform and direct engines: workspace
path, account path, deny-listed proxy under `/accounts/`, `--account`,
`--workspace-id`, `?o=` query, `workspace_id = none`. Each test asserts
header presence/absence explicitly via `print_requests.py |
contains.py`.
- [x] `make checks`

Author: simonfaltum

NEXT_CHANGELOG.md

@@ -4,6 +4,8 @@
 
 ### CLI
 
+* `databricks api` now works against unified hosts. Adds `--account` to scope a call to the account API and `--workspace-id` to override the workspace routing identifier per call. A `?o=<workspace-id>` query parameter on the path (the SPOG URL convention used by the Databricks UI) is also recognized as a per-call workspace override, so URLs pasted from the browser route correctly.
+
 ### Bundles
 
 ### Dependency updates

acceptance/cmd/api/account-flag/out.test.toml

@@ -0,0 +1,15 @@
+{}
+
+>>> print_requests.py --get //api/2.0/clusters/list
+{
+  "headers": {
+    "Authorization": [
+      "Bearer [DATABRICKS_TOKEN]"
+    ],
+    "User-Agent": [
+      "cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/api_get cmd-exec-id/[UUID] interactive/none auth/pat"
+    ]
+  },
+  "method": "GET",
+  "path": "/api/2.0/clusters/list"
+}

acceptance/cmd/api/account-flag/output.txt

@@ -0,0 +1,2 @@
+MSYS_NO_PATHCONV=1 $CLI api get /api/2.0/clusters/list --account
+trace print_requests.py --get //api/2.0/clusters/list | contains.py "!X-Databricks-Org-Id"

acceptance/cmd/api/account-flag/script

@@ -0,0 +1,15 @@
+{}
+
+>>> print_requests.py --get //api/2.0/accounts/abc-123/network-policies
+{
+  "headers": {
+    "Authorization": [
+      "Bearer [DATABRICKS_TOKEN]"
+    ],
+    "User-Agent": [
+      "cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/api_get cmd-exec-id/[UUID] interactive/none auth/pat"
+    ]
+  },
+  "method": "GET",
+  "path": "/api/2.0/accounts/abc-123/network-policies"
+}

acceptance/cmd/api/account-path/out.test.toml

@@ -0,0 +1,2 @@
+MSYS_NO_PATHCONV=1 $CLI api get /api/2.0/accounts/abc-123/network-policies
+trace print_requests.py --get //api/2.0/accounts/abc-123/network-policies | contains.py "!X-Databricks-Org-Id"

acceptance/cmd/api/account-path/output.txt

@@ -0,0 +1,40 @@
+RecordRequests = true
+IncludeRequestHeaders = ["Authorization", "User-Agent", "X-Databricks-Org-Id"]
+
+# Normalize OS-dependent and CI-only User-Agent segments so the recorded
+# requests are stable across local macOS/Linux runs and GitHub Actions.
+[[Repls]]
+Old = '(linux|darwin|windows)'
+New = '[OS]'
+
+[[Repls]]
+Old = " cicd/[A-Za-z0-9.-]+"
+New = ""
+
+[[Repls]]
+Old = " upstream/[A-Za-z0-9.-]+"
+New = ""
+
+[[Repls]]
+Old = " upstream-version/[A-Za-z0-9.-]+"
+New = ""
+
+# Common stubs for paths used across variants. Each returns an empty JSON
+# object; the variants assert on the *recorded request* (out.requests.txt),
+# not on the response body, so any well-formed JSON is fine.
+
+[[Server]]
+Pattern = "GET /api/2.0/clusters/list"
+Response.Body = '{}'
+
+[[Server]]
+Pattern = "GET /api/2.0/accounts/abc-123/network-policies"
+Response.Body = '{}'
+
+[[Server]]
+Pattern = "GET /api/2.0/accounts/abc-123/oauth2/published-app-integrations"
+Response.Body = '{}'
+
+[[Server]]
+Pattern = "GET /api/2.0/preview/accounts/access-control/rule-sets"
+Response.Body = '{}'

acceptance/cmd/api/account-path/script

@@ -0,0 +1,18 @@
+{}
+
+>>> print_requests.py --get //api/2.0/clusters/list
+{
+  "headers": {
+    "Authorization": [
+      "Bearer [DATABRICKS_TOKEN]"
+    ],
+    "User-Agent": [
+      "cli/[DEV_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/api_get cmd-exec-id/[UUID] interactive/none auth/pat"
+    ],
+    "X-Databricks-Org-Id": [
+      "999"
+    ]
+  },
+  "method": "GET",
+  "path": "/api/2.0/clusters/list"
+}