feat(managed-databases): migrate to /v1/databases API and fix query catalog

[eddietejeda]

Summary

• Migrate managed databases to /v1/databases API — replaces the legacy /v1/connections approach with dedicated DatabasesApi endpoints (create_database, delete_database, list_databases, get_database). CreateConnectionRequest / build_managed_config are gone; CreateDatabaseRequest with schema/table declarations is used instead.
• Fix managed table queries — the Hotdata server requires "default"."schema"."table" as the SQL catalog for managed databases (not the raw connection_id). The backend now uses "default" in generated SQL while mapping it back to the real connection_id for information-schema API calls via the new _resolve_database_connection_id helper.
• Fix unknown query run statuses — the async polling loop in execute_query now raises immediately on statuses like "cancelled" or "timed_out" instead of spinning until the 600 s timeout.
• Pass database_id through query execution — _safe_raw_sql, to_pyarrow, and _get_schema_using_query now forward self._database_id as the X-Database-Id header so managed-database context is preserved end-to-end.
• README: new Managed databases section — complete working example covering create → upload → sync-wait → ibis expression query → raw SQL → drop, plus key points about the "default" catalog, table pre-declaration, and async load behaviour.

Test plan

• Smoke-tested locally end-to-end: create database, upload table, wait for sync, ibis expression filter+order, con.sql() sum, to_pyarrow, drop database — all 8 steps pass
• Unit tests updated to match new API shapes (test_hotdata_backend.py, test_hotdata_http.py)
• Run uv run pytest to verify no regressions

🤖 Generated with Claude Code

[claude]

nit: (not blocking) the bare except HotdataAPIError: pass swallows every API error — including 500s, auth failures, and network errors — and silently falls through to using "default" as the connection_id in subsequent info-schema calls, producing a confusing downstream error. Consider letting non-404s propagate so the original cause surfaces:

if self._database_connection_id is None:
    try:
        db = self._http.get_database(self._database_id)
    except HotdataAPIError as exc:
        if exc.status_code == 404:
            return None
        raise _ibis_err_from_hotdata(exc) from exc
    self._database_connection_id = db.get("default_connection_id")

[claude]

nit: (not blocking) asymmetric update — _database_id is "sticky" (only set when unset), but _database_connection_id is overwritten unconditionally. If a caller works with two managed databases through the same con (e.g. calls _table_location against db_A then db_B), _database_id keeps pointing at db_A while _database_connection_id points at db_B's connection. Subsequent execute_query calls then send X-Database-Id: <db_A_id> to the server while info-schema lookups go through db_B's connection_id — a hidden split-brain.

The README documents a one-managed-db-per-connection workflow, so this isn't hit on the happy path, but the two fields should move together. Either always update both, or always preserve both — whichever matches the intended invariant.

[claude]

nit: (not blocking) the PR description calls out this new behavior (raise immediately on "cancelled" / "timed_out" instead of spinning until the 600 s timeout) as a deliberate bug fix, but there's no test covering it. A small unit test that mocks /v1/query-runs/{id} to return status: "cancelled" and asserts HotdataAPIError raises promptly would lock the behavior in and prevent a future regression back to the 600s-spin.

[claude]

super nit: (not blocking) the combined or/ternary parses correctly but is harder to read at a glance — the or and the if/else are operating at different levels. A plain branch would be clearer:

if conn == "default":
    api_conn = self._resolve_database_connection_id() or conn
else:
    api_conn = conn