Skip to content

hotdata-ibis 0.3.0

Latest

Choose a tag to compare

@github-actions github-actions released this 13 Jul 12:06
89acaf6

Changed

  • BREAKING: Managed-database operations are now id-addressed only.
    create_database returns the created database's id (previously returned
    None); create_table, drop_table, and drop_database all require that
    id in place of the database's display name. Hotdata database names are not
    unique, so the previous behavior — falling back to a list_databases()
    scan by name whenever a name/id lookup 404'd — could silently resolve to
    the wrong database on a name collision, risking a write or delete against
    the wrong data. There is no longer a name-based fallback anywhere in the
    managed-database path; callers must track the id create_database returns.
    create_database's force parameter is now a no-op (kept only for
    interface compatibility with ibis.backends.CanCreateDatabase) since an
    "already exists" check by name is no longer meaningful.
  • Bump ibis-framework >=10.0,<11>=12,<13 to unblock co-installing
    hotdata-ibis alongside hotdata-dlt-destination (which needs dlt 1.28 /
    ibis 12) in the same environment. No API drift: the compiler handle
    (sc.postgres.compiler), SQLBackend/mixin imports, PyArrowSchema.to_ibis,
    PandasData.convert_table, and the exception surface used by this backend
    are unchanged between ibis 10 and 12. Verified with the full offline test
    suite and a live read/write round trip (managed database created via
    database_id bound at connect time, catalog "default", con.table(...)
    and con.sql(...) to both pandas and pyarrow) against a running Hotdata
    workspace.
  • Raise the minimum hotdata SDK version to >=0.6,<0.7 (was >=0.5.0).
    hotdata 0.6.0 made X-Database-Id required on the query-run and result
    endpoints (get_query_run, get_result*); execute_query now forwards the
    same database_id used to submit a query to its poll and result-fetch
    calls, matching the fix already applied on the hotdata-dlt-destination
    side.
  • Raise the minimum pyarrow version to >=16 (was >=15) — required for
    pyarrow.string_view(), used by the fix below.

Fixed

  • Map Arrow's Utf8View (the StringView layout PyArrow/Arrow >=16 introduced,
    which RuntimeDB's information schema now reports for plain string columns —
    both federated and managed-database) to dt.String instead of falling
    through to dt.unknown(...). Confirmed against a live workspace: this
    affected both an existing federated TPC-H connection and a managed database
    populated via dlt, i.e. exactly the read path hotdata-dlt-destination's
    live ibis backend depends on.

Added

  • examples/05_roundtrip_demo.py: a runnable example demonstrating the
    managed-database lifecycle end to end (create_databasecreate_table
    → bind database_id at connect time → read via con.table() / con.sql()
    drop_database) — the specific contract hotdata-dlt-destination's live
    ibis backend wrapper relies on. Distinct from the existing 01-04
    examples, which all read pre-existing federated connections and never
    exercise the managed-database write/bind path.