Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
125 changes: 48 additions & 77 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,151 +1,122 @@
# hotdata-marimo

Marimo widgets for [Hotdata](https://hotdata.dev): run SQL, browse catalogs, load managed databases, and display results in notebooks.

Requires Python 3.10+, [Marimo](https://marimo.io/), and [hotdata-runtime](https://github.com/hotdata-dev/hotdata-runtime) (installed automatically).

## Supported widgets

Importing `hotdata_marimo` registers `mo.ui.hotdata_*` aliases for discoverability.

| Widget | Function | Notes |
|--------|----------|-------|
| SQL editor | `hm.sql_editor(client)` | Returns `.ui` and `.result` |
| Table browser | `hm.table_browser(client)` | Browse connections, schemas, tables, columns |
| Managed databases panel | `hm.databases_panel(client)` | Create catalogs and load parquet files |
| Managed database writer | `hm.managed_database_writer(client)` | Lower-level create/load UI |
| Workspace selector | `hm.workspace_selector_from_env()` | Pick workspace when `HOTDATA_WORKSPACE` is unset |
| Connection picker | `hm.connection_picker(client)` | Dropdown of workspace connections |
| Connection status | `hm.connection_status(client)` | API / workspace health callout |
| Connections panel | `hm.connections_panel(client)` | Status callout plus connection list |
| Query result | `hm.query_result(result)` | Render a `QueryResult` as a table |
| Recent results | `hm.recent_results(client)` | Browse past query results |
| Run history | `hm.run_history(client)` | Recent query runs |

Each widget also has a `mo.ui.hotdata_*` alias (e.g. `mo.ui.hotdata_sql_editor`). Native Marimo SQL cells are supported via `hm.register_hotdata_sql_engine()` and `mo.sql(..., engine=client)`.
[Marimo](https://marimo.io/) widgets for [Hotdata](https://hotdata.dev) — run SQL, browse your schema, and work with managed databases in reactive notebooks.

## Install

```bash
pip install hotdata-marimo
```

Set `HOTDATA_API_KEY`. Optionally set `HOTDATA_WORKSPACE`, `HOTDATA_API_URL`, or `HOTDATA_SANDBOX`.

## Connect

```python
import hotdata_marimo as hm

client = hm.from_env()
```

If `HOTDATA_WORKSPACE` is unset, pick a workspace interactively:

```python
ws = hm.workspace_selector_from_env()
client = ws.client
```
## Authentication

## SQL editor widget
Set `HOTDATA_API_KEY` in your environment. Optionally set `HOTDATA_WORKSPACE` to pin a specific workspace (the first available workspace is used if unset).

Run SQL in one cell; show results in the next. Marimo only renders what you **`return`**.
## Quickstart

**Cell 1 — editor**
Because Marimo reruns cells reactively, construct a widget in one cell and read its `.ui` or `.result` in the next.

```python
import marimo as mo
# Cell 1
import hotdata_marimo as hm

client = hm.from_env()
editor = hm.sql_editor(client, default_sql="SELECT 1 AS ok")
return editor.ui
```

**Cell 2 — result**

```python
# Cell 2
return hm.query_result(editor.result)
```

Click **Run on Hotdata** after changing SQL. The editor caches the last successful result so downstream cells do not re-query on every refresh.
Click **Run on Hotdata** after editing SQL. The editor caches the last successful result so downstream cells don't re-query on every refresh.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: The PR description says this change "covers scoping SQL to a managed database and controlling result size", but the README never mentions the new database= parameter on hm.sql_editor() / HotdataMarimoEngine, and there's no section on result-size controls. If those features are intentionally part of the same PR, the Quickstart or Managed databases section is a natural place to show hm.sql_editor(client, database="<id>"). (not blocking)


## Native Marimo SQL cells
## Workspace selection

If you have multiple workspaces or `HOTDATA_WORKSPACE` is unset, add an interactive picker. `ws.client` updates when the selection changes:

```python
ws = hm.workspace_selector_from_env()
client = ws.client
return ws.ui
```

Register the Hotdata engine once, then pass `engine=client` to `mo.sql`. Hotdata appears as **Hotdata** in the SQL connection picker.
## Native Marimo SQL cells

**Setup cell**
Register the Hotdata engine once and Hotdata will appear as a selectable engine in the SQL connection picker:

```python
# Setup cell
import marimo as mo
import hotdata_marimo as hm

hm.register_hotdata_sql_engine()
client = hm.from_env()
```

**SQL cell**

```python
_df = mo.sql(
"""
SELECT 1 AS example_value
""",
engine=client,
)
# Any SQL cell
_df = mo.sql("SELECT * FROM orders LIMIT 10", engine=client)
```

![Marimo SQL cell with Hotdata selected in the database connections picker](docs/images/mo-sql-hotdata-connection.png)

## Browse tables
## Browse your schema

The table browser lets you pick a connection, search for a table, and inspect its columns — with a starter query ready to copy:

```python
browser = hm.table_browser(client)
return browser.ui
```

Pick a connection, schema, and table to inspect columns. Use `browser.selected_table` in downstream cells.
Use `browser.selected_table` in downstream cells to reference the chosen table.

## Managed databases

Create a Hotdata-owned catalog and load a parquet file from the notebook:
View existing managed databases and load new parquet files from a single tabbed panel:

```python
panel = hm.databases_panel(client)
return panel
writer = hm.managed_database_writer(client)
return writer.tab_ui
```

Or use the lower-level writer API:
Or show just the read-only panel:

```python
writer = hm.managed_database_writer(client)
return writer.ui
return hm.databases_panel(client)
```

## Other helpers

See [Supported widgets](#supported-widgets) for the full list. Quick examples:
## All widgets

| Widget | Code | What you get |
|--------|------|-------------|
| SQL editor | `hm.sql_editor(client)` | `.ui` to show the editor, `.result` to read rows |
| Query result | `hm.query_result(result)` | Renders a `QueryResult` as a table |
| Table browser | `hm.table_browser(client)` | Browse connections, tables, and column metadata |
| Managed databases | `hm.databases_panel(client)` | Read-only list of managed databases |
| Database writer | `hm.managed_database_writer(client)` | Create databases and load parquet files |
| Workspace picker | `hm.workspace_selector_from_env()` | Dropdown to switch workspaces |
| Connection picker | `hm.connection_picker(client)` | Dropdown of connections in the workspace |
| Connection status | `hm.connection_status(client)` | Health callout for the API and workspace |
| Connections panel | `hm.connections_panel(client)` | Status + list of connections |
| Recent results | `hm.recent_results(client)` | Browse past query results |
| Run history | `hm.run_history(client)` | Recent query runs |

```python
return hm.connection_status(client)
return hm.connections_panel(client)
return hm.recent_results(client).ui
return hm.run_history(client)
```
All widgets are also available as `mo.ui.hotdata_*` aliases (e.g. `mo.ui.hotdata_sql_editor`) for discovery via Marimo's autocomplete.

## Demo notebook

```bash
uv run marimo edit examples/demo.py --no-token
```

`examples/demo.py` combines workspace selection, catalog browsing, managed databases, query history, and a native `mo.sql` cell.
The demo combines workspace selection, schema browsing, managed databases, query history, and a native SQL cell in a single tabbed interface.

## Development

```bash
uv sync --locked
uv run pytest
```

See [hotdata-runtime](https://github.com/hotdata-dev/hotdata-runtime) for the underlying API client.
15 changes: 9 additions & 6 deletions hotdata_marimo/databases.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ def databases_panel(client: HotdataClient):
gap=1,
)
rows: list[dict[str, object]] = [
{"name": db.name, "id": db.id, "sql_prefix": f"{db.name}.{{schema}}.{{table}}"}
{"description": db.description or db.id, "id": db.id, "sql_prefix": f"{db.id}.{{schema}}.{{table}}"}
for db in dbs
]
return mo.vstack(
Expand Down Expand Up @@ -127,13 +127,16 @@ def _rebuild_database_pick(self) -> None:
message="(create one first)",
)
return
options = {db.name: db.name for db in dbs}
value = current if current in options else next(iter(options))
options = {db.description or db.id: db.id for db in dbs}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Two managed databases with the same description (or both falling back to the same db.id collision pattern) will silently collapse to one dropdown entry, and the second one becomes unselectable. workspace_selector.py already uses unique_label_options to disambiguate — consider reusing the same helper here (e.g. label as f"{description} ({id})" on collision). (not blocking)

# current holds the previously selected database ID (.value returns the dict value).
# mo.ui.dropdown validates value= against option keys (labels), not values.
default_key = next(iter(options))
selected_key = next((k for k, v in options.items() if v == current), default_key)
self.database = mo.ui.dropdown(
options=options,
label="Database",
full_width=True,
value=value,
value=selected_key,
)

def _maybe_create(self) -> None:
Expand All @@ -153,7 +156,7 @@ def _maybe_create(self) -> None:
tables = _parse_table_names(self.tables.value)
try:
self._create_result = self._client.create_managed_database(
db_name,
description=db_name,
schema=schema,
tables=tables or None,
)
Expand Down Expand Up @@ -209,7 +212,7 @@ def result_panel(self):
db = self._create_result
return mo.callout(
mo.md(
f"Created **{db.name}** (`{db.id}`). "
f"Created **{db.description or db.id}** (`{db.id}`). "
"Load parquet into a declared table below."
),
kind="success",
Expand Down
9 changes: 6 additions & 3 deletions hotdata_marimo/sql_editor.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,10 @@ def __init__(
default_sql: str = "",
label: str = "SQL",
run_label: str = "Run on Hotdata",
database: str | None = None,
) -> None:
self._client = client
self._database = database
self.sql = mo.ui.text_area(default_sql, label=label)
self.run = mo.ui.button(
value=0,
Expand Down Expand Up @@ -103,7 +105,7 @@ def _execute_or_cached(self) -> QueryResult | None:
title="Running on Hotdata",
subtitle="Re-running last query and waiting for results…",
):
result = self._client.execute_sql(self._cached_sql or "")
result = self._client.execute_sql(self._cached_sql, database=self._database)
self._result_cache = result
self._last_rerun_n = rerun_n
return result
Expand All @@ -113,7 +115,7 @@ def _execute_or_cached(self) -> QueryResult | None:
title="Running on Hotdata",
subtitle="Executing query and waiting for results…",
):
result = self._client.execute_sql(sql_text)
result = self._client.execute_sql(sql_text, database=self._database)
self._result_cache = result
self._cached_sql = sql_text
self._last_run_n = run_n
Expand Down Expand Up @@ -195,7 +197,8 @@ def sql_editor(
default_sql: str = "",
label: str = "SQL",
run_label: str = "Run on Hotdata",
database: str | None = None,
) -> SqlEditor:
return SqlEditor(
client, default_sql=default_sql, label=label, run_label=run_label
client, default_sql=default_sql, label=label, run_label=run_label, database=database
)
9 changes: 8 additions & 1 deletion hotdata_marimo/sql_engine.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,12 @@ def __init__(
self,
connection: HotdataClient,
engine_name: VariableName | None = None,
*,
default_database: str | None = None,
) -> None:
super().__init__(connection, engine_name)
self._connections_cache: list[Any] | None = None
self._default_database = default_database
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: default_database is only honored when users manually instantiate HotdataMarimoEngine. Through register_hotdata_sql_engine() Marimo's registry constructs the engine itself with just (connection, engine_name), so this kwarg never gets a value via the documented flow. Worth either documenting how to inject it (e.g. by registering a pre-built engine) or dropping it until there's a real path. (not blocking)


@property
def source(self) -> str:
Expand Down Expand Up @@ -291,7 +294,7 @@ def get_table_details(
)

def execute(self, query: str) -> Any:
qr = self._connection.execute_sql(query)
qr = self._connection.execute_sql(query, database=self._default_database)
fmt = self.sql_output_format()

def to_polars() -> Any:
Expand Down Expand Up @@ -365,7 +368,11 @@ def register_hotdata_sql_engine() -> None:

def unregister_hotdata_sql_engine() -> None:
"""Remove :class:`HotdataMarimoEngine` from Marimo's registry (mostly for tests)."""
global _ORIGINAL_ENGINE_TO_CONNECTION
from marimo._sql.get_engines import SUPPORTED_ENGINES

while HotdataMarimoEngine in SUPPORTED_ENGINES:
SUPPORTED_ENGINES.remove(HotdataMarimoEngine)
if _ORIGINAL_ENGINE_TO_CONNECTION is not None:
_set_engine_to_data_source_connection(_ORIGINAL_ENGINE_TO_CONNECTION)
_ORIGINAL_ENGINE_TO_CONNECTION = None
Loading
Loading