-
Notifications
You must be signed in to change notification settings - Fork 0
docs: improve README for developer experience #9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
f1f6188
b1f9d5c
1e56b56
d78bb80
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,151 +1,122 @@ | ||
| # hotdata-marimo | ||
|
|
||
| Marimo widgets for [Hotdata](https://hotdata.dev): run SQL, browse catalogs, load managed databases, and display results in notebooks. | ||
|
|
||
| Requires Python 3.10+, [Marimo](https://marimo.io/), and [hotdata-runtime](https://github.com/hotdata-dev/hotdata-runtime) (installed automatically). | ||
|
|
||
| ## Supported widgets | ||
|
|
||
| Importing `hotdata_marimo` registers `mo.ui.hotdata_*` aliases for discoverability. | ||
|
|
||
| | Widget | Function | Notes | | ||
| |--------|----------|-------| | ||
| | SQL editor | `hm.sql_editor(client)` | Returns `.ui` and `.result` | | ||
| | Table browser | `hm.table_browser(client)` | Browse connections, schemas, tables, columns | | ||
| | Managed databases panel | `hm.databases_panel(client)` | Create catalogs and load parquet files | | ||
| | Managed database writer | `hm.managed_database_writer(client)` | Lower-level create/load UI | | ||
| | Workspace selector | `hm.workspace_selector_from_env()` | Pick workspace when `HOTDATA_WORKSPACE` is unset | | ||
| | Connection picker | `hm.connection_picker(client)` | Dropdown of workspace connections | | ||
| | Connection status | `hm.connection_status(client)` | API / workspace health callout | | ||
| | Connections panel | `hm.connections_panel(client)` | Status callout plus connection list | | ||
| | Query result | `hm.query_result(result)` | Render a `QueryResult` as a table | | ||
| | Recent results | `hm.recent_results(client)` | Browse past query results | | ||
| | Run history | `hm.run_history(client)` | Recent query runs | | ||
|
|
||
| Each widget also has a `mo.ui.hotdata_*` alias (e.g. `mo.ui.hotdata_sql_editor`). Native Marimo SQL cells are supported via `hm.register_hotdata_sql_engine()` and `mo.sql(..., engine=client)`. | ||
| [Marimo](https://marimo.io/) widgets for [Hotdata](https://hotdata.dev) — run SQL, browse your schema, and work with managed databases in reactive notebooks. | ||
|
|
||
| ## Install | ||
|
|
||
| ```bash | ||
| pip install hotdata-marimo | ||
| ``` | ||
|
|
||
| Set `HOTDATA_API_KEY`. Optionally set `HOTDATA_WORKSPACE`, `HOTDATA_API_URL`, or `HOTDATA_SANDBOX`. | ||
|
|
||
| ## Connect | ||
|
|
||
| ```python | ||
| import hotdata_marimo as hm | ||
|
|
||
| client = hm.from_env() | ||
| ``` | ||
|
|
||
| If `HOTDATA_WORKSPACE` is unset, pick a workspace interactively: | ||
|
|
||
| ```python | ||
| ws = hm.workspace_selector_from_env() | ||
| client = ws.client | ||
| ``` | ||
| ## Authentication | ||
|
|
||
| ## SQL editor widget | ||
| Set `HOTDATA_API_KEY` in your environment. Optionally set `HOTDATA_WORKSPACE` to pin a specific workspace (the first available workspace is used if unset). | ||
|
|
||
| Run SQL in one cell; show results in the next. Marimo only renders what you **`return`**. | ||
| ## Quickstart | ||
|
|
||
| **Cell 1 — editor** | ||
| Because Marimo reruns cells reactively, construct a widget in one cell and read its `.ui` or `.result` in the next. | ||
|
|
||
| ```python | ||
| import marimo as mo | ||
| # Cell 1 | ||
| import hotdata_marimo as hm | ||
|
|
||
| client = hm.from_env() | ||
| editor = hm.sql_editor(client, default_sql="SELECT 1 AS ok") | ||
| return editor.ui | ||
| ``` | ||
|
|
||
| **Cell 2 — result** | ||
|
|
||
| ```python | ||
| # Cell 2 | ||
| return hm.query_result(editor.result) | ||
| ``` | ||
|
|
||
| Click **Run on Hotdata** after changing SQL. The editor caches the last successful result so downstream cells do not re-query on every refresh. | ||
| Click **Run on Hotdata** after editing SQL. The editor caches the last successful result so downstream cells don't re-query on every refresh. | ||
|
|
||
| ## Native Marimo SQL cells | ||
| ## Workspace selection | ||
|
|
||
| If you have multiple workspaces or `HOTDATA_WORKSPACE` is unset, add an interactive picker. `ws.client` updates when the selection changes: | ||
|
|
||
| ```python | ||
| ws = hm.workspace_selector_from_env() | ||
| client = ws.client | ||
| return ws.ui | ||
| ``` | ||
|
|
||
| Register the Hotdata engine once, then pass `engine=client` to `mo.sql`. Hotdata appears as **Hotdata** in the SQL connection picker. | ||
| ## Native Marimo SQL cells | ||
|
|
||
| **Setup cell** | ||
| Register the Hotdata engine once and Hotdata will appear as a selectable engine in the SQL connection picker: | ||
|
|
||
| ```python | ||
| # Setup cell | ||
| import marimo as mo | ||
| import hotdata_marimo as hm | ||
|
|
||
| hm.register_hotdata_sql_engine() | ||
| client = hm.from_env() | ||
| ``` | ||
|
|
||
| **SQL cell** | ||
|
|
||
| ```python | ||
| _df = mo.sql( | ||
| """ | ||
| SELECT 1 AS example_value | ||
| """, | ||
| engine=client, | ||
| ) | ||
| # Any SQL cell | ||
| _df = mo.sql("SELECT * FROM orders LIMIT 10", engine=client) | ||
| ``` | ||
|
|
||
|  | ||
|
|
||
| ## Browse tables | ||
| ## Browse your schema | ||
|
|
||
| The table browser lets you pick a connection, search for a table, and inspect its columns — with a starter query ready to copy: | ||
|
|
||
| ```python | ||
| browser = hm.table_browser(client) | ||
| return browser.ui | ||
| ``` | ||
|
|
||
| Pick a connection, schema, and table to inspect columns. Use `browser.selected_table` in downstream cells. | ||
| Use `browser.selected_table` in downstream cells to reference the chosen table. | ||
|
|
||
| ## Managed databases | ||
|
|
||
| Create a Hotdata-owned catalog and load a parquet file from the notebook: | ||
| View existing managed databases and load new parquet files from a single tabbed panel: | ||
|
|
||
| ```python | ||
| panel = hm.databases_panel(client) | ||
| return panel | ||
| writer = hm.managed_database_writer(client) | ||
| return writer.tab_ui | ||
| ``` | ||
|
|
||
| Or use the lower-level writer API: | ||
| Or show just the read-only panel: | ||
|
|
||
| ```python | ||
| writer = hm.managed_database_writer(client) | ||
| return writer.ui | ||
| return hm.databases_panel(client) | ||
| ``` | ||
|
|
||
| ## Other helpers | ||
|
|
||
| See [Supported widgets](#supported-widgets) for the full list. Quick examples: | ||
| ## All widgets | ||
|
|
||
| | Widget | Code | What you get | | ||
| |--------|------|-------------| | ||
| | SQL editor | `hm.sql_editor(client)` | `.ui` to show the editor, `.result` to read rows | | ||
| | Query result | `hm.query_result(result)` | Renders a `QueryResult` as a table | | ||
| | Table browser | `hm.table_browser(client)` | Browse connections, tables, and column metadata | | ||
| | Managed databases | `hm.databases_panel(client)` | Read-only list of managed databases | | ||
| | Database writer | `hm.managed_database_writer(client)` | Create databases and load parquet files | | ||
| | Workspace picker | `hm.workspace_selector_from_env()` | Dropdown to switch workspaces | | ||
| | Connection picker | `hm.connection_picker(client)` | Dropdown of connections in the workspace | | ||
| | Connection status | `hm.connection_status(client)` | Health callout for the API and workspace | | ||
| | Connections panel | `hm.connections_panel(client)` | Status + list of connections | | ||
| | Recent results | `hm.recent_results(client)` | Browse past query results | | ||
| | Run history | `hm.run_history(client)` | Recent query runs | | ||
|
|
||
| ```python | ||
| return hm.connection_status(client) | ||
| return hm.connections_panel(client) | ||
| return hm.recent_results(client).ui | ||
| return hm.run_history(client) | ||
| ``` | ||
| All widgets are also available as `mo.ui.hotdata_*` aliases (e.g. `mo.ui.hotdata_sql_editor`) for discovery via Marimo's autocomplete. | ||
|
|
||
| ## Demo notebook | ||
|
|
||
| ```bash | ||
| uv run marimo edit examples/demo.py --no-token | ||
| ``` | ||
|
|
||
| `examples/demo.py` combines workspace selection, catalog browsing, managed databases, query history, and a native `mo.sql` cell. | ||
| The demo combines workspace selection, schema browsing, managed databases, query history, and a native SQL cell in a single tabbed interface. | ||
|
|
||
| ## Development | ||
|
|
||
| ```bash | ||
| uv sync --locked | ||
| uv run pytest | ||
| ``` | ||
|
|
||
| See [hotdata-runtime](https://github.com/hotdata-dev/hotdata-runtime) for the underlying API client. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -47,7 +47,7 @@ def databases_panel(client: HotdataClient): | |
| gap=1, | ||
| ) | ||
| rows: list[dict[str, object]] = [ | ||
| {"name": db.name, "id": db.id, "sql_prefix": f"{db.name}.{{schema}}.{{table}}"} | ||
| {"description": db.description or db.id, "id": db.id, "sql_prefix": f"{db.id}.{{schema}}.{{table}}"} | ||
| for db in dbs | ||
| ] | ||
| return mo.vstack( | ||
|
|
@@ -127,13 +127,16 @@ def _rebuild_database_pick(self) -> None: | |
| message="(create one first)", | ||
| ) | ||
| return | ||
| options = {db.name: db.name for db in dbs} | ||
| value = current if current in options else next(iter(options)) | ||
| options = {db.description or db.id: db.id for db in dbs} | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nit: Two managed databases with the same description (or both falling back to the same |
||
| # current holds the previously selected database ID (.value returns the dict value). | ||
| # mo.ui.dropdown validates value= against option keys (labels), not values. | ||
| default_key = next(iter(options)) | ||
| selected_key = next((k for k, v in options.items() if v == current), default_key) | ||
| self.database = mo.ui.dropdown( | ||
| options=options, | ||
| label="Database", | ||
| full_width=True, | ||
| value=value, | ||
| value=selected_key, | ||
| ) | ||
|
|
||
| def _maybe_create(self) -> None: | ||
|
|
@@ -153,7 +156,7 @@ def _maybe_create(self) -> None: | |
| tables = _parse_table_names(self.tables.value) | ||
| try: | ||
| self._create_result = self._client.create_managed_database( | ||
| db_name, | ||
| description=db_name, | ||
| schema=schema, | ||
| tables=tables or None, | ||
| ) | ||
|
|
@@ -209,7 +212,7 @@ def result_panel(self): | |
| db = self._create_result | ||
| return mo.callout( | ||
| mo.md( | ||
| f"Created **{db.name}** (`{db.id}`). " | ||
| f"Created **{db.description or db.id}** (`{db.id}`). " | ||
| "Load parquet into a declared table below." | ||
| ), | ||
| kind="success", | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -37,9 +37,12 @@ def __init__( | |
| self, | ||
| connection: HotdataClient, | ||
| engine_name: VariableName | None = None, | ||
| *, | ||
| default_database: str | None = None, | ||
| ) -> None: | ||
| super().__init__(connection, engine_name) | ||
| self._connections_cache: list[Any] | None = None | ||
| self._default_database = default_database | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nit: |
||
|
|
||
| @property | ||
| def source(self) -> str: | ||
|
|
@@ -291,7 +294,7 @@ def get_table_details( | |
| ) | ||
|
|
||
| def execute(self, query: str) -> Any: | ||
| qr = self._connection.execute_sql(query) | ||
| qr = self._connection.execute_sql(query, database=self._default_database) | ||
| fmt = self.sql_output_format() | ||
|
|
||
| def to_polars() -> Any: | ||
|
|
@@ -365,7 +368,11 @@ def register_hotdata_sql_engine() -> None: | |
|
|
||
| def unregister_hotdata_sql_engine() -> None: | ||
| """Remove :class:`HotdataMarimoEngine` from Marimo's registry (mostly for tests).""" | ||
| global _ORIGINAL_ENGINE_TO_CONNECTION | ||
| from marimo._sql.get_engines import SUPPORTED_ENGINES | ||
|
|
||
| while HotdataMarimoEngine in SUPPORTED_ENGINES: | ||
| SUPPORTED_ENGINES.remove(HotdataMarimoEngine) | ||
| if _ORIGINAL_ENGINE_TO_CONNECTION is not None: | ||
| _set_engine_to_data_source_connection(_ORIGINAL_ENGINE_TO_CONNECTION) | ||
| _ORIGINAL_ENGINE_TO_CONNECTION = None | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: The PR description says this change "covers scoping SQL to a managed database and controlling result size", but the README never mentions the new
database=parameter onhm.sql_editor()/HotdataMarimoEngine, and there's no section on result-size controls. If those features are intentionally part of the same PR, the Quickstart or Managed databases section is a natural place to showhm.sql_editor(client, database="<id>"). (not blocking)