diff --git a/skills/hotdata/SKILL.md b/skills/hotdata/SKILL.md index 60ea906..d1cd4a3 100644 --- a/skills/hotdata/SKILL.md +++ b/skills/hotdata/SKILL.md @@ -88,16 +88,18 @@ Returns workspaces with `public_id`, `name`, `active`, `favorite`, `provision_st **Parquet only:** `databases tables load` accepts **parquet** files (local `--file`, remote `--url`, or a pre-staged `--upload-id`). -**Active database:** `hotdata databases set ` saves the active database to config. All `databases tables` subcommands and all `context` commands default to the active database; pass **`--database `** to override per-command. +**Active database:** `hotdata databases set ` saves the active database to config. All `databases tables` subcommands and all `context` commands default to the active database; pass **`--database `** to override per-command. + +**Always select databases by id** (`dbid...`, from `databases list`). Display names and catalog aliases are not unique — several databases can share a name, and a fork answers to the same catalog as its source — so name-based selection is ambiguous. ``` hotdata databases list [--workspace-id ] [--output table|json|yaml] hotdata databases create [--name ] [--catalog ] [--table ...] [--schema public] [--expires-at ] [--workspace-id ] [--output table|json|yaml] -hotdata databases fork [] [--name ] [--expires-at ] [--workspace-id ] [--output table|json|yaml] -hotdata databases set +hotdata databases fork [] [--name ] [--expires-at ] [--workspace-id ] [--output table|json|yaml] +hotdata databases set hotdata databases unset -hotdata databases [--workspace-id ] [--output table|json|yaml] -hotdata databases delete [--workspace-id ] +hotdata databases [--workspace-id ] [--output table|json|yaml] +hotdata databases delete [--workspace-id ] hotdata databases run [--database ] [--name
...] [--expires-at ] [--workspace-id ] [args...] hotdata databases run [args...] @@ -109,17 +111,17 @@ hotdata databases detach [--database ] hotdata databases load --catalog --table
[--schema public] (--file | --url | --upload-id ) [--workspace-id ] # Also available via tables subcommand -hotdata databases tables list [--database ] [--schema ] [--workspace-id ] [--output table|json|yaml] -hotdata databases tables load
[--database ] [--schema public] (--file | --url | --upload-id ) [--workspace-id ] -hotdata databases tables delete
[--database ] [--schema public] [--workspace-id ] +hotdata databases tables list [--database ] [--schema ] [--workspace-id ] [--output table|json|yaml] +hotdata databases tables load
[--database ] [--schema public] (--file | --url | --upload-id ) [--workspace-id ] +hotdata databases tables delete
[--database ] [--schema public] [--workspace-id ] ``` - `list` — all managed databases in the workspace. Active database is marked with `*` under the DEFAULT column; CREATED shows when each database was made. - `create` — creates a new managed database. `--name` is an optional human-readable display name. `--catalog` sets the SQL alias used in queries (`SELECT … FROM .schema.table`); must be `[a-z_][a-z0-9_]*`. `--expires-at` accepts relative durations (`24h`, `7d`, `90m`) or an RFC 3339 timestamp; omitting means no expiry. Repeat `--table` to declare tables up front. -- `fork` — creates a new managed database that is an independent deep copy of an existing one (same schemas, tables, and data); the source is left unchanged and the two diverge freely afterwards. The source defaults to the active database; pass `` (id, catalog, or name) to fork another. `--name` defaults to `-fork` (so the two stay distinguishable in `list`); `--expires-at` accepts a relative duration or RFC 3339 timestamp, and when omitted a still-future source expiry is carried over. The fork becomes the active database on success. The fork answers to the **same catalog alias** as its source inside its own scope; indexes are **not** carried over. Only databases created with the current (DuckLake) storage engine can be forked — older parquet-backed databases return an error. -- `set` — saves `` as the active database. Subsequent `databases tables` and `context` commands use it automatically. +- `fork` — creates a new managed database that is an independent deep copy of an existing one (same schemas, tables, and data); the source is left unchanged and the two diverge freely afterwards. The source defaults to the active database; pass the database `` to fork another. `--name` defaults to `-fork` (so the two stay distinguishable in `list`); `--expires-at` accepts a relative duration or RFC 3339 timestamp, and when omitted a still-future source expiry is carried over. The fork becomes the active database on success. The fork answers to the **same catalog alias** as its source inside its own scope; connection catalogs attached to the source are **re-attached** to the fork, but indexes are **not** carried over. Only databases created with the current (DuckLake) storage engine can be forked — older parquet-backed databases return an error. +- `set` — saves the database **id** as the active database. Subsequent `databases tables` and `context` commands use it automatically. Note that a successful `fork` also updates this: the fork becomes the active database. - `unset` — clears the active database from config. -- `` — inspect one database (id, catalog, name, expires_at). +- `` — inspect one database (returns id, catalog, name, expires_at). - `delete` — removes the managed database; clears the active-database config if it matched. - `load` (top-level shorthand) — loads parquet into `--catalog.--schema.--table`. Accepts `--file`, `--url`, or `--upload-id`. If the table was not declared at create time, the CLI automatically deletes and recreates the database with the table declared, then retries the load. - `tables list` — lists tables with `TABLE` (`..
`), `SYNCED`, `LAST_SYNC`. Uses active database when `--database` is omitted. diff --git a/skills/hotdata/references/WORKFLOWS.md b/skills/hotdata/references/WORKFLOWS.md index 90cde70..cf367f6 100644 --- a/skills/hotdata/references/WORKFLOWS.md +++ b/skills/hotdata/references/WORKFLOWS.md @@ -121,7 +121,7 @@ A `hotdata query` runs inside **one** managed database; its scope sees that data hotdata databases load --catalog sales --table customers --url https://example.com/customers.parquet ``` - > Auto-declaring a *new* table recreates the database (no add-table API), which **changes its `id`**. Always reference a managed database by its **catalog** (stable), not the `id` returned by `databases create` — that id goes stale after the next `load` of an undeclared table. Declare tables up front (`databases create --table orders --table customers`) to avoid the recreate. + > Auto-declaring a *new* table recreates the database (no add-table API), which **changes its `id`** — the id returned by `databases create` goes stale after the next `load` of an undeclared table. Declare tables up front (`databases create --table orders --table customers`) to avoid the recreate, and don't cache ids across loads: re-read the current id from `databases list` at time of use. (Selection is still always by id — names and catalogs are not unique.) 3. Confirm and query: @@ -132,6 +132,19 @@ A `hotdata query` runs inside **one** managed database; its scope sees that data For **Chain** materializations into managed databases, see **`hotdata-analytics`**. +### Workflow: fork before risky changes + +Before destructive experimentation (bulk replaces, schema rework, testing a load pipeline), fork the database and experiment on the copy — the source stays untouched and the two diverge freely: + +```bash +hotdata databases list # note the source database id (dbid...) +hotdata databases set # source to protect (`set` takes an id) +hotdata databases fork --expires-at 24h # deep copy; becomes the active database — note the fork id it prints +hotdata databases load --catalog sales --table orders --file ./risky.parquet # hits the fork +``` + +**Capture both ids.** After the fork, both databases answer to the same catalog alias (here `sales`), so ids are the only unambiguous way to refer to either one — the source id comes from `databases list` up front, the fork id from the `fork` output. The shared alias means experimental SQL runs unchanged against the fork. Attached connections are re-attached to the fork; indexes are not carried over. When done, keep the fork (`databases set ` to switch back to the source) or delete it (`databases delete `). Only DuckLake-backed databases can be forked — see `fork` in the main skill for details. + --- ## Model