Skip to content

docs(skills): document fork re-attach behavior and add fork-before-risky-changes workflow #229

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
eddietejeda merged 4 commits into from
Jul 16, 2026
Merged

docs(skills): document fork re-attach behavior and add fork-before-risky-changes workflow #229

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
5f78374
docs(skills): document fork connection re-attach, active-db side effe…
eddietejeda Jul 16, 2026
0392d57
docs(skills): reference source database by id in fork workflow; docum…
eddietejeda Jul 16, 2026
93db544
docs(skills): database selection is always by id — names and catalogs…
eddietejeda Jul 16, 2026
c63bb38
docs(skills): fork workflow — capture the fork id too; delete by fork id
eddietejeda Jul 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 13 additions & 11 deletions skills/hotdata/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,16 +88,18 @@ Returns workspaces with `public_id`, `name`, `active`, `favorite`, `provision_st

**Parquet only:** `databases tables load` accepts **parquet** files (local `--file`, remote `--url`, or a pre-staged `--upload-id`).

**Active database:** `hotdata databases set <id_or_description>` saves the active database to config. All `databases tables` subcommands and all `context` commands default to the active database; pass **`--database <id>`** to override per-command.
**Active database:** `hotdata databases set <id>` saves the active database to config. All `databases tables` subcommands and all `context` commands default to the active database; pass **`--database <id>`** to override per-command.

**Always select databases by id** (`dbid...`, from `databases list`). Display names and catalog aliases are not unique — several databases can share a name, and a fork answers to the same catalog as its source — so name-based selection is ambiguous.

```
hotdata databases list [--workspace-id <workspace_id>] [--output table|json|yaml]
hotdata databases create [--name <display_name>] [--catalog <alias>] [--table <table> ...] [--schema public] [--expires-at <duration|timestamp>] [--workspace-id <workspace_id>] [--output table|json|yaml]
hotdata databases fork [<id_or_name>] [--name <display_name>] [--expires-at <duration|timestamp>] [--workspace-id <workspace_id>] [--output table|json|yaml]
hotdata databases set <id_or_name>
hotdata databases fork [<id>] [--name <display_name>] [--expires-at <duration|timestamp>] [--workspace-id <workspace_id>] [--output table|json|yaml]
hotdata databases set <id>
hotdata databases unset
hotdata databases <id_or_name> [--workspace-id <workspace_id>] [--output table|json|yaml]
hotdata databases delete <id_or_name> [--workspace-id <workspace_id>]
hotdata databases <id> [--workspace-id <workspace_id>] [--output table|json|yaml]
hotdata databases delete <id> [--workspace-id <workspace_id>]
hotdata databases run [--database <id>] [--name <label>] [--schema public] [--table <table> ...] [--expires-at <duration|timestamp>] [--workspace-id <workspace_id>] <cmd> [args...]
hotdata databases <id> run <cmd> [args...]

Expand All @@ -109,17 +111,17 @@ hotdata databases detach <connection_id|name|alias> [--database <id>]
hotdata databases load --catalog <alias> --table <table> [--schema public] (--file <path> | --url <url> | --upload-id <id>) [--workspace-id <workspace_id>]

# Also available via tables subcommand
hotdata databases tables list [--database <id_or_name>] [--schema <name>] [--workspace-id <workspace_id>] [--output table|json|yaml]
hotdata databases tables load <table> [--database <id_or_name>] [--schema public] (--file <path> | --url <url> | --upload-id <id>) [--workspace-id <workspace_id>]
hotdata databases tables delete <table> [--database <id_or_name>] [--schema public] [--workspace-id <workspace_id>]
hotdata databases tables list [--database <id>] [--schema <name>] [--workspace-id <workspace_id>] [--output table|json|yaml]
hotdata databases tables load <table> [--database <id>] [--schema public] (--file <path> | --url <url> | --upload-id <id>) [--workspace-id <workspace_id>]
hotdata databases tables delete <table> [--database <id>] [--schema public] [--workspace-id <workspace_id>]
```

- `list` — all managed databases in the workspace. Active database is marked with `*` under the DEFAULT column; CREATED shows when each database was made.
- `create` — creates a new managed database. `--name` is an optional human-readable display name. `--catalog` sets the SQL alias used in queries (`SELECT … FROM <catalog>.schema.table`); must be `[a-z_][a-z0-9_]*`. `--expires-at` accepts relative durations (`24h`, `7d`, `90m`) or an RFC 3339 timestamp; omitting means no expiry. Repeat `--table` to declare tables up front.
- `fork` — creates a new managed database that is an independent deep copy of an existing one (same schemas, tables, and data); the source is left unchanged and the two diverge freely afterwards. The source defaults to the active database; pass `<id_or_name>` (id, catalog, or name) to fork another. `--name` defaults to `<source>-fork` (so the two stay distinguishable in `list`); `--expires-at` accepts a relative duration or RFC 3339 timestamp, and when omitted a still-future source expiry is carried over. The fork becomes the active database on success. The fork answers to the **same catalog alias** as its source inside its own scope; indexes are **not** carried over. Only databases created with the current (DuckLake) storage engine can be forked — older parquet-backed databases return an error.
- `set` — saves `<id_or_name>` as the active database. Subsequent `databases tables` and `context` commands use it automatically.
- `fork` — creates a new managed database that is an independent deep copy of an existing one (same schemas, tables, and data); the source is left unchanged and the two diverge freely afterwards. The source defaults to the active database; pass the database `<id>` to fork another. `--name` defaults to `<source>-fork` (so the two stay distinguishable in `list`); `--expires-at` accepts a relative duration or RFC 3339 timestamp, and when omitted a still-future source expiry is carried over. The fork becomes the active database on success. The fork answers to the **same catalog alias** as its source inside its own scope; connection catalogs attached to the source are **re-attached** to the fork, but indexes are **not** carried over. Only databases created with the current (DuckLake) storage engine can be forked — older parquet-backed databases return an error.
- `set` — saves the database **id** as the active database. Subsequent `databases tables` and `context` commands use it automatically. Note that a successful `fork` also updates this: the fork becomes the active database.
- `unset` — clears the active database from config.
- `<id_or_name>` — inspect one database (id, catalog, name, expires_at).
- `<id>` — inspect one database (returns id, catalog, name, expires_at).
- `delete` — removes the managed database; clears the active-database config if it matched.
- `load` (top-level shorthand) — loads parquet into `--catalog.--schema.--table`. Accepts `--file`, `--url`, or `--upload-id`. If the table was not declared at create time, the CLI automatically deletes and recreates the database with the table declared, then retries the load.
- `tables list` — lists tables with `TABLE` (`<catalog>.<schema>.<table>`), `SYNCED`, `LAST_SYNC`. Uses active database when `--database` is omitted.
Expand Down
15 changes: 14 additions & 1 deletion skills/hotdata/references/WORKFLOWS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ A `hotdata query` runs inside **one** managed database; its scope sees that data
hotdata databases load --catalog sales --table customers --url https://example.com/customers.parquet
```

> Auto-declaring a *new* table recreates the database (no add-table API), which **changes its `id`**. Always reference a managed database by its **catalog** (stable), not the `id` returned by `databases create` — that id goes stale after the next `load` of an undeclared table. Declare tables up front (`databases create --table orders --table customers`) to avoid the recreate.
> Auto-declaring a *new* table recreates the database (no add-table API), which **changes its `id`** — the id returned by `databases create` goes stale after the next `load` of an undeclared table. Declare tables up front (`databases create --table orders --table customers`) to avoid the recreate, and don't cache ids across loads: re-read the current id from `databases list` at time of use. (Selection is still always by id — names and catalogs are not unique.)

3. Confirm and query:

Expand All @@ -132,6 +132,19 @@ A `hotdata query` runs inside **one** managed database; its scope sees that data

For **Chain** materializations into managed databases, see **`hotdata-analytics`**.

### Workflow: fork before risky changes

Before destructive experimentation (bulk replaces, schema rework, testing a load pipeline), fork the database and experiment on the copy — the source stays untouched and the two diverge freely:

```bash
hotdata databases list # note the source database id (dbid...)
hotdata databases set <source_id> # source to protect (`set` takes an id)
hotdata databases fork --expires-at 24h # deep copy; becomes the active database
hotdata databases load --catalog sales --table orders --file ./risky.parquet # hits the fork
```

**Capture the source database id up front.** After the fork, both databases answer to the same catalog alias (here `sales`), so the id is the only unambiguous way to refer back to the source. The shared alias means experimental SQL runs unchanged against the fork. Attached connections are re-attached to the fork; indexes are not carried over. When done, keep the fork (`databases set <source_id>` to switch back to the source) or `databases delete` it. Only DuckLake-backed databases can be forked — see `fork` in the main skill for details.

@claude claude Bot Jul 16, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

super nit: (not blocking) The databases delete cleanup path has the same catalog-ambiguity problem this workflow was rewritten to avoid. After the fork, both source and fork answer to catalog sales, so databases delete sales hits resolve_database and errors with "multiple databases have catalog 'sales' — use the database id instead" (src/commands/databases.rs:457-466). Deleting the fork therefore needs the fork id, but the workflow only tells the agent to capture the source id. The fork id is available (the fork command prints it / it's the new active db), so consider noting that databases delete also needs the fork's dbid..., e.g. "or databases delete <fork_id> it (the id fork printed)."

@eddietejeda eddietejeda Jul 16, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — addressed in the latest commit: the fork step now says to note the fork id from the command output, and the cleanup line reads databases delete <fork_id>. The prose now says to capture both ids (source from databases list, fork from the fork output).


---

## Model
Expand Down
Loading