restructure CLAUDE.md (#382)

Author: thomasp85

CHANGELOG.md

@@ -25,6 +25,7 @@ shapes (#368)
 before registering them back to the backend. We now keep the data purely on the
 backend until the layer query as was always intended (#363)
 - Simplified internal approach to DataFrame with DuckDB reader (#365)
+- Restructured CLAUDE.md to better deal with the rising complexity of the project (#382)
 
 ### Removed
 

CLAUDE.md

@@ -0,0 +1,80 @@
+# `doc/` — Quarto documentation site
+
+Source for the public ggsql documentation at <https://ggsql.org>. **This directory is the authoritative reference for ggsql syntax and semantics.** Other CLAUDE.md files in the repo link here rather than restate.
+
+If you are about to document a clause, layer type, scale, or aesthetic anywhere else: stop and put it here instead.
+
+## Layout
+
+```
+doc/
+├── _quarto.yml             Quarto project config (navbar, sidebar, theme, llms-txt)
+├── _brand.yml, styles.scss, ggsql.xml   Branding + the ggsql syntax-highlight definition
+├── index.qmd               Landing page
+├── faq.qmd
+├── get_started/
+│   ├── installation.qmd    Platform-specific install + CLI quick reference
+│   ├── first_plot.qmd      Tutorial: first visualization
+│   ├── grammar.qmd         Grammar of Graphics conceptual foundation
+│   ├── anatomy.qmd         Anatomy of a ggsql query
+│   ├── tooling.qmd         VS Code / Positron / Jupyter / Python / R / CLI integrations
+│   └── the_rest.qmd        Advanced features
+├── syntax/
+│   ├── index.qmd
+│   ├── clause/             draw, facet, label, place, project, scale, visualise
+│   ├── layer/
+│   │   ├── type/           one .qmd per layer type (point, line, bar, …)
+│   │   └── position/       identity, stack, dodge, jitter
+│   ├── scale/
+│   │   ├── type/           binned, continuous, discrete, identity, ordinal
+│   │   └── aesthetic/      position, color, opacity, size, linewidth, shape, linetype, faceting
+│   └── coord/              cartesian, polar
+├── gallery/
+│   ├── index.qmd
+│   └── examples/           Runnable example queries
+├── assets/, vendor/        Static resources
+├── data/CSVs               Sample datasets used by examples (data.csv, sales.csv, …)
+└── wasm/                   Built playground (copied from /ggsql-wasm/demo/dist by build-wasm.sh)
+```
+
+Generated artefacts not to edit by hand:
+
+- `_site/` — Quarto build output. Not committed (see `.gitignore`).
+- `*.quarto_ipynb*` — Quarto's intermediate notebook files for `.qmd` pages with executable cells. Regenerated on build.
+- `wasm/` — produced by [`/ggsql-wasm/build-wasm.sh`](../ggsql-wasm/build-wasm.sh).
+
+## Authoring conventions
+
+- Pages are Quarto markdown (`.qmd`) — markdown plus YAML front-matter.
+- Code blocks tagged ```` ```ggsql ```` use the syntax definition in `ggsql.xml` (referenced from `_quarto.yml` under `syntax-definitions`). Use that fence for runnable / illustrative ggsql snippets.
+- Each clause page under `syntax/clause/` follows the same shape: short narrative, a "Clause syntax" code block listing all subclauses, then sections per subclause with examples. Mirror that shape when adding pages.
+- Examples that need data reference the CSV files at the top of `doc/` (e.g. `data.csv`, `sales.csv`, `metrics.csv`, `timeseries.csv`).
+- The site has `llms-txt: true` in `_quarto.yml`, so an `llms.txt` is generated for AI tooling — keep page titles and descriptions clean.
+
+## Site structure (from `_quarto.yml`)
+
+- **Navbar**: Get started · Syntax (drop-down per clause) · Gallery · FAQ · News · Playground · Python · R.
+- **Syntax sidebar**: clauses → layers (type + position) → scales (type + aesthetic) → coordinate systems. New `.qmd` files in those folders are picked up automatically via `auto:` glob entries.
+- **Get-started sidebar**: hand-ordered list (`installation` → `first_plot` → `grammar` → `anatomy` → `tooling` → `the_rest`).
+
+## Build & preview
+
+```sh
+cd doc
+quarto preview     # local server with hot reload
+quarto render      # full site build → _site/
+```
+
+The site is published from the `gh-pages` branch (see `_quarto.yml`'s `repo-branch`). Quarto extensions live in `_extensions/`.
+
+## What goes here vs. CLAUDE.md
+
+- **Here**: anything a ggsql user might want to read — clause syntax, layer types, scales, aesthetics, examples, CLI usage, installation.
+- **In CLAUDE.md files**: implementation details for contributors — module layouts, build pipelines, where to add a new geom, internal types.
+
+If you find clause/layer/scale syntax described in any CLAUDE.md file, that is a duplication bug — move it here and link from there.
+
+## See also
+
+- [`/CLAUDE.md`](../CLAUDE.md) — workspace overview.
+- [`/ggsql-wasm/CLAUDE.md`](../ggsql-wasm/CLAUDE.md) — how the embedded playground at `wasm/` is built.

doc/CLAUDE.md

@@ -0,0 +1,88 @@
+# `ggsql-jupyter/` — Jupyter kernel
+
+Standalone Rust binary that speaks the Jupyter messaging protocol over ZeroMQ. Embeds the `ggsql` library and renders results as Vega-Lite (visual queries) or HTML tables (pure SQL). Workspace member; published to crates.io and to PyPI (as a binary wheel via maturin).
+
+End-user installation and usage live in [`README.md`](README.md). End-user notebook docs live in [`/doc/get_started/tooling.qmd`](../doc/get_started/tooling.qmd). This file describes the *implementation*.
+
+## Layout
+
+```
+ggsql-jupyter/
+├── Cargo.toml          Rust binary + library, depends on ggsql with duckdb + vegalite
+├── pyproject.toml      maturin config (bindings = "bin") for the PyPI wheel
+├── README.md           User-facing install + usage
+├── src/
+│   ├── main.rs         Binary entry: clap CLI (start kernel, --install)
+│   ├── lib.rs          Library root (so internals can be unit-tested)
+│   ├── kernel.rs       Jupyter messaging loop (ZMQ, message dispatch)
+│   ├── executor.rs     Runs queries via ggsql::Reader, returns rendered output
+│   ├── connection.rs   Reader lifecycle, connection-string parsing
+│   ├── data_explorer.rs Positron data-explorer comm channel
+│   ├── display.rs      Output formatting (Vega-Lite + vega-embed HTML, SQL → HTML table)
+│   ├── message.rs      Jupyter message structs (ZMQ frames, HMAC signing)
+│   └── util.rs
+└── tests/
+    ├── test_compliance.py   Jupyter protocol conformance
+    ├── test_integration.py  End-to-end via jupyter_client
+    ├── fixtures/            Sample notebooks / queries
+    └── requirements.txt
+```
+
+## How it runs
+
+1. `ggsql-jupyter --install` writes a kernelspec into the active Python environment (Jupyter, conda, uv, virtualenv — auto-detected).
+2. `ggsql-jupyter <connection-file>` is the entry point Jupyter invokes; it reads the connection JSON, opens the five ZMQ sockets (shell, control, iopub, stdin, heartbeat), and runs `kernel.rs`'s message loop.
+3. Each `execute_request` is dispatched through `executor.rs` → `ggsql::reader::DuckDBReader::execute(...)`. The kernel keeps a single persistent in-memory DuckDB session so cells share state.
+4. The result is wrapped by `display.rs` into a Jupyter `display_data` message — Vega-Lite specs go through vega-embed in an HTML payload (works in classic Jupyter, JupyterLab, and Positron); pure SQL goes out as an HTML table.
+
+## Positron-specific bits
+
+- Kernel info advertises `"output_location": "plot"` so visualizations route to Positron's Plot pane.
+- `data_explorer.rs` implements Positron's data-explorer comm channel (registered query results become explorable tables).
+- The companion VS Code extension (`ggsql-vscode/`) discovers this binary via the `ggsql.kernelPath` setting, the active Jupyter kernelspec, or `PATH`.
+
+## Build & install
+
+```sh
+# Dev: build the binary and register with the active env
+cargo build --release --package ggsql-jupyter
+./target/release/ggsql-jupyter --install
+
+# Run a one-off install from crates.io
+cargo install ggsql-jupyter
+ggsql-jupyter --install
+
+# PyPI distribution (built via maturin in CI; pyproject.toml is a wheel-builder shim)
+pip install ggsql-jupyter && ggsql-jupyter --install
+```
+
+`pyproject.toml` declares `bindings = "bin"` — there is no Python module, the wheel just delivers the binary cross-platform.
+
+## Features
+
+```toml
+default = ["all-readers"]
+all-readers = ["sqlite", "odbc", "duckdb"]
+```
+
+Each feature passes through to `ggsql/<feature>`. The default install therefore supports DuckDB, SQLite, and ODBC connection strings.
+
+## Testing
+
+The Rust side has unit tests inline (`cargo test -p ggsql-jupyter`). The Jupyter protocol tests are Python:
+
+```sh
+cd ggsql-jupyter/tests
+python -m venv .venv && source .venv/bin/activate
+pip install -r requirements.txt
+pytest
+```
+
+`test_compliance.py` verifies handler coverage (`execute_request`, `kernel_info_request`, `is_complete_request`, `shutdown_request`); `test_integration.py` drives a real kernel via `jupyter_client`.
+
+## See also
+
+- [`/CLAUDE.md`](../CLAUDE.md) — workspace overview.
+- [`/ggsql-vscode/CLAUDE.md`](../ggsql-vscode/CLAUDE.md) — the VS Code / Positron extension that drives this kernel.
+- [`/src/CLAUDE.md`](../src/CLAUDE.md) — the underlying `ggsql` library.
+- [`/doc/get_started/tooling.qmd`](../doc/get_started/tooling.qmd) — user-facing notebook docs.

ggsql-jupyter/CLAUDE.md

@@ -0,0 +1,87 @@
+# `ggsql-vscode/` — VS Code / Positron extension
+
+TypeScript extension that adds ggsql language support to VS Code and Positron: syntax highlighting, code-cell execution, connection management, and (in Positron) a registered language runtime that drives the `ggsql-jupyter` kernel.
+
+Not a Cargo workspace member — this is a standalone npm project. End-user docs live in [`README.md`](README.md). Tooling overview for users: [`/doc/get_started/tooling.qmd`](../doc/get_started/tooling.qmd). This file describes the *implementation*.
+
+## Layout
+
+```
+ggsql-vscode/
+├── package.json              Extension manifest (commands, keybindings, languages, runtime)
+├── tsconfig.json
+├── esbuild.js                Bundler config (builds out/extension.js)
+├── eslint.config.mjs
+├── language-configuration.json   Bracket pairs, comment markers
+├── logo.png, icon.png
+├── src/
+│   ├── extension.ts          activate(): registers commands, manager, code lenses
+│   ├── manager.ts            Kernel discovery + Positron language-runtime registration
+│   ├── connections.ts        Connection-string handling for the Connections pane
+│   ├── cellParser.ts         Splits .ggsql files into cells for Run-Cell commands
+│   ├── codelens.ts           "▶ Run cell" lens above each cell
+│   ├── decorations.ts        Cell separator decorations
+│   ├── context.ts            Sets editor context keys (e.g. ggsql.hasCodeCells)
+│   └── types.ts              Shared interfaces
+├── syntaxes/
+│   └── ggsql.tmLanguage.json TextMate grammar (used for tokenization in VS Code)
+├── examples/                 Sample .ggsql files
+├── resources/                Static assets bundled with the extension
+└── ggsql-0.1.0.vsix          Packaged extension (build artifact, may be stale)
+```
+
+## File extensions and language ID
+
+`package.json` registers `id: ggsql` for `.ggsql`, `.ggsql.sql`, and `.gsql`. The TextMate grammar at `syntaxes/ggsql.tmLanguage.json` provides tokenization. Tree-sitter highlights — used by editors that prefer the grammar package directly — live in [`/tree-sitter-ggsql/queries/highlights.scm`](../tree-sitter-ggsql/queries/highlights.scm).
+
+## Commands and keybindings
+
+Declared in `package.json` and wired up in `extension.ts`:
+
+| Command | Default key | Purpose |
+| --- | --- | --- |
+| `ggsql.runCurrentAdvance` | Cmd/Ctrl+Enter, Shift+Enter | Run current cell, advance to next |
+| `ggsql.runQuery` | Cmd/Ctrl+Shift+Enter | Run current cell only |
+| `ggsql.runNextCell` | — | Run the next cell |
+| `ggsql.runCellsAbove` | — | Run all cells above the cursor |
+| `ggsql.sourceCurrentFile` | — | Run the entire file (also exposed as the editor "Run" button) |
+
+Cells are detected by `cellParser.ts`; `codelens.ts` puts a CodeLens above each cell.
+
+## Positron integration
+
+The extension declares `contributes.languageRuntimes` for `ggsql` (see `package.json`) and depends on `@posit-dev/positron`. When activated under Positron, `manager.ts`:
+
+1. Discovers a `ggsql-jupyter` binary via, in order: the `ggsql.kernelPath` setting, an installed Jupyter kernelspec named `ggsql`, or `ggsql-jupyter` on `PATH`.
+2. Registers it as a Positron language runtime so `▶ Run` and the Console route to the kernel.
+3. Routes plot output to Positron's Plot pane via metadata coming back from the kernel (`output_location: "plot"`).
+
+Outside Positron, the same commands fall back to writing query output to the active terminal.
+
+## Settings
+
+```json
+{
+  "ggsql.kernelPath": "string"   // empty → use 'ggsql-jupyter' from PATH
+}
+```
+
+## Build & package
+
+```sh
+cd ggsql-vscode
+npm install                # one-time
+npm run check-types        # tsc --noEmit
+npm run package            # esbuild → out/extension.js (production)
+npx vsce package           # produces ggsql-<version>.vsix
+code --install-extension ggsql-<version>.vsix
+```
+
+Watch mode for development: `npm run watch` (runs esbuild + tsc in parallel).
+
+## See also
+
+- [`/CLAUDE.md`](../CLAUDE.md) — workspace overview.
+- [`/ggsql-jupyter/CLAUDE.md`](../ggsql-jupyter/CLAUDE.md) — the kernel this extension drives.
+- [`/tree-sitter-ggsql/CLAUDE.md`](../tree-sitter-ggsql/CLAUDE.md) — grammar that powers more advanced editor highlighting.
+- [`/doc/get_started/tooling.qmd`](../doc/get_started/tooling.qmd) — user-facing tooling docs.

ggsql-vscode/CLAUDE.md

@@ -0,0 +1,80 @@
+# `ggsql-wasm/` — WebAssembly bindings
+
+Compiles the `ggsql` core to WebAssembly so it can run in browsers. Used by the playground at [`/doc/wasm/`](../doc/wasm/) and published as an npm package. Workspace member.
+
+End-user playground: <https://ggsql.org/wasm/>. This file describes the *build*.
+
+## Layout
+
+```
+ggsql-wasm/
+├── Cargo.toml            cdylib; ggsql with default-features = false + vegalite, sqlite, builtin-data
+├── build-wasm.sh         End-to-end build orchestrator (library + wasm + demo → doc/wasm)
+├── src/
+│   └── lib.rs            wasm-bindgen entry points (the only Rust here)
+├── library/              TypeScript wrapper distributed on npm
+│   ├── package.json      npm package (build with `npm run build`)
+│   ├── build.mjs         esbuild script
+│   └── src/
+├── demo/                 Browser demo + playground used by the doc site
+│   ├── package.json
+│   ├── build.mjs
+│   └── src/              UI code (editor + Vega-Lite preview)
+└── pkg/                  wasm-pack output (committed; consumed by library/ and demo/)
+    ├── ggsql_wasm_bg.wasm
+    ├── ggsql_wasm.js, .d.ts
+    └── package.json
+```
+
+`pkg/` is generated but committed so contributors don't need a wasm toolchain just to run the docs.
+
+## Toolchain
+
+- Rust target `wasm32-unknown-unknown` and [`wasm-pack`](https://rustwasm.github.io/wasm-pack/) for compilation.
+- A clang/llvm with wasm backend support (the build script verifies this with a one-line probe).
+- `wasm-opt` (from binaryen) for the `-Oz` optimization step.
+- Node.js for `library/` and `demo/`.
+
+## Build
+
+The full build:
+
+```sh
+cd ggsql-wasm
+./build-wasm.sh
+```
+
+This sequentially:
+
+1. `npm install && npm run build` in `library/` — produces the typed JS wrapper.
+2. `wasm-pack build --target web --profile wasm --no-opt` — compiles `src/lib.rs` to `pkg/`. The `wasm` profile is defined in the workspace `Cargo.toml` (release-style, `opt-level = "z"`, LTO, `panic = "abort"`).
+3. `wasm-opt pkg/ggsql_wasm_bg.wasm -o pkg/ggsql_wasm_bg.wasm -Oz` — shrinks the binary further.
+4. `npm install && npm run build` in `demo/` — bundles the playground UI.
+5. Copies `demo/dist/` to `/doc/wasm/` so Quarto can serve it under the docs site.
+
+Flags:
+
+- `--skip-binary` — reuse the existing `pkg/` (skip steps 2–3); useful when iterating on `library/` or `demo/`.
+- `--skip-opt` — compile but skip `wasm-opt` (faster, larger binary).
+
+## Wasm-specific feature constraints
+
+`Cargo.toml` carves out wasm32-only dependency overrides:
+
+- `getrandom` and `uuid` are forced to the `js` feature so they get randomness from the browser.
+- `sqlite-wasm-rs` replaces `rusqlite` for SQLite support in the browser.
+- `tokio` is reduced to `default-features = false` (no I/O reactor on wasm).
+
+ODBC is not enabled here — it requires host APIs that aren't available in the browser.
+
+## Distribution
+
+- **npm**: `library/` is published as the user-facing JS/TS wrapper. The `pkg/` artifact is bundled with it.
+- **GitHub Releases**: the wasm binary is also attached to releases (see commit `071cff6`).
+- **Docs site**: `demo/dist/` is committed into [`/doc/wasm/`](../doc/wasm/) by `build-wasm.sh` and embedded in Quarto pages via `_quarto.yml`.
+
+## See also
+
+- [`/CLAUDE.md`](../CLAUDE.md) — workspace overview.
+- [`/src/CLAUDE.md`](../src/CLAUDE.md) — the underlying `ggsql` library.
+- [`/doc/CLAUDE.md`](../doc/CLAUDE.md) — how the playground gets embedded into the Quarto site.