Skip to content

Commit 889ed5b

Browse files
feat: query JSON/recipes, golden & benchmark tooling, docs hub (#8)
* feat: query JSON/recipes, golden & benchmark tooling, docs hub CLI (codemap query) - Add --json (JSON array; {"error"} on failure) and --recipe <id> with bundled SQL. - Wire query-recipes, bootstrap/main/cmd-query, index-engine error handling. Golden query regression - scripts/query-golden.ts + Zod schema; --corpus minimal|external, --update, --strict-budget. - fixtures/golden scenarios + minimal goldens; external example + gitignore. Benchmark - Split benchmark into benchmark-common/defaults/config; CODEMAP_BENCHMARK_CONFIG JSON. - fixtures/benchmark/scenarios.example.json; validate CODEMAP_ROOT on benchmark. QA & tooling - scripts/qa-external-repo.ts; fixtures/qa prompts template; package.json scripts. Docs & agents - docs/README.md hub: single-source rows for golden, benchmark config, qa:external. - Root README, benchmark.md, architecture, plan-query-golden, CONTRIBUTING, templates. Tests & CI - cli/cmd-query tests; test:golden in check; changesets for query/json/recipe/golden. * docs: consolidate hub and golden queries; clarify API docs and incremental git - Add golden-queries doc, remove superseded plan; refresh docs hub and links - Restore multi-line JSDoc on public API and CLI query surfaces - Document getChangedFiles strategy and git status porcelain path slicing * chore(changeset): consolidate query and golden into one patch entry * feat(cli): recipe catalog JSON and print-sql; align docs with hub - Add codemap query --recipes-json and --print-sql <id> (no DB); QUERY_RECIPES stays sole SQL source; listQueryRecipeCatalog() - Wire main.ts to handle catalog modes before runQueryCmd; extend parseQueryRest and tests - Hydrate changeset with discovery flags; README CLI examples; SKILL and agent rules/templates - docs/README.md conventions: README § CLI canonical; trim duplicate CLI prose from architecture, why-codemap, benchmark * fix: address PR review (query JSON contract, benchmark QA, recipes) - Open DB inside printQueryResult try; guard closeDb - runQueryCmd: try/catch for bootstrap errors with --json; use process.exitCode - Deduplicate globFiles hits; tighten shop/ components SQL filter - assertReadOnlyIndexedSql for CODEMAP_BENCHMARK_CONFIG; RegExp reuse + JSDoc - Stable ORDER BY in bundled recipes; align example benchmark SQL - Validate qa:external --max-files/--max-symbols as positive integers - Changeset: drop brittle "scenario 8" wording * docs: sync changeset and guides for query/benchmark/qa release - Expand changeset with robustness, benchmark validation, recipes, qa:external - README + hub + architecture: --json error contract (SQL, DB, bootstrap) - benchmark.md: custom config path/SQL validation, metrics, fix scenario source paths - packaging: pre-publish checklist; CONTRIBUTING: qa sampling flags * fix: align shop benchmark with tsx+jsx; tighten query parse test - Traditional glob includes components/shop **/*.jsx alongside **/*.tsx (matches components table: parser only emits components from TSX/JSX). - Assert missing SQL/recipe error when --json has no SQL. - Note changeset bullet for shop scenario alignment. * style: brace multiline if bodies; align shop benchmark SQL - Add braces for multiline if/else-if in index-engine, css-parser, and scripts - Assert parseQueryRest error text for --json without SQL - Restrict default shop/ indexed components query to .tsx/.jsx like traditional globs
1 parent 340bbd9 commit 889ed5b

51 files changed

Lines changed: 2285 additions & 340 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.agents/rules/codemap.mdc

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,9 @@ A local database (default **`.codemap.db`**) indexes structure: symbols, imports
1616
| ------- | ----------------- | ----- |
1717
| **Default** — from this clone | `bun src/index.ts` | `bun src/index.ts query "<SQL>"` |
1818
| Same entry | `bun run dev` | (same as first row) |
19+
| JSON output | — | `bun src/index.ts query --json "<SQL>"` |
20+
| Recipe | — | `bun src/index.ts query --recipe fan-out` (see **`bun src/index.ts query --help`**) |
21+
| Recipe catalog / SQL | — | `bun src/index.ts query --recipes-json` · `bun src/index.ts query --print-sql fan-out` |
1922

2023
After **`bun run build`**, **`node dist/index.mjs`** matches the published **`codemap`** binary (same flags). **`bun link`** / global **`codemap`** also work when testing the packaged CLI.
2124

@@ -66,6 +69,10 @@ If the question looks like any of these → use the index:
6669
bun src/index.ts query "<SQL>"
6770
```
6871

72+
**Row count:** The CLI does **not** impose a maximum number of rows. Add **`LIMIT`** (and **`ORDER BY`**) in SQL when you need a bounded list. For automation and multi-row answers, prefer **`bun src/index.ts query --json`** — stdout is a JSON array; on failure, stdout is **`{"error":"<message>"}`** and the process exits **1**.
73+
74+
**Verbatim answers:** When the user asks for lists, counts, or enumerated structural data from the index, **paste or summarize from the query output without inventing rows** — do not substitute a prose “summary” that omits rows the user asked to see. Prefer **`--json`** so the full result set is unambiguous.
75+
6976
## Quick reference queries
7077

7178
| I need to... | Query |

.agents/skills/codemap/SKILL.md

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,44 @@ bun src/index.ts query "<SQL>"
1616

1717
After **`bun run build`**, **`node dist/index.mjs query …`** or a linked **`codemap`** binary matches the published CLI. Use **`--root`** / **`CODEMAP_ROOT`** to index another tree.
1818

19+
## Query output and agents
20+
21+
- **`bun src/index.ts query --json`** prints a **JSON array** of row objects to stdout. On SQL error, stdout is **`{"error":"<message>"}`** and the process exits **1**.
22+
- The CLI **does not cap** how many rows SQLite returns — add **`LIMIT`** and **`ORDER BY`** in SQL when you need a bounded list.
23+
- When answering with structural facts from the index (lists of paths, symbols, dependency edges), **ground the answer in the query rows** — do not invent or silently drop rows. Prefer **`--json`** for large or multi-column results.
24+
25+
## Agent-friendly SQL recipes
26+
27+
Replace placeholders (`'...'`) with your module path, file glob, or symbol name.
28+
29+
**CLI shortcuts:** **`bun src/index.ts query --recipe <id>`** runs bundled SQL (optional **`--json`**). **`bun src/index.ts query --recipes-json`** prints every bundled recipe (**`id`**, **`description`**, **`sql`**) as JSON (no index / DB required). **`bun src/index.ts query --print-sql <id>`** prints one recipe’s SQL only. Ids include **`fan-out`**, **`fan-out-sample`** (**`GROUP_CONCAT`** samples), **`fan-out-sample-json`** (same, but **`json_group_array`** — needs SQLite JSON1), **`fan-in`**, **`index-summary`**, **`files-largest`**, **`components-by-hooks`**, **`markers-by-kind`** — see **`bun src/index.ts query --help`**. The fan-out rows match the SQL below; others align with “Conditional aggregation”, “Codebase statistics”, and component sections later in this skill.
30+
31+
**Top files by dependency fan-out:**
32+
33+
```sql
34+
SELECT from_path, COUNT(*) AS deps
35+
FROM dependencies
36+
GROUP BY from_path
37+
ORDER BY deps DESC
38+
LIMIT 10;
39+
```
40+
41+
**Same ranking, plus up to five sample targets per file** (uses a correlated subquery; adjust **`LIMIT 5`** as needed):
42+
43+
```sql
44+
SELECT d.from_path,
45+
COUNT(*) AS deps,
46+
(SELECT GROUP_CONCAT(to_path, ' | ')
47+
FROM (SELECT to_path FROM dependencies d2 WHERE d2.from_path = d.from_path LIMIT 5))
48+
AS sample_targets
49+
FROM dependencies d
50+
GROUP BY d.from_path
51+
ORDER BY deps DESC
52+
LIMIT 10;
53+
```
54+
55+
**JSON array samples (JSON1):** replace **`GROUP_CONCAT`** with **`json_group_array(to_path)`** in that subquery if your SQLite build has JSON1 — or use **`bun src/index.ts query --recipe fan-out-sample-json`**.
56+
1957
## Schema
2058

2159
### `files` — Every indexed file
@@ -196,6 +234,8 @@ SELECT name, file_path, hooks_used
196234
FROM components WHERE hooks_used LIKE '%useTheme%';
197235

198236
-- Components with most hooks (complexity indicator)
237+
-- `json_array_length` requires SQLite JSON1. For a portable ranking, use
238+
-- `bun src/index.ts query --recipe components-by-hooks` (comma-based count on the stored JSON array).
199239
SELECT name, file_path,
200240
json_array_length(hooks_used) as hook_count
201241
FROM components ORDER BY hook_count DESC LIMIT 15;
Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
---
2+
"@stainless-code/codemap": patch
3+
---
4+
5+
**Query CLI**
6+
7+
- **`codemap query --json`**: print a JSON array of result rows to stdout (and **`{"error":"…"}`** on SQL errors) for agents and automation. Document that the query subcommand does **not** cap rows — use SQL **`LIMIT`** for bounded results. Update bundled agent rule and skill with **`--json`** preference, verbatim structural answers, and generic SQL recipes (fan-out + sample targets).
8+
9+
- **`codemap query --recipe <id>`** for bundled read-only SQL so agents can run common structural queries without embedding SQL on the command line. **`--json`** works with recipes the same way as ad-hoc SQL. Bundled ids include dependency **`fan-out`** / **`fan-out-sample`** / **`fan-out-sample-json`** (JSON1 **`json_group_array`**) / **`fan-in`**, index **`index-summary`**, **`files-largest`**, React **`components-by-hooks`** (comma-based hook count, no JSON1), and **`markers-by-kind`**. The benchmark suite uses the **`fan-out`** recipe SQL for an indexed-path scenario; docs clarify that recipes add no extra query cost vs pasting the same SQL.
10+
11+
- **Recipe discovery (no index / DB):** **`codemap query --recipes-json`** prints all bundled recipes (**`id`**, **`description`**, **`sql`**) as JSON. **`codemap query --print-sql <id>`** prints one recipe’s SQL. **`listQueryRecipeCatalog()`** in **`src/cli/query-recipes.ts`** is the single derived view of **`QUERY_RECIPES`** for the JSON output.
12+
13+
**Golden tests**
14+
15+
- **`bun run test:golden`**: index **`fixtures/minimal`**, run scenarios from **`fixtures/golden/scenarios.json`**, and compare query JSON to **`fixtures/golden/minimal/`**. Use **`bun scripts/query-golden.ts --update`** after intentional fixture or schema changes. Documented in **benchmark.md** and **CONTRIBUTING**.
16+
17+
**Query robustness**
18+
19+
- With **`--json`**, **`{"error":"…"}`** is printed for invalid SQL, database open failures, and **`codemap query`** bootstrap failures (config / resolver setup), not only bad SQL. The CLI sets **`process.exitCode`** instead of **`process.exit`** so piped stdout is not cut off mid-stream.
20+
21+
**Benchmark & `CODEMAP_BENCHMARK_CONFIG`**
22+
23+
- Each **`indexedSql`** in custom scenario JSON is validated as a single read-only **`SELECT`** (or **`WITH``SELECT`**) — DDL/DML and **`RETURNING`** are rejected before execution.
24+
- Config file paths are resolved from **`process.cwd()`** (see **benchmark.md**). **`traditional.regex`** strings are developer-controlled (local JSON); **`files`** mode compiles the regex once per scenario.
25+
- Overlapping **globs** in the traditional path are **deduplicated** so **Files read** / **Bytes read** count each path once.
26+
- The default **components in `shop/`** scenario uses a **`LIKE`** filter aligned with the traditional globs under **`components/shop/`** (**\*.tsx** and **\*.jsx**, matching **`components`** rows from the parser) and avoids unrelated paths such as **`workshop`**.
27+
28+
**Recipes (determinism)**
29+
30+
- Bundled recipe SQL adds stable secondary **`ORDER BY`** columns (and orders inner **`LIMIT`** samples) so **`--recipe`** / **`--json`** output does not vary on aggregate ties.
31+
32+
**External QA**
33+
34+
- **`bun run qa:external`**: **`--max-files`** and **`--max-symbols`** must be positive integers (invalid values throw before indexing).

.github/CONTRIBUTING.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,9 @@ Codemap is in **bootstrap / extraction** phase. Before large PRs, please open an
1111
bun install # runs `prepare` → Husky git hooks
1212
bun run dev # same as `bun src/index.ts` — CLI from source
1313
bun test
14-
bun run check # format + lint + tests + typecheck + build
14+
bun run test:golden # golden SQL vs fixtures/minimal (also runs at end of `bun run check`)
15+
bun run test:golden:external # Tier B: local tree via CODEMAP_ROOT / --root (not in CI)
16+
bun run check # format + lint + tests + typecheck + build + test:golden
1517
bun run clean # remove untracked/ignored build artifacts (keeps `.env`, `.codemap.db*`)
1618
bun run check-updates # interactive dependency updates (`bun update -i --latest`)
1719
```
@@ -36,6 +38,8 @@ Then open a PR on GitHub into **`main`**.
3638
- **Public API** — Anything exported from the package entry (`src/index.ts``src/api.ts`, `config.ts`, shared types) should have **JSDoc** that reads well in hovers and in published typings.
3739
- **Layers** — Keep boundaries clear: [architecture.md](../docs/architecture.md) (`cli``application` → infrastructure). Don’t let CLI concerns leak into parsers or the DB layer.
3840
- **Before you open / update a PR**`bun run check` (or at least `bun run test` + `bun run typecheck` while iterating).
41+
- **Golden queries (Tier A)** — If you change `fixtures/minimal/` or schema/query behavior expected by [fixtures/golden/](../fixtures/golden/), run `bun scripts/query-golden.ts --update`, review diffs, and commit updated JSON under `fixtures/golden/minimal/`. Prefer **fixing the indexer** when output changes for the wrong reason; only refresh goldens when the new rows are correct. See [docs/golden-queries.md](../docs/golden-queries.md).
42+
- **Golden queries (Tier B)** — Against a **local** clone, use `bun run test:golden:external` with `CODEMAP_ROOT` / `--root`. Copy [fixtures/golden/scenarios.external.example.json](../fixtures/golden/scenarios.external.example.json) to `scenarios.external.json` if you need custom scenarios; goldens under `fixtures/golden/external/` are gitignored — do not commit snapshots from proprietary trees.
3943
- **Style** — Match Oxfmt/Oxlint; prefer **straight-line code** and extracted helpers over long nested blocks.
4044

4145
**Editor (VS Code):** [`.vscode/extensions.json`](../.vscode/extensions.json) lists recommended extensions (Bun, Oxc, TypeScript native preview, etc.). [`.vscode/settings.json`](../.vscode/settings.json) enables Oxc format on save and `tsgo`. Formatting and lint rules live in [`.oxfmtrc.json`](../.oxfmtrc.json) and [`.oxlintrc.json`](../.oxlintrc.json) (no framework-specific options beyond defaults).
@@ -46,6 +50,8 @@ Then open a PR on GitHub into **`main`**.
4650

4751
Do **not** add Codemap as a dependency to the bench repo. In **this** repo, copy `.env.example` to `.env` and set **`CODEMAP_TEST_BENCH`** to an **absolute path** to the other clone, then run `bun src/index.ts` as usual. See [docs/benchmark.md § Indexing another project](../docs/benchmark.md#indexing-another-project).
4852

53+
**One-shot QA (index + disk checks + benchmark):** `CODEMAP_ROOT=/absolute/path/to/app bun run qa:external` (or set **`CODEMAP_TEST_BENCH`** in `.env`; optional `--root` overrides). Optional **`--max-files`** / **`--max-symbols`** (positive integers; default caps sampling). Validates indexed paths exist, spot-checks symbol lines vs files, prints sample SQL rows, then runs `src/benchmark.ts`. Do **not** add external app source into this repository.
54+
4955
Releases: **[@changesets/cli](https://github.com/changesets/changesets)** — run **`bun run changeset`** when your PR should bump the version; see [docs/packaging.md § Releases](../docs/packaging.md#releases).
5056

5157
**Issues:** use [GitHub issue templates](https://github.com/stainless-code/codemap/issues/new/choose)**Core bug** vs **Adapter proposal** (see `.github/ISSUE_TEMPLATE/`).

.github/workflows/ci.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,9 @@ jobs:
9292
- name: Run unit tests with coverage
9393
run: bun run test:coverage
9494

95+
- name: Golden query regression (fixtures/minimal)
96+
run: bun run test:golden
97+
9598
build:
9699
name: 🧰 Build
97100
needs: skip-ci

.gitignore

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,3 +8,11 @@ dist/
88
.env.*
99
!.env.example
1010
tsconfig.lint-staged.json
11+
12+
# Tier B golden query outputs (local / private trees — see docs/golden-queries.md)
13+
fixtures/golden/external/
14+
fixtures/golden/scenarios.external.json
15+
16+
# QA chat prompts tied to a private/local index (paths + product names)
17+
fixtures/qa/*.local.md
18+
fixtures/benchmark/*.local.json

README.md

Lines changed: 24 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
- **Not** full-text search or grep on arbitrary strings — use those when you need raw file-body search.
66
- **Is** a fast, token-efficient way to navigate **structure**: definitions, imports, dependency direction, components, and other extracted facts.
77

8-
**Documentation:** [docs/README.md](docs/README.md) is the hub (topic index + single-source rules). Topics: [architecture](docs/architecture.md), [agents](docs/agents.md) (`codemap agents init`), [benchmark](docs/benchmark.md), [packaging](docs/packaging.md), [roadmap](docs/roadmap.md), [why Codemap](docs/why-codemap.md). **Bundled rules/skills:** [`.agents/rules/`](.agents/rules/), [`.agents/skills/codemap/SKILL.md`](.agents/skills/codemap/SKILL.md). **Consumers:** [.github/CONTRIBUTING.md](.github/CONTRIBUTING.md).
8+
**Documentation:** [docs/README.md](docs/README.md) is the hub (topic index + single-source rules). Topics: [architecture](docs/architecture.md), [agents](docs/agents.md) (`codemap agents init`), [benchmark](docs/benchmark.md), [golden queries](docs/golden-queries.md), [packaging](docs/packaging.md), [roadmap](docs/roadmap.md), [why Codemap](docs/why-codemap.md). **Bundled rules/skills:** [`.agents/rules/`](.agents/rules/), [`.agents/skills/codemap/SKILL.md`](.agents/skills/codemap/SKILL.md). **Consumers:** [.github/CONTRIBUTING.md](.github/CONTRIBUTING.md).
99

1010
---
1111

@@ -37,6 +37,16 @@ codemap --full
3737

3838
# SQL against the index (after at least one index run)
3939
codemap query "SELECT name, file_path FROM symbols LIMIT 10"
40+
# JSON array on stdout (agents / scripts); {"error":"..."} for bad SQL, DB open, or query bootstrap (config/resolver) when using --json
41+
codemap query --json "SELECT name, file_path FROM symbols LIMIT 10"
42+
# Query is not row-capped — add LIMIT in SQL for large selects
43+
# Bundled SQL (same as skill examples): fan-out rankings
44+
codemap query --recipe fan-out
45+
codemap query --json --recipe fan-out-sample
46+
# List bundled recipes as JSON, or print one recipe's SQL (no DB required)
47+
codemap query --recipes-json
48+
codemap query --print-sql fan-out
49+
# `components-by-hooks` ranks by hook count without SQLite JSON1 (comma-based count on the stored JSON array).
4050

4151
# Another project
4252
codemap --root /path/to/repo --full
@@ -81,12 +91,15 @@ const rows = cm.query("SELECT name FROM symbols LIMIT 5");
8191

8292
Tooling: **Oxfmt**, **Oxlint**, **tsgo** (`@typescript/native-preview`).
8393

84-
| Command | Purpose |
85-
| ------------------------------------ | ---------------------------------------------------------------- |
86-
| `bun run dev` | Run the CLI from source (same as `bun src/index.ts`) |
87-
| `bun run check` | Build, format check, lint, tests, typecheck — run before pushing |
88-
| `bun run fix` | Apply lint fixes, then format |
89-
| `bun run test` / `bun run typecheck` | Focused checks |
94+
| Command | Purpose |
95+
| ------------------------------------ | -------------------------------------------------------------------------- |
96+
| `bun run dev` | Run the CLI from source (same as `bun src/index.ts`) |
97+
| `bun run check` | Build, format check, lint, tests, typecheck — run before pushing |
98+
| `bun run fix` | Apply lint fixes, then format |
99+
| `bun run test` / `bun run typecheck` | Focused checks |
100+
| `bun run test:golden` | SQL snapshot regression on `fixtures/minimal` (included in `check`) |
101+
| `bun run test:golden:external` | Tier B: local tree via `CODEMAP_*` / `--root` (not in default `check`) |
102+
| `bun run qa:external` | Index + sanity checks + benchmark on `CODEMAP_ROOT` / `CODEMAP_TEST_BENCH` |
90103

91104
```bash
92105
bun install
@@ -100,11 +113,13 @@ bun run fix # oxlint --fix, then oxfmt
100113

101114
## Benchmark
102115

116+
Use a **real** project path (the repo must exist on disk). See [docs/benchmark.md § Indexing another project](docs/benchmark.md#indexing-another-project).
117+
103118
```bash
104-
CODEMAP_ROOT=/path/to/indexed-repo bun src/benchmark.ts
119+
CODEMAP_ROOT=/absolute/path/to/indexed-repo bun src/benchmark.ts
105120
```
106121

107-
Details: [docs/benchmark.md](docs/benchmark.md).
122+
Optional **`CODEMAP_BENCHMARK_CONFIG`** for repo-specific scenarios: [docs/benchmark.md § Custom scenarios](docs/benchmark.md#custom-scenarios-codemap_benchmark_config).
108123

109124
---
110125

0 commit comments

Comments
 (0)