Skip to content

Commit 4ca4267

Browse files
SutuSebastianSutuSebastian
committed
feat: Bun-first glob and JSON config; align engines; consolidate docs
- Add glob-sync (Bun Glob vs fast-glob) for collectFiles and benchmark - Load codemap.config.json with readFile on Node; Bun.file on Bun - Set engines from oxc-parser/better-sqlite3; trim packaging doc - Merge bench-repo into benchmark.md; remove bun-reference and extraction - Update agent rules, README, CONTRIBUTING, .env.example
1 parent 3dc6519 commit 4ca4267

23 files changed

Lines changed: 212 additions & 179 deletions

‎.agents/rules/codemap.mdc‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ A local database (default **`.codemap.db`**) indexes structure: symbols, imports
1818
| After `bun link` / global `codemap` | `codemap` | `codemap query "<SQL>"` |
1919
| Published package (when available) | `bunx @stainless-code/codemap` | `bunx @stainless-code/codemap query "<SQL>"` |
2020

21-
Index another project: **`--root /path/to/repo`**, or set **`CODEMAP_ROOT`** or **`CODEMAP_TEST_BENCH`** (e.g. in **`.env`** in this repo — see [docs/bench-repo.md](../../docs/bench-repo.md)). Full rebuild: **`--full`**. Targeted re-index: **`--files path/to/a.ts path/to/b.tsx`**.
21+
Index another project: **`--root /path/to/repo`**, or set **`CODEMAP_ROOT`** or **`CODEMAP_TEST_BENCH`** (e.g. in **`.env`** in this repo — see [docs/benchmark.md § Indexing another project](../../docs/benchmark.md#indexing-another-project)). Full rebuild: **`--full`**. Targeted re-index: **`--files path/to/a.ts path/to/b.tsx`**.
2222

2323
## Session start (do this ONCE per conversation)
2424

‎.agents/rules/verify-after-each-step.mdc‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,5 +42,5 @@ A "step" is any self-contained unit of work where you've finished editing and ar
4242
2. **Fix before moving on** — If any check fails, fix it immediately while context is fresh. Never carry forward known failures.
4343
3. **Use the right scope** — Run `bun run lint` and `bun run format:check` on specific files when possible. Prefer `bun run typecheck` project-wide when types may depend on unstaged files.
4444
4. **Run affected tests** — If you modified or created `*.test.ts` files, run `bun test <file>` on them.
45-
5. **Re-index before querying Codemap** — If you changed indexed source and plan to run SQL against the structural index next, run `bun src/index.ts --files <paths>` with paths **relative to the indexed project root** (set `CODEMAP_TEST_BENCH` / `CODEMAP_ROOT` or `--root` so that root is correct — see [docs/bench-repo.md](../../docs/bench-repo.md)).
45+
5. **Re-index before querying Codemap** — If you changed indexed source and plan to run SQL against the structural index next, run `bun src/index.ts --files <paths>` with paths **relative to the indexed project root** (set `CODEMAP_TEST_BENCH` / `CODEMAP_ROOT` or `--root` so that root is correct — see [docs/benchmark.md § Indexing another project](../../docs/benchmark.md#indexing-another-project)).
4646
6. **Don't duplicate the hook's job** — You don't need to re-verify at commit time; the pre-commit hook (`lint-staged`) handles that automatically when AI/agent env vars trigger it. Your job is to stay green *between* commits.

‎.env.example‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
# Optional: when developing Codemap in this repo, index another project (QA / “testing bench”).
1+
# Optional: when developing Codemap in this repo, index another project (QA).
22
# Copy to `.env` and set an absolute path. Bun loads `.env` automatically for `bun src/index.ts`.
3-
# Same effect as CODEMAP_ROOT; use whichever name you prefer.
3+
# Same effect as CODEMAP_ROOT; use whichever name you prefer. See docs/benchmark.md#indexing-another-project.
44
#
5-
# CODEMAP_TEST_BENCH=/absolute/path/to/your-bench-repo
6-
# CODEMAP_ROOT=/absolute/path/to/your-bench-repo
5+
# CODEMAP_TEST_BENCH=/absolute/path/to/the-indexed-project
6+
# CODEMAP_ROOT=/absolute/path/to/the-indexed-project

‎.github/CONTRIBUTING.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
Codemap is in **bootstrap / extraction** phase. Before large PRs, please open an issue so we can align on:
44

55
- **Core vs adapter** — core should stay small; language-specific logic belongs in **adapters** (see [docs/roadmap.md](../docs/roadmap.md)).
6-
- **Runtimes** — **Node ≥20** and **Bun ≥1.1** are both supported for the CLI and library; SQLite is **`better-sqlite3`** on Node and **`bun:sqlite`** on Bun ([docs/architecture.md](../docs/architecture.md), [docs/packaging.md § Node vs Bun](../docs/packaging.md#node-vs-bun)).
6+
- **Runtimes** — **Node** `^20.19.0 || >=22.12.0` and **Bun** `>=1.0.0` (`package.json` **engines**); SQLite is **`better-sqlite3`** on Node and **`bun:sqlite`** on Bun ([docs/architecture.md](../docs/architecture.md), [docs/packaging.md § Node vs Bun](../docs/packaging.md#node-vs-bun)).
77

88
## Dev workflow
99

@@ -27,7 +27,7 @@ bun run check # format + lint + tests + typecheck + build
2727

2828
### QA against a real app (testing bench)
2929

30-
Do **not** add Codemap as a dependency to the bench repo. In **this** repo, copy `.env.example` to `.env` and set **`CODEMAP_TEST_BENCH`** to an **absolute path** to the other clone, then run `bun src/index.ts` as usual. See [docs/bench-repo.md](../docs/bench-repo.md).
30+
Do **not** add Codemap as a dependency to the bench repo. In **this** repo, copy `.env.example` to `.env` and set **`CODEMAP_TEST_BENCH`** to an **absolute path** to the other clone, then run `bun src/index.ts` as usual. See [docs/benchmark.md § Indexing another project](../docs/benchmark.md#indexing-another-project).
3131

3232
Releases: **[@changesets/cli](https://github.com/changesets/changesets)** — run **`bun run changeset`** when your PR should bump the version; see [docs/packaging.md § Releases](../docs/packaging.md#releases).
3333

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ bun add @stainless-code/codemap
2020

2121
The package exposes a **`codemap`** binary, a **library** entry (`import` / `exports`), compiled **`dist/`**, and **`templates/agents`** for **`codemap agents init`** — see [docs/packaging.md](docs/packaging.md).
2222

23-
**Engines:** Node **≥20** and/or Bun **≥1.1** (see `package.json`).
23+
**Engines:** Node **`^20.19.0 || >=22.12.0`** and/or Bun **`>=1.0.0`** — see `package.json` and [docs/packaging.md](docs/packaging.md).
2424

2525
---
2626

@@ -56,7 +56,7 @@ codemap agents init
5656
codemap agents init --force
5757
```
5858

59-
**Environment / flags:** `--root` overrides **`CODEMAP_ROOT`** / **`CODEMAP_TEST_BENCH`**, then **`process.cwd()`**. Using a separate app as a QA target: [docs/bench-repo.md](docs/bench-repo.md).
59+
**Environment / flags:** `--root` overrides **`CODEMAP_ROOT`** / **`CODEMAP_TEST_BENCH`**, then **`process.cwd()`**. Indexing a project outside this clone: [docs/benchmark.md § Indexing another project](docs/benchmark.md#indexing-another-project).
6060

6161
**Configuration:** optional **`codemap.config.ts`** (default export object or async factory) or **`codemap.config.json`**. Shape: [codemap.config.example.json](codemap.config.example.json). When developing inside this repo you can use `defineConfig` from `@stainless-code/codemap` or `./src/config`. If you set **`include`**, it **replaces** the default glob list entirely.
6262

‎docs/README.md‎

Lines changed: 10 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -2,15 +2,14 @@
22

33
Technical docs for **[@stainless-code/codemap](https://github.com/stainless-code/codemap)**. Quick start: [../README.md](../README.md).
44

5-
| File | Topic |
6-
| -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
7-
| [architecture.md](./architecture.md) | Schema, layering, CLI, programmatic API, parsers |
8-
| [bench-repo.md](./bench-repo.md) | `CODEMAP_TEST_BENCH` / `.env` QA workflow |
9-
| [benchmark.md](./benchmark.md) | Benchmark script; [CLI vs benchmark scope](./benchmark.md#cli-and-runtime-startup); [`fixtures/minimal/`](../fixtures/minimal/) |
10-
| [bun-reference.md](./bun-reference.md) | Upstream **`bun:sqlite`** doc links |
11-
| [extraction.md](./extraction.md) | Extraction history → layout in [architecture § Key Files](./architecture.md#key-files) |
12-
| [packaging.md](./packaging.md) | `dist/`, npm entry, Node vs Bun, [Changesets](https://github.com/changesets/changesets) |
13-
| [roadmap.md](./roadmap.md) | Forward-looking backlog (not a `src/` inventory) |
14-
| [why-codemap.md](./why-codemap.md) | Why index + SQL for agents |
5+
| File | Topic |
6+
| ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
7+
| [architecture.md](./architecture.md) | Schema, layering, CLI, programmatic API, parsers, [Key Files](./architecture.md#key-files) |
8+
| [benchmark.md](./benchmark.md) | **[Indexing another project](./benchmark.md#indexing-another-project)** (`CODEMAP_*`, `.env`) · **[benchmark script](./benchmark.md#the-benchmark-script)** (`src/benchmark.ts`) · [`fixtures/minimal/`](../fixtures/minimal/) |
9+
| [packaging.md](./packaging.md) | `dist/`, npm **`files`**, **engines**, **[Node vs Bun](./packaging.md#node-vs-bun)**, **[Releases](./packaging.md#releases)** (Changesets) |
10+
| [roadmap.md](./roadmap.md) | Forward-looking backlog (not a `src/` inventory) |
11+
| [why-codemap.md](./why-codemap.md) | Why index + SQL for agents (speed, tokens, accuracy) |
1512

16-
**Conventions:** one topic per file; relative links; no symbol/file counts or source line numbers in docs (use `codemap query` / `bun run dev query` to measure). **Contributors:** `bun run check`, JSDoc on public API — [.github/CONTRIBUTING.md](../.github/CONTRIBUTING.md) (tooling, `.agents/` / `.cursor/`, `.gitignore` / format config).
13+
**Cross-cutting:** SQLite, workers, include globs, and JSON config use Bun when available — **[packaging.md § Node vs Bun](./packaging.md#node-vs-bun)** (single table; don’t duplicate elsewhere).
14+
15+
**Conventions:** one topic per file; relative links; no symbol/file counts or source line numbers (use `codemap query` / `bun run dev query` after indexing). **This repo:** `bun run dev` → `bun src/index.ts`; **`bun run build`** → tsdown → `dist/`. **Contributors:** `bun run check`, JSDoc on public API — [.github/CONTRIBUTING.md](../.github/CONTRIBUTING.md).

‎docs/architecture.md‎

Lines changed: 24 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# Codemap — Architecture
22

3+
See [documentation index](./README.md).
4+
35
## Overview
46

57
A local SQLite database (`.codemap.db`) indexes the project tree and stores structural metadata (symbols, imports, exports, components, dependencies, CSS tokens, markers) for SQL queries instead of repeated full-tree scans.
@@ -8,7 +10,9 @@ A local SQLite database (`.codemap.db`) indexes the project tree and stores stru
810

911
**`src/sqlite-db.ts`:** Node uses **`better-sqlite3`**; Bun uses **`bun:sqlite`**. Same schema everywhere. **`better-sqlite3`** allows **one SQL statement per `prepare()`**; **`bun:sqlite`** accepts **multiple statements** in one `run()`. On Node, **`runSql()`** splits multi-statement strings on **`;`** and runs each fragment. Do **not** put **`;`** inside **`--` line comments** in **`db.ts`** DDL strings (naive split would break). Details: [packaging.md § Node vs Bun](./packaging.md#node-vs-bun).
1012

11-
**`src/worker-pool.ts`:** Bun `Worker` or Node `worker_threads`. **Shipped artifact:** **`dist/`** — `package.json` **`bin`** and **`exports`** both point at **`dist/index.mjs`** ([packaging.md](./packaging.md)); tsdown also emits **lazy CLI chunks** (`cmd-index`, `cmd-query`, `cmd-agents`, …) loaded via **`import()`** from **`src/cli/main.ts`**.
13+
**`src/worker-pool.ts`:** Bun `Worker` or Node `worker_threads`. **`src/glob-sync.ts`:** Bun **`Glob`** or **`fast-glob`** for include patterns. **`src/config.ts`:** JSON user config via **`Bun.file`** on Bun, **`readFile` + `JSON.parse`** on Node. Full table: [packaging.md § Node vs Bun](./packaging.md#node-vs-bun).
14+
15+
**Shipped artifact:** **`dist/`** — `package.json` **`bin`** and **`exports`** both point at **`dist/index.mjs`** ([packaging.md](./packaging.md)); tsdown also emits **lazy CLI chunks** (`cmd-index`, `cmd-query`, `cmd-agents`, …) loaded via **`import()`** from **`src/cli/main.ts`**.
1216

1317
## Layering
1418

@@ -88,24 +92,25 @@ A local SQLite database (`.codemap.db`) indexes the project tree and stores stru
8892

8993
## Key Files
9094

91-
| File | Purpose |
92-
| ----------------- | ------------------------------------------------------------------------------------------------ |
93-
| `index.ts` | Package entry — re-exports `api` / `config`, runs CLI when main |
94-
| `cli/` | CLI — bootstrap argv, lazy command modules, `query` / `agents init` / index modes |
95-
| `api.ts` | Programmatic API — `createCodemap`, `Codemap`, `runCodemapIndex` |
96-
| `application/` | Indexing use cases and engine (`run-index`, `index-engine`, types) |
97-
| `worker-pool.ts` | Parallel parse workers (Bun / Node) |
98-
| `db.ts` | SQLite adapter — schema DDL, typed CRUD, connection management |
99-
| `parser.ts` | TS/TSX/JS/JSX extraction via `oxc-parser` — symbols, imports, exports, components, markers |
100-
| `css-parser.ts` | CSS extraction via `lightningcss` — custom properties, classes, keyframes, `@theme` blocks |
101-
| `resolver.ts` | Import path resolution via `oxc-resolver` — respects `tsconfig` aliases, builds dependency graph |
102-
| `constants.ts` | Shared constants — e.g. `LANG_MAP` |
103-
| `markers.ts` | Shared marker extraction (`TODO`/`FIXME`/`HACK`/`NOTE`) — used by all parsers |
104-
| `parse-worker.ts` | Worker thread entry point — reads, parses, and extracts file data in parallel |
105-
| `adapters/` | `LanguageAdapter` types and built-in TS/CSS/text implementations |
106-
| `parsed-types.ts` | Shared `ParsedFile` shape for workers and adapters |
107-
| `agents-init.ts` | `codemap agents init` — copies `templates/agents` → `.agents/` |
108-
| `benchmark.ts` | Performance comparison script — see [benchmark.md](./benchmark.md) |
95+
| File | Purpose |
96+
| ----------------- | ----------------------------------------------------------------------------------------------------------------- |
97+
| `index.ts` | Package entry — re-exports `api` / `config`, runs CLI when main |
98+
| `cli/` | CLI — bootstrap argv, lazy command modules, `query` / `agents init` / index modes |
99+
| `api.ts` | Programmatic API — `createCodemap`, `Codemap`, `runCodemapIndex` |
100+
| `application/` | Indexing use cases and engine (`run-index`, `index-engine`, types) |
101+
| `worker-pool.ts` | Parallel parse workers (Bun / Node) |
102+
| `db.ts` | SQLite adapter — schema DDL, typed CRUD, connection management |
103+
| `parser.ts` | TS/TSX/JS/JSX extraction via `oxc-parser` — symbols, imports, exports, components, markers |
104+
| `css-parser.ts` | CSS extraction via `lightningcss` — custom properties, classes, keyframes, `@theme` blocks |
105+
| `resolver.ts` | Import path resolution via `oxc-resolver` — respects `tsconfig` aliases, builds dependency graph |
106+
| `constants.ts` | Shared constants — e.g. `LANG_MAP` |
107+
| `glob-sync.ts` | Include globs — Bun `Glob` vs `fast-glob` on Node ([packaging § Node vs Bun](./packaging.md#node-vs-bun)) |
108+
| `markers.ts` | Shared marker extraction (`TODO`/`FIXME`/`HACK`/`NOTE`) — used by all parsers |
109+
| `parse-worker.ts` | Worker thread entry point — reads, parses, and extracts file data in parallel |
110+
| `adapters/` | `LanguageAdapter` types and built-in TS/CSS/text implementations |
111+
| `parsed-types.ts` | Shared `ParsedFile` shape for workers and adapters |
112+
| `agents-init.ts` | `codemap agents init` — copies `templates/agents` → `.agents/` |
113+
| `benchmark.ts` | SQL vs traditional timing script — see [benchmark.md § The benchmark script](./benchmark.md#the-benchmark-script) |
109114

110115
## CLI usage
111116

‎docs/bench-repo.md‎

Lines changed: 0 additions & 30 deletions
This file was deleted.

0 commit comments

Comments
 (0)