diff --git a/.agents/skills/codemap/SKILL.md b/.agents/skills/codemap/SKILL.md index 79ef897a..41a5c503 100644 --- a/.agents/skills/codemap/SKILL.md +++ b/.agents/skills/codemap/SKILL.md @@ -68,7 +68,7 @@ LIMIT 10 | Column | Type | Description | | ------------- | ------- | -------------------------------------------------------------------------------- | | path | TEXT PK | Relative path from project root | -| content_hash | TEXT | Wyhash fingerprint (base-36) | +| content_hash | TEXT | SHA-256 hex digest | | size | INTEGER | File size in bytes | | line_count | INTEGER | Number of lines | | language | TEXT | `ts`, `tsx`, `js`, `jsx`, `css`, `md`, `mdx`, `mdc`, `json`, `yaml`, `sh`, `txt` | @@ -77,17 +77,17 @@ LIMIT 10 ### `symbols` — Functions, types, interfaces, enums, constants, classes -| Column | Type | Description | -| ----------------- | ---------- | --------------------------------------------------------------------- | -| id | INTEGER PK | Auto-increment ID | -| file_path | TEXT FK | References `files(path)` | -| name | TEXT | Symbol name | -| kind | TEXT | `function`, `class`, `type`, `interface`, `enum`, `const`, `variable` | -| line_start | INTEGER | Start line (1-based) | -| line_end | INTEGER | End line (1-based) | -| signature | TEXT | e.g. `createHandler()`, `type UserProps` | -| is_exported | INTEGER | 1 if exported | -| is_default_export | INTEGER | 1 if default export | +| Column | Type | Description | +| ----------------- | ---------- | --------------------------------------------------------- | +| id | INTEGER PK | Auto-increment ID | +| file_path | TEXT FK | References `files(path)` | +| name | TEXT | Symbol name | +| kind | TEXT | `function`, `class`, `type`, `interface`, `enum`, `const` | +| line_start | INTEGER | Start line (1-based) | +| line_end | INTEGER | End line (1-based) | +| signature | TEXT | e.g. `createHandler()`, `type UserProps` | +| is_exported | INTEGER | 1 if exported | +| is_default_export | INTEGER | 1 if default export | ### `imports` — Import statements diff --git a/.changeset/audit-findings.md b/.changeset/audit-findings.md new file mode 100644 index 00000000..d4c77660 --- /dev/null +++ b/.changeset/audit-findings.md @@ -0,0 +1,38 @@ +--- +"@stainless-code/codemap": patch +--- + +Fix three HIGH-severity bugs found via cross-audit triangulation, plus performance and docs improvements. + +**Bug fixes** + +- Add missing `onerror` handler on Bun Worker — prevents silent promise hang when a parse worker crashes +- Require JSX return or hook usage for component detection — eliminates false positives (e.g. `FormatCurrency()` in `.tsx` files no longer indexed as a component) +- Include previously-indexed files in incremental and `--files` modes — custom-extension files indexed during `--full` no longer silently go stale + +**Performance** + +- Batch CSS imports instead of inserting one-at-a-time (both full-rebuild and incremental paths) +- Add `Map` cache for `better-sqlite3` `run()`/`query()` — avoids ~2,000+ redundant `prepare()` calls on large projects +- Hoist `inner.query()` in `wrap()` to prepare once per call instead of per `.get()`/`.all()` +- Skip `PRAGMA optimize` on `closeDb` for read-only query paths + +**Docs** + +- Fix Wyhash → SHA-256 in architecture.md and SKILL.md (3 locations) +- Correct `symbols.kind` values (`variable` → `const`, `type_alias` → `type`) and `exports.kind` values +- Clarify `Database.query()` caching is Bun-only; Node statement cache via wrapper +- Update architecture.md: component heuristic, statement cache, `closeDb` readonly, incremental/`--files` custom extensions +- Update benchmark.md and golden-queries.md for enriched fixture + +**Testing** + +- Enrich `fixtures/minimal/` to cover all 10 indexed tables (CSS module, `@keyframes`, `@import`, non-component PascalCase export, FIXME marker) +- Add 7 new golden scenarios (exports, css_variables, css_classes, css_keyframes, css_imports, markers-all-kinds, components-no-false-positives) + +**Cleanup** + +- Remove unused `analyzeDependencies: true` from CSS parser +- Deduplicate `fetchTableStats` (was duplicated across `index-engine.ts` and `run-index.ts`) +- Remove dead `eslint-disable-next-line` directives (oxlint doesn't enforce those rules) +- Fix `SCHEMA_VERSION` comment (said "2", value is `1`) diff --git a/docs/architecture.md b/docs/architecture.md index f13aa710..e8c4a3eb 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -129,7 +129,7 @@ A local SQLite database (`.codemap.db`) indexes the project tree and stores stru ### `--files` (targeted reindex) -When specific file paths are passed via `--files`, the indexer skips git diff, git status, and the full filesystem glob scan. It reads the set of already-indexed paths from the database (for import resolution), then only processes the listed files. Files that no longer exist on disk are automatically removed from the index via `ON DELETE CASCADE`. +When specific file paths are passed via `--files`, the indexer skips git diff, git status, and the full filesystem glob scan. It reads the set of already-indexed paths from the database (for import resolution), then only processes the listed files. Files with non-standard extensions (e.g. custom `include` globs) are accepted and indexed as text; a warning is printed but they are not skipped. Files that no longer exist on disk are automatically removed from the index via `ON DELETE CASCADE`. ## Programmatic usage @@ -171,14 +171,14 @@ All tables use `STRICT` mode. Tables marked with `WITHOUT ROWID` store data dire | last_modified | INTEGER | File mtime (epoch ms) | | indexed_at | INTEGER | When this row was written | -### `symbols` — Functions, variables, classes, interfaces, type aliases, enums (`STRICT`) +### `symbols` — Functions, constants, classes, interfaces, types, enums (`STRICT`) | Column | Type | Description | | ----------------- | ---------- | --------------------------------------------------------------------- | | id | INTEGER PK | Auto-increment row id | | file_path | TEXT FK | References `files(path)` ON DELETE CASCADE | | name | TEXT | Symbol name | -| kind | TEXT | `function`, `variable`, `class`, `interface`, `type_alias`, `enum` | +| kind | TEXT | `function`, `const`, `class`, `interface`, `type`, `enum` | | line_start | INTEGER | Start line (1-based) | | line_end | INTEGER | End line | | signature | TEXT | Reconstructed signature (e.g. `usePermissions(): PermissionsContext`) | @@ -204,11 +204,11 @@ All tables use `STRICT` mode. Tables marked with `WITHOUT ROWID` store data dire | id | INTEGER PK | Auto-increment row id | | file_path | TEXT FK | File containing the export | | name | TEXT | Exported name | -| kind | TEXT | `function`, `variable`, etc. | +| kind | TEXT | `value`, `type`, `re-export` | | is_default | INTEGER | 1 if default export | | re_export_source | TEXT | Source module if re-exported | -### `components` — React components (detected by JSX return + PascalCase) (`STRICT`) +### `components` — React components (detected by PascalCase + JSX return or hook usage) (`STRICT`) | Column | Type | Description | | ----------------- | ---------- | ----------------------------- | @@ -286,7 +286,7 @@ Uses the Rust-based `oxc-parser` via NAPI bindings to parse TypeScript/TSX/JS/JS - **Symbols**: Functions, arrow functions, classes, interfaces, type aliases, enums — with reconstructed signatures - **Imports**: All `import` statements with specifiers, source paths, and type-only flags - **Exports**: Named exports, default exports, re-exports -- **Components**: React components detected via PascalCase name + JSX return. Extracts props type and hooks used +- **Components**: React components detected via PascalCase name + (JSX return **or** hook usage). A PascalCase function in `.tsx`/`.jsx` that neither returns JSX nor calls hooks is indexed only as a symbol, not a component. Extracts props type and hooks used - **Markers**: `TODO`, `FIXME`, `HACK`, `NOTE` comments with line numbers ### CSS — `css-parser.ts` (`lightningcss`) @@ -327,9 +327,10 @@ The indexer uses git to detect changes since the last indexed commit: 1. **Stores `last_indexed_commit`** (HEAD SHA) in the `meta` table after each run 2. On next run, computes `git diff --name-only ..HEAD` + `git status --porcelain` -3. Only re-indexes changed files (Wyhash content comparison), using DB-sourced `indexedPaths` for import resolution (skips full `collectFiles()` glob scan) -4. Deleted files are removed via `ON DELETE CASCADE` — deleting from `files` cascades to all related tables -5. Falls back to full rebuild if commit history is incompatible (e.g. force push, branch switch) +3. Filters changed files to those with a known extension **or** already present in the `files` table (so custom-extension files indexed during `--full` are re-indexed on subsequent incremental runs) +4. Only re-indexes changed files (SHA-256 content comparison), using DB-sourced `indexedPaths` for import resolution (skips full `collectFiles()` glob scan) +5. Deleted files are removed via `ON DELETE CASCADE` — deleting from `files` cascades to all related tables +6. Falls back to full rebuild if commit history is incompatible (e.g. force push, branch switch) ## File Artifacts @@ -436,7 +437,7 @@ Until the first release, Codemap keeps **`SCHEMA_VERSION` at 1**; pull `--full` ### `bun:sqlite` API -All DDL and PRAGMA statements use `Database.run()` (not the deprecated `Database.exec()` alias). Parameterized insert/update statements use `Database.query()` (which caches compiled statements) instead of `Database.prepare()` (which does not cache). Read queries also use `Database.query().all()` or `.get()`. Bulk inserts use the generic `batchInsert()` helper with multi-row `INSERT ... VALUES (...),(...),(...)` in batches of 100, pre-computed placeholders, and zero-copy index-bounds iteration. +All DDL and PRAGMA statements use `Database.run()`. The `sqlite-db.ts` wrapper abstracts both Bun (`bun:sqlite`) and Node (`better-sqlite3`). On Bun, `Database.query()` caches compiled statements internally. On Node, the wrapper maintains a `Map` cache so repeated `run()` and `query()` calls with the same SQL reuse a single prepared statement. Read queries use the wrapper's `.query().all()` or `.get()`. Bulk inserts use the generic `batchInsert()` helper with multi-row `INSERT ... VALUES (...),(...),(...)` in batches of 100, pre-computed placeholders, and zero-copy index-bounds iteration. ### PRAGMAs (set on every `openDb()`) @@ -457,6 +458,8 @@ All DDL and PRAGMA statements use `Database.run()` (not the deprecated `Database | `analysis_limit` | `400` | Caps rows sampled by `optimize` to keep it fast | | `optimize` | — | Gathers query planner statistics (`sqlite_stat1`) for better plans | +Read-only query paths (`printQueryResult`, `queryRows`) call `closeDb` with `{ readonly: true }`, which skips both PRAGMAs to avoid write contention under concurrent `codemap query` processes. + ### WITHOUT ROWID tables Tables with a TEXT PRIMARY KEY and no auto-increment benefit from `WITHOUT ROWID` — the data is stored directly in the primary key B-tree instead of a separate rowid B-tree, eliminating a lookup indirection: diff --git a/docs/benchmark.md b/docs/benchmark.md index b4d1c90e..79bea11b 100644 --- a/docs/benchmark.md +++ b/docs/benchmark.md @@ -206,9 +206,12 @@ The benchmark also measures the cost of keeping the index fresh (3 runs each, sa #### `fixtures/minimal/` -Small **private** package (not published) with intentional: +Small **private** package (not published) with intentional coverage of all indexed tables: -- `usePermissions`, `~/api/client` import, `components/shop/*`, `utils/date`, CSS variables, and a TODO marker. +- **Symbols / imports / dependencies**: `usePermissions`, `~/api/client` path alias, `components/shop/*`, `utils/date` +- **Components**: `ShopButton` (JSX return) + `FormatPrice` (PascalCase non-component — validates detection heuristic) +- **CSS**: variables (`--color-brand`, `--spacing-md`), class selectors (`.container`, `.primary` in a `.module.css`), `@keyframes fadeIn`, `@import` edge +- **Markers**: `TODO` (in `notes.md`) + `FIXME` (in `consumer.ts`) **Local:** diff --git a/docs/golden-queries.md b/docs/golden-queries.md index 867dbff9..8571ddef 100644 --- a/docs/golden-queries.md +++ b/docs/golden-queries.md @@ -69,12 +69,13 @@ Scenarios live in **`fixtures/golden/scenarios.json`** (Tier A) or optional **`s ## Status -| Area | State | -| ----------------------------- | ----------------------------------------------------------------------- | -| Tier A runner + CI | **`bun run test:golden`** in `check` | -| Tier B external + schema | **`test:golden:external`**, Zod in **`scripts/query-golden/schema.ts`** | -| Subset matchers + budgets | **`match`**, **`budgetMs`**, **`--strict-budget`** | -| Optional CI for public corpus | Deferred — [roadmap § Backlog](./roadmap.md#backlog) | +| Area | State | +| ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | +| Tier A runner + CI | **`bun run test:golden`** in `check` | +| Tier A scenario coverage | 13 scenarios across all indexed tables: files, symbols, imports, exports, components, dependencies, markers, CSS vars/classes/keyframes | +| Tier B external + schema | **`test:golden:external`**, Zod in **`scripts/query-golden/schema.ts`** | +| Subset matchers + budgets | **`match`**, **`budgetMs`**, **`--strict-budget`** | +| Optional CI for public corpus | Deferred — [roadmap § Backlog](./roadmap.md#backlog) | --- diff --git a/fixtures/golden/minimal/components-no-false-positives.json b/fixtures/golden/minimal/components-no-false-positives.json new file mode 100644 index 00000000..dce72f82 --- /dev/null +++ b/fixtures/golden/minimal/components-no-false-positives.json @@ -0,0 +1,5 @@ +[ + { + "name": "ShopButton" + } +] diff --git a/fixtures/golden/minimal/css-classes-module.json b/fixtures/golden/minimal/css-classes-module.json new file mode 100644 index 00000000..aab1605f --- /dev/null +++ b/fixtures/golden/minimal/css-classes-module.json @@ -0,0 +1,12 @@ +[ + { + "name": "container", + "file_path": "src/theme.css", + "is_module": 0 + }, + { + "name": "primary", + "file_path": "src/styles/button.module.css", + "is_module": 1 + } +] diff --git a/fixtures/golden/minimal/css-imports.json b/fixtures/golden/minimal/css-imports.json new file mode 100644 index 00000000..308e6f61 --- /dev/null +++ b/fixtures/golden/minimal/css-imports.json @@ -0,0 +1,6 @@ +[ + { + "file_path": "src/styles/button.module.css", + "source": "../theme.css" + } +] diff --git a/fixtures/golden/minimal/css-keyframes.json b/fixtures/golden/minimal/css-keyframes.json new file mode 100644 index 00000000..5882a36d --- /dev/null +++ b/fixtures/golden/minimal/css-keyframes.json @@ -0,0 +1,6 @@ +[ + { + "name": "fadeIn", + "file_path": "src/styles/button.module.css" + } +] diff --git a/fixtures/golden/minimal/css-variables.json b/fixtures/golden/minimal/css-variables.json new file mode 100644 index 00000000..b15218bb --- /dev/null +++ b/fixtures/golden/minimal/css-variables.json @@ -0,0 +1,10 @@ +[ + { + "name": "--color-brand", + "value": "rgb(51, 102, 204)" + }, + { + "name": "--spacing-md", + "value": "1rem" + } +] diff --git a/fixtures/golden/minimal/exports-client.json b/fixtures/golden/minimal/exports-client.json new file mode 100644 index 00000000..ed86df2a --- /dev/null +++ b/fixtures/golden/minimal/exports-client.json @@ -0,0 +1,6 @@ +[ + { + "name": "createClient", + "kind": "value" + } +] diff --git a/fixtures/golden/minimal/files-count.json b/fixtures/golden/minimal/files-count.json index 4a98986c..ecc2960a 100644 --- a/fixtures/golden/minimal/files-count.json +++ b/fixtures/golden/minimal/files-count.json @@ -1,5 +1,5 @@ [ { - "n": 10 + "n": 11 } ] diff --git a/fixtures/golden/minimal/index-summary.json b/fixtures/golden/minimal/index-summary.json index fc688241..d360b2c0 100644 --- a/fixtures/golden/minimal/index-summary.json +++ b/fixtures/golden/minimal/index-summary.json @@ -1,8 +1,8 @@ [ { - "files": 10, - "symbols": 5, - "imports": 2, + "files": 11, + "symbols": 6, + "imports": 3, "components": 1, "dependencies": 2 } diff --git a/fixtures/golden/minimal/markers-all-kinds.json b/fixtures/golden/minimal/markers-all-kinds.json new file mode 100644 index 00000000..db59b2c7 --- /dev/null +++ b/fixtures/golden/minimal/markers-all-kinds.json @@ -0,0 +1,10 @@ +[ + { + "kind": "FIXME", + "n": 1 + }, + { + "kind": "TODO", + "n": 2 + } +] diff --git a/fixtures/golden/scenarios.json b/fixtures/golden/scenarios.json index 908a31b9..526c2bf7 100644 --- a/fixtures/golden/scenarios.json +++ b/fixtures/golden/scenarios.json @@ -28,5 +28,40 @@ "id": "markers-notes-todo", "prompt": "TODO marker in fixture notes markdown", "sql": "SELECT file_path, line_number, kind, content FROM markers WHERE file_path = 'src/notes.md'" + }, + { + "id": "exports-client", + "prompt": "What does src/api/client.ts export?", + "sql": "SELECT name, kind FROM exports WHERE file_path = 'src/api/client.ts'" + }, + { + "id": "css-variables", + "prompt": "Which CSS custom properties (design tokens) are defined?", + "sql": "SELECT name, value FROM css_variables ORDER BY name" + }, + { + "id": "css-classes-module", + "prompt": "Which CSS classes exist and are any from CSS modules?", + "sql": "SELECT name, file_path, is_module FROM css_classes ORDER BY name" + }, + { + "id": "css-keyframes", + "prompt": "Which @keyframes animations are defined?", + "sql": "SELECT name, file_path FROM css_keyframes" + }, + { + "id": "css-imports", + "prompt": "Which CSS files import other stylesheets?", + "sql": "SELECT file_path, source FROM imports WHERE file_path LIKE '%.css'" + }, + { + "id": "markers-all-kinds", + "prompt": "How many markers of each kind exist?", + "sql": "SELECT kind, COUNT(*) as n FROM markers GROUP BY kind ORDER BY kind" + }, + { + "id": "components-no-false-positives", + "prompt": "Which components are detected (FormatPrice should not appear)?", + "sql": "SELECT name FROM components ORDER BY name" } ] diff --git a/fixtures/minimal/src/components/shop/ShopButton.tsx b/fixtures/minimal/src/components/shop/ShopButton.tsx index be1a4673..d8380d7c 100644 --- a/fixtures/minimal/src/components/shop/ShopButton.tsx +++ b/fixtures/minimal/src/components/shop/ShopButton.tsx @@ -1,3 +1,7 @@ +export function FormatPrice(cents: number): string { + return `$${(cents / 100).toFixed(2)}`; +} + export function ShopButton() { return ; } diff --git a/fixtures/minimal/src/consumer.ts b/fixtures/minimal/src/consumer.ts index ea9fc28c..9d33a817 100644 --- a/fixtures/minimal/src/consumer.ts +++ b/fixtures/minimal/src/consumer.ts @@ -2,6 +2,7 @@ import { createClient } from "~/api/client"; import { now } from "./utils/date"; +// FIXME: handle errors export function run() { createClient(); return now(); diff --git a/fixtures/minimal/src/styles/button.module.css b/fixtures/minimal/src/styles/button.module.css new file mode 100644 index 00000000..70638780 --- /dev/null +++ b/fixtures/minimal/src/styles/button.module.css @@ -0,0 +1,15 @@ +@import "../theme.css"; + +.primary { + background: var(--color-brand); + color: white; +} + +@keyframes fadeIn { + from { + opacity: 0; + } + to { + opacity: 1; + } +} diff --git a/fixtures/minimal/src/theme.css b/fixtures/minimal/src/theme.css index 2a0e4674..ce39d194 100644 --- a/fixtures/minimal/src/theme.css +++ b/fixtures/minimal/src/theme.css @@ -1,3 +1,8 @@ :root { --color-brand: #3366cc; + --spacing-md: 1rem; +} + +.container { + max-width: 960px; } diff --git a/src/application/index-engine.ts b/src/application/index-engine.ts index 9a82ff09..21a35940 100644 --- a/src/application/index-engine.ts +++ b/src/application/index-engine.ts @@ -126,10 +126,11 @@ export function getChangedFiles(db: CodemapDatabase): { .filter(Boolean) .map((line: string) => line.slice(3).trim()); + const indexedPaths = new Set(getAllFileHashes(db).keys()); const allChanged = [...new Set([...diffFiles, ...statusFiles])].filter( (f) => { const ext = extname(f); - return ext in LANG_MAP; + return ext in LANG_MAP || indexedPaths.has(f); }, ); @@ -186,19 +187,18 @@ function insertParsedResults( } if (parsed.markers?.length) insertMarkers(db, parsed.markers); - if (parsed.cssImportSources) { - for (const importSource of parsed.cssImportSources) { - insertImports(db, [ - { - file_path: parsed.relPath, - source: importSource, - resolved_path: null, - specifiers: "[]", - is_type_only: 0, - line_number: 0, - }, - ]); - } + if (parsed.cssImportSources?.length) { + insertImports( + db, + parsed.cssImportSources.map((importSource) => ({ + file_path: parsed.relPath, + source: importSource, + resolved_path: null, + specifiers: "[]", + is_type_only: 0, + line_number: 0, + })), + ); } } else { if (parsed.symbols?.length) insertSymbols(db, parsed.symbols); @@ -230,7 +230,7 @@ function insertParsedResults( return indexed; } -function fetchTableStats(db: CodemapDatabase): IndexTableStats { +export function fetchTableStats(db: CodemapDatabase): IndexTableStats { const row = db .query>( `SELECT @@ -333,17 +333,18 @@ export async function indexFiles( insertCssKeyframes(db, cssData.keyframes); } if (cssData.markers.length) insertMarkers(db, cssData.markers); - for (const importSource of cssData.importSources) { - insertImports(db, [ - { + if (cssData.importSources.length) { + insertImports( + db, + cssData.importSources.map((importSource) => ({ file_path: relPath, source: importSource, resolved_path: null, specifiers: "[]", is_type_only: 0, line_number: 0, - }, - ]); + })), + ); } } else { const data = extractFileData(absPath, source, relPath); @@ -475,7 +476,7 @@ export function printQueryResult( } return 1; } finally { - if (db !== undefined) closeDb(db); + if (db !== undefined) closeDb(db, { readonly: true }); } } @@ -488,6 +489,6 @@ export function queryRows(sql: string): unknown[] { try { return db.query(sql).all(); } finally { - closeDb(db); + closeDb(db, { readonly: true }); } } diff --git a/src/application/run-index.ts b/src/application/run-index.ts index 627d8ad5..7291b85d 100644 --- a/src/application/run-index.ts +++ b/src/application/run-index.ts @@ -1,15 +1,13 @@ -import { extname } from "node:path"; - import { createSchema, getAllFileHashes, setMeta } from "../db"; import type { CodemapDatabase } from "../db"; import { collectFiles, deleteFilesFromIndex, + fetchTableStats, getChangedFiles, getCurrentCommit, indexFiles, targetedReindex, - VALID_EXTENSIONS, } from "./index-engine"; import type { IndexResult, IndexTableStats } from "./types"; @@ -42,7 +40,7 @@ export interface RunIndexOptions { mode?: IndexMode; /** * Paths relative to the project root; used only when `mode === "files"`. - * Non-indexable extensions are filtered out. + * All paths are forwarded as-is; non-standard extensions are indexed as text. */ files?: string[]; /** @@ -82,10 +80,7 @@ export async function runCodemapIndex( } if (mode === "files") { - const targetFiles = (options.files ?? []).filter((f) => { - const ext = extname(f); - return VALID_EXTENSIONS.has(ext); - }); + const targetFiles = options.files ?? []; if (targetFiles.length === 0) { return { mode: "files", @@ -139,7 +134,7 @@ export async function runCodemapIndex( indexed: 0, skipped: 0, elapsedMs: 0, - stats: fetchStats(db), + stats: fetchTableStats(db), idle: true, }; } @@ -149,7 +144,7 @@ export async function runCodemapIndex( indexed: 0, skipped: 0, elapsedMs: 0, - stats: fetchStats(db), + stats: fetchTableStats(db), idle: true, }; } @@ -169,22 +164,3 @@ export async function runCodemapIndex( stats: run.stats, }; } - -function fetchStats(db: CodemapDatabase): IndexTableStats { - const row = db - .query>( - `SELECT - (SELECT COUNT(*) FROM files) as files, - (SELECT COUNT(*) FROM symbols) as symbols, - (SELECT COUNT(*) FROM imports) as imports, - (SELECT COUNT(*) FROM exports) as exports, - (SELECT COUNT(*) FROM components) as components, - (SELECT COUNT(*) FROM dependencies) as dependencies, - (SELECT COUNT(*) FROM markers) as markers, - (SELECT COUNT(*) FROM css_variables) as css_vars, - (SELECT COUNT(*) FROM css_classes) as css_classes, - (SELECT COUNT(*) FROM css_keyframes) as css_keyframes`, - ) - .get()!; - return row as IndexTableStats; -} diff --git a/src/cli/cmd-index.ts b/src/cli/cmd-index.ts index 1e6e3dbc..46230a8c 100644 --- a/src/cli/cmd-index.ts +++ b/src/cli/cmd-index.ts @@ -20,14 +20,12 @@ export async function runIndexCmd(opts: { const db = openDb(); try { if (args[0] === "--files" && args.length > 1) { - const targetFiles = args.slice(1).filter((f) => { - const ext = extname(f); - if (!VALID_EXTENSIONS.has(ext)) { - console.warn(` Skipping ${f}: unsupported extension "${ext}"`); - return false; + const targetFiles = args.slice(1); + for (const f of targetFiles) { + if (!VALID_EXTENSIONS.has(extname(f))) { + console.warn(` ${f}: non-standard extension, indexing as text`); } - return true; - }); + } if (targetFiles.length > 0) { await runCodemapIndex(db, { mode: "files", diff --git a/src/css-parser.ts b/src/css-parser.ts index 6c776d53..c7038a46 100644 --- a/src/css-parser.ts +++ b/src/css-parser.ts @@ -34,7 +34,6 @@ export function extractCssData( filename: filePath, code: Buffer.from(source), errorRecovery: true, - analyzeDependencies: true, customAtRules: { // Tailwind v4 @theme blocks theme: { body: "declaration-list" }, diff --git a/src/db.ts b/src/db.ts index 8d5d3385..c4ec7277 100644 --- a/src/db.ts +++ b/src/db.ts @@ -5,7 +5,7 @@ import { } from "./sqlite-db"; /** - * Pre-release: keep at **2** until the first npm release — do not bump for DDL + * Pre-release: keep at **1** until the first npm release — do not bump for DDL * tweaks; run `--full` locally after pulling. After v1.0, bump in lockstep with * `createTables` / `createIndexes` when the on-disk schema changes. */ @@ -17,10 +17,15 @@ export function openDb(): CodemapDatabase { return openCodemapDatabase(); } -export function closeDb(db: CodemapDatabase) { - db.run("PRAGMA analysis_limit = 400"); - db.run("PRAGMA optimize"); - db.close(); +export function closeDb(db: CodemapDatabase, opts?: { readonly?: boolean }) { + try { + if (!opts?.readonly) { + db.run("PRAGMA analysis_limit = 400"); + db.run("PRAGMA optimize"); + } + } finally { + db.close(); + } } export function createTables(db: CodemapDatabase) { diff --git a/src/glob-sync.ts b/src/glob-sync.ts index 47593909..1ee6a3bf 100644 --- a/src/glob-sync.ts +++ b/src/glob-sync.ts @@ -6,7 +6,6 @@ import { globSync as tinyglobbySync } from "tinyglobby"; */ export function globSync(pattern: string, cwd: string): string[] { if (typeof Bun !== "undefined") { - // eslint-disable-next-line @typescript-eslint/no-require-imports const { Glob } = require("bun") as typeof import("bun"); const glob = new Glob(pattern); return Array.from(glob.scanSync({ cwd, dot: true })); diff --git a/src/parser.test.ts b/src/parser.test.ts index 8dde7437..49626f3a 100644 --- a/src/parser.test.ts +++ b/src/parser.test.ts @@ -35,9 +35,46 @@ describe("extractFileData", () => { }); it("treats .tsx as JSX for PascalCase function components", () => { - const src = `export function Button() { return null; }\n`; + const src = `export function Button() { return ; }\n`; const d = extractFileData("/proj/x.tsx", src, "x.tsx"); expect(d.components.some((c) => c.name === "Button")).toBe(true); }); }); + + describe("component detection heuristic", () => { + it("detects components that return JSX", () => { + const src = `export function Card() { return
card
; }\n`; + const d = extractFileData("/proj/x.tsx", src, "x.tsx"); + expect(d.components.some((c) => c.name === "Card")).toBe(true); + }); + + it("detects arrow components that return JSX", () => { + const src = `export const Card = () =>
card
;\n`; + const d = extractFileData("/proj/x.tsx", src, "x.tsx"); + expect(d.components.some((c) => c.name === "Card")).toBe(true); + }); + + it("detects components that use hooks", () => { + const src = `export function Timer() { useEffect(() => {}); return null; }\n`; + const d = extractFileData("/proj/x.tsx", src, "x.tsx"); + expect(d.components.some((c) => c.name === "Timer")).toBe(true); + }); + + it("rejects PascalCase functions without JSX or hooks", () => { + const src = [ + `export function FormatCurrency(n: number): string { return "$"+n; }`, + `export function ValidateEmail(e: string): boolean { return e.includes("@"); }`, + ].join("\n"); + const d = extractFileData("/proj/x.tsx", src, "x.tsx"); + expect(d.components).toHaveLength(0); + }); + + it("rejects PascalCase functions that return null without hooks", () => { + const src = `export function EmptyPlaceholder() { return null; }\n`; + const d = extractFileData("/proj/x.tsx", src, "x.tsx"); + expect(d.components.some((c) => c.name === "EmptyPlaceholder")).toBe( + false, + ); + }); + }); }); diff --git a/src/parser.ts b/src/parser.ts index 4ebaa71b..15f5d329 100644 --- a/src/parser.ts +++ b/src/parser.ts @@ -103,6 +103,7 @@ export function extractFileData( } const hookCalls = new Map>(); // function scope name -> hook names + const jsxScopes = new Set(); // function scopes that contain JSX let currentFunctionScope: string | null = null; const visitor = new Visitor({ @@ -255,6 +256,13 @@ export function extractFileData( hookCalls.get(currentFunctionScope)?.add(callee.name); } }, + + JSXElement() { + if (currentFunctionScope) jsxScopes.add(currentFunctionScope); + }, + JSXFragment() { + if (currentFunctionScope) jsxScopes.add(currentFunctionScope); + }, }); visitor.visit(result.program); @@ -264,6 +272,8 @@ export function extractFileData( function maybeAddComponent(name: string, node: any, _isArrow: boolean) { if (!isTsx || !/^[A-Z]/.test(name)) return; const hooks = hookCalls.get(name); + const hasJsx = jsxScopes.has(name); + if (!hasJsx && !(hooks && hooks.size > 0)) return; const isDefault = defaultExportedNames.has(name); let propsType: string | null = null; diff --git a/src/sqlite-db.ts b/src/sqlite-db.ts index 456330c2..6593cc91 100644 --- a/src/sqlite-db.ts +++ b/src/sqlite-db.ts @@ -60,7 +60,6 @@ function runSql(inner: SqliteInner, sql: string, params?: BindValues): void { function openRaw(path: string): SqliteInner { if (typeof Bun !== "undefined") { - // eslint-disable-next-line @typescript-eslint/no-require-imports const { Database } = require("bun:sqlite") as { Database: new (path: string, opts?: { create?: boolean }) => unknown; }; @@ -68,13 +67,22 @@ function openRaw(path: string): SqliteInner { } type BetterSqlite = typeof import("better-sqlite3"); - // eslint-disable-next-line @typescript-eslint/no-require-imports const BetterSqlite = require("better-sqlite3") as BetterSqlite; const rawDb = new BetterSqlite(path); + const stmtCache = new Map(); + + function cachedPrepare(sql: string) { + let stmt = stmtCache.get(sql); + if (!stmt) { + stmt = rawDb.prepare(sql); + stmtCache.set(sql, stmt); + } + return stmt; + } return { run(sql: string, params?: BindValues) { - const stmt = rawDb.prepare(sql); + const stmt = cachedPrepare(sql); if (params !== undefined && params.length > 0) { stmt.run(...params); } else { @@ -82,7 +90,7 @@ function openRaw(path: string): SqliteInner { } }, query(sql: string) { - const stmt = rawDb.prepare(sql); + const stmt = cachedPrepare(sql); return { get(...params: unknown[]) { return stmt.get(...params); @@ -107,12 +115,13 @@ function wrap(inner: SqliteInner): CodemapDatabase { runSql(inner, sql, params); }, query(sql: string) { + const stmt = inner.query(sql); return { get(...params: unknown[]) { - return inner.query(sql).get(...params) as T | undefined; + return stmt.get(...params) as T | undefined; }, all(...params: unknown[]) { - return inner.query(sql).all(...params) as T[]; + return stmt.all(...params) as T[]; }, }; }, diff --git a/src/worker-pool.ts b/src/worker-pool.ts index 8167158f..c51f3d67 100644 --- a/src/worker-pool.ts +++ b/src/worker-pool.ts @@ -44,6 +44,10 @@ export function parseFilesParallel(filePaths: string[]): Promise { resolve(event.data.results); worker.terminate(); }; + worker.onerror = (event: ErrorEvent) => { + reject(new Error(event.message)); + worker.terminate(); + }; worker.postMessage(input); return; } diff --git a/templates/agents/skills/codemap/SKILL.md b/templates/agents/skills/codemap/SKILL.md index 4cabb7b1..0ac8ae18 100644 --- a/templates/agents/skills/codemap/SKILL.md +++ b/templates/agents/skills/codemap/SKILL.md @@ -68,7 +68,7 @@ LIMIT 10 | Column | Type | Description | | ------------- | ------- | -------------------------------------------------------------------------------- | | path | TEXT PK | Relative path from project root | -| content_hash | TEXT | Wyhash fingerprint (base-36) | +| content_hash | TEXT | SHA-256 hex digest | | size | INTEGER | File size in bytes | | line_count | INTEGER | Number of lines | | language | TEXT | `ts`, `tsx`, `js`, `jsx`, `css`, `md`, `mdx`, `mdc`, `json`, `yaml`, `sh`, `txt` | @@ -77,17 +77,17 @@ LIMIT 10 ### `symbols` — Functions, types, interfaces, enums, constants, classes -| Column | Type | Description | -| ----------------- | ---------- | --------------------------------------------------------------------- | -| id | INTEGER PK | Auto-increment ID | -| file_path | TEXT FK | References `files(path)` | -| name | TEXT | Symbol name | -| kind | TEXT | `function`, `class`, `type`, `interface`, `enum`, `const`, `variable` | -| line_start | INTEGER | Start line (1-based) | -| line_end | INTEGER | End line (1-based) | -| signature | TEXT | e.g. `createHandler()`, `type UserProps` | -| is_exported | INTEGER | 1 if exported | -| is_default_export | INTEGER | 1 if default export | +| Column | Type | Description | +| ----------------- | ---------- | --------------------------------------------------------- | +| id | INTEGER PK | Auto-increment ID | +| file_path | TEXT FK | References `files(path)` | +| name | TEXT | Symbol name | +| kind | TEXT | `function`, `class`, `type`, `interface`, `enum`, `const` | +| line_start | INTEGER | Start line (1-based) | +| line_end | INTEGER | End line (1-based) | +| signature | TEXT | e.g. `createHandler()`, `type UserProps` | +| is_exported | INTEGER | 1 if exported | +| is_default_export | INTEGER | 1 if default export | ### `imports` — Import statements