Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 12 additions & 12 deletions .agents/skills/codemap/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ LIMIT 10
| Column | Type | Description |
| ------------- | ------- | -------------------------------------------------------------------------------- |
| path | TEXT PK | Relative path from project root |
| content_hash | TEXT | Wyhash fingerprint (base-36) |
| content_hash | TEXT | SHA-256 hex digest |
| size | INTEGER | File size in bytes |
| line_count | INTEGER | Number of lines |
| language | TEXT | `ts`, `tsx`, `js`, `jsx`, `css`, `md`, `mdx`, `mdc`, `json`, `yaml`, `sh`, `txt` |
Expand All @@ -77,17 +77,17 @@ LIMIT 10

### `symbols` — Functions, types, interfaces, enums, constants, classes

| Column | Type | Description |
| ----------------- | ---------- | --------------------------------------------------------------------- |
| id | INTEGER PK | Auto-increment ID |
| file_path | TEXT FK | References `files(path)` |
| name | TEXT | Symbol name |
| kind | TEXT | `function`, `class`, `type`, `interface`, `enum`, `const`, `variable` |
| line_start | INTEGER | Start line (1-based) |
| line_end | INTEGER | End line (1-based) |
| signature | TEXT | e.g. `createHandler()`, `type UserProps` |
| is_exported | INTEGER | 1 if exported |
| is_default_export | INTEGER | 1 if default export |
| Column | Type | Description |
| ----------------- | ---------- | --------------------------------------------------------- |
| id | INTEGER PK | Auto-increment ID |
| file_path | TEXT FK | References `files(path)` |
| name | TEXT | Symbol name |
| kind | TEXT | `function`, `class`, `type`, `interface`, `enum`, `const` |
| line_start | INTEGER | Start line (1-based) |
| line_end | INTEGER | End line (1-based) |
| signature | TEXT | e.g. `createHandler()`, `type UserProps` |
| is_exported | INTEGER | 1 if exported |
| is_default_export | INTEGER | 1 if default export |

### `imports` — Import statements

Expand Down
38 changes: 38 additions & 0 deletions .changeset/audit-findings.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
---
"@stainless-code/codemap": patch
---

Fix three HIGH-severity bugs found via cross-audit triangulation, plus performance and docs improvements.

**Bug fixes**

- Add missing `onerror` handler on Bun Worker — prevents silent promise hang when a parse worker crashes
- Require JSX return or hook usage for component detection — eliminates false positives (e.g. `FormatCurrency()` in `.tsx` files no longer indexed as a component)
- Include previously-indexed files in incremental and `--files` modes — custom-extension files indexed during `--full` no longer silently go stale

**Performance**

- Batch CSS imports instead of inserting one-at-a-time (both full-rebuild and incremental paths)
- Add `Map<string, Statement>` cache for `better-sqlite3` `run()`/`query()` — avoids ~2,000+ redundant `prepare()` calls on large projects
- Hoist `inner.query()` in `wrap()` to prepare once per call instead of per `.get()`/`.all()`
- Skip `PRAGMA optimize` on `closeDb` for read-only query paths

**Docs**

- Fix Wyhash → SHA-256 in architecture.md and SKILL.md (3 locations)
- Correct `symbols.kind` values (`variable` → `const`, `type_alias` → `type`) and `exports.kind` values
- Clarify `Database.query()` caching is Bun-only; Node statement cache via wrapper
- Update architecture.md: component heuristic, statement cache, `closeDb` readonly, incremental/`--files` custom extensions
- Update benchmark.md and golden-queries.md for enriched fixture

**Testing**

- Enrich `fixtures/minimal/` to cover all 10 indexed tables (CSS module, `@keyframes`, `@import`, non-component PascalCase export, FIXME marker)
- Add 7 new golden scenarios (exports, css_variables, css_classes, css_keyframes, css_imports, markers-all-kinds, components-no-false-positives)

**Cleanup**

- Remove unused `analyzeDependencies: true` from CSS parser
- Deduplicate `fetchTableStats` (was duplicated across `index-engine.ts` and `run-index.ts`)
- Remove dead `eslint-disable-next-line` directives (oxlint doesn't enforce those rules)
- Fix `SCHEMA_VERSION` comment (said "2", value is `1`)
23 changes: 13 additions & 10 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ A local SQLite database (`.codemap.db`) indexes the project tree and stores stru

### `--files` (targeted reindex)

When specific file paths are passed via `--files`, the indexer skips git diff, git status, and the full filesystem glob scan. It reads the set of already-indexed paths from the database (for import resolution), then only processes the listed files. Files that no longer exist on disk are automatically removed from the index via `ON DELETE CASCADE`.
When specific file paths are passed via `--files`, the indexer skips git diff, git status, and the full filesystem glob scan. It reads the set of already-indexed paths from the database (for import resolution), then only processes the listed files. Files with non-standard extensions (e.g. custom `include` globs) are accepted and indexed as text; a warning is printed but they are not skipped. Files that no longer exist on disk are automatically removed from the index via `ON DELETE CASCADE`.

## Programmatic usage

Expand Down Expand Up @@ -171,14 +171,14 @@ All tables use `STRICT` mode. Tables marked with `WITHOUT ROWID` store data dire
| last_modified | INTEGER | File mtime (epoch ms) |
| indexed_at | INTEGER | When this row was written |

### `symbols` — Functions, variables, classes, interfaces, type aliases, enums (`STRICT`)
### `symbols` — Functions, constants, classes, interfaces, types, enums (`STRICT`)

| Column | Type | Description |
| ----------------- | ---------- | --------------------------------------------------------------------- |
| id | INTEGER PK | Auto-increment row id |
| file_path | TEXT FK | References `files(path)` ON DELETE CASCADE |
| name | TEXT | Symbol name |
| kind | TEXT | `function`, `variable`, `class`, `interface`, `type_alias`, `enum` |
| kind | TEXT | `function`, `const`, `class`, `interface`, `type`, `enum` |
| line_start | INTEGER | Start line (1-based) |
| line_end | INTEGER | End line |
| signature | TEXT | Reconstructed signature (e.g. `usePermissions(): PermissionsContext`) |
Expand All @@ -204,11 +204,11 @@ All tables use `STRICT` mode. Tables marked with `WITHOUT ROWID` store data dire
| id | INTEGER PK | Auto-increment row id |
| file_path | TEXT FK | File containing the export |
| name | TEXT | Exported name |
| kind | TEXT | `function`, `variable`, etc. |
| kind | TEXT | `value`, `type`, `re-export` |
| is_default | INTEGER | 1 if default export |
| re_export_source | TEXT | Source module if re-exported |

### `components` — React components (detected by JSX return + PascalCase) (`STRICT`)
### `components` — React components (detected by PascalCase + JSX return or hook usage) (`STRICT`)

| Column | Type | Description |
| ----------------- | ---------- | ----------------------------- |
Expand Down Expand Up @@ -286,7 +286,7 @@ Uses the Rust-based `oxc-parser` via NAPI bindings to parse TypeScript/TSX/JS/JS
- **Symbols**: Functions, arrow functions, classes, interfaces, type aliases, enums — with reconstructed signatures
- **Imports**: All `import` statements with specifiers, source paths, and type-only flags
- **Exports**: Named exports, default exports, re-exports
- **Components**: React components detected via PascalCase name + JSX return. Extracts props type and hooks used
- **Components**: React components detected via PascalCase name + (JSX return **or** hook usage). A PascalCase function in `.tsx`/`.jsx` that neither returns JSX nor calls hooks is indexed only as a symbol, not a component. Extracts props type and hooks used
- **Markers**: `TODO`, `FIXME`, `HACK`, `NOTE` comments with line numbers

### CSS — `css-parser.ts` (`lightningcss`)
Expand Down Expand Up @@ -327,9 +327,10 @@ The indexer uses git to detect changes since the last indexed commit:

1. **Stores `last_indexed_commit`** (HEAD SHA) in the `meta` table after each run
2. On next run, computes `git diff --name-only <last_commit>..HEAD` + `git status --porcelain`
3. Only re-indexes changed files (Wyhash content comparison), using DB-sourced `indexedPaths` for import resolution (skips full `collectFiles()` glob scan)
4. Deleted files are removed via `ON DELETE CASCADE` — deleting from `files` cascades to all related tables
5. Falls back to full rebuild if commit history is incompatible (e.g. force push, branch switch)
3. Filters changed files to those with a known extension **or** already present in the `files` table (so custom-extension files indexed during `--full` are re-indexed on subsequent incremental runs)
4. Only re-indexes changed files (SHA-256 content comparison), using DB-sourced `indexedPaths` for import resolution (skips full `collectFiles()` glob scan)
5. Deleted files are removed via `ON DELETE CASCADE` — deleting from `files` cascades to all related tables
6. Falls back to full rebuild if commit history is incompatible (e.g. force push, branch switch)

## File Artifacts

Expand Down Expand Up @@ -436,7 +437,7 @@ Until the first release, Codemap keeps **`SCHEMA_VERSION` at 1**; pull `--full`

### `bun:sqlite` API

All DDL and PRAGMA statements use `Database.run()` (not the deprecated `Database.exec()` alias). Parameterized insert/update statements use `Database.query()` (which caches compiled statements) instead of `Database.prepare()` (which does not cache). Read queries also use `Database.query().all()` or `.get()`. Bulk inserts use the generic `batchInsert<T>()` helper with multi-row `INSERT ... VALUES (...),(...),(...)` in batches of 100, pre-computed placeholders, and zero-copy index-bounds iteration.
All DDL and PRAGMA statements use `Database.run()`. The `sqlite-db.ts` wrapper abstracts both Bun (`bun:sqlite`) and Node (`better-sqlite3`). On Bun, `Database.query()` caches compiled statements internally. On Node, the wrapper maintains a `Map<string, Statement>` cache so repeated `run()` and `query()` calls with the same SQL reuse a single prepared statement. Read queries use the wrapper's `.query().all()` or `.get()`. Bulk inserts use the generic `batchInsert<T>()` helper with multi-row `INSERT ... VALUES (...),(...),(...)` in batches of 100, pre-computed placeholders, and zero-copy index-bounds iteration.

### PRAGMAs (set on every `openDb()`)

Expand All @@ -457,6 +458,8 @@ All DDL and PRAGMA statements use `Database.run()` (not the deprecated `Database
| `analysis_limit` | `400` | Caps rows sampled by `optimize` to keep it fast |
| `optimize` | — | Gathers query planner statistics (`sqlite_stat1`) for better plans |

Read-only query paths (`printQueryResult`, `queryRows`) call `closeDb` with `{ readonly: true }`, which skips both PRAGMAs to avoid write contention under concurrent `codemap query` processes.

### WITHOUT ROWID tables

Tables with a TEXT PRIMARY KEY and no auto-increment benefit from `WITHOUT ROWID` — the data is stored directly in the primary key B-tree instead of a separate rowid B-tree, eliminating a lookup indirection:
Expand Down
7 changes: 5 additions & 2 deletions docs/benchmark.md
Original file line number Diff line number Diff line change
Expand Up @@ -206,9 +206,12 @@ The benchmark also measures the cost of keeping the index fresh (3 runs each, sa

#### `fixtures/minimal/`

Small **private** package (not published) with intentional:
Small **private** package (not published) with intentional coverage of all indexed tables:

- `usePermissions`, `~/api/client` import, `components/shop/*`, `utils/date`, CSS variables, and a TODO marker.
- **Symbols / imports / dependencies**: `usePermissions`, `~/api/client` path alias, `components/shop/*`, `utils/date`
- **Components**: `ShopButton` (JSX return) + `FormatPrice` (PascalCase non-component — validates detection heuristic)
- **CSS**: variables (`--color-brand`, `--spacing-md`), class selectors (`.container`, `.primary` in a `.module.css`), `@keyframes fadeIn`, `@import` edge
- **Markers**: `TODO` (in `notes.md`) + `FIXME` (in `consumer.ts`)

**Local:**

Expand Down
13 changes: 7 additions & 6 deletions docs/golden-queries.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,12 +69,13 @@ Scenarios live in **`fixtures/golden/scenarios.json`** (Tier A) or optional **`s

## Status

| Area | State |
| ----------------------------- | ----------------------------------------------------------------------- |
| Tier A runner + CI | **`bun run test:golden`** in `check` |
| Tier B external + schema | **`test:golden:external`**, Zod in **`scripts/query-golden/schema.ts`** |
| Subset matchers + budgets | **`match`**, **`budgetMs`**, **`--strict-budget`** |
| Optional CI for public corpus | Deferred — [roadmap § Backlog](./roadmap.md#backlog) |
| Area | State |
| ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| Tier A runner + CI | **`bun run test:golden`** in `check` |
| Tier A scenario coverage | 13 scenarios across all indexed tables: files, symbols, imports, exports, components, dependencies, markers, CSS vars/classes/keyframes |
| Tier B external + schema | **`test:golden:external`**, Zod in **`scripts/query-golden/schema.ts`** |
| Subset matchers + budgets | **`match`**, **`budgetMs`**, **`--strict-budget`** |
| Optional CI for public corpus | Deferred — [roadmap § Backlog](./roadmap.md#backlog) |

---

Expand Down
5 changes: 5 additions & 0 deletions fixtures/golden/minimal/components-no-false-positives.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[
{
"name": "ShopButton"
}
]
12 changes: 12 additions & 0 deletions fixtures/golden/minimal/css-classes-module.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
[
{
"name": "container",
"file_path": "src/theme.css",
"is_module": 0
},
{
"name": "primary",
"file_path": "src/styles/button.module.css",
"is_module": 1
}
]
6 changes: 6 additions & 0 deletions fixtures/golden/minimal/css-imports.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[
{
"file_path": "src/styles/button.module.css",
"source": "../theme.css"
}
]
6 changes: 6 additions & 0 deletions fixtures/golden/minimal/css-keyframes.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[
{
"name": "fadeIn",
"file_path": "src/styles/button.module.css"
}
]
10 changes: 10 additions & 0 deletions fixtures/golden/minimal/css-variables.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[
{
"name": "--color-brand",
"value": "rgb(51, 102, 204)"
},
{
"name": "--spacing-md",
"value": "1rem"
}
]
6 changes: 6 additions & 0 deletions fixtures/golden/minimal/exports-client.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[
{
"name": "createClient",
"kind": "value"
}
]
2 changes: 1 addition & 1 deletion fixtures/golden/minimal/files-count.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[
{
"n": 10
"n": 11
}
]
6 changes: 3 additions & 3 deletions fixtures/golden/minimal/index-summary.json
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
[
{
"files": 10,
"symbols": 5,
"imports": 2,
"files": 11,
"symbols": 6,
"imports": 3,
"components": 1,
"dependencies": 2
}
Expand Down
10 changes: 10 additions & 0 deletions fixtures/golden/minimal/markers-all-kinds.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[
{
"kind": "FIXME",
"n": 1
},
{
"kind": "TODO",
"n": 2
}
]
35 changes: 35 additions & 0 deletions fixtures/golden/scenarios.json
Original file line number Diff line number Diff line change
Expand Up @@ -28,5 +28,40 @@
"id": "markers-notes-todo",
"prompt": "TODO marker in fixture notes markdown",
"sql": "SELECT file_path, line_number, kind, content FROM markers WHERE file_path = 'src/notes.md'"
},
{
"id": "exports-client",
"prompt": "What does src/api/client.ts export?",
"sql": "SELECT name, kind FROM exports WHERE file_path = 'src/api/client.ts'"
},
{
"id": "css-variables",
"prompt": "Which CSS custom properties (design tokens) are defined?",
"sql": "SELECT name, value FROM css_variables ORDER BY name"
},
{
"id": "css-classes-module",
"prompt": "Which CSS classes exist and are any from CSS modules?",
"sql": "SELECT name, file_path, is_module FROM css_classes ORDER BY name"
},
{
"id": "css-keyframes",
"prompt": "Which @keyframes animations are defined?",
"sql": "SELECT name, file_path FROM css_keyframes"
},
{
"id": "css-imports",
"prompt": "Which CSS files import other stylesheets?",
"sql": "SELECT file_path, source FROM imports WHERE file_path LIKE '%.css'"
},
{
"id": "markers-all-kinds",
"prompt": "How many markers of each kind exist?",
"sql": "SELECT kind, COUNT(*) as n FROM markers GROUP BY kind ORDER BY kind"
},
{
"id": "components-no-false-positives",
"prompt": "Which components are detected (FormatPrice should not appear)?",
"sql": "SELECT name FROM components ORDER BY name"
}
]
4 changes: 4 additions & 0 deletions fixtures/minimal/src/components/shop/ShopButton.tsx
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
export function FormatPrice(cents: number): string {
return `$${(cents / 100).toFixed(2)}`;
}

export function ShopButton() {
return <button type="button">Buy</button>;
}
1 change: 1 addition & 0 deletions fixtures/minimal/src/consumer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { createClient } from "~/api/client";

import { now } from "./utils/date";

// FIXME: handle errors
export function run() {
createClient();
return now();
Expand Down
15 changes: 15 additions & 0 deletions fixtures/minimal/src/styles/button.module.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
@import "../theme.css";

.primary {
background: var(--color-brand);
color: white;
}

@keyframes fadeIn {
from {
opacity: 0;
}
to {
opacity: 1;
}
}
5 changes: 5 additions & 0 deletions fixtures/minimal/src/theme.css
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
:root {
--color-brand: #3366cc;
--spacing-md: 1rem;
}

.container {
max-width: 960px;
}
Loading
Loading