Skip to content
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .agents/skills/codemap/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ LIMIT 10
| Column | Type | Description |
| ------------- | ------- | -------------------------------------------------------------------------------- |
| path | TEXT PK | Relative path from project root |
| content_hash | TEXT | Wyhash fingerprint (base-36) |
| content_hash | TEXT | SHA-256 hex digest |
| size | INTEGER | File size in bytes |
| line_count | INTEGER | Number of lines |
| language | TEXT | `ts`, `tsx`, `js`, `jsx`, `css`, `md`, `mdx`, `mdc`, `json`, `yaml`, `sh`, `txt` |
Expand Down
31 changes: 31 additions & 0 deletions .changeset/audit-findings.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
---
"@stainless-code/codemap": patch
---

Fix three HIGH-severity bugs found via cross-audit triangulation, plus performance and docs improvements.

**Bug fixes**

- Add missing `onerror` handler on Bun Worker — prevents silent promise hang when a parse worker crashes
- Require JSX return or hook usage for component detection — eliminates false positives (e.g. `FormatCurrency()` in `.tsx` files no longer indexed as a component)
- Include previously-indexed files in incremental and `--files` modes — custom-extension files indexed during `--full` no longer silently go stale

**Performance**

- Batch CSS imports instead of inserting one-at-a-time (both full-rebuild and incremental paths)
- Add `Map<string, Statement>` cache for `better-sqlite3` `run()`/`query()` — avoids ~2,000+ redundant `prepare()` calls on large projects
- Hoist `inner.query()` in `wrap()` to prepare once per call instead of per `.get()`/`.all()`
- Skip `PRAGMA optimize` on `closeDb` for read-only query paths

**Docs**

- Fix Wyhash → SHA-256 in architecture.md and SKILL.md (3 locations)
- Correct `symbols.kind` values (`variable` → `const`, `type_alias` → `type`) and `exports.kind` values
- Clarify `Database.query()` caching is Bun-only; Node re-prepares via wrapper

**Cleanup**

- Remove unused `analyzeDependencies: true` from CSS parser
- Deduplicate `fetchTableStats` (was duplicated across `index-engine.ts` and `run-index.ts`)
- Remove dead `eslint-disable-next-line` directives (oxlint doesn't enforce those rules)
- Fix `SCHEMA_VERSION` comment (said "2", value is `1`)
10 changes: 5 additions & 5 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -171,14 +171,14 @@ All tables use `STRICT` mode. Tables marked with `WITHOUT ROWID` store data dire
| last_modified | INTEGER | File mtime (epoch ms) |
| indexed_at | INTEGER | When this row was written |

### `symbols` — Functions, variables, classes, interfaces, type aliases, enums (`STRICT`)
### `symbols` — Functions, constants, classes, interfaces, types, enums (`STRICT`)

| Column | Type | Description |
| ----------------- | ---------- | --------------------------------------------------------------------- |
| id | INTEGER PK | Auto-increment row id |
| file_path | TEXT FK | References `files(path)` ON DELETE CASCADE |
| name | TEXT | Symbol name |
| kind | TEXT | `function`, `variable`, `class`, `interface`, `type_alias`, `enum` |
| kind | TEXT | `function`, `const`, `class`, `interface`, `type`, `enum` |
| line_start | INTEGER | Start line (1-based) |
| line_end | INTEGER | End line |
| signature | TEXT | Reconstructed signature (e.g. `usePermissions(): PermissionsContext`) |
Expand All @@ -204,7 +204,7 @@ All tables use `STRICT` mode. Tables marked with `WITHOUT ROWID` store data dire
| id | INTEGER PK | Auto-increment row id |
| file_path | TEXT FK | File containing the export |
| name | TEXT | Exported name |
| kind | TEXT | `function`, `variable`, etc. |
| kind | TEXT | `value`, `type`, `re-export` |
| is_default | INTEGER | 1 if default export |
| re_export_source | TEXT | Source module if re-exported |

Expand Down Expand Up @@ -327,7 +327,7 @@ The indexer uses git to detect changes since the last indexed commit:

1. **Stores `last_indexed_commit`** (HEAD SHA) in the `meta` table after each run
2. On next run, computes `git diff --name-only <last_commit>..HEAD` + `git status --porcelain`
3. Only re-indexes changed files (Wyhash content comparison), using DB-sourced `indexedPaths` for import resolution (skips full `collectFiles()` glob scan)
3. Only re-indexes changed files (SHA-256 content comparison), using DB-sourced `indexedPaths` for import resolution (skips full `collectFiles()` glob scan)
4. Deleted files are removed via `ON DELETE CASCADE` — deleting from `files` cascades to all related tables
5. Falls back to full rebuild if commit history is incompatible (e.g. force push, branch switch)

Expand Down Expand Up @@ -436,7 +436,7 @@ Until the first release, Codemap keeps **`SCHEMA_VERSION` at 1**; pull `--full`

### `bun:sqlite` API

All DDL and PRAGMA statements use `Database.run()` (not the deprecated `Database.exec()` alias). Parameterized insert/update statements use `Database.query()` (which caches compiled statements) instead of `Database.prepare()` (which does not cache). Read queries also use `Database.query().all()` or `.get()`. Bulk inserts use the generic `batchInsert<T>()` helper with multi-row `INSERT ... VALUES (...),(...),(...)` in batches of 100, pre-computed placeholders, and zero-copy index-bounds iteration.
All DDL and PRAGMA statements use `Database.run()`. The `sqlite-db.ts` wrapper abstracts both Bun (`bun:sqlite`) and Node (`better-sqlite3`). On Bun, `Database.query()` caches compiled statements; on Node, `better-sqlite3` re-prepares on each call via the wrapper's `run()` method. Read queries use the wrapper's `.query().all()` or `.get()`. Bulk inserts use the generic `batchInsert<T>()` helper with multi-row `INSERT ... VALUES (...),(...),(...)` in batches of 100, pre-computed placeholders, and zero-copy index-bounds iteration.

### PRAGMAs (set on every `openDb()`)

Expand Down
45 changes: 23 additions & 22 deletions src/application/index-engine.ts
Original file line number Diff line number Diff line change
Expand Up @@ -126,10 +126,11 @@ export function getChangedFiles(db: CodemapDatabase): {
.filter(Boolean)
.map((line: string) => line.slice(3).trim());

const indexedPaths = new Set(getAllFileHashes(db).keys());
const allChanged = [...new Set([...diffFiles, ...statusFiles])].filter(
(f) => {
const ext = extname(f);
return ext in LANG_MAP;
return ext in LANG_MAP || indexedPaths.has(f);
},
Comment thread
SutuSebastian marked this conversation as resolved.
);

Expand Down Expand Up @@ -186,19 +187,18 @@ function insertParsedResults(
}
if (parsed.markers?.length) insertMarkers(db, parsed.markers);

if (parsed.cssImportSources) {
for (const importSource of parsed.cssImportSources) {
insertImports(db, [
{
file_path: parsed.relPath,
source: importSource,
resolved_path: null,
specifiers: "[]",
is_type_only: 0,
line_number: 0,
},
]);
}
if (parsed.cssImportSources?.length) {
insertImports(
db,
parsed.cssImportSources.map((importSource) => ({
file_path: parsed.relPath,
source: importSource,
resolved_path: null,
specifiers: "[]",
is_type_only: 0,
line_number: 0,
})),
);
}
} else {
if (parsed.symbols?.length) insertSymbols(db, parsed.symbols);
Expand Down Expand Up @@ -230,7 +230,7 @@ function insertParsedResults(
return indexed;
}

function fetchTableStats(db: CodemapDatabase): IndexTableStats {
export function fetchTableStats(db: CodemapDatabase): IndexTableStats {
const row = db
.query<Record<string, number>>(
`SELECT
Expand Down Expand Up @@ -333,17 +333,18 @@ export async function indexFiles(
insertCssKeyframes(db, cssData.keyframes);
}
if (cssData.markers.length) insertMarkers(db, cssData.markers);
for (const importSource of cssData.importSources) {
insertImports(db, [
{
if (cssData.importSources.length) {
insertImports(
db,
cssData.importSources.map((importSource) => ({
file_path: relPath,
source: importSource,
resolved_path: null,
specifiers: "[]",
is_type_only: 0,
line_number: 0,
},
]);
})),
);
}
} else {
const data = extractFileData(absPath, source, relPath);
Expand Down Expand Up @@ -475,7 +476,7 @@ export function printQueryResult(
}
return 1;
} finally {
if (db !== undefined) closeDb(db);
if (db !== undefined) closeDb(db, { readonly: true });
}
}

Expand All @@ -488,6 +489,6 @@ export function queryRows(sql: string): unknown[] {
try {
return db.query(sql).all();
} finally {
closeDb(db);
closeDb(db, { readonly: true });
}
}
32 changes: 4 additions & 28 deletions src/application/run-index.ts
Original file line number Diff line number Diff line change
@@ -1,15 +1,13 @@
import { extname } from "node:path";

import { createSchema, getAllFileHashes, setMeta } from "../db";
import type { CodemapDatabase } from "../db";
import {
collectFiles,
deleteFilesFromIndex,
fetchTableStats,
getChangedFiles,
getCurrentCommit,
indexFiles,
targetedReindex,
VALID_EXTENSIONS,
} from "./index-engine";
import type { IndexResult, IndexTableStats } from "./types";

Expand Down Expand Up @@ -82,10 +80,7 @@ export async function runCodemapIndex(
}

if (mode === "files") {
const targetFiles = (options.files ?? []).filter((f) => {
const ext = extname(f);
return VALID_EXTENSIONS.has(ext);
});
const targetFiles = options.files ?? [];
if (targetFiles.length === 0) {
return {
mode: "files",
Expand Down Expand Up @@ -139,7 +134,7 @@ export async function runCodemapIndex(
indexed: 0,
skipped: 0,
elapsedMs: 0,
stats: fetchStats(db),
stats: fetchTableStats(db),
idle: true,
};
}
Expand All @@ -149,7 +144,7 @@ export async function runCodemapIndex(
indexed: 0,
skipped: 0,
elapsedMs: 0,
stats: fetchStats(db),
stats: fetchTableStats(db),
idle: true,
};
}
Expand All @@ -169,22 +164,3 @@ export async function runCodemapIndex(
stats: run.stats,
};
}

function fetchStats(db: CodemapDatabase): IndexTableStats {
const row = db
.query<Record<string, number>>(
`SELECT
(SELECT COUNT(*) FROM files) as files,
(SELECT COUNT(*) FROM symbols) as symbols,
(SELECT COUNT(*) FROM imports) as imports,
(SELECT COUNT(*) FROM exports) as exports,
(SELECT COUNT(*) FROM components) as components,
(SELECT COUNT(*) FROM dependencies) as dependencies,
(SELECT COUNT(*) FROM markers) as markers,
(SELECT COUNT(*) FROM css_variables) as css_vars,
(SELECT COUNT(*) FROM css_classes) as css_classes,
(SELECT COUNT(*) FROM css_keyframes) as css_keyframes`,
)
.get()!;
return row as IndexTableStats;
}
12 changes: 5 additions & 7 deletions src/cli/cmd-index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,12 @@ export async function runIndexCmd(opts: {
const db = openDb();
try {
if (args[0] === "--files" && args.length > 1) {
const targetFiles = args.slice(1).filter((f) => {
const ext = extname(f);
if (!VALID_EXTENSIONS.has(ext)) {
console.warn(` Skipping ${f}: unsupported extension "${ext}"`);
return false;
const targetFiles = args.slice(1);
for (const f of targetFiles) {
if (!VALID_EXTENSIONS.has(extname(f))) {
console.warn(` ${f}: non-standard extension, indexing as text`);
}
return true;
});
}
if (targetFiles.length > 0) {
await runCodemapIndex(db, {
mode: "files",
Expand Down
1 change: 0 additions & 1 deletion src/css-parser.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,6 @@ export function extractCssData(
filename: filePath,
code: Buffer.from(source),
errorRecovery: true,
analyzeDependencies: true,
customAtRules: {
// Tailwind v4 @theme blocks
theme: { body: "declaration-list" },
Expand Down
10 changes: 6 additions & 4 deletions src/db.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import {
} from "./sqlite-db";

/**
* Pre-release: keep at **2** until the first npm release — do not bump for DDL
* Pre-release: keep at **1** until the first npm release — do not bump for DDL
* tweaks; run `--full` locally after pulling. After v1.0, bump in lockstep with
* `createTables` / `createIndexes` when the on-disk schema changes.
*/
Expand All @@ -17,9 +17,11 @@ export function openDb(): CodemapDatabase {
return openCodemapDatabase();
}

export function closeDb(db: CodemapDatabase) {
db.run("PRAGMA analysis_limit = 400");
db.run("PRAGMA optimize");
export function closeDb(db: CodemapDatabase, opts?: { readonly?: boolean }) {
if (!opts?.readonly) {
db.run("PRAGMA analysis_limit = 400");
db.run("PRAGMA optimize");
}
db.close();
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated
}

Expand Down
1 change: 0 additions & 1 deletion src/glob-sync.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ import { globSync as tinyglobbySync } from "tinyglobby";
*/
export function globSync(pattern: string, cwd: string): string[] {
if (typeof Bun !== "undefined") {
// eslint-disable-next-line @typescript-eslint/no-require-imports
const { Glob } = require("bun") as typeof import("bun");
const glob = new Glob(pattern);
return Array.from(glob.scanSync({ cwd, dot: true }));
Expand Down
39 changes: 38 additions & 1 deletion src/parser.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,46 @@ describe("extractFileData", () => {
});

it("treats .tsx as JSX for PascalCase function components", () => {
const src = `export function Button() { return null; }\n`;
const src = `export function Button() { return <button>click</button>; }\n`;
const d = extractFileData("/proj/x.tsx", src, "x.tsx");
expect(d.components.some((c) => c.name === "Button")).toBe(true);
});
});

describe("component detection heuristic", () => {
it("detects components that return JSX", () => {
const src = `export function Card() { return <div>card</div>; }\n`;
const d = extractFileData("/proj/x.tsx", src, "x.tsx");
expect(d.components.some((c) => c.name === "Card")).toBe(true);
});

it("detects arrow components that return JSX", () => {
const src = `export const Card = () => <div>card</div>;\n`;
const d = extractFileData("/proj/x.tsx", src, "x.tsx");
expect(d.components.some((c) => c.name === "Card")).toBe(true);
});

it("detects components that use hooks", () => {
const src = `export function Timer() { useEffect(() => {}); return null; }\n`;
const d = extractFileData("/proj/x.tsx", src, "x.tsx");
expect(d.components.some((c) => c.name === "Timer")).toBe(true);
});

it("rejects PascalCase functions without JSX or hooks", () => {
const src = [
`export function FormatCurrency(n: number): string { return "$"+n; }`,
`export function ValidateEmail(e: string): boolean { return e.includes("@"); }`,
].join("\n");
const d = extractFileData("/proj/x.tsx", src, "x.tsx");
expect(d.components).toHaveLength(0);
});

it("rejects PascalCase functions that return null without hooks", () => {
const src = `export function EmptyPlaceholder() { return null; }\n`;
const d = extractFileData("/proj/x.tsx", src, "x.tsx");
expect(d.components.some((c) => c.name === "EmptyPlaceholder")).toBe(
false,
);
});
});
});
10 changes: 10 additions & 0 deletions src/parser.ts
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,7 @@ export function extractFileData(
}

const hookCalls = new Map<string, Set<string>>(); // function scope name -> hook names
const jsxScopes = new Set<string>(); // function scopes that contain JSX
let currentFunctionScope: string | null = null;

const visitor = new Visitor({
Expand Down Expand Up @@ -255,6 +256,13 @@ export function extractFileData(
hookCalls.get(currentFunctionScope)?.add(callee.name);
}
},

JSXElement() {
if (currentFunctionScope) jsxScopes.add(currentFunctionScope);
},
JSXFragment() {
if (currentFunctionScope) jsxScopes.add(currentFunctionScope);
},
});

visitor.visit(result.program);
Expand All @@ -264,6 +272,8 @@ export function extractFileData(
function maybeAddComponent(name: string, node: any, _isArrow: boolean) {
if (!isTsx || !/^[A-Z]/.test(name)) return;
const hooks = hookCalls.get(name);
const hasJsx = jsxScopes.has(name);
if (!hasJsx && !(hooks && hooks.size > 0)) return;
const isDefault = defaultExportedNames.has(name);

let propsType: string | null = null;
Expand Down
Loading
Loading