perf: cache prepared statements for hot-path edge repository functions (#402)

* feat: unified AST analysis framework with pluggable visitor pattern

Implement Phase 3.1 of the architectural refactoring roadmap. Replace 4
independent AST analysis passes (complexity, dataflow, AST-store, CFG)
with a shared visitor framework that runs complexity, dataflow, and
AST-store visitors in a single DFS walk per file.

New framework files:
- visitor.js: shared DFS walker with enterNode/exitNode/enterFunction/
  exitFunction hooks, per-visitor skipChildren, nesting/scope tracking
- engine.js: orchestrates all analyses in one coordinated pass, replaces
  4 sequential buildXxx blocks + WASM pre-parse in builder.js
- metrics.js: extracted Halstead derived math, LOC, MI from complexity.js
- visitor-utils.js: extracted shared helpers from dataflow.js
- visitors/complexity-visitor.js: cognitive/cyclomatic/nesting + Halstead
- visitors/ast-store-visitor.js: new/throw/await/string/regex extraction
- visitors/dataflow-visitor.js: scope stack + define-use chains
- tests/unit/visitor.test.js: 9 tests for walker framework

Modified modules:
- builder.js: 4 sequential buildXxx blocks → single runAnalyses call
- complexity.js: delegates walk to complexity visitor
- dataflow.js: delegates walk to dataflow visitor
- ast.js: delegates walk to ast-store visitor, removed dead code

Bug fixes:
- Fix Ruby dataflow parity: skip nested return keyword tokens inside
  return statements (tree-sitter Ruby grammar nests return/return)

Hook fixes:
- check-dead-exports.sh: exclude public API (index.js) and dynamic
  import() consumers from dead export detection
- check-readme.sh: match ROADMAP.md at any path; allow commit when at
  least one doc is staged (developer reviewed which docs need updating)

Docs:
- ROADMAP.md: update 3.1 with completed items and remaining work
- BACKLOG.md: add Tier 1h with dynamic import/re-export tracking (ID 71)
- CLAUDE.md: add ast-analysis/ to architecture table

Impact: 51 functions changed, 47 affected

* revert: restore original check-dead-exports.sh hook

The hook correctly catches a real limitation — codegraph doesn't track
dynamic import() consumers as graph edges. The proper fix is to add
dynamic import edge tracking (backlog ID 81), not to exclude them
from the hook. Reverting the workaround.

* fix: address Greptile review — indexOf→m.index, precise publicAPI regex, remove unused var

Impact: 2 functions changed, 23 affected

* fix: address Greptile review — halsteadSkip depth counter, debug logging, perf import

- Replace halsteadSkip boolean with halsteadSkipDepth counter to handle
  nested skip-type nodes correctly (e.g. return_type containing
  type_annotation in TypeScript)
- Add debug() logging to all silent catch blocks in engine.js so
  failures are discoverable with --verbose
- Explicitly import performance from node:perf_hooks for clarity

Impact: 5 functions changed, 24 affected

* fix: remove function nodes from nestingNodeTypes and eliminate O(n²) lookup

Address two Greptile review comments on PR #388:

1. Function nodes in nestingNodeTypes inflates cognitive complexity by +1
   for every function body — funcDepth already tracks function-level
   nesting, so adding functionNodes to the walker's nestingNodeTypes
   double-counts.

2. Replace O(n²) complexityResults.find() with O(1) resultByLine map
   lookup by storing the full result object (metrics + funcNode) instead
   of just metrics.

Impact: 1 functions changed, 0 affected

* fix: remove function nesting inflation in computeAllMetrics and pass langId to LOC

Address two Greptile review comments:

1. computeAllMetrics added functionNodes to nestingNodeTypes, inflating
   cognitive complexity by +1 for every function body (same bug as the
   engine.js fix in a47eb47, but in the per-function code path).

2. collectResult in complexity-visitor passed null as langId to
   computeLOCMetrics, causing C-style comment detection for all
   languages. Now accepts langId via options and forwards it, so
   Python/Ruby/PHP get correct comment-line counts and MI values.

Impact: 4 functions changed, 5 affected

* fix: guard runAnalyses call, fix nested function nesting, rename _engineOpts

Address three Greptile review comments:

1. Wrap runAnalyses call in builder.js with try/catch + debug() so a
   walk-phase throw doesn't crash the entire build — matches the
   resilience of the previous individual buildXxx try/catch blocks.

2. In function-level mode, enterFunction/exitFunction were no-ops, so
   funcDepth stayed 0 for nested functions (callbacks, closures).
   Now increments/decrements funcDepth in function-level mode too,
   restoring parity with the original computeAllMetrics algorithm.

3. Rename _engineOpts to engineOpts in engine.js since it's actively
   forwarded to all four buildXxx delegate calls — the underscore
   prefix falsely signaled an unused parameter.

Impact: 5 functions changed, 4 affected

* fix: remove redundant processed Set and fix multi-line destructuring in hook

Address two Greptile review comments:

1. Remove the `processed` Set from dataflow-visitor.js — walkWithVisitors
   visits each node exactly once via DFS on a proper tree, so the
   deduplication guard never fires and just wastes memory.

2. Replace single-line destructuring regex in check-dead-exports.sh with
   a multi-line-safe pattern (using the `s` flag) so multi-line
   `const { ... } = await import(...)` patterns are correctly detected
   and their names added to publicAPI.

Impact: 2 functions changed, 0 affected

* refactor: migrate raw SQL into repository pattern (Phase 3.3)

Split src/db/repository.js into src/db/repository/ directory with 10
domain files: nodes, edges, build-stmts, complexity, cfg, dataflow,
cochange, embeddings, graph-read, and barrel index.

Extracted ~120 inline db.prepare() calls from 14 consumer modules
into shared repository functions, eliminating duplication of common
patterns like getNodeId (5 files), bulkNodeIdsByFile (3 files),
callee/caller joins (6+ call sites), and file purge cascades.

Net reduction: -437 lines across the codebase. All 1573 tests pass.

Impact: 73 functions changed, 112 affected

* fix: address Greptile review — deduplicate relatedTests, hoist prepared stmts, fix .raw() no-op

- queries.js: restore DISTINCT-by-file deduplication for relatedTests in
  explainFunctionImpl (lost when switching to findCallers)
- build-stmts.js: prepare SQL statements once in preparePurgeStmts() and
  loop with runPurge() instead of calling db.prepare() per file per table
- cfg.js: replace misleading .raw() with .get() in hasCfgTables

Impact: 7 functions changed, 14 affected

* perf: cache prepared statements in hot-path repository functions

bulkNodeIdsByFile, getNodeId, and getFunctionNodeId each called
db.prepare() on every invocation. These are called inside per-file
loops in engine.js, ast.js, builder.js, complexity.js, and cfg.js,
producing O(N) prepare() calls instead of O(1).

Add WeakMap caches keyed on the db instance so each statement is
prepared once per connection and reused on subsequent calls. For a
300-file project this eliminates ~900 redundant prepare() calls per
build pass.

Also fix commit-msg hook: IMPACT=$(...||...) under sh -e exits with
code 1 when both diff-impact fallbacks return non-zero; append || true
so the assignment is always tolerated.

docs check acknowledged

* fix: guard pre-push hook against sh -e failure when diff-impact unavailable

Same issue as commit-msg: IMPACT=$(...||...) exits non-zero under sh -e
when both codegraph and node fallbacks fail. Append || true to absorb it.

docs check acknowledged

* fix: add DISTINCT to findCallees and cache prepared statements in cfg repository

* perf: cache prepared statements for findCallees, findCallers, findDistinctCallers

* refactor: extract cachedStmt utility, cache all edge/cfg/node statements

- Extract the WeakMap cache-check pattern into a shared cachedStmt()
  helper in cached-stmt.js, replacing the local _cached() in edges.js
  and the inline cache-check boilerplate in cfg.js and nodes.js.
- Cache all 14 edge functions (previously only 6 were cached), all 4
  cfg functions, and all 3 node lookup functions via cachedStmt().
- No SQL or behavioral changes — purely structural.

Impact: 21 functions changed, 72 affected

* perf: cache all remaining repository prepared statements

Cache db.prepare() calls in nodes.js (7), graph-read.js (4),
complexity.js (1), embeddings.js (3), cochange.js (3), dataflow.js (1)
using the cachedStmt WeakMap utility. Every query-path function in
src/db/repository/ now reuses a single prepared statement per db instance.

Impact: 19 functions changed, 0 affected

Author: carlos-alm

src/db/repository/cached-stmt.js

@@ -0,0 +1,19 @@
+/**
+ * Resolve a cached prepared statement, compiling on first use per db.
+ * Each `cache` WeakMap must always be called with the same `sql` —
+ * the sql argument is only used on the first compile; subsequent calls
+ * return the cached statement regardless of the sql passed.
+ *
+ * @param {WeakMap} cache  - WeakMap keyed by db instance
+ * @param {object}  db     - better-sqlite3 database instance
+ * @param {string}  sql    - SQL to compile on first use
+ * @returns {object} prepared statement
+ */
+export function cachedStmt(cache, db, sql) {
+  let stmt = cache.get(db);
+  if (!stmt) {
+    stmt = db.prepare(sql);
+    cache.set(db, stmt);
+  }
+  return stmt;
+}

src/db/repository/cfg.js

@@ -1,5 +1,6 @@
+import { cachedStmt } from './cached-stmt.js';
+
 // ─── Statement caches (one prepared statement per db instance) ────────────
-// WeakMap keys on the db object so statements are GC'd when the db closes.
 const _getCfgBlocksStmt = new WeakMap();
 const _getCfgEdgesStmt = new WeakMap();
 const _deleteCfgEdgesStmt = new WeakMap();
@@ -26,16 +27,13 @@ export function hasCfgTables(db) {
  * @returns {object[]}
  */
 export function getCfgBlocks(db, functionNodeId) {
-  let stmt = _getCfgBlocksStmt.get(db);
-  if (!stmt) {
-    stmt = db.prepare(
-      `SELECT id, block_index, block_type, start_line, end_line, label
-       FROM cfg_blocks WHERE function_node_id = ?
-       ORDER BY block_index`,
-    );
-    _getCfgBlocksStmt.set(db, stmt);
-  }
-  return stmt.all(functionNodeId);
+  return cachedStmt(
+    _getCfgBlocksStmt,
+    db,
+    `SELECT id, block_index, block_type, start_line, end_line, label
+     FROM cfg_blocks WHERE function_node_id = ?
+     ORDER BY block_index`,
+  ).all(functionNodeId);
 }
 
 /**
@@ -45,21 +43,18 @@ export function getCfgBlocks(db, functionNodeId) {
  * @returns {object[]}
  */
 export function getCfgEdges(db, functionNodeId) {
-  let stmt = _getCfgEdgesStmt.get(db);
-  if (!stmt) {
-    stmt = db.prepare(
-      `SELECT e.kind,
-              sb.block_index AS source_index, sb.block_type AS source_type,
-              tb.block_index AS target_index, tb.block_type AS target_type
-       FROM cfg_edges e
-       JOIN cfg_blocks sb ON e.source_block_id = sb.id
-       JOIN cfg_blocks tb ON e.target_block_id = tb.id
-       WHERE e.function_node_id = ?
-       ORDER BY sb.block_index, tb.block_index`,
-    );
-    _getCfgEdgesStmt.set(db, stmt);
-  }
-  return stmt.all(functionNodeId);
+  return cachedStmt(
+    _getCfgEdgesStmt,
+    db,
+    `SELECT e.kind,
+            sb.block_index AS source_index, sb.block_type AS source_type,
+            tb.block_index AS target_index, tb.block_type AS target_type
+     FROM cfg_edges e
+     JOIN cfg_blocks sb ON e.source_block_id = sb.id
+     JOIN cfg_blocks tb ON e.target_block_id = tb.id
+     WHERE e.function_node_id = ?
+     ORDER BY sb.block_index, tb.block_index`,
+  ).all(functionNodeId);
 }
 
 /**
@@ -68,16 +63,10 @@ export function getCfgEdges(db, functionNodeId) {
  * @param {number} functionNodeId
  */
 export function deleteCfgForNode(db, functionNodeId) {
-  let delEdges = _deleteCfgEdgesStmt.get(db);
-  if (!delEdges) {
-    delEdges = db.prepare('DELETE FROM cfg_edges WHERE function_node_id = ?');
-    _deleteCfgEdgesStmt.set(db, delEdges);
-  }
-  let delBlocks = _deleteCfgBlocksStmt.get(db);
-  if (!delBlocks) {
-    delBlocks = db.prepare('DELETE FROM cfg_blocks WHERE function_node_id = ?');
-    _deleteCfgBlocksStmt.set(db, delBlocks);
-  }
-  delEdges.run(functionNodeId);
-  delBlocks.run(functionNodeId);
+  cachedStmt(_deleteCfgEdgesStmt, db, 'DELETE FROM cfg_edges WHERE function_node_id = ?').run(
+    functionNodeId,
+  );
+  cachedStmt(_deleteCfgBlocksStmt, db, 'DELETE FROM cfg_blocks WHERE function_node_id = ?').run(
+    functionNodeId,
+  );
 }

src/db/repository/cochange.js

@@ -1,11 +1,18 @@
+import { cachedStmt } from './cached-stmt.js';
+
+// ─── Statement caches (one prepared statement per db instance) ────────────
+const _hasCoChangesStmt = new WeakMap();
+const _getCoChangeMetaStmt = new WeakMap();
+const _upsertCoChangeMetaStmt = new WeakMap();
+
 /**
  * Check whether the co_changes table has data.
  * @param {object} db
  * @returns {boolean}
  */
 export function hasCoChanges(db) {
   try {
-    return !!db.prepare('SELECT 1 FROM co_changes LIMIT 1').get();
+    return !!cachedStmt(_hasCoChangesStmt, db, 'SELECT 1 FROM co_changes LIMIT 1').get();
   } catch {
     return false;
   }
@@ -19,7 +26,11 @@ export function hasCoChanges(db) {
 export function getCoChangeMeta(db) {
   const meta = {};
   try {
-    for (const row of db.prepare('SELECT key, value FROM co_change_meta').all()) {
+    for (const row of cachedStmt(
+      _getCoChangeMetaStmt,
+      db,
+      'SELECT key, value FROM co_change_meta',
+    ).all()) {
       meta[row.key] = row.value;
     }
   } catch {
@@ -35,7 +46,9 @@ export function getCoChangeMeta(db) {
  * @param {string} value
  */
 export function upsertCoChangeMeta(db, key, value) {
-  db.prepare(
+  cachedStmt(
+    _upsertCoChangeMetaStmt,
+    db,
     'INSERT INTO co_change_meta (key, value) VALUES (?, ?) ON CONFLICT(key) DO UPDATE SET value = excluded.value',
   ).run(key, value);
 }

src/db/repository/complexity.js

@@ -1,3 +1,8 @@
+import { cachedStmt } from './cached-stmt.js';
+
+// ─── Statement caches (one prepared statement per db instance) ────────────
+const _getComplexityForNodeStmt = new WeakMap();
+
 /**
  * Get complexity metrics for a node.
  * Used by contextData and explainFunctionImpl in queries.js.
@@ -6,10 +11,10 @@
  * @returns {{ cognitive: number, cyclomatic: number, max_nesting: number, maintainability_index: number, halstead_volume: number }|undefined}
  */
 export function getComplexityForNode(db, nodeId) {
-  return db
-    .prepare(
-      `SELECT cognitive, cyclomatic, max_nesting, maintainability_index, halstead_volume
-       FROM function_complexity WHERE node_id = ?`,
-    )
-    .get(nodeId);
+  return cachedStmt(
+    _getComplexityForNodeStmt,
+    db,
+    `SELECT cognitive, cyclomatic, max_nesting, maintainability_index, halstead_volume
+     FROM function_complexity WHERE node_id = ?`,
+  ).get(nodeId);
 }

src/db/repository/dataflow.js

@@ -1,11 +1,16 @@
+import { cachedStmt } from './cached-stmt.js';
+
+// ─── Statement caches (one prepared statement per db instance) ────────────
+const _hasDataflowTableStmt = new WeakMap();
+
 /**
  * Check whether the dataflow table exists and has data.
  * @param {object} db
  * @returns {boolean}
  */
 export function hasDataflowTable(db) {
   try {
-    return db.prepare('SELECT COUNT(*) AS c FROM dataflow').get().c > 0;
+    return cachedStmt(_hasDataflowTableStmt, db, 'SELECT COUNT(*) AS c FROM dataflow').get().c > 0;
   } catch {
     return false;
   }