feat(watch): codemap watch — live reindex + mcp/serve --watch killer combo (impl of PR #46 plan) (#47)

* feat(watch): application/watcher.ts skeleton — pure debouncer + path filter + chokidar backend (Tracer 1 of 5)

Per docs/plans/watch-mode.md the engine is split into pure helpers + an injectable backend so tests don't need real fs watches:

- shouldIndexPath(relPath, excludeDirNames) — pure predicate. Same indexed extensions as the indexer (.ts/.tsx/.mts/.cts/.js/.jsx/.mjs/.cjs/.css) plus project-local recipes (<root>/.codemap/recipes/<id>.{sql,md}). Hand-rolled path-segment scan (no .split('/') alloc per call — watcher fires on every unrelated edit).
- createDebouncer(onFlush, delayMs) — sliding-window timer; trigger() resets, flushNow() forces, reset() clears without firing. Coalesces a burst (git checkout, npm install) into one onChange call.
- WatchBackend interface — production wires chokidar v5 (atomic + awaitWriteFinish for chunked-write detection); tests inject a fake backend that drives events deterministically (no flake in CI containers).
- runWatchLoop({root, excludeDirNames, onChange, debounceMs?, backend?}) — wires the three together. Returns {stop} that drains the debouncer + closes the backend (so SIGINT loses no events).
- DEFAULT_DEBOUNCE_MS = 250 (long enough to coalesce one editor save burst, short enough that agents don't perceive lag).

18 unit tests cover: extension whitelist, exclude-dir-names exact match (not substring — 'distill' is fine even when 'dist' is excluded), recipe paths, .codemap/codemap.db rejected, debouncer sliding window + flushNow + reset, backend dispatch, abs→rel POSIX conversion, dedup within burst, stop flushes pending.

chokidar v5 added (1 dep, 82 KB) per docs/plans/watch-mode.md decision. Tracer 2 wires cmd-watch.ts; Tracer 3 wires --watch into serve / mcp.

* feat(watch): cmd-watch.ts CLI verb + main/bootstrap wiring (Tracer 2 of 5)

Adds the standalone 'codemap watch' command. Wires the engine from Tracer 1:
- parseWatchRest() with --debounce <ms> + --quiet flag (12 unit tests cover space/equals forms, validation, defaults, composition, error paths).
- printWatchCmdHelp() — explains the --debounce trade-off (lower = snappier, higher = fewer cycles during git checkout / npm install) and points at 'codemap serve --watch' / 'codemap mcp --watch' as the killer combo (Tracer 3).
- runWatchCmd() — bootstraps codemap (initCodemap + configureResolver), starts runWatchLoop with onChange = runCodemapIndex({mode: 'files', files: [...paths]}), awaits SIGINT/SIGTERM, drains pending edits before close.
- main.ts: dispatch on rest[0] === 'watch'; bootstrap.ts: validateIndexModeArgs accepts 'watch'; printCliUsage lists 'Watch mode' between HTTP server and Targeted reads.

Per-batch stderr line: 'codemap watch: reindex N file(s) in Mms' unless --quiet.

Smoke verified: 'bun src/index.ts watch' boots, logs the bind line, drains cleanly on SIGTERM. Tracer 3 wires --watch into serve / mcp.

* feat(watch): --watch flag on serve + mcp + shared createReindexOnChange helper (Tracer 3 of 5)

Killer combo: codemap mcp --watch / codemap serve --watch boots the transport AND a co-process file watcher in one process. Removes the 'is the index stale?' friction agents hit today (per docs/plans/watch-mode.md § Agent-experience win).

Factored helper to keep cmd-watch / mcp-server / http-server identical:
- application/watcher.ts: createReindexOnChange({quiet, label?}) — opens DB, runs targeted reindex on the changed paths, logs 'reindex N file(s) in Mms' to stderr unless quiet, catches errors so a transient parse failure doesn't kill the loop. Caller passes a label so 'codemap mcp' / 'codemap serve' / 'codemap watch' lines are distinguishable in interleaved logs.
- cmd-watch.ts now uses createReindexOnChange (DRY with the embedders).

CLI surface:
- cmd-mcp.ts: new --watch flag + --debounce <ms> override (default 250). Help text + parser tests + propagation through runMcpCmd → runMcpServer.
- cmd-serve.ts: same flags. Parser tests + 'serve: ... (watch: on)' marker on the bind line.
- CODEMAP_WATCH=1 / 'true' env shortcut for IDE / CI launches that can't easily edit the agent host's tool spawn command (per the plan's sketched API).

Embedder lifecycle:
- runMcpServer: starts watcher AFTER server.connect(transport); on shutdown awaits stopWatch() (drains pending reindex) before resolving.
- runHttpServer: starts watcher AFTER listen succeeds; on SIGINT/SIGTERM awaits stopWatch() then closes the listener.

146 tests pass (cmd-watch + cmd-mcp + cmd-serve + watcher + mcp-server + http-server). No new code paths in the existing engines — just the boot-time wiring.

* feat(watch): handleAudit skips incremental-index prelude when watcher is active (Tracer 4 of 5)

Closes the wasted-I/O loop the plan called out: today MCP audit's default behavior is to run an incremental-index prelude (so 'head' reflects the on-disk source) — but with mcp --watch / serve --watch the watcher already keeps the index fresh, so the prelude is pure overhead.

- application/watcher.ts: module-level watchActive flag toggled by runWatchLoop start/stop. isWatchActive() exposed for handleAudit; _resetWatchStateForTests + _markWatchActiveForTests for test seam.
- application/tool-handlers.ts handleAudit: shouldRunPrelude = !args.no_index && !(isWatchActive() && args.no_index !== false). Hoisted to function scope so the inner finally can also pass it as the readonly hint to closeDb (avoids a wasted checkpoint pass).
- Explicit no_index: false still forces the prelude even when watch is on (escape hatch for 'force re-index right now').
- 1 new watcher test + 1 new MCP-server integration test (audit succeeds with no_index unset when watcher is marked active — would have failed if the prelude tried to run on the test's freshly-created DB without git history).

62 watcher + MCP tests pass.

* docs: sync README + architecture + glossary + roadmap + agents (Rule 10) + delete plan + changeset (Tracer 5 of 5)

- README.md 'Daily commands' stripe: extended with codemap mcp --watch / serve --watch / watch standalone / CODEMAP_WATCH=1 examples.
- docs/architecture.md: new 'Watch wiring' paragraph after MCP / HTTP wiring; covers chokidar selection, debounce + filter, audit prelude optimization. application/ table extended with watcher.ts.
- docs/glossary.md: new 'codemap watch / watch mode' entry under ## C (alphabetically before 'codemap mcp' / 'codemap serve' since 'watch' < 'serve' but the entry naming convention puts it after the existing CLI verbs).
- docs/roadmap.md: 'Watch mode for dev' line removed (shipped per Rule 2).
- .agents/rules/codemap.md + templates/agents/rules/codemap.md (Rule 10): new 'Watch mode (live reindex)' table row + --watch / --debounce flags appended to the mcp + serve rows.
- .agents/skills/codemap/SKILL.md + templates/agents/skills/codemap/SKILL.md: --watch / --debounce + CODEMAP_WATCH semantics on the MCP + HTTP server bullets; new 'Watch mode' bullet covering standalone vs combined shape choice and the audit prelude optimization.
- .changeset/codemap-watch.md: minor changeset (new top-level CLI verb + new --watch flag on mcp + serve).
- docs/plans/watch-mode.md: deleted on ship per docs-governance Rule 3.
- src/{application/{watcher,mcp-server,http-server}.ts, cli/cmd-{mcp,watch}.ts}: replaced dangling cross-refs to the deleted plan with cross-refs to architecture.md § Watch wiring.

* fix(watch): drain in-flight + prime gating + onError clears flag + 6 robustness fixes (CodeRabbit on #47)

9 of 10 CodeRabbit threads, all verified ✅ correct. The 10th (#4 — anchor) is ⚠️ partial: their suggested anchor (#watch-wiring) doesn't exist; #cli-usage is correct (precedent for every wiring paragraph). Pushing back with evidence in the reply.

**Major correctness fixes:**

- (#5, heavy) handleAudit treated 'watch active' as 'index definitely fresh' — but the watcher only sees NEW events, not historical drift. On boot before catch-up, audit could read a months-stale index. New onPrime opt on runWatchLoop runs an incremental catch-up BEFORE flipping watchActive=true. Embedders pass createPrimeIndex({label}) — same pattern as createReindexOnChange. Without onPrime, flag flips immediately (test-friendly default).
- (#7, heavy) stop() didn't drain async reindex work — fire-and-forget meant a stop() could resolve while onChange was mid-DB-write. Now: serialize onChange via inFlight chain, await it on stop. Also await primingDone so we don't tear down a DB connection out from under the prime catch-up.
- (#8) Backend onError left watchActive=true → handleAudit kept skipping prelude even when chokidar died. Now clears the flag.
- (#1) http-server: watcher boot throw after listen() leaked the listener. try/catch closes server on failure.
- (#2 + #10) http-server + cmd-watch: stopWatch().then(closeServer) never fired closeServer if stopWatch rejected — process hang on SIGTERM. Now .catch(log).finally(...) so progress is guaranteed.
- (#3) mcp-server.test: _markWatchActiveForTests ran outside the try guard — a thrown makeClient() would leak the singleton flag into sibling tests. Hoisted into try/finally.

**Minor:**

- (#6) shouldIndexPath built recipe prefix with platform sep, but relPath is POSIX-normalized. Windows recipe edits got skipped. Fixed to literal '.codemap/recipes/'.
- (#9) printWatchCmdHelp lacked JSDoc; added.

**Push-back:**

- (#4) CodeRabbit suggested changing #cli-usage anchor to #watch-wiring. Verified: there's no '## Watch wiring' heading — the wiring sections are bold-prefix paragraphs under '## CLI usage'. Their fix would 404. Keeping #cli-usage matches the precedent for MCP / HTTP / SARIF / serve wiring paragraphs.

5 new watcher tests cover the prime-gating race, onError flag clear, in-flight drain on stop (both auto-fire and flushNow paths). 152 tests pass total.

* docs(watch): outside-diff + 2 nitpicks from CodeRabbit follow-up review on #47

All 3 verified ✅ correct (the inline 10 actionable were already addressed in 207c05d):

- (outside-diff, mcp-server.ts:165) audit tool description still claimed prelude always runs first; updated to document watch-mode default ('default true-equivalent without watch, default false-equivalent with --watch active') and the 'pass no_index: false to force a re-index even when watch is active' escape hatch.
- (nitpick, glossary.md:395) widened wording from 'codemap mcp audit' to 'audit tool ... on both transports' since the same skip applies to 'codemap serve --watch' POST /tool/audit.
- (nitpick, cmd-serve.test.ts:14 + cmd-mcp.test.ts:11) replaced hardcoded debounceMs: 250 with imported DEFAULT_DEBOUNCE_MS so future default changes don't silently break tests.

Author: SutuSebastian

.agents/rules/codemap.md

@@ -12,24 +12,25 @@ A local database (default **`.codemap.db`**) indexes structure: symbols, imports
 
 ## CLI (this repository)
 
-| Context                        | Incremental index  | Query                                                                                                                     |
-| ------------------------------ | ------------------ | ------------------------------------------------------------------------------------------------------------------------- |
-| **Default** — from this clone  | `bun src/index.ts` | `bun src/index.ts query --json "<SQL>"`                                                                                   |
-| Same entry                     | `bun run dev`      | (same as first row)                                                                                                       |
-| Query (ASCII table — optional) | —                  | `bun src/index.ts query "<SQL>"`                                                                                          |
-| Recipe                         | —                  | `bun src/index.ts query --json --recipe fan-out` (see **`bun src/index.ts query --help`**)                                |
-| Recipe catalog / SQL           | —                  | `bun src/index.ts query --recipes-json` · `bun src/index.ts query --print-sql fan-out`                                    |
-| Counts only                    | —                  | `bun src/index.ts query --json --summary -r deprecated-symbols`                                                           |
-| PR-scoped rows                 | —                  | `bun src/index.ts query --json --changed-since origin/main -r fan-out`                                                    |
-| Bucket by owner / dir / pkg    | —                  | `bun src/index.ts query --json --group-by directory -r fan-in`                                                            |
-| Save / diff a baseline         | —                  | `bun src/index.ts query --save-baseline -r visibility-tags` then `… --json --baseline -r visibility-tags`                 |
-| List / drop baselines          | —                  | `bun src/index.ts query --baselines` · `bun src/index.ts query --drop-baseline <name>`                                    |
-| Per-delta audit                | —                  | `bun src/index.ts audit --json --baseline base` (auto-resolves `base-files` / `base-dependencies` / `base-deprecated`)    |
-| MCP server (for agent hosts)   | —                  | `bun src/index.ts mcp` — JSON-RPC on stdio; one tool per CLI verb. See **MCP** section below.                             |
-| HTTP server (for non-MCP)      | —                  | `bun src/index.ts serve [--host 127.0.0.1] [--port 7878] [--token <secret>]` — same tool taxonomy over POST /tool/{name}. |
-| Targeted read (metadata)       | —                  | `bun src/index.ts show <name> [--kind <k>] [--in <path>] [--json]` — file:line + signature                                |
-| Targeted read (source text)    | —                  | `bun src/index.ts snippet <name> [--kind <k>] [--in <path>] [--json]` — same lookup + source from disk + stale flag       |
-| SARIF / GH annotations         | —                  | `bun src/index.ts query --recipe deprecated-symbols --format sarif` · `… --format annotations`                            |
+| Context                        | Incremental index  | Query                                                                                                                                                                                                                                    |
+| ------------------------------ | ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| **Default** — from this clone  | `bun src/index.ts` | `bun src/index.ts query --json "<SQL>"`                                                                                                                                                                                                  |
+| Same entry                     | `bun run dev`      | (same as first row)                                                                                                                                                                                                                      |
+| Query (ASCII table — optional) | —                  | `bun src/index.ts query "<SQL>"`                                                                                                                                                                                                         |
+| Recipe                         | —                  | `bun src/index.ts query --json --recipe fan-out` (see **`bun src/index.ts query --help`**)                                                                                                                                               |
+| Recipe catalog / SQL           | —                  | `bun src/index.ts query --recipes-json` · `bun src/index.ts query --print-sql fan-out`                                                                                                                                                   |
+| Counts only                    | —                  | `bun src/index.ts query --json --summary -r deprecated-symbols`                                                                                                                                                                          |
+| PR-scoped rows                 | —                  | `bun src/index.ts query --json --changed-since origin/main -r fan-out`                                                                                                                                                                   |
+| Bucket by owner / dir / pkg    | —                  | `bun src/index.ts query --json --group-by directory -r fan-in`                                                                                                                                                                           |
+| Save / diff a baseline         | —                  | `bun src/index.ts query --save-baseline -r visibility-tags` then `… --json --baseline -r visibility-tags`                                                                                                                                |
+| List / drop baselines          | —                  | `bun src/index.ts query --baselines` · `bun src/index.ts query --drop-baseline <name>`                                                                                                                                                   |
+| Per-delta audit                | —                  | `bun src/index.ts audit --json --baseline base` (auto-resolves `base-files` / `base-dependencies` / `base-deprecated`)                                                                                                                   |
+| MCP server (for agent hosts)   | —                  | `bun src/index.ts mcp [--watch] [--debounce <ms>]` — JSON-RPC on stdio; one tool per CLI verb. See **MCP** section below.                                                                                                                |
+| HTTP server (for non-MCP)      | —                  | `bun src/index.ts serve [--host 127.0.0.1] [--port 7878] [--token <secret>] [--watch] [--debounce <ms>]` — same tool taxonomy over POST /tool/{name}.                                                                                    |
+| Watch mode (live reindex)      | —                  | `bun src/index.ts watch [--debounce 250] [--quiet]` — long-running; debounced reindex on file changes. Combine with `mcp --watch` / `serve --watch` (or `CODEMAP_WATCH=1`) so every tool reads a live index without per-request prelude. |
+| Targeted read (metadata)       | —                  | `bun src/index.ts show <name> [--kind <k>] [--in <path>] [--json]` — file:line + signature                                                                                                                                               |
+| Targeted read (source text)    | —                  | `bun src/index.ts snippet <name> [--kind <k>] [--in <path>] [--json]` — same lookup + source from disk + stale flag                                                                                                                      |
+| SARIF / GH annotations         | —                  | `bun src/index.ts query --recipe deprecated-symbols --format sarif` · `… --format annotations`                                                                                                                                           |
 
 **Recipe `actions`:** with **`--json`**, recipes that define an `actions` template append it to every row (kebab-case verb + description — e.g. `fan-out` → `review-coupling`). Under `--baseline`, actions attach to the **`added`** rows only. Inspect via **`--recipes-json`**. Ad-hoc SQL never carries actions.
 

.agents/skills/codemap/SKILL.md

@@ -54,9 +54,11 @@ Replace placeholders (`'...'`) with your module path, file glob, or symbol name.
 
 Each emitted delta carries its own `base` metadata so mixed-baseline audits are first-class. `--summary` collapses each delta to `{added: N, removed: N}`. `--no-index` skips the auto-incremental-index prelude (default is to re-index first so `head` reflects current source). v1 ships no `verdict` / threshold config — `codemap audit --json | jq -e '.deltas.dependencies.added | length <= 50'` is the CI exit-code idiom until v1.x ships native thresholds. Each delta pins a canonical SQL projection and validates baseline column-set membership before diffing — schema-bump-resilient (extras dropped, missing columns surface a clean re-save command).
 
-**MCP server (`bun src/index.ts mcp`)** — separate top-level command that exposes the entire CLI surface to agent hosts (Claude Code, Cursor, Codex, generic MCP clients) as JSON-RPC tools over stdio. Eliminates the bash round-trip on every agent call. Bootstrap once at server boot; tool handlers reuse the existing engine entry-points (`executeQuery`, `runAudit`, etc.) so output shape is verbatim from each tool's CLI counterpart's `--json` envelope.
+**MCP server (`bun src/index.ts mcp [--watch] [--debounce <ms>]`)** — separate top-level command that exposes the entire CLI surface to agent hosts (Claude Code, Cursor, Codex, generic MCP clients) as JSON-RPC tools over stdio. Eliminates the bash round-trip on every agent call. Bootstrap once at server boot; tool handlers reuse the existing engine entry-points (`executeQuery`, `runAudit`, etc.) so output shape is verbatim from each tool's CLI counterpart's `--json` envelope. With `--watch` (or `CODEMAP_WATCH=1`), boots a co-process file watcher so every tool reads a live index — and `audit`'s incremental-index prelude becomes a no-op (saves the per-request reindex cost).
 
-**HTTP server (`bun src/index.ts serve [--host 127.0.0.1] [--port 7878] [--token <secret>]`)** — same tool taxonomy as MCP, exposed over `POST /tool/{name}` for non-MCP consumers (CI scripts, simple `curl`, IDE plugins that don't speak MCP). Loopback-default; optional Bearer-token auth. Output shape is the `codemap query --json` envelope (NOT MCP's `{content: [...]}` wrapper); SARIF / annotations payloads ship with `application/sarif+json` / `text/plain` Content-Type. Resources mirrored at `GET /resources/{encoded-uri}`. `GET /health` is auth-exempt; `GET /tools` / `GET /resources` are catalogs. Same `application/tool-handlers.ts` + `resource-handlers.ts` MCP uses — no engine duplication.
+**HTTP server (`bun src/index.ts serve [--host 127.0.0.1] [--port 7878] [--token <secret>] [--watch] [--debounce <ms>]`)** — same tool taxonomy as MCP, exposed over `POST /tool/{name}` for non-MCP consumers (CI scripts, simple `curl`, IDE plugins that don't speak MCP). Loopback-default; optional Bearer-token auth. Output shape is the `codemap query --json` envelope (NOT MCP's `{content: [...]}` wrapper); SARIF / annotations payloads ship with `application/sarif+json` / `text/plain` Content-Type. Resources mirrored at `GET /resources/{encoded-uri}`. `GET /health` is auth-exempt; `GET /tools` / `GET /resources` are catalogs. Same `application/tool-handlers.ts` + `resource-handlers.ts` MCP uses — no engine duplication. Same `--watch` + `--debounce` semantics as `mcp` — for IDE / CI scripts that hit the API repeatedly, `serve --watch` removes the per-query staleness anxiety.
+
+**Watch mode (`bun src/index.ts watch [--debounce 250] [--quiet]`)** — standalone long-running process that debounces file changes and re-indexes only the changed paths via `runCodemapIndex({mode: 'files'})`. SIGINT/SIGTERM drains pending edits before exit. Use `mcp --watch` / `serve --watch` for the in-process killer combo; use `codemap watch` standalone when you want the watcher decoupled from a transport (e.g. running alongside an editor that already speaks MCP via a different process).
 
 **Tools (snake_case keys — Codemap convention matching MCP spec examples + reference servers; spec is convention-agnostic. CLI stays kebab; translation lives at the MCP-arg layer.):**
 

.changeset/codemap-watch.md

@@ -0,0 +1,19 @@
+---
+"@stainless-code/codemap": minor
+---
+
+`codemap watch` — long-running process that re-indexes changed files in real time so every CLI / MCP / HTTP query reads live data without a per-query reindex prelude. Eliminates the single biggest source of agent-side friction: "is the index stale right now?"
+
+**Three shapes:**
+
+- **Standalone**: `codemap watch [--debounce 250] [--quiet]` — foreground process; logs `reindex N file(s) in Mms` per batch unless `--quiet`. SIGINT / SIGTERM drains pending edits.
+- **MCP killer combo**: `codemap mcp --watch [--debounce <ms>]` — boots stdio MCP server + watcher in one process. Long Cursor / Claude Code sessions never hit a stale index; agents stop having to remember to reindex between edit + query.
+- **HTTP killer combo**: `codemap serve --watch [--debounce <ms>]` — same shape for non-MCP consumers (CI scripts, IDE plugins, simple `curl`).
+
+**Audit prelude optimization:** when watch is active, `mcp audit`'s default incremental-index prelude becomes a no-op (the watcher already keeps the index fresh — saves the per-request reindex cost). Explicit `no_index: false` still forces the prelude.
+
+**Env shortcut:** `CODEMAP_WATCH=1` (or `"true"`) implies `--watch` for `mcp` / `serve` — useful for IDE / CI launches that can't easily edit the spawn command.
+
+**Backend:** [chokidar v5](https://github.com/paulmillr/chokidar) (selected via 6-watcher audit in PR #46). Pure JS — runs identically on Bun + Node, no per-runtime branching, no native compile matrix on top of `bun:sqlite` / `better-sqlite3`. Cross-platform (macOS / Linux / Windows / WSL). Atomic-write + chunked-write detection out of the box. 1 dep (`readdirp`), 82 KB.
+
+**Filtering:** Only paths the indexer cares about trigger a reindex (TS / TSX / JS / JSX / CSS + project-local recipes under `<root>/.codemap/recipes/`). `node_modules` / `.git` / `dist` / configured `excludeDirNames` are skipped.

README.md

@@ -119,6 +119,12 @@ curl -s -X POST http://127.0.0.1:7878/tool/query \
   -H 'Content-Type: application/json' \
   -H "Authorization: Bearer $TOKEN" \
   -d '{"sql":"SELECT name, file_path FROM symbols LIMIT 5"}'
+# Watch mode — long-running process; debounced reindex on file changes (default 250ms).
+# Combine with mcp / serve so every tool reads a live index without per-request prelude:
+codemap mcp --watch                   # killer combo for agent hosts
+codemap serve --watch --port 7878     # killer combo for CI / IDE plugins
+codemap watch --quiet                 # standalone (decoupled from a transport)
+CODEMAP_WATCH=1 codemap mcp           # env-var shortcut for IDE / CI launches
 # List bundled recipes as JSON, or print one recipe's SQL (no DB required)
 codemap query --recipes-json
 codemap query --print-sql fan-out