feat(mcp): session lifecycle hygiene (#153)

* feat(mcp): session lifecycle hygiene without idle timeout

Add stdio disconnect detection and refcount-gated watcher sessions for
mcp/serve; document explicit no-MCP-idle-timeout policy in architecture
and agent docs.

* fix(session-lifecycle): harden acquire/stop refcount races

Roll back client count when watch start fails; await in-flight stop
before acquire; align serve help with non-/health acquire scope.

* fix(mcp): prime watch before connect on --watch

Restore main's watch-ready-before-tools ordering via acquireClient
before server.connect; align agents.md HTTP grace wording.

* docs(mcp): watcher primes before connect, not after

Align architecture, glossary, and cmd-mcp help with acquireClient
ordering before server.connect.

* fix(session-lifecycle): await in-flight startup on stop

stopWatcher now waits for starting before checking handle, so
forceStop during HTTP first-request prime cannot orphan chokidar.
Also observe fire-and-forget release/stop rejections and harden
the parent-pid spy test with try/finally.

Author: SutuSebastian

.changeset/mcp-session-lifecycle-hygiene.md

@@ -0,0 +1,5 @@
+---
+"codemap": minor
+---
+
+MCP session lifecycle hygiene: stdio disconnect detection (stdin EOF, stdout EPIPE, parent-PID poll) with graceful watcher shutdown on client exit; HTTP `serve --watch` refcount-gates the watcher per request (5s release grace, `/health` excluded). No MCP idle shutdown.

docs/agents.md

@@ -101,6 +101,16 @@ All three transports resolve to the same `assembleAgentContent(kind)` function i
 
 Recipe ids cited in the playbook are machine-validated in tests against the live catalog (`extractMcpInstructionRecipeIds`).
 
+## MCP session lifecycle
+
+Long-running **`codemap mcp`** stays up for the whole IDE session while the stdio pipe is open — **there is no idle timeout** that exits after N minutes without tool calls. Cursor / Claude Code spawn MCP once; they do not reliably respawn it mid-conversation, so an idle shutdown would break long pauses with no recovery path.
+
+**Exit triggers (MCP):** client disconnect only — stdin EOF, stdout broken pipe (`EPIPE`), boot parent process gone, or SIGINT/SIGTERM. Implementation: `src/application/session-lifecycle.ts` (`createStdioDisconnectMonitor`).
+
+**Not idle timeout:** HTTP **`serve --watch`** uses a 5s **watch release grace** after the last non-`/health` request — that stops chokidar between stateless requests, not the MCP/HTTP process. **`GET /health`** never acquires a watch client.
+
+See [architecture.md § Session lifecycle wiring](./architecture.md#cli-usage).
+
 ## MCP tool allowlist
 
 **`context.index_freshness`** — session bootstrap includes index-level freshness metadata: `commit_drift` (HEAD ≠ `last_indexed_commit`), `pending_sync` (watcher debounce queue or in-flight reindex), optional disk-drift counts when watch is off, and a single `warning` string when agents should pause or re-index. **`context.start_here`** (non-compact) adds inline index summary, intent-ranked `query_recipe` cards, and top hub files with export signatures (adaptive caps by file count; optional MCP/HTTP `include_snippets` for one-line previews). Debug intent biases `sample_markers` toward FIXME/TODO. **MCP:** array-shaped JSON tools (`query`, …) keep row payloads verbatim and append a second `content` block prefixed `@codemap/index_freshness`; object-shaped tools merge `index_freshness` inline. **HTTP:** `POST /tool/*` adds `X-Codemap-Pending-Sync`, `X-Codemap-Commit-Drift`, and `X-Codemap-Warning` headers without changing JSON bodies; **`GET /health`** includes full cheap `index_freshness` when the DB is readable. Complements per-file `validate` / snippet `stale`. See [`architecture.md` § Context wiring](./architecture.md).

docs/architecture.md

@@ -353,7 +353,7 @@ Rust-based CSS parser (NAPI bindings). Codemap's `src/css-parser.ts` uses its vi
 
 ### `codemap mcp` / MCP server
 
-Stdio MCP (Model Context Protocol) server exposing codemap's structural-query surface to agent hosts (Claude Code, Cursor, Codex, generic MCP clients) as JSON-RPC tools — eliminates the bash round-trip on every agent invocation. **17 tools:** `query`, `query_batch` (no CLI verb), `query_recipe`, `audit`, `save_baseline`, `list_baselines`, `drop_baseline`, `context`, `validate`, `show`, `snippet`, `impact`, `affected`, `trace`, `explore`, `node`, `apply`. Subset via **`CODEMAP_MCP_TOOLS`** ([agents.md § MCP tool allowlist](./agents.md#mcp-tool-allowlist)). **Resources:** `codemap://schema`, `codemap://skill`, `codemap://rule`, `codemap://mcp-instructions`, `codemap://recipes`, `codemap://recipes/{id}`, `codemap://files/{path}`, `codemap://symbols/{name}`. Resource freshness is split by contract: schema / skill / rule / mcp-instructions are lazy-cached per server process; recipes, files, and symbols are live read-per-call so inline recency fields and index mutations under `--watch` don't freeze at first read. HTTP's `GET /resources/{encoded-uri}` uses the same resource handler. **Baseline tools** (`save_baseline`, `list_baselines`, `drop_baseline`) mirror `query --save-baseline` / `--baselines` / `--drop-baseline`. **`query_batch`**, **`trace`**, **`explore`**, and **`node`** have no CLI verb (MCP/HTTP composers only). Tool input/output keys are snake_case — Codemap's convention, matching the patterns in MCP spec examples and reference servers (GitHub MCP, Cursor built-ins); the spec itself doesn't mandate it. CLI stays kebab — translation lives at the MCP-arg layer. Output shape matches each tool's CLI `--json` payload where a CLI verb exists (no CLI verb: `query_batch`, `trace`, `explore`, `node`); MCP wraps payloads in `{content: [{type: "text", text: …}]}`. Bootstrap once at server boot; tool handlers (in `application/tool-handlers.ts`) and resource handlers (in `application/resource-handlers.ts`) are pure transport-agnostic — the same handlers serve `codemap serve` (HTTP) via `POST /tool/{name}` and `GET /resources/{encoded-uri}`. Implementation: `src/cli/cmd-mcp.ts` (CLI shell) + `src/application/mcp-server.ts` (engine). See [`architecture.md` § MCP wiring](./architecture.md#cli-usage).
+Stdio MCP (Model Context Protocol) server exposing codemap's structural-query surface to agent hosts (Claude Code, Cursor, Codex, generic MCP clients) as JSON-RPC tools — eliminates the bash round-trip on every agent invocation. **17 tools:** `query`, `query_batch` (no CLI verb), `query_recipe`, `audit`, `save_baseline`, `list_baselines`, `drop_baseline`, `context`, `validate`, `show`, `snippet`, `impact`, `affected`, `trace`, `explore`, `node`, `apply`. Subset via **`CODEMAP_MCP_TOOLS`** ([agents.md § MCP tool allowlist](./agents.md#mcp-tool-allowlist)). **Resources:** `codemap://schema`, `codemap://skill`, `codemap://rule`, `codemap://mcp-instructions`, `codemap://recipes`, `codemap://recipes/{id}`, `codemap://files/{path}`, `codemap://symbols/{name}`. Resource freshness is split by contract: schema / skill / rule / mcp-instructions are lazy-cached per server process; recipes, files, and symbols are live read-per-call so inline recency fields and index mutations under `--watch` don't freeze at first read. HTTP's `GET /resources/{encoded-uri}` uses the same resource handler. **Baseline tools** (`save_baseline`, `list_baselines`, `drop_baseline`) mirror `query --save-baseline` / `--baselines` / `--drop-baseline`. **`query_batch`**, **`trace`**, **`explore`**, and **`node`** have no CLI verb (MCP/HTTP composers only). Tool input/output keys are snake_case — Codemap's convention, matching the patterns in MCP spec examples and reference servers (GitHub MCP, Cursor built-ins); the spec itself doesn't mandate it. CLI stays kebab — translation lives at the MCP-arg layer. Output shape matches each tool's CLI `--json` payload where a CLI verb exists (no CLI verb: `query_batch`, `trace`, `explore`, `node`); MCP wraps payloads in `{content: [{type: "text", text: …}]}`. Bootstrap once at server boot; tool handlers (in `application/tool-handlers.ts`) and resource handlers (in `application/resource-handlers.ts`) are pure transport-agnostic — the same handlers serve `codemap serve` (HTTP) via `POST /tool/{name}` and `GET /resources/{encoded-uri}`. **Session lifecycle:** exits on client disconnect (stdin EOF, stdout broken pipe, parent process exit, SIGINT/SIGTERM) via `session-lifecycle.ts`; **no idle timeout** — the process stays up while the pipe is open even without tool calls (see [§ Session lifecycle](./architecture.md#cli-usage)). With `--watch`, the watcher starts before connect and drains on exit. Implementation: `src/cli/cmd-mcp.ts` (CLI shell) + `src/application/mcp-server.ts` (engine). See [`architecture.md` § MCP wiring](./architecture.md#cli-usage).
 
 ### `query_batch` (no CLI verb; MCP + HTTP)
 
@@ -508,6 +508,10 @@ Conceptually, the structure of the SQLite database — every table, column, cons
 
 Integer constant in `src/db.ts`. Bumped only for rebuild-forcing DDL changes; additive tables / columns / indexes can land through `CREATE ... IF NOT EXISTS` without a version bump. `createSchema()` reads `meta.schema_version` and triggers a full rebuild on mismatch.
 
+### Session lifecycle
+
+Long-running transport shutdown rules in `src/application/session-lifecycle.ts`. **MCP:** `createStdioDisconnectMonitor` exits `codemap mcp` on client disconnect (stdin EOF, stdout `EPIPE`, boot parent PID gone, SIGINT/SIGTERM). **No idle timeout** — the process stays up while stdio is open even without tool calls; IDE hosts do not respawn MCP mid-session. **HTTP:** `createManagedWatchSession` refcount-gates chokidar per request (`GET /health` excluded); **`HTTP_WATCH_RELEASE_GRACE_MS`** (5s) stops the watcher between stateless requests without shutting down the HTTP listener — not an MCP-style idle kill. See [architecture.md § Session lifecycle wiring](./architecture.md#cli-usage).
+
 ### show
 
 `codemap show <name>` — one-step lookup that returns metadata (`file_path:line_start-line_end` + `signature` + `kind`) for symbol(s). **Exact mode:** `<name>` is case-sensitive; flags `--kind`, `--in`. **Field-qualified mode:** `--query 'kind:… name:… path:… in:…'` with optional free text; `--with-fts` (or `fts5: true` when indexed) searches file bodies via `source_fts` and returns every symbol in matching files; `--print-sql` prints Moat-A equivalent SQL. Output: `{matches, disambiguation?, warning?}` (`warning` when FTS was requested but `source_fts` is empty). MCP: `show` with `{name}` or `{query, with_fts?}`. Distinct from **snippet** (adds source text) and from hand-composed `query` SQL. See [`architecture.md` § Show / snippet wiring](./architecture.md#cli-usage).
@@ -525,14 +529,14 @@ Opt-in substrate. Markers parser recognises `// codemap-ignore-next-line <recipe
 Long-running process that subscribes to filesystem changes via [chokidar v5](https://github.com/paulmillr/chokidar) and re-indexes only the changed files via `runCodemapIndex({mode: 'files'})`. Eliminates the "is the index stale?" friction every CLI / MCP / HTTP query rides on today: agents in long sessions or multi-step refactors can `query` immediately after editing without remembering to reindex. Debounced (default 250 ms) so a burst — `git checkout`, `npm install`, multi-file save — collapses to one reindex call. Filters event paths the same way the indexer does (TS / TSX / JS / JSX / CSS + project-local recipes; skips `node_modules`, `.git`, `dist`, etc.). SIGINT / SIGTERM drains pending edits before exit. Three shapes:
 
 - **Standalone**: `codemap watch` — foreground process; logs `reindex N file(s) in Mms` per batch unless `--quiet`.
-- **Combined with MCP**: `codemap mcp` — boots stdio MCP server + watcher in one process by default since 2026-05; agents never hit a stale index. Pass `--no-watch` to disable.
-- **Combined with HTTP**: `codemap serve` — boots HTTP server + watcher by default; CI scripts / IDE plugins read live data. Pass `--no-watch` to disable.
+- **Combined with MCP**: `codemap mcp` — boots stdio MCP server + watcher in one process by default since 2026-05; agents never hit a stale index. Pass `--no-watch` to disable. Watcher starts before MCP connect; stops on client disconnect (not on idle silence — see [§ Session lifecycle](./architecture.md#cli-usage)).
+- **Combined with HTTP**: `codemap serve` — boots HTTP server + watcher by default; CI scripts / IDE plugins read live data. Pass `--no-watch` to disable. Watcher is refcount-gated per request (`GET /health` excluded) with a 5s release grace between requests — stops chokidar, not the HTTP listener.
 
 `CODEMAP_WATCH=0` (or `"false"`) is the env-shortcut for opting out of the default-ON watcher on `codemap mcp` / `codemap serve` — useful for IDE / CI launches that can't easily edit the spawn command. `CODEMAP_WATCH=1` still parses for backwards-compat but is now a no-op (it matches the new default). When watch is active, the audit tool's incremental-index prelude becomes a no-op on both transports (the watcher already keeps the index fresh — saves the per-request reindex cost on every `mcp audit` and every `POST /tool/audit`). Implementation: `src/cli/cmd-watch.ts` (CLI shell) + `src/application/watcher.ts` (engine — pure debouncer + chokidar backend; injectable backend for tests). See [`architecture.md` § Watch wiring](./architecture.md#cli-usage).
 
 ### `codemap serve` / HTTP server
 
-Long-running HTTP server exposing the same tool taxonomy as `codemap mcp` over `POST /tool/{name}` for non-MCP consumers (CI scripts, simple `curl`, IDE plugins that don't speak MCP). Default bind **`127.0.0.1:7878`** (loopback only — refuse `0.0.0.0` unless explicitly opted in via `--host 0.0.0.0`); optional `--token <secret>` requires `Authorization: Bearer <secret>` on every request. HTTP returns each tool's native JSON payload directly (NOT MCP's `{content: [...]}` wrapper); `query` / `query_recipe` match `codemap query --json` row arrays unless `summary` / `group_by` reshape the envelope (baseline save/compare is separate tools — not on MCP/HTTP `query` / `query_recipe`); other tools match their CLI `--json` payloads; `format: "sarif"` payloads ship as `application/sarif+json`, `format: "annotations"` / `"mermaid"` / `"diff"` as `text/plain; charset=utf-8`, `format: "diff-json"` as `application/json; charset=utf-8`. Routes: `POST /tool/{name}` (every MCP tool), `GET /resources/{encoded-uri}` (resource handler for `codemap://recipes`, `codemap://recipes/{id}`, `codemap://schema`, `codemap://skill`, `codemap://rule`, `codemap://mcp-instructions`, `codemap://files/{path}`, and `codemap://symbols/{name}`), `GET /health` (auth-exempt liveness probe), `GET /tools` / `GET /resources` (catalogs). Pure transport — same `tool-handlers.ts` / `resource-handlers.ts` MCP uses; no engine duplication. Errors → `{"error": "..."}` with HTTP status 400 / 401 / 403 / 404 / 500. SIGINT / SIGTERM → graceful drain. Every response carries `X-Codemap-Version: <semver>`. **CSRF + DNS-rebinding guard:** every request (including auth-exempt `/health`) is evaluated against `Sec-Fetch-Site` / `Origin` / `Host` when present — modern browsers send `Sec-Fetch-Site` and `Origin` on cross-origin fetches (header presence varies by request type, browser, and privacy settings), so the guard rejects browser-driven cross-origin requests like a malicious local webpage `fetch`-ing `http://127.0.0.1:7878/tool/save_baseline` to mutate `.codemap/index.db`. `Host` mismatch on a loopback bind blocks DNS rebinding (an attacker resolving `evil.com` to `127.0.0.1` post-load). Non-browser clients (curl, fetch from Node, MCP hosts, CI scripts) typically omit these headers and pass through. Implementation: `src/cli/cmd-serve.ts` (CLI shell) + `src/application/http-server.ts` (transport). See [`architecture.md` § HTTP wiring](./architecture.md#cli-usage).
+Long-running HTTP server exposing the same tool taxonomy as `codemap mcp` over `POST /tool/{name}` for non-MCP consumers (CI scripts, simple `curl`, IDE plugins that don't speak MCP). Default bind **`127.0.0.1:7878`** (loopback only — refuse `0.0.0.0` unless explicitly opted in via `--host 0.0.0.0`); optional `--token <secret>` requires `Authorization: Bearer <secret>` on every request. HTTP returns each tool's native JSON payload directly (NOT MCP's `{content: [...]}` wrapper); `query` / `query_recipe` match `codemap query --json` row arrays unless `summary` / `group_by` reshape the envelope (baseline save/compare is separate tools — not on MCP/HTTP `query` / `query_recipe`); other tools match their CLI `--json` payloads; `format: "sarif"` payloads ship as `application/sarif+json`, `format: "annotations"` / `"mermaid"` / `"diff"` as `text/plain; charset=utf-8`, `format: "diff-json"` as `application/json; charset=utf-8`. Routes: `POST /tool/{name}` (every MCP tool), `GET /resources/{encoded-uri}` (resource handler for `codemap://recipes`, `codemap://recipes/{id}`, `codemap://schema`, `codemap://skill`, `codemap://rule`, `codemap://mcp-instructions`, `codemap://files/{path}`, and `codemap://symbols/{name}`), `GET /health` (auth-exempt liveness probe — does not start the watcher), `GET /tools` / `GET /resources` (catalogs). With `--watch`, chokidar is refcount-gated per request and stops 5s after the last client (`HTTP_WATCH_RELEASE_GRACE_MS`) — distinct from MCP idle shutdown; the HTTP process keeps listening. Pure transport — same `tool-handlers.ts` / `resource-handlers.ts` MCP uses; no engine duplication. Errors → `{"error": "..."}` with HTTP status 400 / 401 / 403 / 404 / 500. SIGINT / SIGTERM → graceful drain. Every response carries `X-Codemap-Version: <semver>`. **CSRF + DNS-rebinding guard:** every request (including auth-exempt `/health`) is evaluated against `Sec-Fetch-Site` / `Origin` / `Host` when present — modern browsers send `Sec-Fetch-Site` and `Origin` on cross-origin fetches (header presence varies by request type, browser, and privacy settings), so the guard rejects browser-driven cross-origin requests like a malicious local webpage `fetch`-ing `http://127.0.0.1:7878/tool/save_baseline` to mutate `.codemap/index.db`. `Host` mismatch on a loopback bind blocks DNS rebinding (an attacker resolving `evil.com` to `127.0.0.1` post-load). Non-browser clients (curl, fetch from Node, MCP hosts, CI scripts) typically omit these headers and pass through. Implementation: `src/cli/cmd-serve.ts` (CLI shell) + `src/application/http-server.ts` (transport). See [`architecture.md` § HTTP wiring](./architecture.md#cli-usage).
 
 ### SARIF
 

docs/glossary.md

@@ -80,7 +80,7 @@ Long-running MCP / HTTP sessions dominate agent workflows; one-shot CLI keeps th
 - [ ] **Codebase map in bootstrap responses** — hash-stable structural summary (top hubs, CLI entry hints, schema version, index freshness) auto-included in `context` / MCP initialize payload. **Partial:** hubs + `start_here.index_summary` + `index_freshness` ship on `context`; CLI entry hints + hash-stable map id still open. Opt-out via flag. Effort: S–M.
 - [x] **Index staleness surfacing** — `index_freshness.pending_sync` on `context`, MCP tool metadata, and HTTP headers when the watcher debounce queue or in-flight reindex is active. Shipped [#149](https://github.com/stainless-code/codemap/pull/149).
 - [x] **Adaptive output budgets** — scale trace/explore/node snippet char caps (and explore row limits) from indexed file counts via **`resolveOutputBudget(file_count)`** in `output-budget.ts`. Shipped [#152](https://github.com/stainless-code/codemap/pull/152). **`context`** hub/signature caps remain in **`resolveContextBudget()`**.
-- [ ] **MCP session lifecycle hygiene** — idle timeout, client disconnect detection, graceful watcher shutdown on last client; avoid orphaned watchers after agent host crashes. Effort: S–M.
+- [x] **MCP session lifecycle hygiene** — stdio disconnect detection (stdin EOF, stdout EPIPE, parent-PID poll, SIGINT/SIGTERM) and refcount-gated watcher stop on MCP client exit; HTTP `serve --watch` starts/stops the watcher per client (5s release grace between stateless requests; `/health` excluded). **Explicitly no MCP idle timeout** — process stays up while the stdio pipe is open even without tool calls (IDE hosts do not respawn mid-session). See [architecture.md § Session lifecycle wiring](./architecture.md#cli-usage). Effort: S–M.
 - [ ] **`agents init` uninstall (teardown)** — symmetric inverse of init for failed pilots, template mistakes, or leaving a repo: remove codemap-managed MCP entries, pointer sections, and IDE symlinks only (same scoped paths as init; never delete user-authored `.agents/` siblings). `--target` filter, `--yes` non-interactive. Not the happy-path docs story — adoption stays `init --mcp --git-hooks` + committed `.agents/`. Effort: S.
 - [x] **HEAD / index freshness warning** — `index_freshness.commit_drift` + `warning` on `context` / tool metadata; boot stderr on `codemap mcp` / `serve` when concerns remain after prime. Shipped [#149](https://github.com/stainless-code/codemap/pull/149).