fix(ingest): self-healing graph memo + commit-triggered re-analyze

Make codebase-graph freshness trustable instead of periodic-and-stale.

1. Self-heal the cached graph_path memo. find_cached_graph returns a
   memoised path only when the graph still exists on disk (non-empty
   dir/file), preferring the most-recent such memo; otherwise None so the
   caller re-analyses. Previously it returned the first tag match
   unconditionally, so a memo outliving its deleted graph made
   ingest_codebase silently project an empty graph. One fix heals all
   three consumers (ensure_graph, pipeline_impact_bump, SessionStart bg
   worker). Dijkstra audit: the pointer is not the thing.

2. Make a commit the freshness signal. New PostToolUse hook
   post_commit_reindex spawns the detached background re-analyze worker
   (ingest_codebase_background --reindex) when a git commit touches
   indexable source. Closes the gap the 24h SessionStart TTL leaves open
   within a working session. Gated on changed source, detached (never
   blocks the commit), 120s coalesce cooldown (also serialises writes to
   the same graph dir).

Two-speed model: facts fresh per commit, consolidation/clustering on the
existing TTL cycles (perception continuous, consolidation in sleep).

Honest scope: the upstream analyzer (ai-automatised-pipeline
index_codebase, src/indexer/mod.rs) re-parses the whole tree -- it has NO
per-file incremental skip. "Incremental" here means trigger-only-on-change,
not reparse-only-the-changed-file. Full re-parse is seconds for
harness-sized repos and runs detached. Extension gate sourced from AP
src/parser/mod.rs Language::from_extension plus the .js light-link pass.

Tests: self-heal memo (existence + recency + dead-path skip), handler-level
re-analyze on dead memo, hook gating (non-commit/failed/docs-only/cooldown/
spawn), --reindex flag threading.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: cdeust

.claude-plugin/plugin.json

@@ -62,6 +62,11 @@
             "type": "command",
             "command": "bash -c 'PY=$(command -v python3 || command -v python) && ROOT=\"${CLAUDE_PLUGIN_ROOT:-$PWD}\" && \"$PY\" \"$ROOT/scripts/launcher.py\" mcp_server.hooks.pipeline_impact_bump'",
             "timeout": 5
+          },
+          {
+            "type": "command",
+            "command": "bash -c 'PY=$(command -v python3 || command -v python) && ROOT=\"${CLAUDE_PLUGIN_ROOT:-$PWD}\" && \"$PY\" \"$ROOT/scripts/launcher.py\" mcp_server.hooks.post_commit_reindex'",
+            "timeout": 10
           }
         ]
       }

mcp_server/handlers/ingest_helpers.py

@@ -36,27 +36,99 @@ def code_graph_tag(project_path: str) -> str:
     return f"{CODE_GRAPH_TAG_PREFIX}{project_key(project_path)}"
 
 
-def find_cached_graph(store, project_path: str) -> str | None:
-    """Return the cached graph_path for a project, or None if not cached.
+def graph_path_is_materialised(graph_path: str | None) -> bool:
+    """True when ``graph_path`` points at a graph that still exists on disk.
+
+    AP writes ``<output_dir>/graph`` as a LadybugDB directory; pre-3.14
+    builds wrote a single file at the same slot. Either form counts as
+    valid only when **non-empty** — an existing-but-empty directory is a
+    half-built or wiped graph, which must read as a cache miss so the
+    caller re-analyses rather than silently projecting zero symbols.
+
+    source: ingest staleness bug Jun-2026 — a memo outlived its graph
+    (the graph directory was deleted) and ``find_cached_graph`` handed the
+    dead path straight back to ``ensure_graph``, which then projected an
+    empty graph. A memo must never outlive the artefact it points at
+    (Dijkstra audit: the pointer is not the thing).
+    """
+    if not graph_path:
+        return False
+    try:
+        p = Path(graph_path).expanduser()
+        if not p.exists():
+            return False
+        if p.is_dir():
+            return any(p.iterdir())
+        return p.stat().st_size > 0
+    except OSError:
+        return False
+
+
+def _memo_tags(mem: dict) -> list:
+    """Tags of a memory row as a list (rows may store JSON-encoded tags)."""
+    raw = mem.get("tags", [])
+    if isinstance(raw, str):
+        try:
+            raw = json.loads(raw)
+        except (ValueError, TypeError):
+            return []
+    return raw if isinstance(raw, list) else []
+
 
-    Reads the most-recent memory tagged with the project's code-graph tag.
+def _memo_recency_key(mem: dict) -> str:
+    """Sortable recency key for a memo row (most-recent sorts highest).
+
+    Prefers ``created_at`` (when the path was memoised); falls back to
+    ``heat_base_set_at`` then ``last_accessed``. Datetimes are
+    ISO-normalised; a missing value sorts oldest. ISO-8601 strings sort
+    lexicographically in chronological order, so plain string comparison
+    is correct here.
+    """
+    for field in ("created_at", "heat_base_set_at", "last_accessed"):
+        val = mem.get(field)
+        if val is None or val == "":
+            continue
+        iso = getattr(val, "isoformat", None)
+        return iso() if callable(iso) else str(val)
+    return ""
+
+
+def find_cached_graph(store, project_path: str) -> str | None:
+    """Return the cached graph_path for a project, or None.
+
+    Returns the path from the MOST-RECENT memo tagged with the project's
+    code-graph tag whose graph **still exists on disk**. A memo whose
+    graph was deleted (path missing or empty) is skipped, never returned
+    — this is the self-heal: a stale memo can no longer make the caller
+    project an empty graph. When no live graph is found, returns None so
+    the caller re-analyses and re-memoises.
+
+    source: ingest staleness bug Jun-2026 (Dijkstra audit). Previously
+    this returned the first tag match unconditionally, with no existence
+    check and no recency ordering.
     """
     tag = code_graph_tag(project_path)
     try:
         mems = store.get_all_memories_for_decay()
     except Exception:
         return None
+
+    candidates: list[tuple[str, str]] = []  # (recency_key, graph_path)
     for mem in mems:
-        raw_tags = mem.get("tags", [])
-        if isinstance(raw_tags, str):
-            try:
-                raw_tags = json.loads(raw_tags)
-            except (ValueError, TypeError):
-                raw_tags = []
-        if tag in raw_tags:
-            content = mem.get("content") or ""
-            if content.startswith("graph_path="):
-                return content[len("graph_path=") :].strip()
+        if tag not in _memo_tags(mem):
+            continue
+        content = mem.get("content") or ""
+        if not content.startswith("graph_path="):
+            continue
+        path = content[len("graph_path=") :].strip()
+        if path:
+            candidates.append((_memo_recency_key(mem), path))
+
+    # Most-recent first; return the first whose graph is materialised.
+    candidates.sort(key=lambda c: c[0], reverse=True)
+    for _, path in candidates:
+        if graph_path_is_materialised(path):
+            return path
     return None
 
 

mcp_server/hooks/ingest_codebase_background.py

@@ -1,11 +1,20 @@
 """Background worker that invokes ``ingest_codebase`` for a project.
 
-Spawned by the SessionStart hook when the cached graph is stale or
-missing. Runs detached from the parent process so SessionStart returns
-immediately.
+Spawned by two triggers, both detached so the parent returns immediately:
+  * the SessionStart hook, when the cached graph is missing or older than
+    the TTL (``pipeline_graph_ttl.graph_is_stale``); and
+  * the PostToolUse ``post_commit_reindex`` hook, after a commit that
+    touched indexable source — there it passes ``--reindex`` because the
+    commit IS the change signal, so the graph must be rebuilt even though
+    a cached one exists.
 
 Invocation:
     python -m mcp_server.hooks.ingest_codebase_background /path/to/project
+    python -m mcp_server.hooks.ingest_codebase_background /path/to/project --reindex
+
+Without ``--reindex`` the handler reuses a fresh cached graph and only
+re-analyses when the cache is stale/absent (identical to interactive
+use). With ``--reindex`` it forces ``analyze_codebase`` to run.
 
 Exit code:
   * 0 on success
@@ -34,6 +43,7 @@ def main() -> None:
         sys.exit(2)
 
     project_root = sys.argv[1]
+    force_reindex = "--reindex" in sys.argv[2:]
 
     # Lazy import so Claude Code hooks can fire even if core deps are
     # still installing on first session.
@@ -43,10 +53,12 @@ def main() -> None:
         print(f"[bg-ingest] ingest_codebase import failed: {exc}", file=sys.stderr)
         sys.exit(1)
 
+    # Without --reindex: reuse a fresh cached graph, auto-reindex when
+    # stale (SessionStart trigger). With --reindex: force analyze_codebase
+    # because a commit already told us the source changed (commit trigger).
     args: dict[str, Any] = {
         "project_path": project_root,
-        # Don't force reindex — handler picks up cached graph when fresh
-        # and auto-reindexes when stale. Identical to interactive use.
+        "force_reindex": force_reindex,
     }
 
     try:
@@ -60,7 +72,8 @@ def main() -> None:
         sys.exit(1)
 
     counts = {k: v for k, v in (result or {}).items() if isinstance(v, (int, float))}
-    print(f"[bg-ingest] ingest_codebase ok: {counts}")
+    mode = "reindex" if force_reindex else "cached-or-stale"
+    print(f"[bg-ingest] ingest_codebase ok ({mode}): {counts}")
     sys.exit(0)