fix(wiki): plug slug/title leaks producing .md.md, timestamp-slugs, path-titles (#26)

* fix(wiki): plug slug/title leaks producing .md.md, timestamp-slugs, path-titles

Audit of the methodology wiki (7,883 pages) on 2026-05-12 found:
  - 58 pages with .md.md double extension
  - 10 ADRs slugged "decision-created-2026-04-15t09-29-10z" from YAML
    frontmatter timestamps leaking through as titles
  - 11+ pages with embedded filesystem paths in slugs, e.g.
    "specs/2026-04-17-also-on-users-cdeust-documents-developments-..."
  - All polluted pages written 2026-04-21 — *after* v3.10.1 (2026-04-15)
    shipped the audit-artefact filter, so these are live bugs not history.

Root causes and fixes:

  1. .md.md (slugify preserves '.') — wiki_layout.slugify now strips a
     trailing chain of ".md" before returning. Single-point repair
     benefits all six filename builders (adr_filename, domain_page_path,
     wiki_sync, draft_compiler, ingest_prd, ingest_codebase_pages).

  2. Timestamp-as-title (YAML "created:" line accepted by derive_title) —
     new _YAML_KV_TITLE_PATTERNS reject lines matching
     "(created|updated|date|...): <value>" and bare ISO-8601 timestamps.

  3. Path-embedded-mid-sentence titles (existing _PATH_OR_URL_TITLE_PATTERNS
     only matched paths at line start) — added two patterns that match
     /Users/, /home/, /opt/, /var/, /etc/, /tmp/, /root/ anywhere in the
     line, plus Windows drive paths anywhere.

  4. derive_title fallback leaked raw content[:80] when no clean line
     existed, defeating wiki_sync's deterministic memory-<hash> fallback.
     derive_title now returns "" when every candidate line is rejected
     (path, URL, YAML, JSON, too-short); callers route to the hash.

Regression tests in test_wiki_layout.py (4 new) and test_wiki_classifier.py
(6 new). 1720 core tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* style: ruff format test_wiki_classifier (CI fix)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: cdeust

mcp_server/core/wiki_classifier.py

@@ -169,6 +169,30 @@
         r"^\s*#*\s*[\w.-]+\.(pdf|png|jpg|jpeg|svg|gif|zip|tar\.gz|docx?|xlsx?|csv|log|yaml|yml)\b",
         re.IGNORECASE,
     ),
+    # Path embedded mid-line ("also on /Users/...", "fix the file at C:\\..."):
+    # any absolute POSIX path or Windows drive path anywhere in the title.
+    # Bug found 2026-05-12: pages like
+    # specs/2026-04-17-also-on-users-cdeust-documents-developments-...md
+    # passed the start-of-line check, then slugify stripped the leading "/"
+    # and folded the entire path into the slug.
+    re.compile(r"(?:^|\s)/(Users|home|root|opt|var|etc|tmp)/", re.IGNORECASE),
+    re.compile(r"(?:^|\s)[A-Za-z]:[\\/]"),
+]
+
+# YAML-frontmatter / key:value lines that leaked through as titles when the
+# real title was missing. Reject lines that are purely "key: value" where
+# the value is a timestamp, identifier, or boolean — they're metadata,
+# not titles.  Bug found 2026-05-12: 10 ADRs slugged as
+# "decision-created-2026-04-15t09-29-10z" because the YAML "created:"
+# line was the first non-{}/[] line in the body.
+_YAML_KV_TITLE_PATTERNS = [
+    # `created: 2026-04-15T09:29:10Z`, `updated: 2026-04-15`, `date: ...`
+    re.compile(
+        r"^\s*(created|updated|date|timestamp|time|id|uuid|version)\s*:\s*\S",
+        re.IGNORECASE,
+    ),
+    # Bare ISO-8601 timestamp anywhere (would slug to t09-29-10z form)
+    re.compile(r"\b\d{4}-\d{2}-\d{2}T\d{2}[:-]\d{2}[:-]\d{2}", re.IGNORECASE),
 ]
 
 # Session / audit artefact tags — these are recall-fodder, not wiki-worthy.
@@ -538,6 +562,27 @@ def classify_memory(content: str, tags: list[str] | None = None) -> str | None:
     return "note"
 
 
+def _line_is_title_candidate(cleaned: str) -> bool:
+    """Return True iff ``cleaned`` is acceptable as a wiki page title.
+
+    Rejects: empty/short, JSON braces, embedded paths/URLs, YAML metadata
+    key:value lines, bare timestamps. Callers that get False from every
+    candidate line should yield an empty title and let the deterministic
+    hash fallback kick in (see ``wiki_sync._sync_to_wiki``).
+    """
+    if len(cleaned) <= 10:
+        return False
+    if cleaned.startswith("{") or cleaned.startswith("["):
+        return False
+    for pat in _PATH_OR_URL_TITLE_PATTERNS:
+        if pat.search(cleaned):
+            return False
+    for pat in _YAML_KV_TITLE_PATTERNS:
+        if pat.search(cleaned):
+            return False
+    return True
+
+
 def derive_title(
     content: str,
     kind: str,
@@ -548,31 +593,25 @@ def derive_title(
 
     Strategy (Alexander P4 + Eco):
     1. Strip known prefixes
-    2. Use kind-specific extraction
-    3. Fall back to entity-based or tag-based title
-    4. Last resort: first meaningful sentence
+    2. Walk lines; accept the first that passes ``_line_is_title_candidate``
+    3. Fall back to entity-based title if 2+ entities are supplied
+    4. Otherwise return "" — caller is responsible for a deterministic
+       fallback (e.g. ``memory-<hash>``). Returning a raw 80-char content
+       prefix here used to leak filesystem paths, timestamps, and sentence
+       fragments into slugs.
     """
-    # Get clean first line
     lines = content.strip().split("\n")
     first_meaningful = ""
     for line in lines:
         cleaned = line.strip()
         for pat in _TITLE_STRIP_PREFIXES:
             cleaned = pat.sub("", cleaned).strip()
-        if (
-            len(cleaned) > 10
-            and not cleaned.startswith("{")
-            and not cleaned.startswith("[")
-        ):
+        if _line_is_title_candidate(cleaned):
             first_meaningful = cleaned
             break
 
-    if not first_meaningful:
-        first_meaningful = content.strip()[:80]
-
     # Truncate to reasonable title length
     if len(first_meaningful) > 80:
-        # Cut at word boundary
         first_meaningful = first_meaningful[:77].rsplit(" ", 1)[0] + "..."
 
     # Kind-specific prefixing for clarity
@@ -591,6 +630,9 @@ def derive_title(
             return f"{prefix}: {entity_title}"
         return entity_title
 
+    if not first_meaningful:
+        return ""
+
     if prefix and not first_meaningful.lower().startswith(prefix.lower()):
         return f"{prefix}: {first_meaningful}"
 

mcp_server/core/wiki_layout.py

@@ -32,16 +32,29 @@
 
 _SAFE = re.compile(r"[^a-zA-Z0-9_.-]+")
 _MAX_SLUG_LEN = 80
+# Trailing extension tokens to strip before slug consumers append ".md".
+# Without this strip, an input title that already looks like a filename
+# (e.g. "001-zero-dependencies.md") produced filenames like
+# "2234-001-zero-dependencies.md.md" because every caller in the wiki
+# layer appends ".md" unconditionally. Strip iteratively so e.g.
+# "foo.md.md" → "foo".
+_TRAILING_MD_EXT = re.compile(r"(?:\.md)+$", re.IGNORECASE)
 
 
 def slugify(value: str, *, max_len: int = _MAX_SLUG_LEN) -> str:
-    """Stable filesystem-safe slug. Deterministic, lowercased, length-capped."""
+    """Stable filesystem-safe slug. Deterministic, lowercased, length-capped.
+
+    Postcondition: the returned slug never ends with ``.md`` (or any chain
+    of ``.md`` suffixes). Callers may safely append ``.md`` without
+    risking ``.md.md``.
+    """
     if not value:
         return "unknown"
     cleaned = _SAFE.sub("-", value.strip().lower()).strip("-")
     if not cleaned:
         return "unknown"
-    return cleaned[:max_len].rstrip("-") or "unknown"
+    cleaned = _TRAILING_MD_EXT.sub("", cleaned).rstrip(".-") or "unknown"
+    return cleaned[:max_len].rstrip("-.") or "unknown"
 
 
 def file_path_slug(file_path: str) -> str:

tests_py/core/test_wiki_classifier.py

@@ -2,7 +2,7 @@
 
 from __future__ import annotations
 
-from mcp_server.core.wiki_classifier import classify_memory
+from mcp_server.core.wiki_classifier import classify_memory, derive_title
 
 
 # ── Audit-tag gate ────────────────────────────────────────────────────
@@ -97,3 +97,76 @@ def test_valid_lesson_admitted() -> None:
         "on path again."
     )
     assert classify_memory(content, tags=["lesson", "bug-fix"]) == "lesson"
+
+
+# ── derive_title regression tests (bugs found 2026-05-12) ─────────────
+
+
+def test_derive_title_rejects_yaml_timestamp_line() -> None:
+    """Regression — 10 ADRs were slugged ``decision-created-2026-04-15t...``
+    because the first non-{}/[] line in the body was a YAML ``created:``
+    timestamp. derive_title must reject metadata key:value lines.
+    """
+    content = (
+        "created: 2026-04-15T09:29:10Z\n"
+        "We adopted pgvector with HNSW because benchmarks showed 3x speedup.\n"
+    )
+    title = derive_title(content, "adr")
+    assert "2026-04-15" not in title
+    assert "created" not in title.lower()
+    assert "pgvector" in title.lower() or "HNSW" in title
+
+
+def test_derive_title_rejects_embedded_posix_path() -> None:
+    """Regression — pages like ``2026-04-17-also-on-users-cdeust-documents-
+    developments-...`` were created because the first body line contained
+    an absolute /Users/ path mid-sentence and the path-as-title regex only
+    matched start-of-line. derive_title must reject path-embedded lines.
+    """
+    content = (
+        "also on /Users/cdeust/Documents/Developments/ai-architect-prd-builder "
+        "the same issue shows up\n"
+        "The real spec: caching keys must include model SHA.\n"
+    )
+    title = derive_title(content, "spec")
+    assert "users-cdeust" not in title.lower()
+    assert "/users/" not in title.lower()
+
+
+def test_derive_title_rejects_windows_path() -> None:
+    content = (
+        "see C:\\Users\\dev\\project\\config.toml for details\n"
+        "Caching policy: include the model SHA in every key.\n"
+    )
+    title = derive_title(content, "spec")
+    # The C:\ line must not appear; the clean second line is used instead.
+    assert "config.toml" not in title.lower()
+    assert "\\users\\dev" not in title.lower()
+    assert "caching" in title.lower()
+
+
+def test_derive_title_returns_empty_when_no_clean_candidate() -> None:
+    """When every candidate line is rejected, return empty so the caller
+    (wiki_sync) routes to the deterministic ``memory-<hash>`` fallback
+    instead of inheriting the v3.10.1 ``content[:80]`` raw-fragment leak.
+    """
+    content = "created: 2026-04-15T09:29:10Z\nupdated: 2026-04-15T09:29:11Z\nid: 1828\n"
+    title = derive_title(content, "adr")
+    assert title == ""
+
+
+def test_derive_title_bare_iso_timestamp_rejected() -> None:
+    content = (
+        "2026-04-15T09:29:10Z is when this happened\n"
+        "Decision: adopt pgvector for ANN.\n"
+    )
+    title = derive_title(content, "adr")
+    assert "2026-04-15" not in title
+
+
+def test_derive_title_still_works_for_clean_input() -> None:
+    """Positive control — happy path must continue to work."""
+    content = "Use pgvector for retrieval — IVFFlat is too slow at our scale."
+    title = derive_title(content, "adr")
+    assert "pgvector" in title.lower()
+    assert title.startswith("Decision:")

tests_py/core/test_wiki_layout.py

@@ -30,6 +30,34 @@ def test_slugify_empty() -> None:
     assert slugify("!!!") == "unknown"
 
 
+def test_slugify_strips_trailing_md_extension() -> None:
+    """Regression — bug found 2026-05-12.
+
+    Inputs that already look like .md filenames produced .md.md pages
+    because every wiki callsite appends ``.md`` to the slug. 58 pages
+    in the wiki had this shape (e.g. ``2234-decision-001-zero-
+    dependencies.md.md``). Slug must never end in ``.md``.
+    """
+    assert slugify("001-zero-dependencies.md") == "001-zero-dependencies"
+    assert slugify("foo.md") == "foo"
+    # Iterated md chain — should collapse to base.
+    assert slugify("foo.md.md") == "foo"
+    assert slugify("foo.md.md.md") == "foo"
+
+
+def test_slugify_preserves_non_md_extensions() -> None:
+    """``file_path_slug`` callers depend on .py/.ts/etc. surviving."""
+    assert slugify("login.py") == "login.py"
+    assert slugify("config.yaml") == "config.yaml"
+
+
+def test_adr_filename_no_double_md() -> None:
+    """End-to-end: title that contains '.md' must not yield .md.md."""
+    slug = slugify("001-zero-dependencies.md")
+    assert adr_filename(2234, slug) == "2234-001-zero-dependencies.md"
+    assert not adr_filename(2234, slug).endswith(".md.md")
+
+
 def test_file_path_slug() -> None:
     assert (
         file_path_slug("mcp_server/handlers/wiki_write.py")