feat(wiki): plain-language top-level README generator

User directive: "readable by non tech while having all information
needed for tech people." Adds a layered documentation surface:

  1. /wiki/README.md          — plain-language entry point (non-tech).
  2. /wiki/.generated/INDEX.md — structured TOC (tech).
  3. /wiki/<kind>/<domain>/<slug>.md — templated detail pages (tech).

Non-tech readers land on README.md and see:
  * One-sentence description of what this wiki IS.
  * Page counts + number of categories + domains.
  * "Last groomed: <timestamp>" — builds trust that it's current.
  * Per-category sections with plain-language "read this when…" hints
    (e.g., "Architecture Decisions — Why we chose X over Y. Read these
    to understand WHY the system looks the way it does.").
  * Domains list sorted by page count.
  * Link down to INDEX.md for the full structured TOC.
  * A "For contributors" section explaining how the grooming works
    and how to opt a page out via `grooming: manual`.

Design invariants (tests cover all):
  * Empty kinds are NOT rendered — no wall of empty sections on a
    fresh wiki.
  * Kind order is deterministic (follows PAGE_KINDS canonical order).
  * Same input produces same output (stable against reindex churn).
  * Pluralization correct: 0 pages, 1 page, 2 pages.
  * Domains section only appears when domain-scoped pages exist.
  * Custom project_name supported (for white-labelled deployments).

Wiring in wiki_store._try_reindex:
  * Writes INDEX.md (existing behavior).
  * Writes README.md IF it doesn't exist OR contains the auto-generated
    marker. Hand-written READMEs are NEVER overwritten — users who
    customize stay customized.

Tests: 15 for README generation + integration with the existing 32
wiki template/groomer tests = 47 wiki-layer tests total passing.

Complements the cortex-wiki-groomer agent shipped in 2b6318e.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: cdeust

mcp_server/core/wiki_readme.py

@@ -0,0 +1,211 @@
+"""Wiki README generation — plain-language top-level entry point.
+
+The wiki's technical content lives in `<kind>/<domain>/<slug>.md` files
+with templated front-matter + section structure — tech-ready, but dense.
+This module generates a top-level ``README.md`` that is readable by
+non-technical stakeholders:
+
+  * What the wiki IS (one paragraph, plain language).
+  * What lives WHERE (kind-labelled sections with a 1-line "what it's
+    for" summary, not "architecture decision record" jargon).
+  * How to NAVIGATE (auto-generated table of contents + link to the
+    detailed technical INDEX.md).
+  * When it was last GROOMED (builds trust: "this is current").
+
+Design principle: non-tech readers see plain language at the top;
+tech readers follow links down to the structured INDEX + per-page
+templates. No information is hidden from either audience — just
+presented at the right depth for each click.
+
+Source: user directive "wiki generation, folder and file management,
+keep this tidy, in order, readable by non tech while having all
+information needed for tech people".
+"""
+
+from __future__ import annotations
+
+from collections import defaultdict
+from datetime import datetime, timezone
+from pathlib import PurePosixPath
+
+from mcp_server.core.wiki_layout import PAGE_KINDS
+
+# Non-tech label + one-line description per kind. The description is
+# what a first-time reader needs to know to decide "do I click here?".
+_KIND_PLAIN: dict[str, tuple[str, str]] = {
+    "adr": (
+        "Architecture Decisions",
+        "Why we chose X over Y — the reasoning behind major design "
+        "choices. Read these to understand WHY the system looks the "
+        "way it does.",
+    ),
+    "specs": (
+        "Specifications & Designs",
+        "What we plan to build before we build it — feature specs, "
+        "design docs, PRDs. Read these to understand WHAT is coming.",
+    ),
+    "guides": (
+        "Guides & How-To",
+        "Step-by-step instructions for common tasks. Read these when "
+        "you want to DO something.",
+    ),
+    "reference": (
+        "Reference",
+        "API signatures, configuration keys, protocol formats. Read "
+        "these to LOOK UP an exact detail.",
+    ),
+    "conventions": (
+        "Conventions",
+        "The rules the team follows (naming, style, contribution "
+        "process). Read these before proposing changes.",
+    ),
+    "lessons": (
+        "Lessons Learned",
+        "Mistakes, root causes, and rules we now follow to avoid "
+        "repeating them. Read these to learn from past incidents.",
+    ),
+    "notes": (
+        "Notes & Investigations",
+        "Work-in-progress thinking, exploratory analyses. Read these "
+        "for context on ongoing work.",
+    ),
+    "journal": (
+        "Journal",
+        "Time-stamped entries of events, experiments, and sessions. "
+        "Read these for what happened and when.",
+    ),
+    "files": (
+        "File Documentation",
+        "Per-source-file documentation (auto-generated from code "
+        "analysis). Read these for a map of the codebase.",
+    ),
+}
+
+
+def _count_pages(page_paths: list[str]) -> dict[str, int]:
+    """Count pages per kind. Only kinds with >0 pages are returned."""
+    counts: dict[str, int] = defaultdict(int)
+    for p in page_paths:
+        first = PurePosixPath(p).parts[0] if p else ""
+        if first in PAGE_KINDS:
+            counts[first] += 1
+    return dict(counts)
+
+
+def _count_by_domain(page_paths: list[str]) -> dict[str, int]:
+    """Count pages per domain, excluding the catch-all ``_general``."""
+    counts: dict[str, int] = defaultdict(int)
+    for p in page_paths:
+        parts = PurePosixPath(p).parts
+        if len(parts) < 3:
+            continue  # root-level or missing domain
+        if parts[0] not in PAGE_KINDS:
+            continue
+        counts[parts[1]] += 1
+    return dict(counts)
+
+
+def build_plain_readme(
+    page_paths: list[str],
+    *,
+    project_name: str = "Cortex",
+    generated_at: datetime | None = None,
+) -> str:
+    """Generate the top-level plain-language README.md for the wiki.
+
+    Pure function — takes a list of wiki-relative page paths, returns
+    Markdown. Caller writes to ``<wiki_root>/README.md``.
+
+    The output is stable (same input → same output, modulo the
+    ``generated_at`` timestamp) so it's safe to write on every
+    reindex without churning the git log.
+    """
+    if generated_at is None:
+        generated_at = datetime.now(timezone.utc)
+
+    total = len([p for p in page_paths if p and not p.startswith(".generated/")])
+    kind_counts = _count_pages(page_paths)
+    domain_counts = _count_by_domain(page_paths)
+
+    lines: list[str] = []
+    lines.append(f"# {project_name} Wiki")
+    lines.append("")
+    lines.append(
+        "This is your project's **living knowledge base** — "
+        "decisions, plans, how-tos, and lessons, kept tidy automatically "
+        "as the work happens."
+    )
+    lines.append("")
+    lines.append(
+        f"There are currently **{total} page{'s' if total != 1 else ''}** "
+        f"across **{len(kind_counts)} categories**"
+        + (f" and **{len(domain_counts)} domains**." if domain_counts else ".")
+    )
+    lines.append("")
+    lines.append(f"_Last groomed: {generated_at.strftime('%Y-%m-%d %H:%M UTC')}._")
+    lines.append("")
+
+    # --- What's here, by category ---
+    lines.append("## What's here")
+    lines.append("")
+    for kind in PAGE_KINDS:
+        if kind_counts.get(kind, 0) == 0:
+            continue
+        label, description = _KIND_PLAIN[kind]
+        count = kind_counts[kind]
+        lines.append(f"### {label} ({count} page{'s' if count != 1 else ''})")
+        lines.append("")
+        lines.append(description)
+        lines.append("")
+        lines.append(f"→ Folder: [`{kind}/`](./{kind}/)")
+        lines.append("")
+
+    # --- Domains ---
+    if domain_counts:
+        lines.append("## Covered domains")
+        lines.append("")
+        lines.append(
+            "Each domain is a distinct area of the project. Pages are "
+            "filed under `<category>/<domain>/<page>.md`."
+        )
+        lines.append("")
+        for domain, count in sorted(domain_counts.items(), key=lambda x: (-x[1], x[0])):
+            lines.append(f"- **{domain}** — {count} page{'s' if count != 1 else ''}")
+        lines.append("")
+
+    # --- Navigation ---
+    lines.append("## Go deeper")
+    lines.append("")
+    lines.append(
+        "The full table of contents (grouped by domain and category) "
+        "lives at [`.generated/INDEX.md`](./.generated/INDEX.md). "
+        "It's rebuilt automatically on every wiki write."
+    )
+    lines.append("")
+    lines.append(
+        "Every page follows a consistent template per category — see "
+        "the [conventions folder](./conventions/) (if present) for the rules."
+    )
+    lines.append("")
+
+    # --- For tech readers ---
+    lines.append("## For contributors")
+    lines.append("")
+    lines.append(
+        "Pages are groomed by `cortex-wiki-groomer` (runs asynchronously "
+        "during `cortex:consolidate`). The groomer:"
+    )
+    lines.append("")
+    lines.append("- Preserves every paragraph you wrote (no information loss).")
+    lines.append("- Fills missing front-matter from context (or marks `unknown`).")
+    lines.append("- Enforces naming conventions (kebab-slugs, 4-digit ADR IDs).")
+    lines.append("- Skips any page whose front-matter declares `grooming: manual`.")
+    lines.append("")
+    lines.append(
+        "To opt a page out of grooming, add `grooming: manual` to its "
+        "front-matter. Nothing you write by hand will ever be rewritten "
+        "without your consent."
+    )
+    lines.append("")
+
+    return "\n".join(lines) + "\n"

mcp_server/infrastructure/wiki_store.py

@@ -248,15 +248,43 @@ def list_pages(root: Path | str, *, kind: str | None = None) -> list[str]:
 
 
 def _try_reindex(root: Path) -> None:
-    """Best-effort index rebuild after wiki write."""
+    """Best-effort index rebuild after wiki write.
+
+    Produces three artefacts:
+      * ``<root>/.generated/INDEX.md`` — structured TOC grouped by
+        domain then kind (for tech readers).
+      * ``<root>/README.md``           — plain-language top-level
+        entry point (for non-tech readers). Only created if no
+        hand-written README exists at the root.
+    """
     try:
         from mcp_server.core.wiki_pages import build_index
+        from mcp_server.core.wiki_readme import build_plain_readme
 
         page_paths = list_pages(root)
         index_md = build_index(page_paths)
         gen_dir = root / ".generated"
         gen_dir.mkdir(exist_ok=True)
         (gen_dir / "INDEX.md").write_text(index_md)
+
+        # README: only overwrite if it doesn't exist OR it's
+        # auto-generated (marker line in the body). Never clobber a
+        # hand-written README.
+        readme_path = root / "README.md"
+        auto_marker = "<!-- cortex-wiki-readme: auto-generated -->"
+        should_write = True
+        if readme_path.exists():
+            try:
+                existing = readme_path.read_text()
+                if auto_marker not in existing:
+                    should_write = False  # hand-written, don't touch
+            except Exception:
+                pass
+        if should_write:
+            readme_md = build_plain_readme(page_paths)
+            readme_md += f"\n{auto_marker}\n"
+            readme_path.write_text(readme_md)
+
         cleanup_id_prefixed_pages(root)
     except Exception:
         pass

tests_py/core/test_wiki_readme.py

@@ -0,0 +1,129 @@
+"""Tests for wiki_readme — plain-language top-level README generation.
+
+Source: user directive "readable by non tech while having all
+information needed for tech people".
+"""
+
+from __future__ import annotations
+
+from datetime import datetime, timezone
+
+import pytest
+
+from mcp_server.core.wiki_readme import (
+    _count_by_domain,
+    _count_pages,
+    build_plain_readme,
+)
+
+
+_FIXED_TIME = datetime(2026, 4, 17, 14, 30, tzinfo=timezone.utc)
+
+
+class TestCounts:
+    def test_count_pages_by_kind(self):
+        paths = [
+            "adr/0001-foo.md",
+            "adr/0002-bar.md",
+            "specs/phase5.md",
+            "notes/x.md",
+            "alien/wrong.md",  # unknown kind — excluded
+        ]
+        assert _count_pages(paths) == {"adr": 2, "specs": 1, "notes": 1}
+
+    def test_count_by_domain_skips_root_level(self):
+        paths = [
+            "adr/cortex/0001.md",
+            "adr/cortex/0002.md",
+            "specs/cortex/phase5.md",
+            "adr/flat.md",  # no domain in path
+            "notes/other-domain/x.md",
+        ]
+        counts = _count_by_domain(paths)
+        assert counts["cortex"] == 3
+        assert counts["other-domain"] == 1
+        assert "flat.md" not in counts
+
+
+class TestReadmeStructure:
+    def test_readme_has_top_heading(self):
+        out = build_plain_readme(["adr/0001-foo.md"], generated_at=_FIXED_TIME)
+        assert "# Cortex Wiki" in out
+
+    def test_custom_project_name(self):
+        out = build_plain_readme([], project_name="MyProject", generated_at=_FIXED_TIME)
+        assert "# MyProject Wiki" in out
+
+    def test_plain_language_intro(self):
+        """Non-tech readers see a living knowledge base pitch, not jargon."""
+        out = build_plain_readme(["adr/0001-foo.md"], generated_at=_FIXED_TIME)
+        assert "living knowledge base" in out
+        # No jargon gate-keepers
+        assert "architecture decision record" not in out.lower()
+        assert "immutable" not in out.lower()
+
+    def test_page_count_displayed(self):
+        paths = ["adr/0001-foo.md", "adr/0002-bar.md", "specs/x.md"]
+        out = build_plain_readme(paths, generated_at=_FIXED_TIME)
+        assert "3 pages" in out
+
+    def test_zero_pages_singular_grammar(self):
+        out = build_plain_readme([], generated_at=_FIXED_TIME)
+        assert "0 pages" in out
+
+    def test_one_page_singular_grammar(self):
+        out = build_plain_readme(["adr/0001-foo.md"], generated_at=_FIXED_TIME)
+        assert "1 page" in out
+        assert "1 pages" not in out
+
+    def test_timestamp_included(self):
+        out = build_plain_readme([], generated_at=_FIXED_TIME)
+        assert "2026-04-17 14:30 UTC" in out
+
+    def test_links_to_detailed_index(self):
+        out = build_plain_readme(["adr/0001-foo.md"], generated_at=_FIXED_TIME)
+        assert ".generated/INDEX.md" in out
+
+    def test_only_populated_kinds_rendered(self):
+        """A category with 0 pages does NOT appear in the README — we
+        don't want a wall of empty sections for a fresh wiki."""
+        out = build_plain_readme(["adr/0001-foo.md"], generated_at=_FIXED_TIME)
+        assert "Architecture Decisions" in out
+        assert "Lessons Learned" not in out  # no lessons pages
+        assert "File Documentation" not in out
+
+    def test_domains_section_only_when_domains_present(self):
+        # Flat paths (no domain) → no domains section
+        out = build_plain_readme(["adr/flat.md"], generated_at=_FIXED_TIME)
+        assert "Covered domains" not in out
+
+        # Domain-scoped paths → section appears
+        out = build_plain_readme(["adr/cortex/0001.md"], generated_at=_FIXED_TIME)
+        assert "Covered domains" in out
+        assert "cortex" in out
+
+
+class TestStability:
+    def test_same_input_same_output(self):
+        paths = ["adr/0001-foo.md", "specs/x.md"]
+        out1 = build_plain_readme(paths, generated_at=_FIXED_TIME)
+        out2 = build_plain_readme(paths, generated_at=_FIXED_TIME)
+        assert out1 == out2
+
+    def test_kind_order_deterministic(self):
+        """Kinds appear in PAGE_KINDS order, not dict-iteration order."""
+        # Paths shuffled — output should still put adr before specs.
+        paths = ["specs/x.md", "adr/0001-foo.md", "notes/y.md"]
+        out = build_plain_readme(paths, generated_at=_FIXED_TIME)
+        adr_pos = out.index("Architecture Decisions")
+        specs_pos = out.index("Specifications & Designs")
+        notes_pos = out.index("Notes & Investigations")
+        assert adr_pos < specs_pos < notes_pos
+
+
+class TestGroomerMention:
+    def test_readme_tells_users_about_manual_override(self):
+        """Tech readers need to know how to opt a page out of grooming."""
+        out = build_plain_readme([], generated_at=_FIXED_TIME)
+        assert "grooming: manual" in out
+        assert "without your consent" in out