feat(plugins): manual-pages flow — manpage seed schema, flow docs, verification fixes (#971)

Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

Author: phernandez

README.md

@@ -590,7 +590,7 @@ retention).
 | `BASIC_MEMORY_IMPORT_UPLOAD_MAX_BYTES` | `104857600` | Max uploaded import size |
 
 ```bash
-BASIC_MEMORY_LOG_LEVEL=DEBUG basic-memory sync
+BASIC_MEMORY_LOG_LEVEL=DEBUG basic-memory reindex
 tail -f ~/.basic-memory/basic-memory.log
 ```
 

docs/Docker.md

@@ -111,7 +111,7 @@ You can run Basic Memory CLI commands inside the container using `docker exec`:
 docker exec basic-memory-server basic-memory status
 
 # Sync files
-docker exec basic-memory-server basic-memory sync
+docker exec basic-memory-server basic-memory reindex
 
 # Show help
 docker exec basic-memory-server basic-memory --help
@@ -137,7 +137,7 @@ When using Docker volumes, you'll need to configure projects to point to your mo
 
 3. **Sync the new project:**
    ```bash
-   docker exec basic-memory-server basic-memory sync
+   docker exec basic-memory-server basic-memory reindex
    ```
 
 ### Example: Setting up an Obsidian Vault
@@ -157,7 +157,7 @@ docker exec basic-memory-server basic-memory project create obsidian /app/data
 docker exec basic-memory-server basic-memory project set-default obsidian
 
 # Sync to index all files
-docker exec basic-memory-server basic-memory sync
+docker exec basic-memory-server basic-memory reindex
 ```
 
 ### Environment Variables

docs/character-handling.md

@@ -184,8 +184,8 @@ finance/           (lowercase f)
 Use Basic Memory's built-in conflict detection:
 
 ```bash
-# Sync will report conflicts
-basic-memory sync
+# Index local file changes (conflicts are handled during the scan)
+basic-memory reindex
 
 # Check sync status for warnings
 basic-memory status

docs/manual-pages.md

@@ -0,0 +1,183 @@
+# Manual Pages
+
+Basic Memory's manual is written in the style of Unix man pages — and
+implemented as Basic Memory notes ([#952](https://github.com/basicmachines-co/basic-memory/issues/952)).
+Every page is a markdown note conforming to the `Manpage` schema, `SEE ALSO`
+entries are real knowledge-graph relations, and every example on every page
+was executed against a live project before the page shipped. The manual
+documents the tools; the tools verify the manual.
+
+## Where it lives
+
+The canonical manual is the **`manual` project in the Basic Memory team
+workspace** (cloud, shared). Anyone can build their own: the schema ships as
+an opt-in seed at `plugins/claude-code/schemas/manpage.md` — copy it into any
+project's folder and start writing pages against it.
+
+Layout:
+
+```
+manual/
+├── schemas/Manpage.md      # the manpage schema (type: schema)
+├── man1/                   # CLI commands        bm(1), bm-status(1), ...
+├── man3/                   # MCP tools           write-note(3), search-notes(3), ...
+├── man5/                   # file formats        bm-note(5), bm-observation(5), ...
+├── man7/                   # concepts            basic-memory(7), semantic-memory(7), ...
+├── playground/             # scratch notes for destructive examples
+└── diagrams/               # canvas visualizations of the manual graph
+```
+
+### Why "man1", "man3", "man5"?
+
+The folder names are Unix's, unchanged since 1971. The manual is divided
+into numbered **sections**, pages physically live in directories named
+after them (`/usr/share/man/man1`, `man5`, ...), and the number tells you
+what *kind* of thing is documented — not importance, not reading order:
+
+- **1** — user commands (`ls`, `grep`)
+- **2** — system calls
+- **3** — library functions / APIs (`printf(3)`)
+- **4** — devices
+- **5** — file formats and config files (`crontab(5)`, `passwd(5)`)
+- **6** — games (really)
+- **7** — miscellanea: concepts, conventions, overviews (`regex(7)`, `signal(7)`)
+- **8** — system administration
+
+That's also why man page names carry the parenthesized number —
+`crontab(1)` is the command, `crontab(5)` is the file format, same name in
+two sections. `man 5 crontab` picks the section explicitly.
+
+This manual copies that layout with the sections that have a Basic Memory
+analog:
+
+- **man1/** — `bm` CLI commands → `bm-status(1)`
+- **man3/** — MCP tools, our equivalent of the "library API" section → `write-note(3)`
+- **man5/** — file formats: note syntax, observations, relations, schemas → `bm-note(5)`
+- **man7/** — concepts → `basic-memory(7)`, `semantic-memory(7)`
+- **8** is reserved for admin/cloud operations but has no pages yet; 2, 4,
+  and 6 have no analog (no system calls, no devices, and no games — yet)
+
+When a page says `see_also [[bm-note(5)]]`, the `(5)` reads "the
+file-format page," exactly the way a Unix manual cross-references — except
+here it's a traversable relation in the graph instead of a typographic
+convention. The manual explains its own conventions in `man-pages(7)` —
+fittingly, the same page name Linux uses for this, and that almost nobody
+ever reads.
+
+## Page anatomy
+
+Pages use the classic headers where applicable: `NAME`, `SYNOPSIS`,
+`DESCRIPTION`, `PARAMETERS`, `MCP USAGE`, `CLI EQUIVALENT`, `EXAMPLES`,
+`GOTCHAS`, `SEE ALSO`. Frontmatter (validated by the schema):
+
+```yaml
+type: manpage
+section: 3                      # 1 | 3 | 5 | 7 | 8
+name: write-note                # page name without section suffix
+summary: create or overwrite a markdown note in the knowledge base
+generated: hand                 # hand | registry | typer  (regeneration ownership)
+tool: write_note                # section-3 pages: the MCP tool documented
+command: basic-memory status    # section-1 pages: the CLI command documented
+verified: 0.21.6 mcp+cli        # version + path(s) that proved the page
+```
+
+Field knowledge accumulates as observations — `[gotcha]`, `[bug]` (with issue
+links), `[pattern]` — and `SEE ALSO` entries are `see_also` relations, so the
+manual is a navigable graph, not a folder of files.
+
+## How to use it
+
+Man-style reads (any MCP client or the CLI):
+
+```bash
+# read a page
+bm tool read-note "man3/write-note-3" --project manual
+
+# apropos — find pages by section, tool, or text
+bm tool search-notes --project manual          # then filter, or via MCP:
+#   search_notes(project="manual", metadata_filters={"type": "manpage", "section": 3})
+#   search_notes(project="manual", metadata_filters={"type": "manpage", "tool": "write_note"})
+
+# traverse SEE ALSO from any page
+#   build_context(url="man3/write-note-3", project="manual")
+```
+
+A future `bm man <topic>` command is thin sugar over exactly these calls.
+
+And for the real thing — `man bm` in an actual terminal:
+
+```bash
+bm man install        # copies bundled groff pages to ~/.local/share/man
+man bm                # the overview page, rendered by man(1)
+man basic-memory      # same page via its alias
+```
+
+`bm man install` warns with a one-line `MANPATH` fix if the install root
+isn't searched by your `man`. Agents with shell access can use `man bm` as
+an offline quick reference; the full per-tool detail stays in the manual
+project's section-3 pages.
+
+## The verification discipline
+
+Two rules make the manual trustworthy:
+
+1. **Examples must have run.** An `EXAMPLES` (or `MCP USAGE` / `CLI
+   EQUIVALENT`) block contains only commands that actually executed against
+   the manual project. Destructive operations (`delete_note`, `move_note`,
+   destructive `edit_note`) run only against `playground/` notes — never
+   against pages. The `verified:` field records the version and which path
+   proved the page: `mcp` (live service), `cli` (dev checkout), or both.
+
+2. **The schema is the linter.** Validate the whole manual any time:
+
+   ```bash
+   bm tool schema-validate manpage --project manual
+   # → {"total_notes": 38, "valid_count": 38, "warning_count": 0, ...}
+   ```
+
+   `bm orphans --project manual` confirms every page is connected to the
+   graph, and `schema_diff`/`schema_infer` report drift between the schema
+   and how pages are actually written.
+
+Because verification exercises real tool calls against the live service,
+building the manual doubles as an end-to-end smoke test. The initial build
+found six bugs in one pass (#954–#959) — including the verification rule
+catching a test that asserted a bug as expected output (#958).
+
+## Adding or updating a page
+
+1. Run the commands you intend to document; keep the actual output.
+2. Write the page with `write_note`, passing frontmatter through the
+   `metadata` parameter (nested YAML in content frontmatter is unreliable on
+   some clients):
+
+   ```
+   write_note(title="my-tool(3)", directory="man3", project="manual",
+              note_type="manpage",
+              metadata={"section": 3, "name": "my-tool",
+                        "summary": "...", "generated": "hand",
+                        "tool": "my_tool", "verified": "<version> mcp"})
+   ```
+
+3. Link related pages in `SEE ALSO` with `see_also [[other-page(3)]]`.
+   Forward references to pages that don't exist yet are fine — they resolve
+   automatically when the target is written.
+4. Validate: `bm tool schema-validate manpage --project manual`.
+
+For mechanical updates to generated sections, prefer `edit_note` with
+`replace_section` / `insert_after_section` so curated content (EXAMPLES,
+GOTCHAS, SEE ALSO, observations) survives — that ownership split is what the
+`generated:` field declares.
+
+## Roadmap
+
+- **Registry generator** — section-3 SYNOPSIS/PARAMETERS generated from the
+  MCP tool registry (docstrings + pydantic schemas), section-1 from Typer
+  help; the hand-written corpus is the template spec. Regenerate-and-diff in
+  CI becomes the drift gate.
+- **`bm man <topic>`** — CLI sugar over `read_note` + metadata search.
+  (`bm man install` + a hand-written `bm.1` already ship — the first slice
+  of [#610](https://github.com/basicmachines-co/basic-memory/issues/610);
+  the generator will produce per-command pages from the same extraction.)
+- **Docs site** — the notes remain canonical for sections 5 and 7, code is
+  canonical for 1 and 3; both render to the hosted docs site.

plugins/claude-code/schemas/manpage.md

@@ -0,0 +1,55 @@
+---
+title: Manpage
+type: schema
+entity: Manpage
+version: 1
+schema:
+  gotcha?(array): string, sharp edges and surprising behavior learned from live verification
+  example?(array): string, worked examples beyond the generated synopsis
+  pattern?(array): string, recommended idioms and usage patterns
+  bug?(array): string, known defects affecting this surface, with issue links
+  see_also?(array): Entity, related manual pages — the SEE ALSO graph
+settings:
+  validation: warn
+  frontmatter:
+    section(enum, Unix manual section number): [1, 3, 5, 7, 8]
+    name: string, page name without section suffix (e.g. write-note)
+    summary: string, one-line NAME description
+    generated?(enum, who owns the mechanical sections): [registry, typer, hand]
+    tool?: string, MCP tool this page documents (section 3 pages)
+    command?: string, CLI command this page documents (section 1 pages)
+    verified?: string, version and path that verified this page (e.g. 0.21.6 mcp+cli)
+    since?: string, version this surface first appeared
+---
+
+# Manpage
+
+A **ManpageNote** is one page of a Unix-style manual implemented as Basic
+Memory notes (issue #952): commands in section 1, MCP tools in section 3,
+file formats in section 5, concepts in section 7, admin in section 8. The
+manual becomes a knowledge graph — `SEE ALSO` entries are typed relations,
+and pages are found by structured recall:
+`search_notes(metadata_filters={"type": "manpage", "section": 3})`.
+
+This schema is an opt-in seed for documentation projects; the canonical
+manual lives in the Basic Memory team workspace `manual` project.
+
+## What makes a good ManpageNote
+
+- **NAME / SYNOPSIS / DESCRIPTION** — classic man-page structure, with
+  PARAMETERS, MCP USAGE, CLI EQUIVALENT, EXAMPLES, GOTCHAS, SEE ALSO where
+  applicable.
+- **Verified examples** — EXAMPLES contain only commands that actually ran;
+  the `verified` field records the version and path (mcp, cli, or both).
+- **generated** — declares regeneration ownership: `registry` (from the MCP
+  tool registry) and `typer` (from CLI help) pages get mechanical sections
+  rewritten; curated sections (EXAMPLES, GOTCHAS, SEE ALSO, observations)
+  are never overwritten.
+- **gotcha / bug observations** — field knowledge accumulates on pages
+  without being clobbered by regeneration; bugs link their tracking issues.
+
+## Frontmatter
+
+`type: manpage` plus `section` makes the manual queryable like `man -k`:
+by section, by `tool`, by `command`, or by missing/stale `verified` stamps.
+Validation is `warn`, never blocking.

src/basic_memory/cli/commands/__init__.py

@@ -4,6 +4,7 @@
 from . import (
     import_claude_projects,
     import_chatgpt,
+    man,
     tool,
     project,
     format,
@@ -29,4 +30,5 @@
     "schema",
     "update",
     "workspace",
+    "man",
 ]

src/basic_memory/cli/commands/man.py

@@ -0,0 +1,76 @@
+"""Install the bundled man pages so `man bm` works."""
+
+import shutil
+import subprocess
+from pathlib import Path
+from typing import Annotated, Optional
+
+import typer
+from rich.console import Console
+
+from basic_memory.cli.app import app
+
+console = Console()
+
+man_app = typer.Typer(help="Manage the bm man pages.")
+app.add_typer(man_app, name="man")
+
+# Bundled groff sources ship inside the package (src/basic_memory/man).
+_MAN_SOURCE_DIR = Path(__file__).parent.parent.parent / "man"
+
+
+def _default_man_root() -> Path:
+    # Why ~/.local/share/man: manpath(1) derives man directories from PATH
+    # entries on both man-db (Linux) and BSD man (macOS), so ~/.local/bin on
+    # PATH — the pipx/uv tool layout — makes this root searchable without any
+    # MANPATH configuration.
+    return Path.home() / ".local" / "share" / "man"
+
+
+def _man_root_on_manpath(man_root: Path) -> Optional[bool]:
+    """Best-effort check whether man(1) will search man_root; None if unknown."""
+    try:
+        result = subprocess.run(["manpath"], capture_output=True, text=True, timeout=5)
+    except (FileNotFoundError, subprocess.TimeoutExpired):
+        return None
+    if result.returncode != 0:
+        return None
+    paths = [entry.rstrip("/") for entry in result.stdout.strip().split(":") if entry]
+    return str(man_root).rstrip("/") in paths
+
+
+@man_app.command()
+def install(
+    directory: Annotated[
+        Optional[Path],
+        typer.Option(
+            "--dir",
+            help="Man root to install into (default: ~/.local/share/man)",
+        ),
+    ] = None,
+) -> None:
+    """Install the bm man pages, then try `man bm`."""
+    man_root = (directory or _default_man_root()).expanduser()
+    man1 = man_root / "man1"
+    man1.mkdir(parents=True, exist_ok=True)
+
+    pages = sorted(_MAN_SOURCE_DIR.glob("*.1"))
+    if not pages:  # pragma: no cover - broken packaging, not a runtime state
+        console.print("[red]No bundled man pages found — broken installation[/red]")
+        raise typer.Exit(1)
+
+    for page in pages:
+        shutil.copyfile(page, man1 / page.name)
+        console.print(f"installed {man1 / page.name}")
+
+    # Trigger: the chosen root is provably absent from manpath output.
+    # Why: a silent install into an unsearched directory looks like success
+    #   but `man bm` still fails; say so and hand over the one-line fix.
+    # Outcome: actionable hint; unknown (None) stays quiet to avoid false alarms.
+    if _man_root_on_manpath(man_root) is False:
+        console.print(
+            f"\n[yellow]{man_root} is not on your manpath.[/yellow] Add it with:\n"
+            f'  export MANPATH="{man_root}:$MANPATH"'
+        )
+
+    console.print("\nTry: [bold]man bm[/bold]")

src/basic_memory/cli/main.py

@@ -24,6 +24,7 @@ def _version_only_invocation(argv: list[str]) -> bool:
         import_claude_conversations,
         import_claude_projects,
         import_memory_json,
+        man,
         mcp,
         orphans,
         project,

src/basic_memory/man/basic-memory.1

@@ -0,0 +1 @@
+.so man1/bm.1