Bring Nous-Wiki into Nous

[susiejojo]

Why

Nous campaigns cost $50-90 each and produce 30-55 research principles. Right now, that knowledge is stuck in a local .nous/ folder. The next campaign starts from zero — it doesn't know what already worked, what failed, or what's left to try.

This causes three concrete problems:

1. Wasted money. New campaigns re-test things that already failed. Each repeated dead-end costs ~$6.
2. Missed combinations. Campaign A proves X works. Campaign B proves Y works. Nobody notices X+Y was never tried together.
3. No map. After several campaigns, there's no way to see which parts of the system are well-understood and which are unexplored.

What We Want

A persistence layer that lets knowledge compound across campaigns. Five skills and four scripts that extract, store, retrieve, and visualize knowledge.

We have a working prototype tested against 3 real campaigns. Below is a strategy to merge this into main step-by-step without disturbing the core Nous codebase.

No existing code is modified. Everything is additive.

How the pieces fit together

[Campaign finishes]
        |
        v
/post-campaign        Reads ledger.json + principles.json from campaign
        |             Writes: dead-ends, frontiers, interactions, concepts,
        |                     summaries, summary.md, campaign viz HTML
        v
/index-wiki           Reads those files
        |             Writes: registry.json (cross-campaign index)
        v
 registry.json        The shared knowledge graph
        |
   +---------+
   |         |
   v         v
/suggest-next       /visualize-registry
  Reads registry      Reads registry
  + campaign files    Calls /suggest-next per cluster
  Writes: markdown    Writes: registry HTML
  recommendations     with opportunity scores

/visualize-campaign re-renders a single campaign's knowledge graph (calls the viz script on pre-existing data, doesn't extract anything).

/visualize-registry renders the full cross-campaign knowledge graph — runs /suggest-next once per entity cluster to compute opportunity scores, then produces an annotated HTML page showing the research landscape.

Deliverables

Deliverable	Type	What it does
Campaign viz script	Python script	Takes concepts.json + summaries.json → interactive HTML
Concepts validator	Python script	Checks a concepts.json graph for integrity errors
Registry viz script	Python script	Takes registry.json → cross-campaign interactive HTML
Wiki retrieval script	Python script	Builds entity-scoped knowledge subgraph for LLM context
/post-campaign	Claude Code skill	Extracts structured knowledge from a finished campaign
/index-wiki	Claude Code skill	Merges one campaign into the cross-campaign registry
/suggest-next	Claude Code skill	Recommends next experiments based on prior knowledge
/visualize-campaign	Claude Code skill	Re-renders a single campaign's viz
/visualize-registry	Claude Code skill	Renders full registry viz with opportunity scores

Where data lives

All wiki data goes to ~/.nous/wiki/ (user-level, never inside a repo):

~/.nous/wiki/
├── registry.json                    # The cross-campaign index
├── campaigns/<name>/
│   ├── concepts.json                # Entities, concepts, parameters
│   ├── summaries.json               # Per-iteration narratives
│   ├── principles.json              # Full principle definitions
│   ├── dead-ends.json               # What failed and why
│   ├── frontiers.json               # Where knowledge ends
│   ├── interactions.json            # Untested combinations
│   └── summary.md                   # Human-readable summary
├── suggestions/<date>-<slug>.md     # suggest-next outputs
└── viz/
    ├── <campaign-name>.html         # Per-campaign graphs
    └── registry.html                # Cross-campaign graph

---

The PRs

Four PRs, merged in order. Each depends on the one before it. Each PR is a complete feature slice — you can run it end-to-end and verify the result.

---

PR 1 — Post-Campaign: Extract + Visualize a Single Campaign

Delivers:

• scripts/visualize_campaign.py — generates interactive campaign HTML from JSON
• scripts/validate_concepts.py — validates concepts.json graph integrity
• .claude/commands/post-campaign.md — the extraction workflow
• .claude/commands/visualize-campaign.md — thin re-render wrapper

Why first: This is the producer. Everything downstream reads what this creates. It's fully self-contained: give it a finished campaign directory, it writes structured knowledge files and a visualization.

Verify: Run /post-campaign on any completed campaign directory.

Done when: All output files appear in ~/.nous/wiki/campaigns/<name>/ (dead-ends.json, frontiers.json, interactions.json, concepts.json, summaries.json, principles.json, summary.md) and the HTML viz is generated.

---

PR 2 — Index-Wiki: Build the Cross-Campaign Registry

Delivers:

• .claude/commands/index-wiki.md — merges one campaign into the registry

Why second: Reads what post-campaign produces. Writes registry.json — the cross-campaign index that everything else queries. This is the only skill that touches the registry.

Verify: Run /index-wiki <campaign-name> twice.

Done when: First run creates/updates ~/.nous/wiki/registry.json with entities, concepts, parameters, dead-ends, frontiers, interactions. Second run on the same campaign is a no-op (idempotent).

---

PR 3 — Suggest-Next: Recommend Experiments from Prior Knowledge

Delivers:

• scripts/retrieve_wiki_context.py — builds entity-scoped knowledge subgraph
• .claude/commands/suggest-next.md — recommends next experiments

Why third: Needs registry.json to exist (PR 2). The script reads the registry + per-campaign files to assemble relevant context. The skill uses that context to generate scored recommendations.

Verify: Run /suggest-next "some research intent".

Done when: Markdown file appears in ~/.nous/wiki/suggestions/ with 3 scored recommendations including per-axis breakdowns.

---

PR 4 — Registry Visualization: The Research Landscape

Delivers:

• scripts/visualize_registry.py — generates cross-campaign HTML from registry
• .claude/commands/visualize-registry.md — orchestrates cluster-level suggestions + viz

Why last: Depends on everything: registry (PR 2), suggest-next (PR 3), per-campaign data (PR 1). The skill calls suggest-next per entity cluster, then feeds results to the viz script.

Verify: Run /visualize-registry.

Done when: ~/.nous/wiki/viz/registry.html is produced with entity clusters and opportunity annotations.

---

After All 4 PRs Merge

The user workflow becomes:

nous run campaign.yaml            # Run campaign
/post-campaign .nous/<name>       # Extract + persist + visualize
/suggest-next "my next idea"      # Get recommendations based on all prior work
/visualize-registry               # See the full research landscape
nous run next-campaign.yaml       # Next campaign, informed by the past

[susiejojo]

Extension: Multi-User Registry Sharing (/sync-registry)

Problem

The wiki registry currently lives at ~/.nous/wiki/registry.json — a user-local file. In an org with 10-50+ engineers running campaigns against the same systems, knowledge stays siloed. User A discovers a dead-end that User B is about to repeat.

Design Constraints (from brainstorming)

• Org-wide (10-50+ users, multiple teams/projects)
• Cloud-native infrastructure available
• Read-all, write-own access model
• Offline-first with eventual sync
• Auto-merge (union) for conflicts — no human review needed
• Keep existing skills unchanged — /post-campaign, /index-wiki, /suggest-next continue reading/writing ~/.nous/wiki/ as-is

Approach: Git-Backed Shared Remote + Local Rebuild

Core idea: A shared git repo stores immutable campaign contribution files. Each user pushes their campaign data as individual files. Pulls fetch others' contributions and rebuild registry.json locally.

Why git:

• Offline-first is native (commit locally, push when connected)
• Concurrent pushes can't conflict (each contribution is its own file)
• Audit trail built-in (git log shows who contributed what)
• No new infrastructure beyond a git remote

Interface

One new skill: /sync-registry

/sync-registry push <project>       ← upload your campaigns for this project
/sync-registry pull <project>       ← download others' campaigns for this project
/sync-registry list-remote          ← see what projects/campaigns exist on the remote

Explicit project names always. No auto-detection magic.

Remote Structure

The shared git repo stores raw contribution files — no computed state:

shared-wiki-repo/
└── projects/
    └── <project-slug>/
        └── contributions/
            ├── alice--saturation-study.json
            ├── bob--latency-tuning.json
            └── carol--scaling-exp.json

Each contribution file contains the campaign data that /index-wiki currently reads from ~/.nous/wiki/campaigns/<name>/ (concepts, principles, dead-ends, frontiers, interactions).

Why per-file contributions: Two users pushing at the same time add different files — git handles this without conflict. No shared mutable state on the remote.

Project Identity

Replace repo_path (machine-specific absolute path) with git remote URL as the canonical project identifier. This is universal across users' machines:

• User A: /Users/alice/code/inference-sim → git@github.com:org/inference-sim
• User B: /home/bob/repos/inference-sim → git@github.com:org/inference-sim
• Same project.

The project slug (used in the shared repo structure) is derived from the remote URL.

What Happens After Pull

Pulling campaign files is step one. After new files land, a full registry rebuild runs:

1. Scan all local ~/.nous/wiki/campaigns/*/concepts.json
2. Deduplicate entities across all campaigns (normalized name matching)
3. Assign fresh global IDs (E-N, C-N, P-N — no collisions since built from scratch)
4. Build campaign entries (concepts, parameters, principles, dead-ends, frontiers, interactions)
5. Recompute entity_clusters (LLM call) only if entity set changed
6. Write registry.json

This is equivalent to /index-wiki --rebuild across all local campaign data. IDs are ephemeral local indexes, regenerated on every rebuild — nothing external references them.

Workflow

# User A finishes a campaign
/post-campaign .nous/my-campaign       # existing — writes locally
/sync-registry push inference-sim      # NEW — shares with org

# User B wants latest knowledge
/sync-registry pull inference-sim      # NEW — fetches others' campaigns + rebuilds registry
/suggest-next inference-sim "..."       # existing — now sees User A's findings

What Changes vs. What Stays the Same

Component	Changes?	Notes
/post-campaign	No	Still writes to ~/.nous/wiki/campaigns/<name>/
/index-wiki	Minor	Needs --rebuild mode (re-index all local campaigns)
/suggest-next	No	Still reads registry.json + campaign files locally
/visualize-campaign	No	Reads local files
/visualize-registry	No	Reads local registry.json
registry.json	Conceptual	Becomes a derived artifact (rebuilt from campaign files) rather than incrementally appended
retrieve_wiki_context.py	No	Still reads local campaign files
NEW: /sync-registry	—	The only component that knows about the shared remote

Discoverability

/sync-registry list-remote shows what projects and campaigns exist on the shared remote without pulling anything. Users can then decide what to pull.

Open Questions

1. Cluster recomputation cost: Entity clusters require an LLM call. Should this be skippable (with a --skip-clusters flag) for frequent pulls?
2. Campaign file format on remote: Store the full ~/.nous/wiki/campaigns/<name>/ directory as-is, or a single consolidated contribution file?
3. Auth: Git SSH keys? GitHub app tokens? Org-level shared credential?
4. Deletion/retraction: Can a user remove a campaign they previously pushed? (Event sourcing says no — append-only. But GDPR might say yes.)