Skip to content

Commit dbd78e6

Browse files
committed
docs(readme): restructure into Wiki Foundation + Generators layers
Reframe the project mental model: the wiki is the substrate, and several primitives (query, chat, skill new) generate output from it. Add 'Drop in a book. Out comes a digital expert.' slogan to the Skill Factory subsection. - Features list split into 'Wiki foundation' (compile + maintain) and 'Generators' (query / chat / Skill Factory) - Quick Start adds step 6 — distill a skill - Architecture diagram extended to show the wiki branching into query/chat + Skill Factory + future generators (ppt/podcast/…) - Usage section regrouped under 🧱 Wiki Foundation and ✨ Generators - Skill Factory subsection promoted with the slogan as its heading and a concrete example folder tree - Chat slash command list updated to include /skill new
1 parent 5d76d58 commit dbd78e6

1 file changed

Lines changed: 77 additions & 44 deletions

File tree

README.md

Lines changed: 77 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -24,16 +24,22 @@ Traditional RAG rediscovers knowledge from scratch on every query. Nothing accum
2424

2525
### Features
2626

27+
OpenKB has two layers: a **wiki foundation** that compiles and maintains your knowledge, and **generators** that turn that wiki into useful artifacts.
28+
29+
**Wiki foundation** — compile and maintain
2730
- **Broad format support** — PDF, Word, Markdown, PowerPoint, HTML, Excel, text, and more via markitdown
28-
- **Scale to long documents**Long and complex documents are handled via [PageIndex](https://github.com/VectifyAI/PageIndex) tree indexing, enabling accurate, vectorless long-context retrieval
31+
- **Scale to long documents**Handled via [PageIndex](https://github.com/VectifyAI/PageIndex) tree indexing, enabling accurate, vectorless long-context retrieval
2932
- **Native multi-modality** — Retrieves and understands figures, tables, and images, not just text
30-
- **Compiled Wiki** — LLM manages and compiles your documents into summaries, concept pages, and cross-links, all kept in sync
31-
- **Query** — Ask questions (one-off) against your wiki. The LLM navigates your compiled knowledge to answer
32-
- **Interactive Chat** — Multi-turn conversations with persisted sessions you can resume across runs
33+
- **Compiled Wiki** — LLM compiles documents into summaries, concept pages, and cross-links, all kept in sync
3334
- **Lint** — Health checks find contradictions, gaps, orphans, and stale content
3435
- **Watch mode** — Drop files into `raw/`, wiki updates automatically
3536
- **Obsidian compatible** — Wiki is plain `.md` files with `[[wikilinks]]`. Open in Obsidian for graph view and browsing
3637

38+
**Generators** — turn the wiki into something useful
39+
- **Query** — Ask questions (one-off) against your wiki. The LLM navigates compiled knowledge to answer
40+
- **Chat** — Multi-turn conversations with persisted sessions you can resume across runs
41+
- **Skill Factory***Drop in a book. Out comes a digital expert.* Compile any subset of your wiki into a redistributable [Anthropic Skill](https://docs.claude.com/en/docs/build-with-claude/skills) installable in Claude Code / Codex / Gemini CLI / Cursor
42+
3743
# 🚀 Getting Started
3844

3945
### Install
@@ -80,6 +86,9 @@ openkb query "What are the main findings?"
8086

8187
# 5. Or chat interactively
8288
openkb chat
89+
90+
# 6. Or distill your wiki into a redistributable skill
91+
openkb skill new my-expert "Reason like an expert on <topic-from-your-docs>"
8392
```
8493

8594
### Set up your LLM
@@ -109,7 +118,7 @@ raw/ You drop files here
109118
│ Wiki Compilation (using LLM)
110119
│ │
111120
▼ ▼
112-
wiki/
121+
wiki/ │ ← the foundation
113122
├── index.md Knowledge base overview
114123
├── log.md Operations timeline
115124
├── AGENTS.md Wiki schema (LLM instructions)
@@ -118,6 +127,13 @@ wiki/
118127
├── concepts/ Cross-document synthesis ← the good stuff
119128
├── explorations/ Saved query results
120129
└── reports/ Lint reports
130+
131+
┌──────────────────────┼──────────────────────┐
132+
▼ ▼ ▼
133+
query / chat Skill Factory (future)
134+
(LLM answers from openkb skill new ppt / podcast /
135+
the wiki) → output/skills/ report / …
136+
+ marketplace.json
121137
```
122138

123139
### Short vs. Long Document Handling
@@ -144,16 +160,15 @@ A single source might touch 10-15 wiki pages. Knowledge accumulates: each docume
144160

145161
# ⚙️ Usage
146162

147-
### Commands
163+
OpenKB commands fall into two layers: the **wiki foundation** (compile + manage your knowledge) and **generators** (turn that wiki into useful output).
164+
165+
## 🧱 Wiki Foundation — compile and maintain
148166

149167
| Command | Description |
150168
|---|---|
151169
| `openkb init` | Initialize a new knowledge base (interactive) |
152170
| <code>openkb&nbsp;add&nbsp;&lt;file_or_dir_or_URL&gt;</code> | Add documents and compile to wiki. URL ingest auto-detects PDF (saved as `.pdf` → PageIndex / markitdown) vs HTML (trafilatura main-content extract → `.md`) |
153-
| <code>openkb&nbsp;skill&nbsp;new&nbsp;&lt;name&gt;&nbsp;"&lt;intent&gt;"</code> | Compile a skill from this KB's wiki into `output/skills/<name>/` and update the marketplace manifest |
154171
| <code>openkb&nbsp;remove&nbsp;&lt;doc&gt;</code> | Remove a document and clean up its wiki pages, images, registry, and PageIndex state (use `--dry-run` to preview, `--keep-raw` / `--keep-empty-concepts` to retain artifacts) |
155-
| <code>openkb&nbsp;query&nbsp;"question"</code> | Ask a question over the knowledge base (use `--save` to save the answer to `wiki/explorations/`) |
156-
| `openkb chat` | Start an interactive multi-turn chat (use `--resume`, `--list`, `--delete` to manage sessions) |
157172
| `openkb watch` | Watch `raw/` and auto-compile new files |
158173
| `openkb lint` | Run structural + knowledge health checks |
159174
| `openkb list` | List indexed documents and concepts |
@@ -162,36 +177,77 @@ A single source might touch 10-15 wiki pages. Knowledge accumulates: each docume
162177

163178
<!-- | `openkb lint --fix` | Auto-fix what it can | -->
164179

165-
### Skills — compile your wiki into a redistributable skill
180+
## ✨ Generators — turn the wiki into output
181+
182+
A "generator" reads from the compiled wiki and produces something usable: an answer, a conversation, a skill folder. The wiki is the substrate; generators are the surfaces.
183+
184+
| Command | Output |
185+
|---|---|
186+
| <code>openkb&nbsp;query&nbsp;"question"</code> | A grounded answer with citations (use `--save` to persist to `wiki/explorations/`) |
187+
| `openkb chat` | Interactive multi-turn session over the wiki (use `--resume`, `--list`, `--delete` to manage sessions) |
188+
| <code>openkb&nbsp;skill&nbsp;new&nbsp;&lt;name&gt;&nbsp;"&lt;intent&gt;"</code> | A redistributable Anthropic Skill at `<kb>/output/skills/<name>/` + auto-updated `marketplace.json` |
189+
190+
### Query & Chat — ask the wiki
191+
192+
`openkb query "..."` answers a single question. `openkb chat` is interactive — each turn carries history, so you can dig into a topic without re-typing context. Both use the same underlying wiki and the same retrieval primitives (PageIndex for long docs, direct concept reads for short).
193+
194+
```bash
195+
openkb query "What does the literature say about attention scaling?"
196+
197+
openkb chat # start a new session
198+
openkb chat --resume # resume the most recent session
199+
openkb chat --resume 20260411 # resume by id (unique prefix works)
200+
openkb chat --list # list all sessions
201+
openkb chat --delete <id> # delete a session
202+
```
203+
204+
Inside a chat, type `/` to access slash commands (Tab to complete):
205+
206+
- `/help` — list available commands
207+
- `/status` — show knowledge base status
208+
- `/list` — list all documents
209+
- `/add <path>` — add a document or directory without leaving the chat
210+
- `/skill new <name> "<intent>"` — compile a skill from this chat (see below)
211+
- `/save [name]` — export the transcript to `wiki/explorations/`
212+
- `/clear` — start a fresh session (the current one stays on disk)
213+
- `/lint` — run knowledge base lint
214+
- `/exit` — exit (Ctrl-D also works)
215+
216+
### 🛠 Skill Factory — *Drop in a book. Out comes a digital expert.*
166217

167-
Once you have a populated wiki, you can compile a subset of it into an
168-
**Anthropic Skill** — a portable folder that Claude Code, Codex CLI,
169-
Gemini CLI, and Cursor all know how to load.
218+
The newest generator. `openkb skill new` distills any subset of your wiki into an [Anthropic Skill](https://docs.claude.com/en/docs/build-with-claude/skills) — a portable folder that **Claude Code, Codex CLI, Gemini CLI, and Cursor** all install and load natively. Drop in a book's worth of papers; out comes a specialist that other agents can call on.
170219

171220
```bash
172221
openkb skill new karpathy-thinking \
173222
"Reason about transformers and attention in Karpathy's style"
174223
```
175224

176-
This produces `output/skills/karpathy-thinking/` with `SKILL.md`,
177-
optional `references/`, and an auto-updated
178-
`.claude-plugin/marketplace.json` for distribution.
225+
This produces:
226+
227+
```
228+
<kb>/output/skills/karpathy-thinking/
229+
├── SKILL.md # YAML frontmatter + when-to-use + approach
230+
├── references/ # depth material the agent loads on demand
231+
│ ├── methodology.md
232+
│ └── key-quotes.md
233+
└── (scripts/) # optional, only if intent implies computation
234+
```
235+
236+
…plus an auto-updated `<kb>/.claude-plugin/marketplace.json` so the whole KB is one-line installable.
179237

180238
**Install locally:**
181239

182240
```bash
183241
cp -r output/skills/karpathy-thinking ~/.claude/skills/
184242
```
185243

186-
**Share with others:**
187-
188-
Push your KB directory to GitHub, then anyone can install all your skills with one command:
244+
**Share with others** — push your KB to GitHub, then anyone runs:
189245

190246
```bash
191247
npx skills@latest add <your-org>/<your-repo>
192248
```
193249

194-
You can also iterate inside chat:
250+
**Iterate from chat** — compilation is one-shot, but follow-up edits aren't. Inside `openkb chat`, you can refine without re-running the whole pipeline:
195251

196252
```
197253
/skill new karpathy-thinking "Reason about transformers like Karpathy"
@@ -200,30 +256,7 @@ You can also iterate inside chat:
200256
[agent edits SKILL.md frontmatter in place]
201257
```
202258

203-
See [CONTRIBUTING.md](CONTRIBUTING.md) for how to submit your compiled skill back to the OpenKB community registry.
204-
205-
### Interactive Chat
206-
207-
`openkb chat` opens an interactive chat session over your wiki knowledge base. Unlike the one-shot `openkb query`, each turn carries the conversation history, so you can dig into a topic without re-typing context.
208-
209-
```bash
210-
openkb chat # start a new session
211-
openkb chat --resume # resume the most recent session
212-
openkb chat --resume 20260411 # resume by id (unique prefix works)
213-
openkb chat --list # list all sessions
214-
openkb chat --delete <id> # delete a session
215-
```
216-
217-
Inside a chat, type `/` to access slash commands (Tab to complete):
218-
219-
- `/help` — list available commands
220-
- `/status` — show knowledge base status
221-
- `/list` — list all documents
222-
- `/add <path>` — add a document or directory without leaving the chat
223-
- `/save [name]` — export the transcript to `wiki/explorations/`
224-
- `/clear` — start a fresh session (the current one stays on disk)
225-
- `/lint` — run knowledge base lint
226-
- `/exit` — exit (Ctrl-D also works)
259+
See [CONTRIBUTING.md](CONTRIBUTING.md) for how to submit your compiled skill back to the community registry at [VectifyAI/OpenKB](https://github.com/VectifyAI/OpenKB).
227260

228261
### Configuration
229262

0 commit comments

Comments
 (0)