You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs(readme): restructure into Wiki Foundation + Generators layers
Reframe the project mental model: the wiki is the substrate, and
several primitives (query, chat, skill new) generate output from it.
Add 'Drop in a book. Out comes a digital expert.' slogan to the
Skill Factory subsection.
- Features list split into 'Wiki foundation' (compile + maintain) and
'Generators' (query / chat / Skill Factory)
- Quick Start adds step 6 — distill a skill
- Architecture diagram extended to show the wiki branching into
query/chat + Skill Factory + future generators (ppt/podcast/…)
- Usage section regrouped under 🧱 Wiki Foundation and ✨ Generators
- Skill Factory subsection promoted with the slogan as its heading
and a concrete example folder tree
- Chat slash command list updated to include /skill new
Copy file name to clipboardExpand all lines: README.md
+77-44Lines changed: 77 additions & 44 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -24,16 +24,22 @@ Traditional RAG rediscovers knowledge from scratch on every query. Nothing accum
24
24
25
25
### Features
26
26
27
+
OpenKB has two layers: a **wiki foundation** that compiles and maintains your knowledge, and **generators** that turn that wiki into useful artifacts.
28
+
29
+
**Wiki foundation** — compile and maintain
27
30
-**Broad format support** — PDF, Word, Markdown, PowerPoint, HTML, Excel, text, and more via markitdown
28
-
-**Scale to long documents** — Long and complex documents are handled via [PageIndex](https://github.com/VectifyAI/PageIndex) tree indexing, enabling accurate, vectorless long-context retrieval
31
+
-**Scale to long documents** — Handled via [PageIndex](https://github.com/VectifyAI/PageIndex) tree indexing, enabling accurate, vectorless long-context retrieval
29
32
-**Native multi-modality** — Retrieves and understands figures, tables, and images, not just text
30
-
-**Compiled Wiki** — LLM manages and compiles your documents into summaries, concept pages, and cross-links, all kept in sync
31
-
-**Query** — Ask questions (one-off) against your wiki. The LLM navigates your compiled knowledge to answer
32
-
-**Interactive Chat** — Multi-turn conversations with persisted sessions you can resume across runs
33
+
-**Compiled Wiki** — LLM compiles documents into summaries, concept pages, and cross-links, all kept in sync
33
34
-**Lint** — Health checks find contradictions, gaps, orphans, and stale content
34
35
-**Watch mode** — Drop files into `raw/`, wiki updates automatically
35
36
-**Obsidian compatible** — Wiki is plain `.md` files with `[[wikilinks]]`. Open in Obsidian for graph view and browsing
36
37
38
+
**Generators** — turn the wiki into something useful
39
+
-**Query** — Ask questions (one-off) against your wiki. The LLM navigates compiled knowledge to answer
40
+
-**Chat** — Multi-turn conversations with persisted sessions you can resume across runs
41
+
-**Skill Factory** — *Drop in a book. Out comes a digital expert.* Compile any subset of your wiki into a redistributable [Anthropic Skill](https://docs.claude.com/en/docs/build-with-claude/skills) installable in Claude Code / Codex / Gemini CLI / Cursor
42
+
37
43
# 🚀 Getting Started
38
44
39
45
### Install
@@ -80,6 +86,9 @@ openkb query "What are the main findings?"
80
86
81
87
# 5. Or chat interactively
82
88
openkb chat
89
+
90
+
# 6. Or distill your wiki into a redistributable skill
91
+
openkb skill new my-expert "Reason like an expert on <topic-from-your-docs>"
83
92
```
84
93
85
94
### Set up your LLM
@@ -109,7 +118,7 @@ raw/ You drop files here
109
118
│ Wiki Compilation (using LLM)
110
119
│ │
111
120
▼ ▼
112
-
wiki/
121
+
wiki/ │ ← the foundation
113
122
├── index.md Knowledge base overview
114
123
├── log.md Operations timeline
115
124
├── AGENTS.md Wiki schema (LLM instructions)
@@ -118,6 +127,13 @@ wiki/
118
127
├── concepts/ Cross-document synthesis ← the good stuff
119
128
├── explorations/ Saved query results
120
129
└── reports/ Lint reports
130
+
│
131
+
┌──────────────────────┼──────────────────────┐
132
+
▼ ▼ ▼
133
+
query / chat Skill Factory (future)
134
+
(LLM answers from openkb skill new ppt / podcast /
135
+
the wiki) → output/skills/ report / …
136
+
+ marketplace.json
121
137
```
122
138
123
139
### Short vs. Long Document Handling
@@ -144,16 +160,15 @@ A single source might touch 10-15 wiki pages. Knowledge accumulates: each docume
144
160
145
161
# ⚙️ Usage
146
162
147
-
### Commands
163
+
OpenKB commands fall into two layers: the **wiki foundation** (compile + manage your knowledge) and **generators** (turn that wiki into useful output).
164
+
165
+
## 🧱 Wiki Foundation — compile and maintain
148
166
149
167
| Command | Description |
150
168
|---|---|
151
169
|`openkb init`| Initialize a new knowledge base (interactive) |
152
170
| <code>openkb add <file_or_dir_or_URL></code> | Add documents and compile to wiki. URL ingest auto-detects PDF (saved as `.pdf` → PageIndex / markitdown) vs HTML (trafilatura main-content extract → `.md`) |
153
-
| <code>openkb skill new <name> "<intent>"</code> | Compile a skill from this KB's wiki into `output/skills/<name>/` and update the marketplace manifest |
154
171
| <code>openkb remove <doc></code> | Remove a document and clean up its wiki pages, images, registry, and PageIndex state (use `--dry-run` to preview, `--keep-raw` / `--keep-empty-concepts` to retain artifacts) |
155
-
| <code>openkb query "question"</code> | Ask a question over the knowledge base (use `--save` to save the answer to `wiki/explorations/`) |
156
-
|`openkb chat`| Start an interactive multi-turn chat (use `--resume`, `--list`, `--delete` to manage sessions) |
157
172
|`openkb watch`| Watch `raw/` and auto-compile new files |
158
173
|`openkb lint`| Run structural + knowledge health checks |
159
174
|`openkb list`| List indexed documents and concepts |
@@ -162,36 +177,77 @@ A single source might touch 10-15 wiki pages. Knowledge accumulates: each docume
162
177
163
178
<!-- | `openkb lint --fix` | Auto-fix what it can | -->
164
179
165
-
### Skills — compile your wiki into a redistributable skill
180
+
## ✨ Generators — turn the wiki into output
181
+
182
+
A "generator" reads from the compiled wiki and produces something usable: an answer, a conversation, a skill folder. The wiki is the substrate; generators are the surfaces.
183
+
184
+
| Command | Output |
185
+
|---|---|
186
+
| <code>openkb query "question"</code> | A grounded answer with citations (use `--save` to persist to `wiki/explorations/`) |
187
+
|`openkb chat`| Interactive multi-turn session over the wiki (use `--resume`, `--list`, `--delete` to manage sessions) |
188
+
| <code>openkb skill new <name> "<intent>"</code> | A redistributable Anthropic Skill at `<kb>/output/skills/<name>/` + auto-updated `marketplace.json`|
189
+
190
+
### Query & Chat — ask the wiki
191
+
192
+
`openkb query "..."` answers a single question. `openkb chat` is interactive — each turn carries history, so you can dig into a topic without re-typing context. Both use the same underlying wiki and the same retrieval primitives (PageIndex for long docs, direct concept reads for short).
193
+
194
+
```bash
195
+
openkb query "What does the literature say about attention scaling?"
196
+
197
+
openkb chat # start a new session
198
+
openkb chat --resume # resume the most recent session
199
+
openkb chat --resume 20260411 # resume by id (unique prefix works)
200
+
openkb chat --list # list all sessions
201
+
openkb chat --delete <id># delete a session
202
+
```
203
+
204
+
Inside a chat, type `/` to access slash commands (Tab to complete):
205
+
206
+
-`/help` — list available commands
207
+
-`/status` — show knowledge base status
208
+
-`/list` — list all documents
209
+
-`/add <path>` — add a document or directory without leaving the chat
210
+
-`/skill new <name> "<intent>"` — compile a skill from this chat (see below)
211
+
-`/save [name]` — export the transcript to `wiki/explorations/`
212
+
-`/clear` — start a fresh session (the current one stays on disk)
213
+
-`/lint` — run knowledge base lint
214
+
-`/exit` — exit (Ctrl-D also works)
215
+
216
+
### 🛠 Skill Factory — *Drop in a book. Out comes a digital expert.*
166
217
167
-
Once you have a populated wiki, you can compile a subset of it into an
168
-
**Anthropic Skill** — a portable folder that Claude Code, Codex CLI,
169
-
Gemini CLI, and Cursor all know how to load.
218
+
The newest generator. `openkb skill new` distills any subset of your wiki into an [Anthropic Skill](https://docs.claude.com/en/docs/build-with-claude/skills) — a portable folder that **Claude Code, Codex CLI, Gemini CLI, and Cursor** all install and load natively. Drop in a book's worth of papers; out comes a specialist that other agents can call on.
170
219
171
220
```bash
172
221
openkb skill new karpathy-thinking \
173
222
"Reason about transformers and attention in Karpathy's style"
174
223
```
175
224
176
-
This produces `output/skills/karpathy-thinking/` with `SKILL.md`,
177
-
optional `references/`, and an auto-updated
178
-
`.claude-plugin/marketplace.json` for distribution.
Push your KB directory to GitHub, then anyone can install all your skills with one command:
244
+
**Share with others** — push your KB to GitHub, then anyone runs:
189
245
190
246
```bash
191
247
npx skills@latest add <your-org>/<your-repo>
192
248
```
193
249
194
-
You can also iterate inside chat:
250
+
**Iterate from chat** — compilation is one-shot, but follow-up edits aren't. Inside `openkb chat`, you can refine without re-running the whole pipeline:
195
251
196
252
```
197
253
/skill new karpathy-thinking "Reason about transformers like Karpathy"
@@ -200,30 +256,7 @@ You can also iterate inside chat:
200
256
[agent edits SKILL.md frontmatter in place]
201
257
```
202
258
203
-
See [CONTRIBUTING.md](CONTRIBUTING.md) for how to submit your compiled skill back to the OpenKB community registry.
204
-
205
-
### Interactive Chat
206
-
207
-
`openkb chat` opens an interactive chat session over your wiki knowledge base. Unlike the one-shot `openkb query`, each turn carries the conversation history, so you can dig into a topic without re-typing context.
208
-
209
-
```bash
210
-
openkb chat # start a new session
211
-
openkb chat --resume # resume the most recent session
212
-
openkb chat --resume 20260411 # resume by id (unique prefix works)
213
-
openkb chat --list # list all sessions
214
-
openkb chat --delete <id># delete a session
215
-
```
216
-
217
-
Inside a chat, type `/` to access slash commands (Tab to complete):
218
-
219
-
-`/help` — list available commands
220
-
-`/status` — show knowledge base status
221
-
-`/list` — list all documents
222
-
-`/add <path>` — add a document or directory without leaving the chat
223
-
-`/save [name]` — export the transcript to `wiki/explorations/`
224
-
-`/clear` — start a fresh session (the current one stays on disk)
225
-
-`/lint` — run knowledge base lint
226
-
-`/exit` — exit (Ctrl-D also works)
259
+
See [CONTRIBUTING.md](CONTRIBUTING.md) for how to submit your compiled skill back to the community registry at [VectifyAI/OpenKB](https://github.com/VectifyAI/OpenKB).
0 commit comments