You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs(readme): document validate / eval / history / rollback commands
Skill Factory section now lists the 4 quality-gate commands borrowed
from Codex (validate) and Anthropic skill-creator (eval + iteration):
openkb skill validate structural lint (frontmatter, sizes, refs)
openkb skill eval trigger-accuracy test of the description
openkb skill history list past iterations
openkb skill rollback restore a previous iteration
The slogan promised a 'digital expert' — these commands are what makes
the output worth that label.
Copy file name to clipboardExpand all lines: README.md
+18Lines changed: 18 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -186,6 +186,9 @@ A "generator" reads from the compiled wiki and produces something usable: an ans
186
186
| <code>openkb query "question"</code> | A grounded answer with citations (use `--save` to persist to `wiki/explorations/`) |
187
187
|`openkb chat`| Interactive multi-turn session over the wiki (use `--resume`, `--list`, `--delete` to manage sessions) |
188
188
| <code>openkb skill new <name> "<intent>"</code> | A redistributable Anthropic Skill at `<kb>/output/skills/<name>/` + auto-updated `marketplace.json`|
189
+
| <code>openkb skill validate [name]</code> | Structural lint of compiled skills (frontmatter, file sizes, wikilinks, scripts/ stdlib check with `--strict`). Auto-runs at end of `skill new`|
190
+
| <code>openkb skill eval <name></code> | Trigger-accuracy evaluation — does the `description:` field actually fire? LLM generates eval prompts; grader LLM scores activation. `--save` persists the eval set |
191
+
| <code>openkb skill history <name></code> / <code>openkb skill rollback <name></code> | Iteration workspace — every overwrite saves the previous version to `output/skills/<name>-workspace/iteration-N/` with a structural diff. Rollback restores any iteration |
**Quality gates** — borrowing from [Codex skill-creator](https://github.com/openai/skills) (structural validation) and [Anthropic skill-creator](https://github.com/anthropics/skills/tree/main/skills/skill-creator) (trigger-accuracy evals):
263
+
264
+
```bash
265
+
# Lint structure (auto-runs at end of `skill new`)
266
+
openkb skill validate karpathy-thinking
267
+
openkb skill validate --strict # treat warnings as failures
268
+
269
+
# Does the description actually fire when it should?
270
+
openkb skill eval karpathy-thinking --save
271
+
272
+
# History + rollback if a new iteration regresses
273
+
openkb skill history karpathy-thinking
274
+
openkb skill rollback karpathy-thinking --to 2
275
+
```
276
+
259
277
See [CONTRIBUTING.md](CONTRIBUTING.md) for how to submit your compiled skill back to the community registry at [VectifyAI/OpenKB](https://github.com/VectifyAI/OpenKB).
0 commit comments