Skip to content

Commit a5fe567

Browse files
committed
docs(readme): document validate / eval / history / rollback commands
Skill Factory section now lists the 4 quality-gate commands borrowed from Codex (validate) and Anthropic skill-creator (eval + iteration): openkb skill validate structural lint (frontmatter, sizes, refs) openkb skill eval trigger-accuracy test of the description openkb skill history list past iterations openkb skill rollback restore a previous iteration The slogan promised a 'digital expert' — these commands are what makes the output worth that label.
1 parent 6f70d02 commit a5fe567

1 file changed

Lines changed: 18 additions & 0 deletions

File tree

README.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -186,6 +186,9 @@ A "generator" reads from the compiled wiki and produces something usable: an ans
186186
| <code>openkb&nbsp;query&nbsp;"question"</code> | A grounded answer with citations (use `--save` to persist to `wiki/explorations/`) |
187187
| `openkb chat` | Interactive multi-turn session over the wiki (use `--resume`, `--list`, `--delete` to manage sessions) |
188188
| <code>openkb&nbsp;skill&nbsp;new&nbsp;&lt;name&gt;&nbsp;"&lt;intent&gt;"</code> | A redistributable Anthropic Skill at `<kb>/output/skills/<name>/` + auto-updated `marketplace.json` |
189+
| <code>openkb&nbsp;skill&nbsp;validate&nbsp;[name]</code> | Structural lint of compiled skills (frontmatter, file sizes, wikilinks, scripts/ stdlib check with `--strict`). Auto-runs at end of `skill new` |
190+
| <code>openkb&nbsp;skill&nbsp;eval&nbsp;&lt;name&gt;</code> | Trigger-accuracy evaluation — does the `description:` field actually fire? LLM generates eval prompts; grader LLM scores activation. `--save` persists the eval set |
191+
| <code>openkb&nbsp;skill&nbsp;history&nbsp;&lt;name&gt;</code> / <code>openkb&nbsp;skill&nbsp;rollback&nbsp;&lt;name&gt;</code> | Iteration workspace — every overwrite saves the previous version to `output/skills/<name>-workspace/iteration-N/` with a structural diff. Rollback restores any iteration |
189192

190193
### Query & Chat — ask the wiki
191194

@@ -256,6 +259,21 @@ npx skills@latest add <your-org>/<your-repo>
256259
[agent edits SKILL.md frontmatter in place]
257260
```
258261

262+
**Quality gates** — borrowing from [Codex skill-creator](https://github.com/openai/skills) (structural validation) and [Anthropic skill-creator](https://github.com/anthropics/skills/tree/main/skills/skill-creator) (trigger-accuracy evals):
263+
264+
```bash
265+
# Lint structure (auto-runs at end of `skill new`)
266+
openkb skill validate karpathy-thinking
267+
openkb skill validate --strict # treat warnings as failures
268+
269+
# Does the description actually fire when it should?
270+
openkb skill eval karpathy-thinking --save
271+
272+
# History + rollback if a new iteration regresses
273+
openkb skill history karpathy-thinking
274+
openkb skill rollback karpathy-thinking --to 2
275+
```
276+
259277
See [CONTRIBUTING.md](CONTRIBUTING.md) for how to submit your compiled skill back to the community registry at [VectifyAI/OpenKB](https://github.com/VectifyAI/OpenKB).
260278

261279
### Configuration

0 commit comments

Comments
 (0)