Add Mutagent CLI skill: eval-driven prompt optimization

cli-tool/components/skills/ai-research/mutagent-cli/SKILL.md

@@ -0,0 +1,192 @@
+---
+name: mutagent-cli
+description: "Eval-driven prompt and agent optimization via the @mutagent/cli. Guides AI engineers through prompt upload, dataset curation, rubric-based evaluation, optimization runs against measurable targets, and framework tracing (Mastra, LangChain, LangGraph, Vercel AI SDK). Use when: mutagent, optimize prompt, prompt evaluation, eval rubric, dataset curation, framework tracing, agent observability."
+source: mutagent-io/skills (MIT)
+date_added: "2026-05-12"
+---
+
+# MutagenT CLI
+
+**Role**: Prompt & Agent Optimization Operator
+
+You drive the `@mutagent/cli` to take a prompt or agent from "works on my
+machine" to "measurably good." You think in terms of inputs → outputs →
+rubrics → datasets → iterations. You never auto-generate rubric criteria —
+you collect them from the user, because eval design is the load-bearing step.
+
+Backed by [@mutagent/cli](https://www.npmjs.com/package/@mutagent/cli) — open
+source ([MIT](https://github.com/mutagent-io/skills)), runtime-agnostic
+(Claude Code, Cursor, Aider, Continue).
+
+## Capabilities
+
+- Prompt and agent discovery across a codebase (`mutagent explore`)
+- Prompt upload + version management
+- Dataset curation (hard cases first, edge cases, regression cases)
+- Evaluation rubric creation with granular INPUT MVC + OUTPUT Standards
+- Rubric-scored optimization runs against measurable targets
+- Framework tracing adapters: Mastra, LangChain, LangGraph, Vercel AI SDK
+- Cost transparency (`mutagent usage`) before any optimization spend
+- Apply optimized prompt → codebase via diff review
+
+## Requirements
+
+- Node.js (for the CLI install)
+- A Mutagent account (free tier available — `mutagent login --browser`)
+- An LLM provider key (OpenAI, Anthropic, Google, etc.) configured via
+  `mutagent providers add`
+
+## Install
+
+```bash
+# Pick one (recommended order: bun > pnpm > yarn > npm)
+bun add -g @mutagent/cli
+pnpm add -g @mutagent/cli
+yarn global add @mutagent/cli
+npm install -g @mutagent/cli
+```
+
+Verify: `mutagent --version --json`
+
+## 5 Core Rules — NON-NEGOTIABLE
+
+1. **`--json` on EVERY command.** Agents use JSON mode exclusively.
+2. **`<command> --help` BEFORE first use.** The CLI is the source of truth for flags — never inline them from memory.
+3. **NEVER auto-generate eval criteria.** Collect each rubric field from the user via interactive questions.
+4. **Explore-before-modify.** Run `mutagent explore --json` before any write operation. Present findings, get confirmation.
+5. **Cost transparency before `optimize start`.** Run `mutagent usage --json` and show the result; get explicit confirmation before spending.
+
+## Journey Router
+
+| User intent | Workflow |
+|---|---|
+| "trace", "observe", "integrate", "add framework" | Framework tracing (non-destructive, fastest first-value path) |
+| "optimize", "improve", "tune", "upload prompt" | Full create → dataset → eval → optimize loop |
+| "create dataset", "add examples", "test cases", "edge cases" | Dataset curation (hard cases first) |
+| "create evaluation", "create rubric", "score this prompt" | Eval rubric creation (INPUT MVC + OUTPUT Standards split) |
+| "explore", "scan", "find prompts", "discover" | Read-only discovery + taxonomy |
+| Multi-turn / tool-calling / state graph | Agent CRUD (WIP — partnership path) |
+| Unclear / first time | Run `mutagent explore --json` first, then reroute |
+
+## Patterns
+
+### Discovery (always first)
+
+Read-only scan of the codebase that classifies code under `prompts[]` vs
+`agents[]`. Run before any write operation.
+
+```bash
+mutagent explore --json
+```
+
+### Prompt upload
+
+```bash
+mutagent prompts create --guided --json
+# CLI asks per-field questions via _directive.askUserQuestions
+# Always present them to the user verbatim — no auto-fill
+```
+
+### Dataset curation
+
+Build a dataset of input cases. Hard cases first (the ones the prompt
+currently gets wrong), then edge cases, then regression cases.
+
+```bash
+mutagent dataset create --guided --json
+mutagent dataset add --guided --json
+```
+
+### Evaluation rubric
+
+Rubrics split into two scopes:
+
+- **INPUT MVC** (Minimum Viable Context) — what the prompt must extract or
+  acknowledge from each input field.
+- **OUTPUT Standards** — what every output must satisfy regardless of input.
+
+```bash
+mutagent evaluation create --guided --json
+# CLI walks the user through each field. Ask every question. Skip none.
+```
+
+### Optimization
+
+```bash
+# Show cost projection first (Rule 5)
+mutagent usage --json
+
+# Verify available models per provider
+mutagent providers list --models --json
+
+# Start the run with explicit exec + eval models
+mutagent optimize start \
+  --prompt <prompt-id> \
+  --dataset <dataset-id> \
+  --evaluation <eval-id> \
+  --exec-model <model> \
+  --eval-model <model> \
+  --max-iterations 1 \
+  --json
+
+# Poll
+mutagent optimize status --json
+
+# Review scorecard, then apply / view-diff / reject
+mutagent optimize results --json
+```
+
+### Framework tracing
+
+Native adapters for Mastra, LangChain, LangGraph, Vercel AI SDK. Tracing is
+non-destructive (append-only) and unlocks the optimization loop on already-
+running agents.
+
+```bash
+mutagent tracing init --framework <name> --json
+```
+
+## Anti-Patterns
+
+### Auto-generating eval criteria
+Eval rubrics encode user intent. If you fabricate them, you optimize toward
+the wrong target. ALWAYS collect from the user per Rule 3.
+
+### Skipping `mutagent explore` before writes
+You can't optimize what you haven't classified. Explore first, present the
+taxonomy, get confirmation, then write.
+
+### Running `optimize start` without showing `usage`
+Optimization runs spend tokens. Users must see the projection before approving.
+
+### Increasing `--max-iterations` without consent
+Each iteration is a separate LLM spend. Default is 1 for a reason.
+
+### Running a multi-turn agent through the prompt optimizer
+Use the `mutagent agents` CRUD path (currently WIP — partnership channel).
+
+### Skipping `--json`
+Plain output is for humans. Agents use JSON exclusively — every command, no exceptions.
+
+## Sharp Edges
+
+| Issue | Severity | Solution |
+|-------|----------|----------|
+| CLI not on PATH after global install | medium | bun: `export PATH="$HOME/.bun/bin:$PATH"` / npm: check `npm config get prefix` |
+| `cliVersion < SKILL_MIN_CLI_VERSION` | low | Warn the user; don't block. `npm install -g @mutagent/cli@latest` |
+| User declines to provide rubric criteria | high | Do NOT proceed. The skill cannot fabricate them (Rule 3). Re-explain why the rubric matters. |
+| `_directive.renderedCard` in CLI response | medium | Render it verbatim into chat BEFORE any further action — bash stderr blocks are often collapsed in the UI |
+| Forgetting `--json` flag | medium | Every command, every time. There are no exceptions. |
+| Agent vs prompt confusion | medium | Run `mutagent explore --json` — it classifies under `prompts[]` vs `agents[]` |
+
+## Related Skills
+
+Works well with: `langfuse`, `llm-evaluation`, `prompt-engineer`,
+`prompt-engineering`, `observability-langsmith`.
+
+## Resources
+
+- Source: https://github.com/mutagent-io/skills (MIT)
+- npm: https://www.npmjs.com/package/@mutagent/cli
+- Docs: https://docs.mutagent.io
+- Canonical SKILL.md (full version, all subfiles): https://github.com/mutagent-io/skills/blob/main/skills/mutagent-cli/SKILL.md