|
| 1 | +# AI Scenario Testing Guide |
| 2 | + |
| 3 | +How to let AI assistants write and run Probitas scenario tests. |
| 4 | + |
| 5 | +## Setup for Claude Code (Recommended) |
| 6 | + |
| 7 | +Install the official marketplace and Probitas plugin: |
| 8 | + |
| 9 | +```bash |
| 10 | +/plugin marketplace add jsr-probitas/claude-plugins |
| 11 | +/plugin install probitas@jsr-probitas |
| 12 | +``` |
| 13 | + |
| 14 | +Enable it in `.claude/settings.json`: |
| 15 | + |
| 16 | +```json |
| 17 | +{ |
| 18 | + "plugins": { |
| 19 | + "marketplaces": ["jsr-probitas/claude-plugins"], |
| 20 | + "installed": ["probitas@jsr-probitas"] |
| 21 | + }, |
| 22 | + "enabledPlugins": { |
| 23 | + "probitas@jsr-probitas": true |
| 24 | + } |
| 25 | +} |
| 26 | +``` |
| 27 | + |
| 28 | +## How to Ask the AI |
| 29 | + |
| 30 | +- Be explicit about the scenario goal, inputs, and expected outcomes. |
| 31 | +- Share API/DB schemas or example payloads; do not let the AI guess. |
| 32 | +- Provide env vars and endpoints (e.g., `API_URL`) up front. |
| 33 | +- Request tags and timeouts if needed (e.g., `tags: ["api", "slow"]`). |
| 34 | +- Ask for `ctx.previous` data flow when steps depend on each other. |
| 35 | + |
| 36 | +Example prompt snippet: |
| 37 | + |
| 38 | +> Write a Probitas scenario for POST /users that returns `{ id: number }`. |
| 39 | +> Use API_URL env, tag as `api`. Step 1: create user, return id. Step 2: GET |
| 40 | +> by id, expect 200 and name match. Use fluent expect chain. |
| 41 | +
|
| 42 | +## Scenario Authoring Rules (What the AI must follow) |
| 43 | + |
| 44 | +- Use the `scenario-writer` agent for `*.probitas.ts`. |
| 45 | +- Keep steps in one scenario only when they depend on `ctx.previous`; otherwise |
| 46 | + split into separate scenarios. |
| 47 | +- Always `export default` and finish builders with `.build()`. |
| 48 | +- Register clients with `.resource()` and use env-driven URLs: |
| 49 | + `Deno.env.get("API_URL") ?? "http://localhost:8080"`. |
| 50 | +- Use fluent `expect()` chains—no manual `if/throw` checks. |
| 51 | +- `.setup()` should return a cleanup function when it creates fixtures. |
| 52 | +- One responsibility per step; return data needed by subsequent steps. |
| 53 | +- Use tags for filtering and retries/timeouts where appropriate. |
| 54 | + |
| 55 | +## Running and Checking via AI |
| 56 | + |
| 57 | +Claude Code commands: |
| 58 | + |
| 59 | +- `/probitas-init` — initialize a project |
| 60 | +- `/probitas-new <type>` — scaffold scenario templates |
| 61 | +- `/probitas-check` — format, lint, and type-check scenarios |
| 62 | +- `/probitas-run [selector]` — run scenarios (use selectors/tags to scope) |
| 63 | + |
| 64 | +Local probitas CLI tasks: |
| 65 | + |
| 66 | +- `probitas fmt` — format scenarios |
| 67 | +- `probitas lint` — lint scenarios |
| 68 | +- `probitas check` — type-check scenarios |
| 69 | +- `probitas run [selector]` — run scenarios (use selectors/tags to scope) |
| 70 | + |
| 71 | +## References to Share with AI |
| 72 | + |
| 73 | +- `/llms.txt` — sitemap for AI agents |
| 74 | +- `/docs/scenario/index.md` — scenario authoring guide |
| 75 | +- `/docs/client/index.md` — client usage |
| 76 | +- `/docs/expect/index.md` — assertion API |
| 77 | +- For API details use `deno doc` directly (preferred over Markdown): |
| 78 | + - `deno doc jsr:@probitas/probitas` |
| 79 | + - `deno doc jsr:@probitas/probitas/client/http` |
| 80 | + - `deno doc jsr:@probitas/probitas/expect` |
| 81 | + |
| 82 | +## Links |
| 83 | + |
| 84 | +- Claude plugin marketplace: https://github.com/jsr-probitas/claude-plugins |
| 85 | +- Probitas plugin docs: https://github.com/jsr-probitas/claude-plugins/tree/main/plugins/probitas |
0 commit comments