readme.md

skillgym Docs

This directory contains end-user documentation for skillgym.

What this project is

skillgym is a benchmarking tool for testing whether coding-agent skills are selected and used correctly during real agent execution.

The tool runs real sessions against supported CLIs, captures session artifacts, normalizes them into a shared session report format, and executes user-provided JavaScript assertions against that session report.

Workspace behavior is documented in workspaces.md, including shared executions, isolated executions, suite-level workspace exports, template directories, and bootstrap commands.

Current V1 scope

Supported in V1:

• OpenCode CLI
• Codex CLI
• Claude Code CLI
• Cursor Agent CLI
• GitHub Copilot CLI

Core principles

• Real execution only
• JavaScript assertions only
• One benchmark metric: success or failure
• Session telemetry is preserved for debugging
• Concurrent execution is capped by available host CPU parallelism unless explicitly overridden
• Best-effort execution limits should still preserve raw artifacts on failure
• TypeScript implementation
• Node.js-compatible APIs only

Assertions

Assertion authoring and the built-in grouped assert API are documented in assertions.md, including:

• standard strict-assert usage like assert.equal and assert.match
• assert.skills.*
• assert.commands.*
• assert.fileReads.*
• assert.toolCalls.*
• assert.output.*

Document map

• cases.md: test suite and case authoring
• assertions.md: assertion reference and matcher semantics
• explain.md: deferred explain workflow and artifact shapes
• workspaces.md: shared and isolated workspace behavior
• reporters.md: reporter lifecycle, loading, and standard reporter behavior
• session-report.md: session report schema
• snapshot.md: token regression snapshots and baseline updates
• skill-detection.md: how skill selection is observed