You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
## What is this?
This PR adds GitHub Copilot CLI as a supported Skillgym runner. It lets
suites run against `agent.type: "copilot"`, keeps Copilot session
artifacts, and documents the new support across the README, docs, and
website.
## How does it work?
The new adapter launches `copilot` in non-interactive JSONL mode with
the configured model, isolates Copilot state under each artifact
directory, and normalizes Copilot messages, tool calls, file reads,
commands, explicit skill loads, and OpenTelemetry token usage into
Skillgym session reports. The runner registry, config parser, public
types, max-step monitor, and examples now recognize the `copilot` agent
type.
## Why is this useful?
Skill authors can now compare GitHub Copilot CLI against the existing
OpenCode, Codex, Claude Code, and Cursor Agent runners using the same
Skillgym cases and assertions. Explicit Copilot skill-load events are
captured as detected skills, so skill-selection suites can verify that
Copilot actually loaded the expected skill instead of relying only on
file-read inference.
@@ -181,7 +188,7 @@ The execution unit is one case x runner pair. `skillgym` expands the suite into
181
188
182
189
For concurrent schedules, `run.maxParallel` defaults to `os.availableParallelism()`. This limits how many Skillgym executions are active at once; it does not pin or limit CPU cores used by an individual agent process.
183
190
184
-
Concurrent schedules do not copy or isolate the workspace by themselves. Overlapping executions may still interact through the same filesystem state and live runner output unless you use isolated workspaces. OpenCode, Codex, and Claude Code runtime state are isolated per execution under each artifact directory.
191
+
Concurrent schedules do not copy or isolate the workspace by themselves. Overlapping executions may still interact through the same filesystem state and live runner output unless you use isolated workspaces. OpenCode, Codex, Claude Code, and Copilot runtime state are isolated per execution under each artifact directory.
185
192
186
193
`run.repeat` is useful when you want stability sampling instead of a single lucky pass. Each case x runner execution keeps running until it records the requested number of successful classified results, or stops early when one repetition still fails after exhausting `run.repeatFailure` retries.
`runner.id` is the configured runner id, and `runner.agent.type` identifies the backing integration such as `opencode`, `codex`, or `claude-code`.
58
+
`runner.id` is the configured runner id, and `runner.agent.type` identifies the backing integration such as `opencode`, `codex`, `claude-code`, `cursor-agent`, or `copilot`.
0 commit comments