Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,9 +177,13 @@ src/github/copilot_sdk/
├── client.clj # Main client API (create-client, create-session, etc.)
├── session.clj # Session operations (send!, send-and-wait!, etc.)
├── helpers.clj # Convenience functions (query, query-seq!, query-chan, etc.)
├── specs.clj # clojure.spec definitions
├── instrument.clj # Function specs and instrumentation
└── util.clj # Internal utilities
├── tools.clj # Helper functions for defining tools (define-tool, result-success, etc.)
├── specs.clj # clojure.spec definitions for all data shapes
├── instrument.clj # Function specs (fdefs) and instrumentation
├── util.clj # Wire conversion (camelCase ↔ kebab-case), MCP helpers
├── protocol.clj # JSON-RPC 2.0 protocol over NIO channels
├── process.clj # CLI process management (spawning, lifecycle)
└── logging.clj # Logging facade via clojure.tools.logging
```

## Documentation
Expand Down
151 changes: 151 additions & 0 deletions .github/skills/update-upstream/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
---
name: update-upstream
description: Sync the Clojure Copilot SDK with upstream copilot-sdk changes. Runs update.sh, performs gap analysis against Node.js and Python SDKs, ports changes with red/green TDD, runs full CI (E2E tests + examples), gets parallel multi-model code reviews, updates docs, and creates a PR. Use when syncing with new upstream releases or checking for unported changes.
compatibility: Requires copilot CLI authenticated, gh CLI, clojure CLI, bb (babashka). Upstream repo at ../copilot-sdk.
---

# Update Upstream Skill

Sync the copilot-sdk-clojure project with upstream [github/copilot-sdk](https://github.com/github/copilot-sdk) changes. This skill codifies the full upstream sync workflow — from discovery through implementation, review, docs, and PR creation.

**Prerequisites**: Read `AGENTS.md` at the repo root for project structure, design philosophy, API compatibility rules, testing commands, and version management. Read `references/PROJECT.md` (relative to this skill) for upstream↔Clojure file mapping and wire conversion notes.

## Process

### Phase 1: Discovery

1. Run `./update.sh` from the repo root to pull the latest upstream and list releases.
2. Check the current Clojure SDK version in `build.clj` (format: `UPSTREAM.CLJ_PATCH` — see AGENTS.md § Version Management).
3. List upstream commits since our last synced version:
```
cd ../copilot-sdk && git log --oneline <last-tag>..HEAD -- nodejs/
```
4. For each commit, classify:
- **Port** — Code changes to `nodejs/src/` (types, client, session, generated)
- **Skip** — CI/tooling, language-specific (Python/Go/.NET only)

### Phase 2: Gap Analysis

Launch three parallel explore agents to build a comprehensive inventory. Use the file mapping in `references/PROJECT.md` to locate the right files.

1. **Node.js SDK** — Read upstream files listed in references/PROJECT.md (types.ts, client.ts, session.ts, index.ts, generated/). Catalog all public types, methods, event types, and event data fields.
2. **Python SDK** — Read `python/copilot/client.py`, `session.py`, `__init__.py`, `generated/`. Note behavioral differences from Node.js.
3. **Clojure SDK** — Read all `src/github/copilot_sdk/*.clj`. Catalog public functions, specs, event sets, wire conversion.

Compare inventories to identify gaps:
- Missing behavioral guards (e.g., `resolvedByHook`)
- Missing spec fields (new event data, new config options)
- Missing permission result kinds, event types
- API signature changes (e.g., handler arity)

### Phase 3: Planning

Create a structured plan in the session plan.md with:
- **Code gaps** — Behavioral changes needed (HIGH priority)
- **Spec gaps** — Missing fields/values (MEDIUM priority)
- **Doc gaps** — Documentation needing updates
- **Example gaps** — Missing examples for new features
- **Idiomatic review** — Areas to check for Clojure idiom adherence

Show the plan to the user. Wait for approval before implementing.

### Phase 4: Implementation (Red/Green TDD)

For each code/spec change:

1. **RED** — Write a failing test first in `test/github/copilot_sdk/integration_test.clj`
2. Run tests: `bb test` — confirm failure (it's OK to just run the tests you added as you iterate)
3. **GREEN** — Implement the minimal change in `src/`
4. Run tests again — confirm all pass (0 failures)

See `references/PROJECT.md` for the upstream↔Clojure file mapping to know which source files to modify. See `AGENTS.md` § Instrumented Testing for the spec/fdef workflow when adding new public functions.

Wire format notes — see `references/PROJECT.md` § Wire Conversion Cheat Sheet. Key rule: camel-snake-kebab does **not** add `?` suffixes for booleans.

### Phase 5: Full Validation

Run the full CI pipeline (see `AGENTS.md` § Before Committing):

```bash
bb ci:full
```

This runs E2E tests, examples, doc validation, and JAR build. If copilot CLI is unavailable, run `bb ci` instead.

Carefully review example output for regressions.

### Phase 6: Multi-Model Code Review

Launch parallel code-review agents using at least three distinct model families (e.g., Claude Opus 4.6, GPT-5.4, and Gemini 3 Pro - or later if available). Different model families catch different categories of issues — use whichever specific models are currently available.

Each reviewer gets the same context: what changed, why, and what to focus on (correctness, spec completeness, test coverage, Clojure idioms, wire conversion accuracy).

Compile a combined assessment table:

| # | Finding | Source | Severity | Validity | Decision |
|---|---------|--------|----------|----------|----------|

For each finding:
- **Valid + actionable** → Fix it, re-run tests
- **Valid + pre-existing** → Note as out of scope, track separately
- **Invalid / false positive** → Document rationale for dismissal

Iterate: fix → re-test → re-review until no actionable findings remain.

### Phase 7: Documentation

Invoke the `update-docs` skill. See `AGENTS.md` § Documentation for the full list of doc files that may need updating.

At minimum:
1. Update `doc/reference/API.md` — new specs, event fields, behavior changes
2. Update `CHANGELOG.md` — entries under `[Unreleased]` (see `AGENTS.md` § Changelog for formatting)
3. Run `bb validate-docs`

### Phase 8: PR Creation

1. Create a feature branch: `git checkout -b upstream-sync/v<version>`
2. Commit changes in logical commits to make them easy to review commit by commit and with descriptive message and `Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>`
3. Push and create PR with `gh pr create`
4. PR body should include: summary, changes list, validation results, review findings table

### Phase 9: Reflecing on code review feedback.

Once the PR is created, Copilot Code Review (and possibly humans) will provide code review feedback.

run: /pr auto Consider Copilot Code Review feedback. For each piece of feedback, determine if its valid and important, or invalid (false positive) and/or not important. For all valid feedback, address it. For each piece of feedback (valid or not), comment in the thread for that particular feedback comment and explain how you addressed the feedback or your rationale for not addressing, or a suggestion for addressing in the future (create issues for future/follow up). Once all feedback is addressed/commented, rerequest a review, and continue iterating this processes until there is no more code review feedback - you should keep iterating this process until you see a review from Copilot Code Review which generates no comments, but no more than 10 rounds (to avoid exploding costs). If more than 10 rounds is needed, prompt the users asking what to do next.

### Phase 10: Skill Self-Review

After each upstream sync, review this skill itself for accuracy and relevance:

1. **Project structure** — Compare the file listing in `AGENTS.md` § Project Structure against the actual contents of `src/github/copilot_sdk/`. Flag any new, renamed, or removed source files.
2. **Upstream file mapping** — Compare the mapping table in `references/PROJECT.md` against the current upstream `nodejs/src/` directory. Flag new or removed upstream files.
3. **Common Pitfalls** — Check whether any pitfalls were encountered during this sync that aren't listed, or whether any listed pitfalls are no longer relevant (e.g., patterns that have been refactored away).
4. **Process phases** — Note any workflow steps that were awkward, missing, or unnecessary during this sync.

If any drift is found, describe your findings to the user and propose specific edits to `SKILL.md`, `references/PROJECT.md`, or `AGENTS.md`. **Do not commit** the skill updates — wait for the maintainer to review and approve them.

## Key Principles

1. **API parity with upstream Node.js SDK.** Only port what the official SDK exposes. Don't add CLI-only features unless clearly marked experimental. See `AGENTS.md` § API Compatibility Rules.

2. **Idiomatic Clojure, not a transliteration.** Use immutable data, core.async for events/concurrency, specs for validation, kebab-case keywords. See `AGENTS.md` § Design Philosophy.

3. **Red/green TDD is mandatory.** Never implement without a failing test first. This catches wire conversion bugs (camelCase → kebab-case) and spec mismatches early.

4. **Multi-model review catches different things.** Different model families tend to find different categories of issues — API contracts, logic/concurrency bugs, spec inconsistencies. Use at least three distinct families.

5. **Wire conversion is the #1 source of bugs.** Always verify camelCase → kebab-case conversion for new fields. See `references/PROJECT.md` § Wire Conversion Cheat Sheet.

6. **Event data specs use open maps.** `s/keys` allows extra keys, so new upstream fields pass through automatically. Add explicit specs for documentation and validation, not to gate functionality.

7. **Pre-existing issues are out of scope.** If a reviewer finds a real issue that wasn't introduced by this sync, note it but don't fix it in the sync PR. Track separately.

## Common Pitfalls

These are verified sources of real bugs in this codebase:

- **Boolean wire fields don't get `?` suffix.** camel-snake-kebab converts `resolvedByHook` → `:resolved-by-hook`, not `:resolved-by-hook?`. Code that manually maps wire booleans to `?`-suffixed keywords (like `:preview?`) must do so explicitly.
- **Forgetting `instrument.clj` allowlists.** Every new public function needs an `s/fdef` and entries in both `instrument-all!` and `unstrument-all!` lists. Integration tests run instrumented, so missing entries cause silent spec gaps.
- **Closed config key sets.** When adding session config options, update both `session-config-keys` and `resume-session-config-keys` in `specs.clj`. Missing a set causes the option to be silently stripped.
- **Mock data vs wire format.** Test fixtures should use wire-shaped data (camelCase) for mock server responses and Clojure-shaped data (kebab-case with `?` suffixes where applicable) for client-side assertions.
41 changes: 41 additions & 0 deletions .github/skills/update-upstream/references/PROJECT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Upstream Sync Reference

This reference supplements `AGENTS.md` (the canonical project reference) with sync-specific context.

For project structure, testing commands, version format, changelog conventions, and code quality expectations, see `AGENTS.md`.

## Upstream ↔ Clojure File Mapping

When syncing, map upstream changes to the corresponding Clojure files:

### Upstream (../copilot-sdk)

| Upstream File | Contains |
|---------------|----------|
| `nodejs/src/types.ts` | Canonical type definitions (`SessionConfig`, `MessageOptions`, etc.) |
| `nodejs/src/client.ts` | `CopilotClient` methods, what params go on the wire |
| `nodejs/src/session.ts` | `CopilotSession` methods, event handling |
| `nodejs/src/index.ts` | Public exports (defines the public API surface) |
| `nodejs/src/generated/session-events.ts` | All event types and data shapes |
| `nodejs/src/generated/rpc.ts` | RPC method signatures |

### Clojure Counterparts

| Upstream Area | Clojure File | Notes |
|---------------|-------------|-------|
| Types / config | `specs.clj` | Session config, event data, permissions, tools |
| Client methods | `client.clj` | Broadcast handlers, create-client, create-session |
| Session methods | `session.clj` | send/receive, UI convenience methods |
| Event types | `client.clj` | `event-types` set, `subscribe-events!` |
| RPC methods | `protocol.clj` | JSON-RPC protocol layer |
| Public exports | `client.clj`, `helpers.clj`, `tools.clj` | Public API surface |
| Function specs | `instrument.clj` | fdefs for all public functions |

## Wire Conversion Cheat Sheet

camelCase → kebab-case conversion is handled by `util/wire->clj` and `util/clj->wire`.

Test a conversion: `(csk/->kebab-case-keyword :yourCamelCaseField)`

**Key rule**: camel-snake-kebab does **not** add a `?` suffix for booleans.
`resolvedByHook` → `:resolved-by-hook` (not `:resolved-by-hook?`).
20 changes: 20 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,26 @@ All notable changes to this project will be documented in this file. This change

## [Unreleased]

### Added
- **Session RPC wrappers** — new experimental functions for session-level RPCs previously only accessible via `proto/send-request!`:
- `mode-get`, `mode-set!` — get/set agent mode (interactive/plan/autopilot)
- `plan-read`, `plan-update!`, `plan-delete!` — read/update/delete session plan file
- `workspace-list-files`, `workspace-read-file`, `workspace-create-file!` — session workspace file operations
- `agent-list`, `agent-get-current`, `agent-select!`, `agent-deselect!`, `agent-reload!` — custom agent management
- `fleet-start!` — start parallel sub-sessions
- **MCP config wrappers** — new experimental server-level functions in `client`:
- `mcp-config-list`, `mcp-config-add!`, `mcp-config-update!`, `mcp-config-remove!` — MCP server configuration management
- **Hooks integration tests** — 6 tests covering all hook types (preToolUse, postToolUse, sessionStart, unknownType, handler exceptions, no-hooks)
- **User input handler tests** — 2 tests for `userInput.request` server→client RPC
- **System message transform tests** — 3 tests for `systemMessage.transform` callback invocation, error fallback, and passthrough
- **Tool result normalization tests** — 3 tests for string, nil, and structured ToolResultObject results via v3 broadcast
- **Session RPC wrapper tests** — 18 integration tests for all new RPC wrapper functions
- **Mock server enhancements** — `send-rpc-request!` for testing server→client RPCs, response routing in server loop, 30+ new method stubs
- Full `s/fdef` instrumentation for all 19 new public functions

### Changed (v0.2.1 sync)
- **`session.error` event data spec enriched** — optional `:status-code` (int), `:provider-call-id` (string), and `:url` (string) fields added to `::session.error-data` spec. These fields carry HTTP status codes, GitHub request tracing IDs, and actionable URLs from upstream error events (upstream PR #999, runtime 1.0.17).

## [0.2.1.0] - 2026-04-04
### Added (v0.2.1 sync)
- **`resolvedByHook` guard on `permission.requested`** — when the runtime resolves a permission request via a `permissionRequest` hook, the broadcast event includes `resolvedByHook: true`. The SDK now skips the client's `:on-permission-request` handler and does not send the `handlePendingPermissionRequest` RPC, preventing duplicate responses. Event subscribers still observe the event (upstream PR #999, runtime 1.0.17).
Expand Down
89 changes: 88 additions & 1 deletion doc/reference/API.md
Original file line number Diff line number Diff line change
Expand Up @@ -486,6 +486,28 @@ Each quota snapshot map contains:
| `:overage-allowed-with-exhausted-quota?` | boolean | Whether overage is allowed when quota is exhausted |
| `:reset-date` | string (optional) | ISO 8601 date when quota resets |

#### `mcp-config-list` / `mcp-config-add!` / `mcp-config-update!` / `mcp-config-remove!`

> **Experimental:** These wrap server-level MCP configuration RPCs and may change.

```clojure
;; List configured MCP servers
(copilot/mcp-config-list client)
;; => {:servers [...]}

;; Add a new MCP server config
(copilot/mcp-config-add! client {:name "my-server"
:command "npx"
:args ["-y" "@modelcontextprotocol/server-filesystem" "/tmp"]
:tools ["*"]})

;; Update an existing config
(copilot/mcp-config-update! client {:name "my-server" :tools ["read_file"]})

;; Remove a config
(copilot/mcp-config-remove! client {:name "my-server"})
```

#### `state`

```clojure
Expand Down Expand Up @@ -930,6 +952,32 @@ Get the client that owns this session.
;; Enable/disable MCP servers
(session/mcp-enable! my-session "my-server")
(session/mcp-disable! my-session "my-server")

;; Get/set agent mode
(session/mode-get my-session)
;; => {:mode "interactive"}
(session/mode-set! my-session "plan")

;; Read/update session plan
(session/plan-read my-session)
;; => {:exists? true :content "# Plan\n..." :file-path "/path/to/plan.md"}
(session/plan-update! my-session "# Updated Plan\n...")
(session/plan-delete! my-session)

;; Workspace file operations
(session/workspace-list-files my-session)
;; => {:files ["notes.md" "data.json"]}
(session/workspace-read-file my-session "notes.md")
;; => {:content "..."}
(session/workspace-create-file! my-session "output.txt" "result data")

;; Custom agent management
(session/agent-list my-session)
;; => {:agents [{:name "researcher" ...} ...]}
(session/agent-select! my-session "researcher")
(session/agent-get-current my-session)
;; => {:name "researcher"}
(session/agent-deselect! my-session)
```

**Skills**
Expand Down Expand Up @@ -959,6 +1007,45 @@ Get the client that owns this session.
| `session/extensions-disable!` | Disable an extension by ID. |
| `session/extensions-reload!` | Reload all extensions. |

**Mode**

| Function | Description |
|----------|-------------|
| `session/mode-get` | Get current agent mode. Returns `{:mode "interactive"\|"plan"\|"autopilot"}`. |
| `session/mode-set!` | Set agent mode. Accepts `"interactive"`, `"plan"`, or `"autopilot"`. |

**Plan**

| Function | Description |
|----------|-------------|
| `session/plan-read` | Read the session plan file. Returns `{:exists? :content :file-path}`. |
| `session/plan-update!` | Update the plan file content. |
| `session/plan-delete!` | Delete the plan file. |

**Workspace**

| Function | Description |
|----------|-------------|
| `session/workspace-list-files` | List files in the session workspace. Returns `{:files [...]}`. |
| `session/workspace-read-file` | Read a workspace file by relative path. Returns `{:content "..."}`. |
| `session/workspace-create-file!` | Create a file in the workspace with given path and content. |

**Agents**

| Function | Description |
|----------|-------------|
| `session/agent-list` | List available custom agents. Returns `{:agents [...]}`. |
| `session/agent-get-current` | Get the currently selected agent. Returns `{:name "..."}` or `{:name nil}`. |
| `session/agent-select!` | Select a custom agent by name. |
| `session/agent-deselect!` | Deselect the current custom agent. |
| `session/agent-reload!` | Reload all custom agents. |

**Fleet**

| Function | Description |
|----------|-------------|
| `session/fleet-start!` | Start parallel sub-sessions. Accepts a params map. |

**Other**

| Function | Description |
Expand Down Expand Up @@ -1120,7 +1207,7 @@ Convert an unqualified event keyword to a namespace-qualified `:copilot/` keywor
|------------|-------------|
| `:copilot/session.start` | Session created |
| `:copilot/session.resume` | Session resumed |
| `:copilot/session.error` | Session error occurred |
| `:copilot/session.error` | Session error occurred; data: `{:error-type "..." :message "..." :stack "..." :status-code 429 :provider-call-id "..." :url "..."}` (`:stack`, `:status-code`, `:provider-call-id`, `:url` optional) |
| `:copilot/session.idle` | Session finished processing |
| `:copilot/session.info` | Informational session update |
| `:copilot/session.model_change` | Session model changed |
Expand Down
Loading
Loading