Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
170 changes: 170 additions & 0 deletions .agents/skills/implement-command-spec/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
---
name: implement-command-spec
description: Orchestrate and verify command-spec Oz runs for Linear tickets. Starts an implementation run using the add-command-spec skill, verifies all edited/created generators have meaningful screenshots, and retries with follow-up runs until validation passes. Use this skill whenever you need to implement and fully validate a command-spec Linear ticket end-to-end, or when a user provides a Linear ticket that involves adding or modifying command completions in the command-signatures repo.
---

# implement-command-spec

You are an orchestration agent managing the full lifecycle of a command-spec ticket: implementation, screenshot verification, and follow-up correction runs. You receive a Linear ticket URL or ID and drive it to completion autonomously.

## Prerequisites

- `gh` CLI (authenticated, with access to `warpdotdev/command-signatures`)
- Linear MCP server connected

## Constants

- **Environment ID**: `Z0IIFcJio9DgsLd71HqTvY` (Command completions environment).
- **Skill**: `warpdotdev/command-signatures:add-command-spec`
- **Max verification cycles**: 4 (initial run + up to 3 follow-ups)
- **Max failure retries per run**: 2 (so up to 3 attempts per run before giving up)
- **Poll interval**: 30 seconds, exponential backoff to at most 480 seconds (8 minutes)

## Phase 1: Extract ticket info

Use the Linear MCP `get_issue` tool to fetch the ticket. Record:
- The ticket identifier (e.g. `APP-3478`)
- The ticket title and description — you will need these to construct follow-up prompts later.

## Phase 2: Start the implementation run

Spawn an agent with computer use enabled.
- In remote environments, use <ENVIRONMENT_ID>.
- Prompt the agent with the full "warpdotdev/command-signatures:add-command-spec" skill, along with the following instructions:
"Use the add_command_spec skill to address this linear ticket: <LINEAR_TICKET>. Screenshots should be included as images in the PR description following the upload process, or posted to the Linear issue and linked to in the GitHub PR description if that's not possible."

Once the agent has started, note its run ID and session links, so that they can reported back to the user.

## Phase 3: Handle the spawned agent

When the agent reaches a terminal state:
- If it **succeeded**: Proceed to Phase 4.
- If it **failed**: Inspect the status message in the output.
- If it contains **"Environment setup failed"**, this is a configuration error. **Stop immediately** and report the error — do not retry.
- For any other failure, restart the run by repeating Phase 2. You may retry up to 2 additional times (3 total attempts per run). If all attempts fail, stop and report the failure with the last error message.

## Phase 4: Discover the PR and identify generators

### Find the PR

The run output includes an **Artifacts** section with the PR reference, branch name, and link:

```
Artifacts:
PR: command-signatures #233
Branch: app-3478/command-spec-kubectl-kubeconfig
Link: https://github.com/warpdotdev/command-signatures/pull/233
```

If artifacts are missing, fall back to searching:

```sh
gh pr list --repo warpdotdev/command-signatures --state open --json number,title,headRefName,url
```

Match by ticket identifier in the branch name or PR title.

### Identify generators from the diff

Fetch the diff:

```sh
gh pr diff <PR_NUMBER> --repo warpdotdev/command-signatures
```

Look for changes in two locations:
- **Generator source files**: `command-signatures/src/generators/<command>.rs` — each file may define multiple generator functions (e.g. `users_generator`, `services_generator`). A single file change can mean multiple generators need verification.
- **JSON specs**: `json/<command>.json` — look for `generatorName` references to identify which generators are being wired up.

Build a list of every distinct generator that was added or modified. Include generator function names when identifiable from the diff, not just file paths — this precision matters for screenshot matching later.

Also check the PR description for an explicit summary of what was added; the add-command-spec skill often lists generators there.

## Phase 5: Validate screenshots

For each generator, determine whether a valid screenshot exists that proves it works.

### 5a: Collect screenshots from all sources

**GitHub PR description:**
```sh
gh pr view <PR_NUMBER> --repo warpdotdev/command-signatures --json body -q '.body'
```

**GitHub PR comments:**
```sh
gh pr view <PR_NUMBER> --repo warpdotdev/command-signatures --json comments -q '.comments[].body'
```

Extract image URLs from both (markdown `![...](URL)` or `<img src="URL">` tags).

**Linear issue**: Use the Linear MCP `get_issue` tool, then `extract_images` on the description markdown.

**Linear issue comments**: Use the Linear MCP `list_comments` tool, then `extract_images` on each comment body.

### 5b: Analyze each screenshot

For each image URL found:

1. Download it:
```sh
curl -sL "<IMAGE_URL>" -o /tmp/screenshot_N.png
```
2. View the downloaded image with `read_files` to trigger vision analysis.
3. Determine:
- **Which generator(s)** does this screenshot demonstrate? Look for: the command being typed, visible completions dropdown entries, flags or subcommands in the input, filenames or paths that indicate which generator was invoked.
- **Is the screenshot meaningful?**
- It must show the completions menu (a dropdown next to the cursor) with real, generator-produced entries.
- A screenshot that shows the command typed but no completions appearing is **not valid**.
- A screenshot showing an empty completions menu or only static (non-generator) completions is **not valid**.
- Each generator needs at least one screenshot showing it producing non-trivial output.

### 5c: Build the verification map

Create a mapping: `generator → valid screenshot(s)`. Any generator with zero valid screenshots is **unverified**.

If all generators are verified, skip to Phase 7.

## Phase 6: Follow-up runs

When generators still lack valid screenshots:

### 6a: Update Linear status

If the ticket was moved to "In Review", move it back to "In Progress" using the Linear MCP `save_issue` tool:

```json
{ "id": "<TICKET_ID>", "state": "In Progress" }
```

### 6b: Construct the follow-up prompt

Read `references/follow-up-prompt-template.md` and fill in the placeholders with:
- The ticket description from Phase 1
- The branch name and PR URL from Phase 4
- The specific list of unverified generators and what each should produce
- Context about what was already verified vs. what's still missing

### 6c: Start the follow-up run

Spawn another agent with computer use enabled.
- In remote environments, use <ENVIRONMENT_ID>.
- Prompt the agent with the full "warpdotdev/command-signatures:add-command-spec and the <CONSTRUCTED_FOLLOW_UP_PROMPT>.

As before, once the agent has started, note its run ID and session links, so that they can be sent back to the user later.

### 6d: Monitor and re-validate

Repeat Phases 3–5 for the follow-up run. The same failure retry limits apply per run.

This verification loop may repeat up to 3 more times after the initial cycle (4 total verification cycles). If generators still lack valid screenshots after all cycles, proceed to Phase 7 and report which remain unverified.

## Phase 7: Completion

### All generators verified
- Advance the Linear ticket's status to "In Review" via `save_issue`.
- Report success in a Linear comment: the PR URL, the verified generators, and how many cycles it took. Include links and run IDs for each spawned agent for traceability.

### Some generators unverified after all attempts
- Leave the Linear ticket in "In Progress".
- Edit the Linear ticket description which generators still need screenshots, the PR URL, what succeeded, and any patterns you noticed (e.g. a generator that never produces output may have a more fundamental testing limitation).
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Follow-up Prompt Template

Use this template to construct follow-up prompts for Oz runs that need to verify generators with missing or invalid screenshots. Fill in the `[BRACKETED]` placeholders.

---

We've been working on [TICKET_DESCRIPTION] in the command-signatures repo, in the [BRANCH_NAME] branch. Now we need to validate all the generators edited in this branch by providing screenshots of them in use and putting them in a comment on the existing PR ([PR_URL]).

The following generators still need valid screenshots showing they produce meaningful completions output:
[GENERATOR_LIST — one item per line, with a brief note about what each generator should produce. For example:
- `kubeconfig_contexts_generator` — should show context names from kubeconfig
- `namespaces_generator` — should show Kubernetes namespace completions]

[PRIOR_VERIFICATION_CONTEXT — describe what was already verified and what's missing. For example: "A prior attempt verified the --kubeconfig flag generator, but the KUBECONFIG environment variable and $KUBECONFIG shell variable methods still need verification." Or: "No valid screenshots were found for any of the generators listed above."]

First, read the add-command-spec skill in command-signatures/.agents/skills/add-command-spec/ to understand the full submission requirements including screenshot upload.

Set up a testing environment with the command installed and configured to produce meaningful results. Do that before attempting to run a local Warp build — the generators need real data to complete against (e.g. running services, existing configs, populated directories).

Use the test-local-warp skill in command-signatures/.agents/skills/test-local-warp/ to patch the command-signatures change into the local warp app build. Then, use a local Warp build to test each generator listed above. To trigger the completions menu, press the Tab key. We are NOT testing autocomplete (ghost text) — we are testing completions, which are dropdown menus that appear next to the cursor. Take a screenshot to show each generator invocation working and attach it as a comment on the PR; your work will not be accepted without it.

Each generator must have its own screenshot showing real, non-trivial output. Empty completions menus, zero results, or generic placeholders are not acceptable — the screenshot must show at least one generator-produced completion entry.
12 changes: 6 additions & 6 deletions .agents/skills/test-local-warp/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,18 @@ description: Test command-signatures changes locally by running Warp terminal ag

## Overview

`warp-command-signatures` is consumed by `warp-internal` as a git dependency. To test changes locally, temporarily point `warp-internal` at a local checkout of this repo, then build and run Warp.
`warp-command-signatures` is consumed by the `warp` app as a git dependency. To test changes locally, temporarily point `warp` at a local checkout of this repo, then build and run Warp.

## Prerequisites

- A local checkout of both `warp-internal` and `command-signatures`.
- A local checkout of both `warp` and `command-signatures`.
- **Computer use must be enabled** — Warp is a GUI application, so verifying completions requires screen interaction.

## Steps

### 1. Patch the dependency in warp-internal
### 1. Patch the dependency in warp

In the `warp-internal` repo's root `Cargo.toml`, find the `warp-command-signatures` line under `[workspace.dependencies]`:
In the `warp` repo's root `Cargo.toml`, find the `warp-command-signatures` line under `[workspace.dependencies]`:

```toml
warp-command-signatures = { git = "ssh://git@github.com/warpdotdev/command-signatures.git", rev = "...", default-features = false }
Expand All @@ -34,7 +34,7 @@ Note the nested `command-signatures/command-signatures` — the outer directory

### 2. Build and run Warp

From the `warp-internal` repo:
From the `warp` repo:

```sh
cargo run --features fast_dev
Expand All @@ -48,4 +48,4 @@ Open the locally-built Warp and test completions for the commands you modified.

### 4. Clean up

Revert the `Cargo.toml` change in `warp-internal` before committing. The local path override should never be checked in.
Revert the `Cargo.toml` change in `warp` before committing. The local path override should never be checked in.
91 changes: 91 additions & 0 deletions .agents/skills/update-command-spec-version/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
---
name: update-command-spec-version
description: >
Update the warp-command-signatures dependency hash in warp to the latest commit from warpdotdev/command-signatures:main.
---

# Update Command Spec Version

This skill automates updating the `warp-command-signatures` git dependency in `warpdotdev/warp` to the latest commit on `warpdotdev/command-signatures:main`, then opens a PR with a summary of what changed.

All scripts live in this skill's `scripts/` directory.

## Prerequisites

- `gh` CLI authenticated with access to `warpdotdev/warp` and `warpdotdev/command-signatures`
- SSH access to both repos (they use `ssh://git@github.com/warpdotdev/...`)
- Python 3 (for `list_merged_prs.py`)
- Rust toolchain with `cargo` (for updating `Cargo.lock`)

## Step-by-step procedure

Run the scripts in order. The skill directory is wherever this SKILL.md lives; reference scripts relative to it.

### 1. Ensure repos are available

```bash
bash <skill-dir>/scripts/ensure_repos.sh
```

This clones or fetches both `~/warp` and `~/command-signatures`.

### 2. Get the current and latest hashes

```bash
OLD_HASH=$(bash <skill-dir>/scripts/get_current_hash.sh)
NEW_HASH=$(bash <skill-dir>/scripts/get_latest_hash.sh)
```

If `OLD_HASH` equals `NEW_HASH`, tell the user that command-signatures is already up to date and stop.

### 3. List merged PRs

```bash
python3 <skill-dir>/scripts/list_merged_prs.py "$OLD_HASH" "$NEW_HASH"
```

This outputs JSON with four keys: `prs` (all PRs), `new_completions`, `bug_fixes`, and `other`. Each entry has `number` and `title`.

Review the categorization. The script uses simple heuristics (titles starting with "add" → new completion, titles containing "fix" or "bug" → bug fix). Move any miscategorized entries before composing the description.

### 4. Create the branch and update dependencies

```bash
bash <skill-dir>/scripts/update_and_branch.sh "$NEW_HASH"
```

This checks out a new branch `completions-bot/update-command-signatures-<first 8 chars of NEW_HASH>` from `origin/master` in `~/warp`, updates the `rev` in `Cargo.toml`, and syncs `Cargo.lock`.

### 5. Compose the PR description

Write a PR description following this structure (match the warp PR template). Save it to a temp file, e.g. `/tmp/completions_pr_body.md`.

```
## Description

Updates `warp-command-signatures` to <NEW_HASH short>.

### Merged PRs
- <PR title> (warpdotdev/command-signatures#<number>)
- <PR title> (warpdotdev/command-signatures#<number>)
- ...

## Changelog Entries for Stable

CHANGELOG-IMPROVEMENT: <single line covering all new completions, subcommands, generators, and updates>
CHANGELOG-BUG-FIX: <single line covering all bug fixes>
```

Guidelines for changelog entries:
- Write **one** `CHANGELOG-IMPROVEMENT` line that summarizes all new completions and improvements. If there are many commands, use only their short names. For example: "Added completions for `aws ec2`, `scp`, `claude`, and `docker-compose` service names."
- Write **one** `CHANGELOG-BUG-FIX` line that summarizes all bug fixes. For example: "Fixed npm install short-form command priority and HTML entity rendering in completion specs."
- Omit a changelog line entirely if there are no PRs in that category.
- If a PR doesn't fit either category, use your judgment — it may belong in the improvement line or may not warrant a changelog entry at all.

### 6. Submit the PR

```bash
bash <skill-dir>/scripts/submit_pr.sh "$NEW_HASH" <absolute_path_to_temp_description_file>
```

This commits the changes, pushes the branch, and opens a draft PR titled "[Completions] Bump command-signatures to <first 8 chars of NEW_HASH>" against `master`.
25 changes: 25 additions & 0 deletions .agents/skills/update-command-spec-version/scripts/ensure_repos.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/bin/bash
set -euo pipefail

WARP_DIR="${HOME}/warp"
CMD_SIGS_DIR="${HOME}/command-signatures"

if [ -d "${WARP_DIR}" ]; then
echo "Fetching warp..." >&2
git -C "${WARP_DIR}" fetch origin
else
echo "Cloning warp..." >&2
git clone ssh://git@github.com/warpdotdev/warp.git "${WARP_DIR}"
fi

if [ -d "${CMD_SIGS_DIR}" ]; then
echo "Fetching command-signatures..." >&2
git -C "${CMD_SIGS_DIR}" fetch origin
git -C "${CMD_SIGS_DIR}" checkout main --quiet
git -C "${CMD_SIGS_DIR}" pull origin main --quiet
else
echo "Cloning command-signatures..." >&2
git clone ssh://git@github.com/warpdotdev/command-signatures.git "${CMD_SIGS_DIR}"
fi

echo "Both repositories are ready." >&2
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#!/bin/bash
set -euo pipefail

WARP_DIR="${HOME}/warp"

# Read Cargo.toml from origin/master so we don't depend on local checkout state.
HASH=$(git -C "${WARP_DIR}" show origin/master:Cargo.toml \
| grep 'warp-command-signatures.*rev' \
| grep -o 'rev = "[^"]*"' \
| grep -o '"[^"]*"' \
| tr -d '"')

if [ -z "${HASH}" ]; then
echo "Error: could not find warp-command-signatures rev in Cargo.toml" >&2
exit 1
fi

echo "${HASH}"
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
#!/bin/bash
set -euo pipefail

CMD_SIGS_DIR="${HOME}/command-signatures"

# After ensure_repos.sh has pulled, origin/main is up to date.
HASH=$(git -C "${CMD_SIGS_DIR}" rev-parse origin/main)

if [ -z "${HASH}" ]; then
echo "Error: could not determine HEAD of command-signatures main" >&2
exit 1
fi

echo "${HASH}"
Loading
Loading