Move operation skills over (#269)

Author: lucieleblanc

.agents/skills/implement-command-spec/SKILL.md

@@ -0,0 +1,170 @@
+---
+name: implement-command-spec
+description: Orchestrate and verify command-spec Oz runs for Linear tickets. Starts an implementation run using the add-command-spec skill, verifies all edited/created generators have meaningful screenshots, and retries with follow-up runs until validation passes. Use this skill whenever you need to implement and fully validate a command-spec Linear ticket end-to-end, or when a user provides a Linear ticket that involves adding or modifying command completions in the command-signatures repo.
+---
+
+# implement-command-spec
+
+You are an orchestration agent managing the full lifecycle of a command-spec ticket: implementation, screenshot verification, and follow-up correction runs. You receive a Linear ticket URL or ID and drive it to completion autonomously.
+
+## Prerequisites
+
+- `gh` CLI (authenticated, with access to `warpdotdev/command-signatures`)
+- Linear MCP server connected
+
+## Constants
+
+- **Environment ID**: `Z0IIFcJio9DgsLd71HqTvY` (Command completions environment).
+- **Skill**: `warpdotdev/command-signatures:add-command-spec`
+- **Max verification cycles**: 4 (initial run + up to 3 follow-ups)
+- **Max failure retries per run**: 2 (so up to 3 attempts per run before giving up)
+- **Poll interval**: 30 seconds, exponential backoff to at most 480 seconds (8 minutes)
+
+## Phase 1: Extract ticket info
+
+Use the Linear MCP `get_issue` tool to fetch the ticket. Record:
+- The ticket identifier (e.g. `APP-3478`)
+- The ticket title and description — you will need these to construct follow-up prompts later.
+
+## Phase 2: Start the implementation run
+
+Spawn an agent with computer use enabled.
+- In remote environments, use <ENVIRONMENT_ID>.
+- Prompt the agent with the full "warpdotdev/command-signatures:add-command-spec" skill, along with the following instructions:
+"Use the add_command_spec skill to address this linear ticket: <LINEAR_TICKET>. Screenshots should be included as images in the PR description following the upload process, or posted to the Linear issue and linked to in the GitHub PR description if that's not possible."
+
+Once the agent has started, note its run ID and session links, so that they can reported back to the user.
+
+## Phase 3: Handle the spawned agent
+
+When the agent reaches a terminal state:
+- If it **succeeded**: Proceed to Phase 4.
+- If it **failed**: Inspect the status message in the output.
+  - If it contains **"Environment setup failed"**, this is a configuration error. **Stop immediately** and report the error — do not retry.
+  - For any other failure, restart the run by repeating Phase 2. You may retry up to 2 additional times (3 total attempts per run). If all attempts fail, stop and report the failure with the last error message.
+
+## Phase 4: Discover the PR and identify generators
+
+### Find the PR
+
+The run output includes an **Artifacts** section with the PR reference, branch name, and link:
+
+```
+Artifacts:
+  PR: command-signatures #233
+    Branch: app-3478/command-spec-kubectl-kubeconfig
+    Link: https://github.com/warpdotdev/command-signatures/pull/233
+```
+
+If artifacts are missing, fall back to searching:
+
+```sh
+gh pr list --repo warpdotdev/command-signatures --state open --json number,title,headRefName,url
+```
+
+Match by ticket identifier in the branch name or PR title.
+
+### Identify generators from the diff
+
+Fetch the diff:
+
+```sh
+gh pr diff <PR_NUMBER> --repo warpdotdev/command-signatures
+```
+
+Look for changes in two locations:
+- **Generator source files**: `command-signatures/src/generators/<command>.rs` — each file may define multiple generator functions (e.g. `users_generator`, `services_generator`). A single file change can mean multiple generators need verification.
+- **JSON specs**: `json/<command>.json` — look for `generatorName` references to identify which generators are being wired up.
+
+Build a list of every distinct generator that was added or modified. Include generator function names when identifiable from the diff, not just file paths — this precision matters for screenshot matching later.
+
+Also check the PR description for an explicit summary of what was added; the add-command-spec skill often lists generators there.
+
+## Phase 5: Validate screenshots
+
+For each generator, determine whether a valid screenshot exists that proves it works.
+
+### 5a: Collect screenshots from all sources
+
+**GitHub PR description:**
+```sh
+gh pr view <PR_NUMBER> --repo warpdotdev/command-signatures --json body -q '.body'
+```
+
+**GitHub PR comments:**
+```sh
+gh pr view <PR_NUMBER> --repo warpdotdev/command-signatures --json comments -q '.comments[].body'
+```
+
+Extract image URLs from both (markdown `![...](URL)` or `<img src="URL">` tags).
+
+**Linear issue**: Use the Linear MCP `get_issue` tool, then `extract_images` on the description markdown.
+
+**Linear issue comments**: Use the Linear MCP `list_comments` tool, then `extract_images` on each comment body.
+
+### 5b: Analyze each screenshot
+
+For each image URL found:
+
+1. Download it:
+   ```sh
+   curl -sL "<IMAGE_URL>" -o /tmp/screenshot_N.png
+   ```
+2. View the downloaded image with `read_files` to trigger vision analysis.
+3. Determine:
+   - **Which generator(s)** does this screenshot demonstrate? Look for: the command being typed, visible completions dropdown entries, flags or subcommands in the input, filenames or paths that indicate which generator was invoked.
+   - **Is the screenshot meaningful?**
+     - It must show the completions menu (a dropdown next to the cursor) with real, generator-produced entries.
+     - A screenshot that shows the command typed but no completions appearing is **not valid**.
+     - A screenshot showing an empty completions menu or only static (non-generator) completions is **not valid**.
+     - Each generator needs at least one screenshot showing it producing non-trivial output.
+
+### 5c: Build the verification map
+
+Create a mapping: `generator → valid screenshot(s)`. Any generator with zero valid screenshots is **unverified**.
+
+If all generators are verified, skip to Phase 7.
+
+## Phase 6: Follow-up runs
+
+When generators still lack valid screenshots:
+
+### 6a: Update Linear status
+
+If the ticket was moved to "In Review", move it back to "In Progress" using the Linear MCP `save_issue` tool:
+
+```json
+{ "id": "<TICKET_ID>", "state": "In Progress" }
+```
+
+### 6b: Construct the follow-up prompt
+
+Read `references/follow-up-prompt-template.md` and fill in the placeholders with:
+- The ticket description from Phase 1
+- The branch name and PR URL from Phase 4
+- The specific list of unverified generators and what each should produce
+- Context about what was already verified vs. what's still missing
+
+### 6c: Start the follow-up run
+
+Spawn another agent with computer use enabled. 
+- In remote environments, use <ENVIRONMENT_ID>.
+- Prompt the agent with the full "warpdotdev/command-signatures:add-command-spec and the <CONSTRUCTED_FOLLOW_UP_PROMPT>.
+
+As before, once the agent has started, note its run ID and session links, so that they can be sent back to the user later.
+
+### 6d: Monitor and re-validate
+
+Repeat Phases 3–5 for the follow-up run. The same failure retry limits apply per run.
+
+This verification loop may repeat up to 3 more times after the initial cycle (4 total verification cycles). If generators still lack valid screenshots after all cycles, proceed to Phase 7 and report which remain unverified.
+
+## Phase 7: Completion
+
+### All generators verified
+- Advance the Linear ticket's status to "In Review" via `save_issue`.
+- Report success in a Linear comment: the PR URL, the verified generators, and how many cycles it took. Include links and run IDs for each spawned agent for traceability.
+
+### Some generators unverified after all attempts
+- Leave the Linear ticket in "In Progress".
+- Edit the Linear ticket description which generators still need screenshots, the PR URL, what succeeded, and any patterns you noticed (e.g. a generator that never produces output may have a more fundamental testing limitation).

.agents/skills/implement-command-spec/references/follow-up-prompt-template.md

@@ -0,0 +1,22 @@
+# Follow-up Prompt Template
+
+Use this template to construct follow-up prompts for Oz runs that need to verify generators with missing or invalid screenshots. Fill in the `[BRACKETED]` placeholders.
+
+---
+
+We've been working on [TICKET_DESCRIPTION] in the command-signatures repo, in the [BRANCH_NAME] branch. Now we need to validate all the generators edited in this branch by providing screenshots of them in use and putting them in a comment on the existing PR ([PR_URL]).
+
+The following generators still need valid screenshots showing they produce meaningful completions output:
+[GENERATOR_LIST — one item per line, with a brief note about what each generator should produce. For example:
+- `kubeconfig_contexts_generator` — should show context names from kubeconfig
+- `namespaces_generator` — should show Kubernetes namespace completions]
+
+[PRIOR_VERIFICATION_CONTEXT — describe what was already verified and what's missing. For example: "A prior attempt verified the --kubeconfig flag generator, but the KUBECONFIG environment variable and $KUBECONFIG shell variable methods still need verification." Or: "No valid screenshots were found for any of the generators listed above."]
+
+First, read the add-command-spec skill in command-signatures/.agents/skills/add-command-spec/ to understand the full submission requirements including screenshot upload.
+
+Set up a testing environment with the command installed and configured to produce meaningful results. Do that before attempting to run a local Warp build — the generators need real data to complete against (e.g. running services, existing configs, populated directories).
+
+Use the test-local-warp skill in command-signatures/.agents/skills/test-local-warp/ to patch the command-signatures change into the local warp app build. Then, use a local Warp build to test each generator listed above. To trigger the completions menu, press the Tab key. We are NOT testing autocomplete (ghost text) — we are testing completions, which are dropdown menus that appear next to the cursor. Take a screenshot to show each generator invocation working and attach it as a comment on the PR; your work will not be accepted without it.
+
+Each generator must have its own screenshot showing real, non-trivial output. Empty completions menus, zero results, or generic placeholders are not acceptable — the screenshot must show at least one generator-produced completion entry.

.agents/skills/test-local-warp/SKILL.md

@@ -7,18 +7,18 @@ description: Test command-signatures changes locally by running Warp terminal ag
 
 ## Overview
 
-`warp-command-signatures` is consumed by `warp-internal` as a git dependency. To test changes locally, temporarily point `warp-internal` at a local checkout of this repo, then build and run Warp.
+`warp-command-signatures` is consumed by the `warp` app as a git dependency. To test changes locally, temporarily point `warp` at a local checkout of this repo, then build and run Warp.
 
 ## Prerequisites
 
-- A local checkout of both `warp-internal` and `command-signatures`.
+- A local checkout of both `warp` and `command-signatures`.
 - **Computer use must be enabled** — Warp is a GUI application, so verifying completions requires screen interaction.
 
 ## Steps
 
-### 1. Patch the dependency in warp-internal
+### 1. Patch the dependency in warp
 
-In the `warp-internal` repo's root `Cargo.toml`, find the `warp-command-signatures` line under `[workspace.dependencies]`:
+In the `warp` repo's root `Cargo.toml`, find the `warp-command-signatures` line under `[workspace.dependencies]`:
 
 ```toml
 warp-command-signatures = { git = "ssh://git@github.com/warpdotdev/command-signatures.git", rev = "...", default-features = false }
@@ -34,7 +34,7 @@ Note the nested `command-signatures/command-signatures` — the outer directory
 
 ### 2. Build and run Warp
 
-From the `warp-internal` repo:
+From the `warp` repo:
 
 ```sh
 cargo run --features fast_dev
@@ -48,4 +48,4 @@ Open the locally-built Warp and test completions for the commands you modified.
 
 ### 4. Clean up
 
-Revert the `Cargo.toml` change in `warp-internal` before committing. The local path override should never be checked in.
+Revert the `Cargo.toml` change in `warp` before committing. The local path override should never be checked in.

.agents/skills/update-command-spec-version/SKILL.md

@@ -0,0 +1,91 @@
+---
+name: update-command-spec-version
+description: >
+  Update the warp-command-signatures dependency hash in warp to the latest commit from warpdotdev/command-signatures:main.
+---
+
+# Update Command Spec Version
+
+This skill automates updating the `warp-command-signatures` git dependency in `warpdotdev/warp` to the latest commit on `warpdotdev/command-signatures:main`, then opens a PR with a summary of what changed.
+
+All scripts live in this skill's `scripts/` directory.
+
+## Prerequisites
+
+- `gh` CLI authenticated with access to `warpdotdev/warp` and `warpdotdev/command-signatures`
+- SSH access to both repos (they use `ssh://git@github.com/warpdotdev/...`)
+- Python 3 (for `list_merged_prs.py`)
+- Rust toolchain with `cargo` (for updating `Cargo.lock`)
+
+## Step-by-step procedure
+
+Run the scripts in order. The skill directory is wherever this SKILL.md lives; reference scripts relative to it.
+
+### 1. Ensure repos are available
+
+```bash
+bash <skill-dir>/scripts/ensure_repos.sh
+```
+
+This clones or fetches both `~/warp` and `~/command-signatures`.
+
+### 2. Get the current and latest hashes
+
+```bash
+OLD_HASH=$(bash <skill-dir>/scripts/get_current_hash.sh)
+NEW_HASH=$(bash <skill-dir>/scripts/get_latest_hash.sh)
+```
+
+If `OLD_HASH` equals `NEW_HASH`, tell the user that command-signatures is already up to date and stop.
+
+### 3. List merged PRs
+
+```bash
+python3 <skill-dir>/scripts/list_merged_prs.py "$OLD_HASH" "$NEW_HASH"
+```
+
+This outputs JSON with four keys: `prs` (all PRs), `new_completions`, `bug_fixes`, and `other`. Each entry has `number` and `title`.
+
+Review the categorization. The script uses simple heuristics (titles starting with "add" → new completion, titles containing "fix" or "bug" → bug fix). Move any miscategorized entries before composing the description.
+
+### 4. Create the branch and update dependencies
+
+```bash
+bash <skill-dir>/scripts/update_and_branch.sh "$NEW_HASH"
+```
+
+This checks out a new branch `completions-bot/update-command-signatures-<first 8 chars of NEW_HASH>` from `origin/master` in `~/warp`, updates the `rev` in `Cargo.toml`, and syncs `Cargo.lock`.
+
+### 5. Compose the PR description
+
+Write a PR description following this structure (match the warp PR template). Save it to a temp file, e.g. `/tmp/completions_pr_body.md`.
+
+```
+## Description
+
+Updates `warp-command-signatures` to <NEW_HASH short>.
+
+### Merged PRs
+- <PR title> (warpdotdev/command-signatures#<number>)
+- <PR title> (warpdotdev/command-signatures#<number>)
+- ...
+
+## Changelog Entries for Stable
+
+CHANGELOG-IMPROVEMENT: <single line covering all new completions, subcommands, generators, and updates>
+CHANGELOG-BUG-FIX: <single line covering all bug fixes>
+```
+
+Guidelines for changelog entries:
+- Write **one** `CHANGELOG-IMPROVEMENT` line that summarizes all new completions and improvements. If there are many commands, use only their short names. For example: "Added completions for `aws ec2`, `scp`, `claude`, and `docker-compose` service names."
+- Write **one** `CHANGELOG-BUG-FIX` line that summarizes all bug fixes. For example: "Fixed npm install short-form command priority and HTML entity rendering in completion specs."
+- Omit a changelog line entirely if there are no PRs in that category.
+- If a PR doesn't fit either category, use your judgment — it may belong in the improvement line or may not warrant a changelog entry at all.
+
+### 6. Submit the PR
+
+```bash
+bash <skill-dir>/scripts/submit_pr.sh "$NEW_HASH" <absolute_path_to_temp_description_file>
+```
+
+This commits the changes, pushes the branch, and opens a draft PR titled "[Completions] Bump command-signatures to <first 8 chars of NEW_HASH>" against `master`.

.agents/skills/update-command-spec-version/scripts/ensure_repos.sh

@@ -0,0 +1,25 @@
+#!/bin/bash
+set -euo pipefail
+
+WARP_DIR="${HOME}/warp"
+CMD_SIGS_DIR="${HOME}/command-signatures"
+
+if [ -d "${WARP_DIR}" ]; then
+    echo "Fetching warp..." >&2
+    git -C "${WARP_DIR}" fetch origin
+else
+    echo "Cloning warp..." >&2
+    git clone ssh://git@github.com/warpdotdev/warp.git "${WARP_DIR}"
+fi
+
+if [ -d "${CMD_SIGS_DIR}" ]; then
+    echo "Fetching command-signatures..." >&2
+    git -C "${CMD_SIGS_DIR}" fetch origin
+    git -C "${CMD_SIGS_DIR}" checkout main --quiet
+    git -C "${CMD_SIGS_DIR}" pull origin main --quiet
+else
+    echo "Cloning command-signatures..." >&2
+    git clone ssh://git@github.com/warpdotdev/command-signatures.git "${CMD_SIGS_DIR}"
+fi
+
+echo "Both repositories are ready." >&2

.agents/skills/update-command-spec-version/scripts/get_current_hash.sh

@@ -0,0 +1,18 @@
+#!/bin/bash
+set -euo pipefail
+
+WARP_DIR="${HOME}/warp"
+
+# Read Cargo.toml from origin/master so we don't depend on local checkout state.
+HASH=$(git -C "${WARP_DIR}" show origin/master:Cargo.toml \
+    | grep 'warp-command-signatures.*rev' \
+    | grep -o 'rev = "[^"]*"' \
+    | grep -o '"[^"]*"' \
+    | tr -d '"')
+
+if [ -z "${HASH}" ]; then
+    echo "Error: could not find warp-command-signatures rev in Cargo.toml" >&2
+    exit 1
+fi
+
+echo "${HASH}"

.agents/skills/update-command-spec-version/scripts/get_latest_hash.sh

@@ -0,0 +1,14 @@
+#!/bin/bash
+set -euo pipefail
+
+CMD_SIGS_DIR="${HOME}/command-signatures"
+
+# After ensure_repos.sh has pulled, origin/main is up to date.
+HASH=$(git -C "${CMD_SIGS_DIR}" rev-parse origin/main)
+
+if [ -z "${HASH}" ]; then
+    echo "Error: could not determine HEAD of command-signatures main" >&2
+    exit 1
+fi
+
+echo "${HASH}"