Skip to content

Commit 9d01958

Browse files
adamfweidmanadamfweidman
authored
feat(skills): add agent-tui and tui-tester skills (#27121)
1 parent 84423e6 commit 9d01958

2 files changed

Lines changed: 395 additions & 0 deletions

File tree

‎.gemini/skills/agent-tui/SKILL.md‎

Lines changed: 330 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,330 @@
1+
---
2+
name: agent-tui
3+
description: >
4+
Main Agents: Do NOT use this skill directly. If you need to test the TUI, invoke the `tui_tester` subagent.
5+
Drive terminal UI (TUI) applications programmatically for testing, automation, and inspection.
6+
Use when: automating CLI/TUI interactions, regression testing terminal apps, or verifying interactive behavior.
7+
Also use when: user asks "what is agent-tui", "what does agent-tui do", "demo agent-tui", "show me agent-tui", "how does agent-tui work", or wants to see it in action.
8+
---
9+
10+
## 🚨 CRITICAL: macOS Daemon Workaround & Gemini CLI Usage 🚨
11+
12+
When using `agent-tui` in this macOS environment, the default background daemonization process crashes, causing `Connection refused (os error 61)` errors.
13+
14+
**You MUST start the daemon manually shielded from TTY hangups before running any `agent-tui` commands.** Using `nohup` is insufficient; you must use `tmux` to provide a fully isolated pseudo-terminal.
15+
16+
To support parallel runs, **only restart the daemon if it is not currently running:**
17+
18+
```bash
19+
# Check if daemon is alive, start it in tmux if it is not
20+
if ! agent-tui sessions >/dev/null 2>&1; then
21+
tmux kill-session -t agent-tui 2>/dev/null || true
22+
agent-tui daemon stop 2>/dev/null || true
23+
rm -f /tmp/agent-tui*
24+
tmux new-session -d -s agent-tui 'agent-tui daemon start --foreground > /tmp/agent-tui-daemon.log 2>&1'
25+
sleep 1
26+
fi
27+
```
28+
29+
### Session ID vs PID (Crucial for Reconnection)
30+
31+
When `agent-tui run` returns JSON, it includes both a `session_id` and a `pid`. The `pid` is purely informational (the OS process ID of the child command). You **do not** use the `pid` to reconnect or issue commands. You must always use the `session_id` (e.g., `--session <id>`).
32+
33+
If the daemon crashes (`os error 61`), the pseudo-terminal is destroyed. Even if the child `pid` survives as an orphan, you cannot reconnect to it. You must restart the daemon using the workaround above and start a completely new session.
34+
35+
### Testing the Gemini CLI
36+
37+
When testing the Gemini CLI with `agent-tui`, there are several strict requirements to ensure deterministic and accurate behavior:
38+
39+
1. **Build Before Running**: `agent-tui` runs the built JS files, not TypeScript. You **MUST** run `npm run build` or `npm run build:all` after making code changes and before launching the CLI with `agent-tui`.
40+
2. **Bypass Trust Modals**: Always pass `GEMINI_CLI_TRUST_WORKSPACE=true` in the environment. If you don't, any new project-level agents or extensions will trigger a full-screen "Acknowledge and Enable" modal. This modal steals focus, swallows automation keystrokes, and causes `agent-tui wait` commands to time out.
41+
3. **Isolated Environments**: If you need to test without real user credentials or existing agents interfering, isolate the global settings using `GEMINI_CLI_HOME=<some-test-dir>`.
42+
4. **Testing State Deltas (e.g., Reloads)**: If you are testing features that report deltas (e.g., `/agents reload` outputting "1 new local subagent"), you **MUST**:
43+
- Start the CLI *first* so it establishes its baseline registry.
44+
- Use a separate shell command (outside of `agent-tui`) to write the new agent `.md`/`.toml` file.
45+
- Use `agent-tui type` and `press` to trigger the `/agents reload` command inside the running session.
46+
- (If you add the files before starting the CLI, they become part of the baseline and won't trigger the delta logic).
47+
48+
```bash
49+
# Example: Standard isolated run (sandboxed config + bypass trust modals)
50+
env GEMINI_CLI_TRUST_WORKSPACE=true GEMINI_CLI_HOME=test-gemini-home agent-tui run -d "$(pwd)" node packages/cli/dist/index.js
51+
```
52+
53+
# Terminal Automation Mastery
54+
55+
## Prerequisites
56+
57+
- **Supported OS**: macOS or Linux (Windows not supported yet).
58+
- **Verify install**:
59+
60+
```bash
61+
agent-tui --version
62+
```
63+
64+
If not installed, use one of:
65+
66+
```bash
67+
# Recommended: one-line install (macOS/Linux)
68+
curl -fsSL https://raw.githubusercontent.com/pproenca/agent-tui/master/install.sh | sh
69+
```
70+
71+
```bash
72+
# Package manager
73+
npm i -g agent-tui
74+
pnpm add -g agent-tui
75+
bun add -g agent-tui
76+
```
77+
78+
```bash
79+
# Build from source
80+
cargo install --git https://github.com/pproenca/agent-tui.git --path cli/crates/agent-tui
81+
```
82+
83+
If you used the install script, ensure `~/.local/bin` is on your PATH.
84+
85+
## Philosophy: Why Terminal Automation Is Different
86+
87+
Terminal UIs are **stateless from the observer's perspective**. Unlike web browsers with a persistent DOM, terminal automation works with a constantly-refreshed character grid. This fundamental difference shapes everything:
88+
89+
| Web Automation | Terminal Automation |
90+
|----------------|---------------------|
91+
| DOM persists across interactions | Screen buffer is redrawn constantly |
92+
| Selectors are stable | Text positions may shift |
93+
| Query once, act many times | Must re-verify before EVERY action |
94+
| Network events signal completion | Must detect visual stability |
95+
96+
**The Core Insight**: agent-tui gives you vision without memory. Each screenshot is a fresh observation. Previous state means nothing after the UI changes. This isn't a limitation—it's the nature of terminal interaction.
97+
98+
## Mental Model: The Feedback Loop
99+
100+
Think of terminal automation as a **closed-loop control system**:
101+
102+
```
103+
┌──────────────────────────────────────────────┐
104+
│ │
105+
▼ │
106+
OBSERVE ──► DECIDE ──► ACT ──► WAIT ──► VERIFY ───┘
107+
│ │
108+
│ │
109+
└─────── NEVER skip ◄────────────────────┘
110+
```
111+
112+
**Each phase is mandatory.** Skipping verification is the #1 cause of flaky automation.
113+
114+
### The "Fresh Eyes" Principle
115+
116+
Every time you need to interact with the UI:
117+
118+
1. **Take a fresh screenshot** — your previous one is now stale
119+
2. **Locate your target visually** — text positions may have changed
120+
3. **Verify the state** — the UI may have changed unexpectedly
121+
4. **Act only when stable** — animations and loading states cause failures
122+
123+
This feels slower, but it's the only reliable approach. Optimistic reuse of stale state causes intermittent failures that are painful to debug.
124+
125+
## Critical Rules (Non-Negotiable)
126+
127+
> **RULE 1: Atomic Execution (No Pipelining)**
128+
> You are FORBIDDEN from chaining commands with `&&` (e.g., `type "x" && press Enter && wait`). Modals or UI updates can intercept your keystrokes. You MUST execute one atomic action, wait, screenshot, and verify before taking the next action in a new turn.
129+
130+
> **RULE 2: Re-snapshot after EVERY action**
131+
> The UI state is invalidated by any change. Always take a fresh screenshot before acting again.
132+
133+
> **RULE 3: Never act on unstable UI**
134+
> If the UI is animating, loading, or transitioning, `wait --stable` first. Acting during transitions because race conditions.
135+
136+
> **RULE 4: Verify before claiming success**
137+
> Use `wait "expected text" --assert` to confirm outcomes. Don't assume an action worked—prove it.
138+
139+
> **RULE 5: Error Recovery**
140+
> If a `wait` command times out, DO NOT blindly restart or kill the session. Execute `screenshot` to visually diagnose what unexpected UI element (modal, error dialog, lost focus) intercepted the flow.
141+
142+
> **RULE 6: Clean up sessions**
143+
> Always end with `agent-tui kill`. Orphaned sessions consume resources and can interfere with future runs.
144+
145+
## Decision Framework
146+
147+
### Which Screenshot Mode?
148+
149+
Use `screenshot --format json` when parsing automation output, or plain `screenshot` for human readable text.
150+
151+
### How to Wait?
152+
153+
```
154+
What are you waiting for?
155+
│
156+
├─► Specific text to appear
157+
│ └─► `wait "text" --assert` (fails if not found)
158+
│
159+
├─► Specific text to disappear
160+
│ └─► `wait "text" --gone --assert`
161+
│
162+
├─► UI to stop changing (animations, loading)
163+
│ └─► `wait --stable`
164+
│
165+
└─► Multiple conditions
166+
└─► Chain waits sequentially
167+
```
168+
169+
### How to Act?
170+
171+
```
172+
What do you need to do?
173+
│
174+
├─► Type text into the terminal
175+
│ └─► `type "text"`
176+
│
177+
├─► Send keyboard shortcuts/navigation
178+
│ └─► `press Ctrl+C` or `press ArrowDown Enter`
179+
```
180+
181+
## Core Workflow
182+
183+
The canonical automation loop:
184+
185+
```bash
186+
# 1. START: Launch the TUI app
187+
agent-tui run <command> [-- args...]
188+
189+
# 2. OBSERVE: Get current UI state
190+
agent-tui screenshot --format json
191+
192+
# 3. DECIDE: Based on text, determine next action
193+
# (This happens in your head/code)
194+
195+
# 4. ACT: Execute the action
196+
agent-tui type "text"
197+
agent-tui press Enter
198+
199+
# 5. WAIT: Synchronize with UI changes
200+
agent-tui wait "Expected" --assert # or wait --stable
201+
202+
# 6. VERIFY: Confirm the outcome (often combined with step 5)
203+
# If verification fails, handle the error
204+
205+
# 7. REPEAT: Go back to step 2 until done
206+
207+
# 8. CLEANUP: Always clean up
208+
agent-tui kill
209+
```
210+
211+
## Anti-Patterns (What NOT to Do)
212+
213+
### ❌ Acting During Animation/Loading
214+
215+
```bash
216+
# WRONG: Acting immediately on dynamic UI
217+
agent-tui run my-app
218+
agent-tui screenshot --format json # UI might still be loading!
219+
agent-tui type "value" # ❌ Might miss the input field
220+
221+
# RIGHT: Wait for stability first
222+
agent-tui run my-app
223+
agent-tui wait --stable # Let UI settle
224+
agent-tui screenshot --format json # Now it's reliable
225+
agent-tui type "value"
226+
```
227+
228+
### ❌ Assuming Success Without Verification
229+
230+
```bash
231+
# WRONG: Assuming the type worked
232+
agent-tui type "value"
233+
agent-tui press Enter
234+
# ...proceed as if success... # ❌ What if it failed silently?
235+
236+
# RIGHT: Verify the outcome
237+
agent-tui type "value"
238+
agent-tui press Enter
239+
agent-tui wait "Success" --assert # ✓ Proves the action worked
240+
```
241+
242+
### ❌ Skipping Cleanup
243+
244+
```bash
245+
# WRONG: Forgetting to kill the session
246+
agent-tui run my-app
247+
# ...do stuff...
248+
# script ends # ❌ Session left running!
249+
250+
# RIGHT: Always clean up
251+
agent-tui run my-app
252+
# ...do stuff...
253+
agent-tui kill # ✓ Clean exit
254+
```
255+
256+
## Before You Start: Clarify Requirements
257+
258+
Before automating any TUI, gather this information:
259+
260+
1. **Command**: What exactly to run? (`my-app --flag` or `npm start`?)
261+
2. **Success criteria**: What text/state indicates success?
262+
3. **Input sequence**: What keystrokes/data to enter, in what order?
263+
4. **Safety**: Is it safe to submit forms, delete data, etc.?
264+
5. **Auth**: Does it need login? Test credentials?
265+
6. **Live preview**: Does the user want to watch? (`agent-tui live start --open`)
266+
267+
If any of these are unclear, ask before running.
268+
269+
## Demo Mode: Showing What agent-tui Can Do
270+
271+
When a user asks what agent-tui is, wants a demo, or asks "show me how it works":
272+
273+
1. **Don't explain—demonstrate.** Actions speak louder than words.
274+
2. **Use the live preview** so they can watch in real-time.
275+
3. **Run `top`**—it's universal and shows dynamic real-time updates.
276+
277+
**Quick demo trigger phrases:**
278+
- "What is agent-tui?" / "What does agent-tui do?"
279+
- "Demo agent-tui" / "Show me agent-tui"
280+
- "How does agent-tui work?" / "See it in action"
281+
282+
## Failure Recovery
283+
284+
| Symptom | Diagnosis | Solution |
285+
|---------|-----------|----------|
286+
| "Text not found" | Stale view or text moved | Re-snapshot, locate text again |
287+
| Wait times out | UI didn't reach expected state | Check screenshot, verify expectations |
288+
| "Daemon not running" | Daemon crashed or not started | `agent-tui daemon start` |
289+
| Unexpected layout | Wrong terminal size | `agent-tui resize --cols 120 --rows 40` |
290+
| Session unresponsive | App crashed or hung | `agent-tui kill`, then re-run |
291+
| Repeated failures | Something fundamentally wrong | Stop after 3-5 attempts, ask user |
292+
293+
## Self-Discovery: Use --help
294+
295+
You don't need to memorize every flag. The CLI is self-documenting:
296+
297+
```bash
298+
agent-tui --help # List all commands
299+
agent-tui run --help # Options for 'run'
300+
agent-tui screenshot --help # Options for 'screenshot'
301+
agent-tui wait --help # Options for 'wait'
302+
```
303+
304+
**When in doubt, ask the CLI.** This skill teaches *when* and *why* to use commands. For exact flags and syntax, `--help` is authoritative.
305+
306+
## Quick Reference
307+
308+
```bash
309+
# Start app
310+
agent-tui run <cmd> [-- args] # Launch TUI under control
311+
312+
# Observe
313+
agent-tui screenshot # Plain text view
314+
agent-tui screenshot --format json # Machine-readable output
315+
316+
# Act
317+
agent-tui press Enter # Press key(s)
318+
agent-tui press Ctrl+C # Keyboard shortcuts
319+
agent-tui type "text" # Type text
320+
321+
# Wait/Verify
322+
agent-tui wait "text" --assert # Wait for text, fail if not found
323+
agent-tui wait "text" --gone --assert # Wait for text to disappear
324+
agent-tui wait --stable # Wait for UI to stop changing
325+
326+
# Manage
327+
agent-tui sessions # List active sessions
328+
agent-tui live start --open # Start live preview
329+
agent-tui kill # End current session
330+
```

‎.gemini/skills/tui-tester/SKILL.md‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
---
2+
name: tui-tester
3+
description: Expert guidance for testing Gemini CLI behavior and visual output using terminal automation.
4+
---
5+
6+
# TUI Tester Skill
7+
8+
This skill provides the operational manual for verifying Gemini CLI behavioral changes and visual output using terminal automation.
9+
10+
## Core Responsibilities
11+
12+
- **Verify Behavior**: Confirm that code changes result in the expected terminal interactions.
13+
- **Visual Validation**: Ensure the TUI renders correctly across different terminal sizes and states.
14+
- **Regression Testing**: Use automation to prevent breaking existing interactive workflows.
15+
16+
## Critical Protocol
17+
18+
When performing TUI testing, you must adhere to these strict rules:
19+
20+
### 1. Initialization
21+
**YOUR ABSOLUTE FIRST ACTION MUST BE:**
22+
Activate the `agent-tui` skill. This provides the underlying tools needed for terminal automation.
23+
24+
### 2. Environment Setup (macOS / Parallel Safe)
25+
Ensure the global daemon is running and the live preview is open:
26+
```bash
27+
if ! agent-tui sessions >/dev/null 2>&1; then
28+
tmux kill-session -t agent-tui 2>/dev/null || true
29+
agent-tui daemon stop 2>/dev/null || true
30+
rm -f /tmp/agent-tui*
31+
tmux new-session -d -s agent-tui 'agent-tui daemon start --foreground > /tmp/agent-tui-daemon.log 2>&1'
32+
sleep 1
33+
fi
34+
agent-tui live start --open
35+
```
36+
37+
### 3. Session Management
38+
- **Session IDs**: Always use the `session_id` returned by `agent-tui run` for subsequent interactions.
39+
- **Atomic Execution**: Execute exactly one command per turn. Do not pipeline actions.
40+
- **The Loop**: Action -> Wait -> Screenshot -> Verify -> Next Action.
41+
42+
### 4. Gemini CLI Specifics
43+
- **Build First**: Always run `npm run build` or `npm run build:all` before testing local changes.
44+
- **Bypass Trust**: Set `GEMINI_CLI_TRUST_WORKSPACE=true` to avoid focus-stealing modals.
45+
- **Isolate Config**: Use `GEMINI_CLI_HOME` to prevent interference with your personal settings.
46+
47+
## Workflow Example
48+
49+
```bash
50+
# Start the CLI
51+
env GEMINI_CLI_TRUST_WORKSPACE=true agent-tui run node packages/cli/dist/index.js
52+
53+
# Wait for the prompt
54+
agent-tui wait "│" --assert
55+
56+
# Send a command
57+
agent-tui type "/help"
58+
agent-tui press Enter
59+
60+
# Verify output
61+
agent-tui wait "Available Commands" --assert
62+
```
63+
64+
## Error Recovery
65+
If a wait times out, take a fresh screenshot to diagnose the state. If you see `os error 61`, restart the daemon using the tmux method.

0 commit comments

Comments
 (0)