diff --git a/SKILL.md b/SKILL.md index 5ee5de76e3..79af51a5c3 100644 --- a/SKILL.md +++ b/SKILL.md @@ -945,6 +945,7 @@ Refs are invalidated on navigation — run `snapshot` again after `goto`. | `disconnect` | Disconnect headed browser, return to headless mode | | `focus [@ref]` | Bring headed browser window to foreground (macOS) | | `handoff [message]` | Open visible Chrome at current page for user takeover | +| `record start [path] [--size WxH] | record stop | record status` | Record video of browser activity to .webm. Useful as repro evidence for interactive bug findings. `start` saves session state, rebuilds the context with recordVideo enabled, and restores state; `stop` closes the context (which flushes the .webm files) and returns the paths; `status` prints the active recording dir. Calling `start` while already recording auto-stops the prior recording first. | | `restart` | Restart server | | `resume` | Re-snapshot after user takeover, return control to AI | | `state save|load ` | Save/load browser state (cookies + URLs) | diff --git a/browse/SKILL.md b/browse/SKILL.md index 1d544756c2..2ac866c674 100644 --- a/browse/SKILL.md +++ b/browse/SKILL.md @@ -625,6 +625,38 @@ $B screenshot /tmp/out.png --selector .tweet-card ``` Scale must be 1-3 (gstack policy cap). Changing `--scale` recreates the browser context; refs from `snapshot` are invalidated (rerun `snapshot`), but `load-html` content is replayed automatically. Not supported in headed mode. +### 14. Record video evidence for interactive bug repros +Static screenshots can't capture the timing of a flicker, the order of clicks that triggers a 500, or the cursor-following animation that breaks. For interactive bug findings, record a `.webm` of the repro: + +```bash +$B goto https://app.example.com/checkout +$B record start # auto-named dir under $TMPDIR +# (or: $B record start /tmp/checkout-bug --size 1280x720) + +$B snapshot -i # interact with the bug +$B fill @e3 "test@example.com" +$B click @e5 +# ... reproduce the bug ... + +$B record stop # flushes the .webm files, prints paths +# Recording saved: +# /tmp/browse-record-2026-05-13T18-42-15-000Z/.webm + +$B record status # confirm no active recording +``` + +`record` runs on the Playwright `recordVideo` context option, so: + +- Recording is **per-context**, not per-page — every page in the browser captures simultaneously. +- The `.webm` is only finalized when `record stop` runs (or when the daemon shuts down). +- Calling `record start` again while a recording is active auto-stops the prior one (single-recording invariant). +- `record start` and `record stop` both rebuild the browser context. The save/restore path preserves cookies and open page URLs, but **`@e` refs from `snapshot` are invalidated** — rerun `snapshot` after `record start` and after `record stop`. +- Output paths default to a timestamped dir under `$TMPDIR`. Pass `[path]` to pick a specific dir. +- `--size WxH` resizes the video frame independently of viewport (rarely needed; default uses current viewport). +- Not supported in headed mode (use a screen-record tool there). + +Use this when the repro involves any of: form submission, drag/drop, async loading state, scroll-triggered behavior, animations, focus management, dialog timing. + ## Puppeteer → browse cheatsheet Migrating from Puppeteer? Here's the 1:1 mapping for the core workflow: @@ -905,6 +937,7 @@ $B prettyscreenshot --cleanup --scroll-to ".pricing" --width 1440 ~/Desktop/hero | `disconnect` | Disconnect headed browser, return to headless mode | | `focus [@ref]` | Bring headed browser window to foreground (macOS) | | `handoff [message]` | Open visible Chrome at current page for user takeover | +| `record start [path] [--size WxH] | record stop | record status` | Record video of browser activity to .webm. Useful as repro evidence for interactive bug findings. `start` saves session state, rebuilds the context with recordVideo enabled, and restores state; `stop` closes the context (which flushes the .webm files) and returns the paths; `status` prints the active recording dir. Calling `start` while already recording auto-stops the prior recording first. | | `restart` | Restart server | | `resume` | Re-snapshot after user takeover, return control to AI | | `state save|load ` | Save/load browser state (cookies + URLs) | diff --git a/browse/SKILL.md.tmpl b/browse/SKILL.md.tmpl index a466fc4468..3bcf02c98b 100644 --- a/browse/SKILL.md.tmpl +++ b/browse/SKILL.md.tmpl @@ -135,6 +135,38 @@ $B screenshot /tmp/out.png --selector .tweet-card ``` Scale must be 1-3 (gstack policy cap). Changing `--scale` recreates the browser context; refs from `snapshot` are invalidated (rerun `snapshot`), but `load-html` content is replayed automatically. Not supported in headed mode. +### 14. Record video evidence for interactive bug repros +Static screenshots can't capture the timing of a flicker, the order of clicks that triggers a 500, or the cursor-following animation that breaks. For interactive bug findings, record a `.webm` of the repro: + +```bash +$B goto https://app.example.com/checkout +$B record start # auto-named dir under $TMPDIR +# (or: $B record start /tmp/checkout-bug --size 1280x720) + +$B snapshot -i # interact with the bug +$B fill @e3 "test@example.com" +$B click @e5 +# ... reproduce the bug ... + +$B record stop # flushes the .webm files, prints paths +# Recording saved: +# /tmp/browse-record-2026-05-13T18-42-15-000Z/.webm + +$B record status # confirm no active recording +``` + +`record` runs on the Playwright `recordVideo` context option, so: + +- Recording is **per-context**, not per-page — every page in the browser captures simultaneously. +- The `.webm` is only finalized when `record stop` runs (or when the daemon shuts down). +- Calling `record start` again while a recording is active auto-stops the prior one (single-recording invariant). +- `record start` and `record stop` both rebuild the browser context. The save/restore path preserves cookies and open page URLs, but **`@e` refs from `snapshot` are invalidated** — rerun `snapshot` after `record start` and after `record stop`. +- Output paths default to a timestamped dir under `$TMPDIR`. Pass `[path]` to pick a specific dir. +- `--size WxH` resizes the video frame independently of viewport (rarely needed; default uses current viewport). +- Not supported in headed mode (use a screen-record tool there). + +Use this when the repro involves any of: form submission, drag/drop, async loading state, scroll-triggered behavior, animations, focus management, dialog timing. + ## Puppeteer → browse cheatsheet Migrating from Puppeteer? Here's the 1:1 mapping for the core workflow: diff --git a/browse/src/browser-manager.ts b/browse/src/browser-manager.ts index cdbd5fc500..ab2fd26046 100644 --- a/browse/src/browser-manager.ts +++ b/browse/src/browser-manager.ts @@ -89,6 +89,15 @@ export class BrowserManager { private deviceScaleFactor: number = 1; private currentViewport: { width: number; height: number } = { width: 1280, height: 720 }; + // ─── Video recording (context option) ──────────────────────── + // Playwright records video at the context level via recordVideo: { dir, size? }. + // The .webm files are written when the context closes, so start/stop both go + // through recreateContext() — start flips the flag and rebuilds; stop collects + // the page.video() paths, clears the flag, and rebuilds again to flush. + private recordVideoDir: string | null = null; + private recordVideoSize: { width: number; height: number } | null = null; + private recordedVideoPaths: string[] = []; + /** Server port — set after server starts, used by cookie-import-browser command */ public serverPort: number = 0; @@ -260,6 +269,12 @@ export class BrowserManager { if (this.customUserAgent) { contextOptions.userAgent = this.customUserAgent; } + if (this.recordVideoDir) { + contextOptions.recordVideo = { + dir: this.recordVideoDir, + ...(this.recordVideoSize ? { size: this.recordVideoSize } : {}), + }; + } this.context = await this.browser.newContext(contextOptions); if (Object.keys(this.extraHeaders).length > 0) { @@ -1166,6 +1181,12 @@ export class BrowserManager { if (this.customUserAgent) { contextOptions.userAgent = this.customUserAgent; } + if (this.recordVideoDir) { + contextOptions.recordVideo = { + dir: this.recordVideoDir, + ...(this.recordVideoSize ? { size: this.recordVideoSize } : {}), + }; + } this.context = await this.browser.newContext(contextOptions); if (Object.keys(this.extraHeaders).length > 0) { @@ -1190,6 +1211,12 @@ export class BrowserManager { if (this.customUserAgent) { contextOptions.userAgent = this.customUserAgent; } + if (this.recordVideoDir) { + contextOptions.recordVideo = { + dir: this.recordVideoDir, + ...(this.recordVideoSize ? { size: this.recordVideoSize } : {}), + }; + } this.context = await this.browser!.newContext(contextOptions); await this.newTab(); this.clearRefs(); @@ -1200,6 +1227,87 @@ export class BrowserManager { } } + // ─── Video Recording ──────────────────────────────────────── + /** + * Begin recording video of subsequent browser activity to `dir`. + * + * Playwright records video at the context level — once a context is created + * with `recordVideo: { dir }`, every page in that context records a .webm. + * The file is finalized when the context closes. To toggle recording on we + * set the flag and recreate the context; the existing save/restore path in + * recreateContext() preserves cookies, URLs, and open pages. + * + * Single-recording invariant: calling startRecording() while already + * recording auto-stops the prior recording and starts a fresh one. Callers + * can call stopRecording() first if they want the paths of the prior + * recording. + */ + async startRecording(dir: string, size?: { width: number; height: number }): Promise { + if (this.connectionMode === 'headed') { + throw new Error('record is not supported in headed mode (use disconnect first).'); + } + if (this.recordVideoDir) { + // Already recording — flush the prior recording before starting fresh. + await this.stopRecording().catch(() => {}); + } + this.recordVideoDir = dir; + this.recordVideoSize = size ?? null; + this.recordedVideoPaths = []; + return this.recreateContext(); + } + + /** + * Stop the current recording and return the paths of the .webm files + * Playwright wrote (one per page that was open during the recording). + * + * Calling stopRecording() when not currently recording is a no-op and + * returns an empty array. + */ + async stopRecording(): Promise { + if (!this.recordVideoDir) return []; + + // Collect video file paths from each live page before tearing the + // context down. Playwright assigns the path eagerly so this resolves + // even before close, but we still need the context to close before + // the files are flushed to disk. + const paths: string[] = []; + for (const page of this.pages.values()) { + const video = page.video(); + if (!video) continue; + try { + const p = await video.path(); + if (p) paths.push(p); + } catch { + // Page may have been torn down already; skip. + } + } + + // Clear the flag so the rebuilt context does NOT continue recording. + this.recordVideoDir = null; + this.recordVideoSize = null; + + // Rebuilding the context closes the old one, which flushes the .webm files. + await this.recreateContext(); + + this.recordedVideoPaths = paths; + return paths; + } + + /** True iff recordVideo is currently enabled on the active context. */ + isRecording(): boolean { + return this.recordVideoDir !== null; + } + + /** The directory videos are written to, or null if not recording. */ + getRecordVideoDir(): string | null { + return this.recordVideoDir; + } + + /** Paths of the last completed recording, or [] if no recording has finished. */ + getRecordedVideoPaths(): string[] { + return [...this.recordedVideoPaths]; + } + /** * Change deviceScaleFactor + viewport size atomically. * diff --git a/browse/src/commands.ts b/browse/src/commands.ts index 1af127d51f..16e7aa0c49 100644 --- a/browse/src/commands.ts +++ b/browse/src/commands.ts @@ -41,6 +41,7 @@ export const META_COMMANDS = new Set([ 'watch', 'state', 'frame', + 'record', 'ux-audit', 'domain-skill', 'skill', @@ -168,6 +169,8 @@ export const COMMAND_DESCRIPTIONS: Record' }, + // Recording + 'record': { category: 'Server', description: 'Record video of browser activity to .webm. Useful as repro evidence for interactive bug findings. `start` saves session state, rebuilds the context with recordVideo enabled, and restores state; `stop` closes the context (which flushes the .webm files) and returns the paths; `status` prints the active recording dir. Calling `start` while already recording auto-stops the prior recording first.', usage: 'record start [path] [--size WxH] | record stop | record status' }, // Frame 'frame': { category: 'Meta', description: 'Switch to iframe context (or main to return)', usage: 'frame ' }, // CSS Inspector @@ -223,6 +226,7 @@ export function canonicalizeCommand(cmd: string): string { */ export const NEW_IN_VERSION: Record = { 'load-html': '0.19.0.0', + 'record': '1.35.0.0', }; /** diff --git a/browse/src/meta-commands.ts b/browse/src/meta-commands.ts index c505d4cf41..3933a177d1 100644 --- a/browse/src/meta-commands.ts +++ b/browse/src/meta-commands.ts @@ -981,6 +981,84 @@ export async function handleMetaCommand( throw new Error('Usage: state save|load '); } + // ─── Record (video) ──────────────────────────────────── + case 'record': { + const [action, ...rest] = args; + if (!action) { + throw new Error('Usage: record start [path] [--size WxH] | record stop | record status'); + } + + if (action === 'status') { + const dir = bm.getRecordVideoDir(); + if (!dir) return 'Not recording.'; + return `Recording → ${dir}`; + } + + if (action === 'stop') { + if (!bm.isRecording()) { + return 'No active recording (record stop is a no-op).'; + } + const paths = await bm.stopRecording(); + if (paths.length === 0) { + return 'Recording stopped. No video files were produced (no pages were open during the recording window?).'; + } + return `Recording saved:\n${paths.map(p => ` ${p}`).join('\n')}`; + } + + if (action === 'start') { + // Parse: record start [] [--size WxH] + let dirArg: string | undefined; + let sizeArg: string | undefined; + for (let i = 0; i < rest.length; i++) { + const tok = rest[i]; + if (tok === '--size') { + sizeArg = rest[++i]; + if (!sizeArg) throw new Error('record start --size: missing value (e.g. --size 1280x720)'); + } else if (tok.startsWith('--')) { + throw new Error(`Unknown record start flag: ${tok}`); + } else if (dirArg === undefined) { + dirArg = tok; + } else { + throw new Error(`Unexpected positional arg: ${tok}. Usage: record start [path] [--size WxH]`); + } + } + + // Default to a timestamped subdirectory under TEMP_DIR so concurrent + // recordings never collide and the user can find the file by mtime. + const defaultDir = path.join( + TEMP_DIR, + `browse-record-${new Date().toISOString().replace(/[:.]/g, '-')}`, + ); + const targetDir = dirArg ? path.resolve(dirArg) : defaultDir; + + // Reuse the standard output-path policy so a malicious caller can't + // smuggle videos to system dirs. + validateOutputPath(targetDir); + mkdirSecure(targetDir); + + let size: { width: number; height: number } | undefined; + if (sizeArg) { + if (!sizeArg.includes('x')) { + throw new Error('record start --size: expected WxH (e.g. 1280x720)'); + } + const [rawW, rawH] = sizeArg.split('x').map(Number); + if (!Number.isFinite(rawW) || !Number.isFinite(rawH) || rawW < 1 || rawH < 1) { + throw new Error(`record start --size: invalid dimensions '${sizeArg}'`); + } + size = { + width: Math.min(Math.max(Math.round(rawW), 1), 16384), + height: Math.min(Math.max(Math.round(rawH), 1), 16384), + }; + } + + const err = await bm.startRecording(targetDir, size); + if (err) return `Recording started with warnings: ${err}`; + return `Recording → ${targetDir}\n(call \`record stop\` to flush and get the .webm paths)`; + } + + throw new Error(`Unknown record action: '${action}'. Usage: record start|stop|status`); + } + // ─── Frame ─────────────────────────────────────── case 'frame': { const target = args[0]; diff --git a/browse/test/record.test.ts b/browse/test/record.test.ts new file mode 100644 index 0000000000..818b9e876b --- /dev/null +++ b/browse/test/record.test.ts @@ -0,0 +1,162 @@ +/** + * `record` command tests — video evidence for interactive bug repros. + * + * Covers: + * - record start enables recording, record stop returns the .webm path(s) + * - the written file is a non-empty WebM (magic bytes 0x1A 0x45 0xDF 0xA3) + * - the browser is still functional after stop (state preserved across recreate) + * - record stop with no active recording is a no-op + * - record start while already recording flushes the prior recording + * - validateOutputPath rejects out-of-sandbox dirs + */ + +import { describe, test, expect, beforeAll, afterAll } from 'bun:test'; +import * as fs from 'fs'; +import * as path from 'path'; +import { startTestServer } from './test-server'; +import { BrowserManager } from '../src/browser-manager'; +import { handleWriteCommand as _handleWriteCommand } from '../src/write-commands'; +import { handleMetaCommand } from '../src/meta-commands'; +import { TEMP_DIR } from '../src/platform'; + +const handleWriteCommand = (cmd: string, args: string[], b: BrowserManager) => + _handleWriteCommand(cmd, args, b.getActiveSession(), b); + +let testServer: ReturnType; +let bm: BrowserManager; +let baseUrl: string; +const shutdown = async () => {}; + +// Unique scratch dir per run — under TEMP_DIR so validateOutputPath's SAFE_DIRECTORIES +// guard accepts it (macOS resolves /tmp → /private/tmp; os.tmpdir() points elsewhere). +const scratchDir = path.join(TEMP_DIR, `gstack-record-test-${process.pid}-${Date.now()}`); + +beforeAll(async () => { + testServer = startTestServer(0); + baseUrl = testServer.url; + fs.mkdirSync(scratchDir, { recursive: true }); + + bm = new BrowserManager(); + await bm.launch(); +}); + +afterAll(() => { + try { testServer.server.stop(); } catch {} + try { fs.rmSync(scratchDir, { recursive: true, force: true }); } catch {} + setTimeout(() => process.exit(0), 500); +}); + +describe('record', () => { + test('record status reports not-recording before start', async () => { + const result = await handleMetaCommand('record', ['status'], bm, shutdown); + expect(result).toContain('Not recording'); + }); + + test('record stop without active recording is a no-op', async () => { + const result = await handleMetaCommand('record', ['stop'], bm, shutdown); + expect(result).toContain('No active recording'); + }); + + test('record start → activity → record stop produces non-empty .webm', async () => { + const target = path.join(scratchDir, 'basic-flow'); + + // Navigate before starting so we have something to record against + await handleWriteCommand('goto', [baseUrl + '/basic.html'], bm); + + const startMsg = await handleMetaCommand('record', ['start', target], bm, shutdown); + expect(startMsg).toContain('Recording → '); + expect(startMsg).toContain(target); + + // status reports active recording + const statusMid = await handleMetaCommand('record', ['status'], bm, shutdown); + expect(statusMid).toContain('Recording → '); + + // Do some real work so the video has frames to capture. + // recreateContext closed the old context and opened a new one — we need to + // re-navigate to populate the new context's page. + await handleWriteCommand('goto', [baseUrl + '/basic.html'], bm); + // Give the recorder a moment to capture frames. + await new Promise(r => setTimeout(r, 400)); + await handleWriteCommand('goto', [baseUrl + '/forms.html'], bm); + await new Promise(r => setTimeout(r, 400)); + + const stopMsg = await handleMetaCommand('record', ['stop'], bm, shutdown); + expect(stopMsg).toContain('Recording saved'); + + // Parse the paths back out of the message. + const lines = stopMsg.split('\n').map(l => l.trim()).filter(l => l.endsWith('.webm')); + expect(lines.length).toBeGreaterThan(0); + for (const p of lines) { + expect(fs.existsSync(p)).toBe(true); + const stat = fs.statSync(p); + expect(stat.size).toBeGreaterThan(0); + + // WebM magic bytes: 0x1A 0x45 0xDF 0xA3 (EBML header) + const fd = fs.openSync(p, 'r'); + const buf = Buffer.alloc(4); + fs.readSync(fd, buf, 0, 4, 0); + fs.closeSync(fd); + expect(buf[0]).toBe(0x1A); + expect(buf[1]).toBe(0x45); + expect(buf[2]).toBe(0xDF); + expect(buf[3]).toBe(0xA3); + } + + // status reports not-recording again + const statusEnd = await handleMetaCommand('record', ['status'], bm, shutdown); + expect(statusEnd).toContain('Not recording'); + }); + + test('browser remains functional after record stop', async () => { + // After the previous test stopped, we should still be able to navigate. + const result = await handleWriteCommand('goto', [baseUrl + '/basic.html'], bm); + expect(result).toBeDefined(); + // A snapshot should still return refs — proves the new context is wired. + const snap = await handleMetaCommand('snapshot', [], bm, shutdown); + expect(snap).toContain('@e'); + }); + + test('record start while already recording auto-stops the prior recording', async () => { + const firstDir = path.join(scratchDir, 'auto-stop-first'); + const secondDir = path.join(scratchDir, 'auto-stop-second'); + + await handleMetaCommand('record', ['start', firstDir], bm, shutdown); + expect(bm.isRecording()).toBe(true); + expect(bm.getRecordVideoDir()).toBe(firstDir); + + await handleWriteCommand('goto', [baseUrl + '/basic.html'], bm); + await new Promise(r => setTimeout(r, 200)); + + // Starting again should not throw and should swap the active dir. + await handleMetaCommand('record', ['start', secondDir], bm, shutdown); + expect(bm.isRecording()).toBe(true); + expect(bm.getRecordVideoDir()).toBe(secondDir); + + await handleMetaCommand('record', ['stop'], bm, shutdown); + expect(bm.isRecording()).toBe(false); + }); + + test('record start rejects unknown flags', async () => { + await expect( + handleMetaCommand('record', ['start', '--bogus'], bm, shutdown), + ).rejects.toThrow(/Unknown record start flag/); + }); + + test('record start rejects malformed --size', async () => { + await expect( + handleMetaCommand('record', ['start', '--size', 'not-a-size'], bm, shutdown), + ).rejects.toThrow(/expected WxH/); + }); + + test('record with no action throws a usage error', async () => { + await expect( + handleMetaCommand('record', [], bm, shutdown), + ).rejects.toThrow(/Usage: record/); + }); + + test('unknown record subaction throws', async () => { + await expect( + handleMetaCommand('record', ['foo'], bm, shutdown), + ).rejects.toThrow(/Unknown record action/); + }); +}); diff --git a/gstack/llms.txt b/gstack/llms.txt index 8c5d4a3924..96d2c07b86 100644 --- a/gstack/llms.txt +++ b/gstack/llms.txt @@ -134,6 +134,7 @@ Run with `browse [args]`. Full reference: `browse/SKILL.md`. - `disconnect`: Disconnect headed browser, return to headless mode - `focus [@ref]`: Bring headed browser window to foreground (macOS) - `handoff [message]`: Open visible Chrome at current page for user takeover +- `record start [path] [--size WxH] | record stop | record status`: Record video of browser activity to .webm. - `restart`: Restart server - `resume`: Re-snapshot after user takeover, return control to AI - `state save|load `: Save/load browser state (cookies + URLs)