Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -945,6 +945,7 @@ Refs are invalidated on navigation — run `snapshot` again after `goto`.
| `disconnect` | Disconnect headed browser, return to headless mode |
| `focus [@ref]` | Bring headed browser window to foreground (macOS) |
| `handoff [message]` | Open visible Chrome at current page for user takeover |
| `record start [path] [--size WxH] | record stop | record status` | Record video of browser activity to .webm. Useful as repro evidence for interactive bug findings. `start` saves session state, rebuilds the context with recordVideo enabled, and restores state; `stop` closes the context (which flushes the .webm files) and returns the paths; `status` prints the active recording dir. Calling `start` while already recording auto-stops the prior recording first. |
| `restart` | Restart server |
| `resume` | Re-snapshot after user takeover, return control to AI |
| `state save|load <name>` | Save/load browser state (cookies + URLs) |
Expand Down
33 changes: 33 additions & 0 deletions browse/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -625,6 +625,38 @@ $B screenshot /tmp/out.png --selector .tweet-card
```
Scale must be 1-3 (gstack policy cap). Changing `--scale` recreates the browser context; refs from `snapshot` are invalidated (rerun `snapshot`), but `load-html` content is replayed automatically. Not supported in headed mode.

### 14. Record video evidence for interactive bug repros
Static screenshots can't capture the timing of a flicker, the order of clicks that triggers a 500, or the cursor-following animation that breaks. For interactive bug findings, record a `.webm` of the repro:

```bash
$B goto https://app.example.com/checkout
$B record start # auto-named dir under $TMPDIR
# (or: $B record start /tmp/checkout-bug --size 1280x720)

$B snapshot -i # interact with the bug
$B fill @e3 "test@example.com"
$B click @e5
# ... reproduce the bug ...

$B record stop # flushes the .webm files, prints paths
# Recording saved:
# /tmp/browse-record-2026-05-13T18-42-15-000Z/<page-id>.webm

$B record status # confirm no active recording
```

`record` runs on the Playwright `recordVideo` context option, so:

- Recording is **per-context**, not per-page — every page in the browser captures simultaneously.
- The `.webm` is only finalized when `record stop` runs (or when the daemon shuts down).
- Calling `record start` again while a recording is active auto-stops the prior one (single-recording invariant).
- `record start` and `record stop` both rebuild the browser context. The save/restore path preserves cookies and open page URLs, but **`@e` refs from `snapshot` are invalidated** — rerun `snapshot` after `record start` and after `record stop`.
- Output paths default to a timestamped dir under `$TMPDIR`. Pass `[path]` to pick a specific dir.
- `--size WxH` resizes the video frame independently of viewport (rarely needed; default uses current viewport).
- Not supported in headed mode (use a screen-record tool there).

Use this when the repro involves any of: form submission, drag/drop, async loading state, scroll-triggered behavior, animations, focus management, dialog timing.

## Puppeteer → browse cheatsheet

Migrating from Puppeteer? Here's the 1:1 mapping for the core workflow:
Expand Down Expand Up @@ -905,6 +937,7 @@ $B prettyscreenshot --cleanup --scroll-to ".pricing" --width 1440 ~/Desktop/hero
| `disconnect` | Disconnect headed browser, return to headless mode |
| `focus [@ref]` | Bring headed browser window to foreground (macOS) |
| `handoff [message]` | Open visible Chrome at current page for user takeover |
| `record start [path] [--size WxH] | record stop | record status` | Record video of browser activity to .webm. Useful as repro evidence for interactive bug findings. `start` saves session state, rebuilds the context with recordVideo enabled, and restores state; `stop` closes the context (which flushes the .webm files) and returns the paths; `status` prints the active recording dir. Calling `start` while already recording auto-stops the prior recording first. |
| `restart` | Restart server |
| `resume` | Re-snapshot after user takeover, return control to AI |
| `state save|load <name>` | Save/load browser state (cookies + URLs) |
Expand Down
32 changes: 32 additions & 0 deletions browse/SKILL.md.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,38 @@ $B screenshot /tmp/out.png --selector .tweet-card
```
Scale must be 1-3 (gstack policy cap). Changing `--scale` recreates the browser context; refs from `snapshot` are invalidated (rerun `snapshot`), but `load-html` content is replayed automatically. Not supported in headed mode.

### 14. Record video evidence for interactive bug repros
Static screenshots can't capture the timing of a flicker, the order of clicks that triggers a 500, or the cursor-following animation that breaks. For interactive bug findings, record a `.webm` of the repro:

```bash
$B goto https://app.example.com/checkout
$B record start # auto-named dir under $TMPDIR
# (or: $B record start /tmp/checkout-bug --size 1280x720)

$B snapshot -i # interact with the bug
$B fill @e3 "test@example.com"
$B click @e5
# ... reproduce the bug ...

$B record stop # flushes the .webm files, prints paths
# Recording saved:
# /tmp/browse-record-2026-05-13T18-42-15-000Z/<page-id>.webm

$B record status # confirm no active recording
```

`record` runs on the Playwright `recordVideo` context option, so:

- Recording is **per-context**, not per-page — every page in the browser captures simultaneously.
- The `.webm` is only finalized when `record stop` runs (or when the daemon shuts down).
- Calling `record start` again while a recording is active auto-stops the prior one (single-recording invariant).
- `record start` and `record stop` both rebuild the browser context. The save/restore path preserves cookies and open page URLs, but **`@e` refs from `snapshot` are invalidated** — rerun `snapshot` after `record start` and after `record stop`.
- Output paths default to a timestamped dir under `$TMPDIR`. Pass `[path]` to pick a specific dir.
- `--size WxH` resizes the video frame independently of viewport (rarely needed; default uses current viewport).
- Not supported in headed mode (use a screen-record tool there).

Use this when the repro involves any of: form submission, drag/drop, async loading state, scroll-triggered behavior, animations, focus management, dialog timing.

## Puppeteer → browse cheatsheet

Migrating from Puppeteer? Here's the 1:1 mapping for the core workflow:
Expand Down
108 changes: 108 additions & 0 deletions browse/src/browser-manager.ts
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,15 @@ export class BrowserManager {
private deviceScaleFactor: number = 1;
private currentViewport: { width: number; height: number } = { width: 1280, height: 720 };

// ─── Video recording (context option) ────────────────────────
// Playwright records video at the context level via recordVideo: { dir, size? }.
// The .webm files are written when the context closes, so start/stop both go
// through recreateContext() — start flips the flag and rebuilds; stop collects
// the page.video() paths, clears the flag, and rebuilds again to flush.
private recordVideoDir: string | null = null;
private recordVideoSize: { width: number; height: number } | null = null;
private recordedVideoPaths: string[] = [];

/** Server port — set after server starts, used by cookie-import-browser command */
public serverPort: number = 0;

Expand Down Expand Up @@ -260,6 +269,12 @@ export class BrowserManager {
if (this.customUserAgent) {
contextOptions.userAgent = this.customUserAgent;
}
if (this.recordVideoDir) {
contextOptions.recordVideo = {
dir: this.recordVideoDir,
...(this.recordVideoSize ? { size: this.recordVideoSize } : {}),
};
}
this.context = await this.browser.newContext(contextOptions);

if (Object.keys(this.extraHeaders).length > 0) {
Expand Down Expand Up @@ -1166,6 +1181,12 @@ export class BrowserManager {
if (this.customUserAgent) {
contextOptions.userAgent = this.customUserAgent;
}
if (this.recordVideoDir) {
contextOptions.recordVideo = {
dir: this.recordVideoDir,
...(this.recordVideoSize ? { size: this.recordVideoSize } : {}),
};
}
this.context = await this.browser.newContext(contextOptions);

if (Object.keys(this.extraHeaders).length > 0) {
Expand All @@ -1190,6 +1211,12 @@ export class BrowserManager {
if (this.customUserAgent) {
contextOptions.userAgent = this.customUserAgent;
}
if (this.recordVideoDir) {
contextOptions.recordVideo = {
dir: this.recordVideoDir,
...(this.recordVideoSize ? { size: this.recordVideoSize } : {}),
};
}
this.context = await this.browser!.newContext(contextOptions);
await this.newTab();
this.clearRefs();
Expand All @@ -1200,6 +1227,87 @@ export class BrowserManager {
}
}

// ─── Video Recording ────────────────────────────────────────
/**
* Begin recording video of subsequent browser activity to `dir`.
*
* Playwright records video at the context level — once a context is created
* with `recordVideo: { dir }`, every page in that context records a .webm.
* The file is finalized when the context closes. To toggle recording on we
* set the flag and recreate the context; the existing save/restore path in
* recreateContext() preserves cookies, URLs, and open pages.
*
* Single-recording invariant: calling startRecording() while already
* recording auto-stops the prior recording and starts a fresh one. Callers
* can call stopRecording() first if they want the paths of the prior
* recording.
*/
async startRecording(dir: string, size?: { width: number; height: number }): Promise<string | null> {
if (this.connectionMode === 'headed') {
throw new Error('record is not supported in headed mode (use disconnect first).');
}
if (this.recordVideoDir) {
// Already recording — flush the prior recording before starting fresh.
await this.stopRecording().catch(() => {});
}
this.recordVideoDir = dir;
this.recordVideoSize = size ?? null;
this.recordedVideoPaths = [];
return this.recreateContext();
}

/**
* Stop the current recording and return the paths of the .webm files
* Playwright wrote (one per page that was open during the recording).
*
* Calling stopRecording() when not currently recording is a no-op and
* returns an empty array.
*/
async stopRecording(): Promise<string[]> {
if (!this.recordVideoDir) return [];

// Collect video file paths from each live page before tearing the
// context down. Playwright assigns the path eagerly so this resolves
// even before close, but we still need the context to close before
// the files are flushed to disk.
const paths: string[] = [];
for (const page of this.pages.values()) {
const video = page.video();
if (!video) continue;
try {
const p = await video.path();
if (p) paths.push(p);
} catch {
// Page may have been torn down already; skip.
}
}

// Clear the flag so the rebuilt context does NOT continue recording.
this.recordVideoDir = null;
this.recordVideoSize = null;

// Rebuilding the context closes the old one, which flushes the .webm files.
await this.recreateContext();

this.recordedVideoPaths = paths;
return paths;
}

/** True iff recordVideo is currently enabled on the active context. */
isRecording(): boolean {
return this.recordVideoDir !== null;
}

/** The directory videos are written to, or null if not recording. */
getRecordVideoDir(): string | null {
return this.recordVideoDir;
}

/** Paths of the last completed recording, or [] if no recording has finished. */
getRecordedVideoPaths(): string[] {
return [...this.recordedVideoPaths];
}

/**
* Change deviceScaleFactor + viewport size atomically.
*
Expand Down
4 changes: 4 additions & 0 deletions browse/src/commands.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ export const META_COMMANDS = new Set([
'watch',
'state',
'frame',
'record',
'ux-audit',
'domain-skill',
'skill',
Expand Down Expand Up @@ -168,6 +169,8 @@ export const COMMAND_DESCRIPTIONS: Record<string, { category: string; descriptio
'watch': { category: 'Meta', description: 'Passive observation — periodic snapshots while user browses', usage: 'watch [stop]' },
// State
'state': { category: 'Server', description: 'Save/load browser state (cookies + URLs)', usage: 'state save|load <name>' },
// Recording
'record': { category: 'Server', description: 'Record video of browser activity to .webm. Useful as repro evidence for interactive bug findings. `start` saves session state, rebuilds the context with recordVideo enabled, and restores state; `stop` closes the context (which flushes the .webm files) and returns the paths; `status` prints the active recording dir. Calling `start` while already recording auto-stops the prior recording first.', usage: 'record start [path] [--size WxH] | record stop | record status' },
// Frame
'frame': { category: 'Meta', description: 'Switch to iframe context (or main to return)', usage: 'frame <sel|@ref|--name n|--url pattern|main>' },
// CSS Inspector
Expand Down Expand Up @@ -223,6 +226,7 @@ export function canonicalizeCommand(cmd: string): string {
*/
export const NEW_IN_VERSION: Record<string, string> = {
'load-html': '0.19.0.0',
'record': '1.35.0.0',
};

/**
Expand Down
78 changes: 78 additions & 0 deletions browse/src/meta-commands.ts
Original file line number Diff line number Diff line change
Expand Up @@ -981,6 +981,84 @@ export async function handleMetaCommand(
throw new Error('Usage: state save|load <name>');
}

// ─── Record (video) ────────────────────────────────────
case 'record': {
const [action, ...rest] = args;
if (!action) {
throw new Error('Usage: record start [path] [--size WxH] | record stop | record status');
}

if (action === 'status') {
const dir = bm.getRecordVideoDir();
if (!dir) return 'Not recording.';
return `Recording → ${dir}`;
}

if (action === 'stop') {
if (!bm.isRecording()) {
return 'No active recording (record stop is a no-op).';
}
const paths = await bm.stopRecording();
if (paths.length === 0) {
return 'Recording stopped. No video files were produced (no pages were open during the recording window?).';
}
return `Recording saved:\n${paths.map(p => ` ${p}`).join('\n')}`;
}

if (action === 'start') {
// Parse: record start [<dir>] [--size WxH]
let dirArg: string | undefined;
let sizeArg: string | undefined;
for (let i = 0; i < rest.length; i++) {
const tok = rest[i];
if (tok === '--size') {
sizeArg = rest[++i];
if (!sizeArg) throw new Error('record start --size: missing value (e.g. --size 1280x720)');
} else if (tok.startsWith('--')) {
throw new Error(`Unknown record start flag: ${tok}`);
} else if (dirArg === undefined) {
dirArg = tok;
} else {
throw new Error(`Unexpected positional arg: ${tok}. Usage: record start [path] [--size WxH]`);
}
}

// Default to a timestamped subdirectory under TEMP_DIR so concurrent
// recordings never collide and the user can find the file by mtime.
const defaultDir = path.join(
TEMP_DIR,
`browse-record-${new Date().toISOString().replace(/[:.]/g, '-')}`,
);
const targetDir = dirArg ? path.resolve(dirArg) : defaultDir;

// Reuse the standard output-path policy so a malicious caller can't
// smuggle videos to system dirs.
validateOutputPath(targetDir);
mkdirSecure(targetDir);

let size: { width: number; height: number } | undefined;
if (sizeArg) {
if (!sizeArg.includes('x')) {
throw new Error('record start --size: expected WxH (e.g. 1280x720)');
}
const [rawW, rawH] = sizeArg.split('x').map(Number);
if (!Number.isFinite(rawW) || !Number.isFinite(rawH) || rawW < 1 || rawH < 1) {
throw new Error(`record start --size: invalid dimensions '${sizeArg}'`);
}
size = {
width: Math.min(Math.max(Math.round(rawW), 1), 16384),
height: Math.min(Math.max(Math.round(rawH), 1), 16384),
};
}

const err = await bm.startRecording(targetDir, size);
if (err) return `Recording started with warnings: ${err}`;
return `Recording → ${targetDir}\n(call \`record stop\` to flush and get the .webm paths)`;
}

throw new Error(`Unknown record action: '${action}'. Usage: record start|stop|status`);
}

// ─── Frame ───────────────────────────────────────
case 'frame': {
const target = args[0];
Expand Down
Loading
Loading