Skip to content

Commit 08a716e

Browse files
authored
✨ Make CLI context handoff production-ready (#269)
## Summary This PR makes the CLI a much more natural local context surface for agents and SDK dogfooding. - Aligns CLI messaging around reviewed UI context and approved baselines - Resets local TDD run context before screenshots so each run reflects the current truth - Removes the reporter highlight CSS warning noise - Improves local context output so agents can consume build, screenshot, comparison, and review state without guessing - Dogfoods the richer context flow through the SDK/e2e projects - Preserves cloud build context in CLI runs so local and cloud paths tell the same story - Pins workflow actions and keeps the GitHub Actions runtime aligned with Node 24 ## Testing - npm run test:cli -- ... focused CLI/context tests - npm run test:e2e -- ... focused SDK/e2e dogfood paths - node --test ... focused command/context tests - npm run lint - npm run build - git diff --check
1 parent 9ec9d12 commit 08a716e

41 files changed

Lines changed: 2010 additions & 178 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

README.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,23 @@
11
# Vizzly CLI
22

3-
> Visual proof that your UI works
3+
> Reviewed UI context for people and LLM agents
44
55
[![npm version](https://img.shields.io/npm/v/@vizzly-testing/cli.svg)](https://www.npmjs.com/package/@vizzly-testing/cli)
66
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
77

8-
Visual bugs slip through code review. They hide in pixel-perfect mockups, sneak past unit tests, and show up right when you're about to ship. Vizzly catches them first.
8+
Vizzly keeps the visual truth behind your UI: approved baselines, meaningful diffs, review state, comments, and preview links in one place. That makes it useful for humans reviewing product changes and for LLM agents that need to understand what the UI is supposed to look like before they edit it.
99

10-
Unlike tools that re-render components in isolation, Vizzly captures screenshots directly from your functional teststhe *real thing*. Whether you're validating AI-generated code or testing manual changes, you get visual proof before anything hits production.
10+
Unlike tools that re-render components in isolation, Vizzly captures screenshots directly from your functional tests: the real thing. Whether you're validating AI-generated code or testing manual changes, you get reviewed UI context before anything hits production.
1111

1212
## Why Vizzly?
1313

14-
**Local TDD workflow.** See changes as you type, not after CI. The `vizzly tdd` command runs a local dashboard that compares screenshots instantly—no cloud roundtrip, no waiting.
14+
**Local TDD workflow.** See changes as you type, not after CI. The `vizzly tdd` command runs a local dashboard that compares screenshots instantly and exposes the current workspace as local context.
1515

16-
**Smart diffing with Honeydiff.** Our Rust-based comparison engine is 12x faster than alternatives and ignores the noise: timestamps, ads, font rendering differences. It finds real changes.
16+
**Meaningful diff metadata.** Vizzly stores rich diff evidence: changed regions, cluster metadata, fingerprints, hotspots, confirmed regions, and image URLs. Agents can inspect what changed instead of guessing from a pass/fail label.
1717

1818
**Any screenshot source.** Playwright, Cypress, Puppeteer, Selenium, native mobile apps, or even design mockups. If you can capture it, Vizzly can compare it.
1919

20-
**Team-based pricing.** Pay for your team, not your screenshots. Test everything without budget anxiety.
20+
**Approved baselines as truth.** Cloud context carries human review state. Local context carries the downloaded or generated baseline metadata. That is the bridge between TDD locally and collaborative review in Vizzly.
2121

2222
## Quick Start
2323

@@ -60,7 +60,7 @@ vizzly run "npm test" --wait
6060

6161
### Visual Context For Agents
6262

63-
Use `vizzly context` when you want Vizzly to act more like visual context than a test runner.
63+
Use `vizzly context` when you want Vizzly to act like a visual context store, not just a test runner.
6464

6565
This is especially useful for LLM agents, automation, and quick debugging loops. Instead of
6666
making a bunch of narrow API calls, you can ask for one build, comparison, screenshot, or review
@@ -73,11 +73,11 @@ vizzly context comparison def456 --json
7373

7474
# Local workspace context from .vizzly/
7575
vizzly context build current --source local
76+
vizzly context build current --source local --agent
7677
vizzly context screenshot build-detail-screenshots --source local --json
7778
```
7879

79-
`--json` is the main automation path. Human-readable output is there for quick terminal use, but
80-
JSON is what you want for scripts, agents, and prompt assembly.
80+
`--json` is the durable automation path. `--agent` gives a compact Markdown handoff for prompt assembly and local dogfooding.
8181

8282
Local context is read-only and file-backed. It reads your existing `.vizzly` workspace state from
8383
TDD runs, including screenshots, diffs, and any saved hotspot or region metadata.

clients/ember/test-app/package-lock.json

Lines changed: 2 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

clients/storybook/src/index.js

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -128,7 +128,7 @@ export async function run(storybookPath, options = {}, context = {}) {
128128
branch = gitInfo.branch;
129129
commit = gitInfo.commit;
130130
message = gitInfo.message;
131-
buildName = gitInfo.buildName;
131+
buildName = vizzlyConfig?.build?.name || gitInfo.buildName;
132132
pullRequestNumber = gitInfo.prNumber;
133133
} else {
134134
// Fallback for older CLI versions - use environment variables
@@ -138,7 +138,9 @@ export async function run(storybookPath, options = {}, context = {}) {
138138
branch = process.env.VIZZLY_BRANCH || 'main';
139139
commit = process.env.VIZZLY_COMMIT_SHA || undefined;
140140
message = process.env.VIZZLY_COMMIT_MESSAGE || undefined;
141-
buildName = `Storybook ${new Date().toISOString()}`;
141+
buildName =
142+
vizzlyConfig?.build?.name ||
143+
`Storybook ${new Date().toISOString()}`;
142144
pullRequestNumber = process.env.VIZZLY_PR_NUMBER
143145
? parseInt(process.env.VIZZLY_PR_NUMBER, 10)
144146
: undefined;

clients/swift/scripts/run-e2e.js

Lines changed: 40 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
import { spawn } from 'node:child_process';
1+
import { spawn, spawnSync } from 'node:child_process';
22
import { existsSync, mkdtempSync, rmSync } from 'node:fs';
33
import { tmpdir } from 'node:os';
44
import { dirname, join, resolve } from 'node:path';
@@ -16,6 +16,7 @@ if (!existsSync(distCliPath)) {
1616
}
1717

1818
let tempDir = mkdtempSync(join(tmpdir(), 'vizzly-swift-e2e-'));
19+
let vizzlyHome = join(tempDir, '.vizzly-home');
1920
let swiftTestCommand = [
2021
'cd',
2122
JSON.stringify(swiftPackageDir),
@@ -26,6 +27,36 @@ let swiftTestCommand = [
2627
'VizzlyE2ETests',
2728
].join(' ');
2829

30+
function cleanupAndExit(code = 1) {
31+
rmSync(tempDir, { recursive: true, force: true });
32+
process.exit(code);
33+
}
34+
35+
function printLocalContext() {
36+
let contextResult = spawnSync(
37+
process.execPath,
38+
[cliPath, 'context', 'build', 'current', '--source', 'local', '--agent'],
39+
{
40+
cwd: tempDir,
41+
encoding: 'utf8',
42+
env: {
43+
...process.env,
44+
VIZZLY_HOME: vizzlyHome,
45+
},
46+
}
47+
);
48+
49+
if (contextResult.stdout) {
50+
process.stdout.write(contextResult.stdout);
51+
}
52+
53+
if (contextResult.stderr) {
54+
process.stderr.write(contextResult.stderr);
55+
}
56+
57+
return contextResult.status ?? 1;
58+
}
59+
2960
let child = spawn(
3061
process.execPath,
3162
[cliPath, 'tdd', 'run', swiftTestCommand, '--no-color'],
@@ -35,18 +66,21 @@ let child = spawn(
3566
env: {
3667
...process.env,
3768
VIZZLY_E2E: '1',
38-
VIZZLY_HOME: join(tempDir, '.vizzly-home'),
69+
VIZZLY_HOME: vizzlyHome,
3970
},
4071
}
4172
);
4273

4374
child.on('exit', code => {
44-
rmSync(tempDir, { recursive: true, force: true });
45-
process.exit(code ?? 1);
75+
if (code !== 0) {
76+
cleanupAndExit(code ?? 1);
77+
}
78+
79+
let contextStatus = printLocalContext();
80+
cleanupAndExit(contextStatus);
4681
});
4782

4883
child.on('error', error => {
49-
rmSync(tempDir, { recursive: true, force: true });
5084
console.error(error);
51-
process.exit(1);
85+
cleanupAndExit(1);
5286
});

docs/json-output.md

Lines changed: 47 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -193,8 +193,8 @@ vizzly tdd list --json
193193
### `vizzly context`
194194

195195
Use `vizzly context` when you want one machine-friendly bundle instead of several narrow calls.
196-
This is the best fit for automation, agents, and scripts that need visual evidence plus a little
197-
bit of memory.
196+
This is the best fit for automation, agents, and scripts that need approved baselines, visual
197+
evidence, review state, comments, preview links, and diff metadata in one place.
198198

199199
Every context payload includes a `source` field. That tells you whether the bundle came from
200200
cloud data or your local `.vizzly` workspace.
@@ -204,8 +204,12 @@ cloud data or your local `.vizzly` workspace.
204204
```bash
205205
vizzly context build abc123 --json
206206
vizzly context build current --source local --json
207+
vizzly context build current --source local --agent
207208
```
208209

210+
Use `--json` for durable automation. Use `--agent` when you want a compact Markdown handoff for
211+
prompt assembly.
212+
209213
```json
210214
{
211215
"resource": "build_context",
@@ -219,6 +223,21 @@ vizzly context build current --source local --json
219223
"status": "completed",
220224
"approval_status": "pending"
221225
},
226+
"baseline": {
227+
"selected": {
228+
"id": "baseline-build",
229+
"name": "Approved Main",
230+
"approval_status": "approved"
231+
},
232+
"selection_reason": "common_ancestor",
233+
"comparison_baseline_build_ids": ["baseline-build"]
234+
},
235+
"status": {
236+
"needs_review": true,
237+
"reasons": ["comparisons_need_review"],
238+
"pending_comparisons": 3,
239+
"unresolved_comments": 0
240+
},
222241
"summary": {
223242
"comparisons": {
224243
"total": 12,
@@ -231,14 +250,30 @@ vizzly context build current --source local --json
231250
"rejected": 0
232251
}
233252
},
253+
"screenshots": [
254+
{
255+
"name": "Dashboard",
256+
"url": "https://...",
257+
"baseline": { "url": "https://..." }
258+
}
259+
],
234260
"comparisons": [
235261
{
236262
"id": "cmp-1",
237-
"name": "Dashboard",
263+
"screenshot_name": "Dashboard",
238264
"result": "changed",
239-
"diff_percentage": 0.42
265+
"needs_review": true,
266+
"diff": {
267+
"percentage": 0.42,
268+
"image_url": "https://...",
269+
"regions": []
270+
}
240271
}
241-
]
272+
],
273+
"comments": {
274+
"build": [],
275+
"screenshot_count": 0
276+
}
242277
}
243278
```
244279

@@ -637,27 +672,19 @@ vizzly init --json
637672
}
638673
```
639674

640-
### `vizzly project:select`
675+
### `vizzly project link`
641676

642677
```bash
643-
vizzly project:select --json
678+
vizzly project link my-org/my-project --json
644679
```
645680

646-
Note: In JSON mode, the interactive prompts still appear because project selection requires user input.
647-
648681
```json
649682
{
650-
"status": "configured",
651-
"project": {
652-
"name": "My Project",
653-
"slug": "my-project"
654-
},
655-
"organization": {
656-
"name": "My Org",
657-
"slug": "my-org"
658-
},
659-
"directory": "/path/to/project",
660-
"tokenCreated": true
683+
"linked": true,
684+
"organizationSlug": "my-org",
685+
"projectSlug": "my-project",
686+
"tokenPrefix": "vzt_abc",
687+
"storage": "keychain"
661688
}
662689
```
663690

package.json

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,14 @@
11
{
22
"name": "@vizzly-testing/cli",
33
"version": "0.30.0",
4-
"description": "Visual review platform for UI developers and designers",
4+
"description": "Reviewed UI context for people and LLM agents",
55
"keywords": [
6+
"llm-context",
7+
"ui-context",
8+
"visual-context",
9+
"approved-baselines",
610
"visual-testing",
711
"screenshot-testing",
8-
"visual-regression",
912
"visual-review",
1013
"ui-testing",
1114
"collaboration",

src/api/client.js

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,10 @@ import {
2323
*/
2424
export const DEFAULT_API_URL = 'https://app.vizzly.dev';
2525

26+
function isProjectToken(token) {
27+
return typeof token === 'string' && token.startsWith('vzt_');
28+
}
29+
2630
/**
2731
* Create an API client with the given configuration
2832
*
@@ -84,7 +88,7 @@ export function createApiClient(options = {}) {
8488
shouldRetryWithRefresh(
8589
response.status,
8690
isRetry,
87-
await hasRefreshToken()
91+
!isProjectToken(token) && (await hasRefreshToken())
8892
)
8993
) {
9094
let refreshed = await attemptTokenRefresh();
@@ -97,7 +101,7 @@ export function createApiClient(options = {}) {
97101
// Auth error
98102
if (isAuthError(response.status)) {
99103
throw new AuthError(
100-
'Invalid or expired API token. Link a project via "vizzly project:select" or set VIZZLY_TOKEN.'
104+
'Invalid or expired API token. Run "vizzly project link <org>/<project>" or set VIZZLY_TOKEN.'
101105
);
102106
}
103107

@@ -146,7 +150,9 @@ export function createApiClient(options = {}) {
146150
await saveAuthTokens({
147151
accessToken: data.accessToken,
148152
refreshToken: data.refreshToken,
149-
expiresAt: data.expiresAt,
153+
expiresAt:
154+
data.expiresAt ||
155+
new Date(Date.now() + data.expiresIn * 1000).toISOString(),
150156
user: auth.user,
151157
});
152158

0 commit comments

Comments
 (0)