Skip to content

Commit db168b2

Browse files
authored
🐛 Fix CLI regressions across JSON, help, and TDD (#296)
## Why The CLI had several rough edges at the boundaries users rely on most: command errors could show misleading help, JSON mode could mix machine-readable output with human text, TDD lifecycle commands were not careful enough around explicit ports, and some commands hid local validation failures behind auth or optional API state. That makes the CLI feel unreliable in automation and confusing in local workflows. This PR tightens those paths so exposed commands fail clearly, preserve their output contracts, and avoid touching unrelated local processes or config state. It also removes the old global TDD `server.json` compatibility path. Local TDD discovery is now project-local or explicit, which matches the current daemon registry model and avoids carrying forward a second server-discovery contract. ## What Changed - Fixed command routing and help behavior so unknown root and nested commands fail with the right recovery hints. - Cleaned up root/TDD help output, including full nested command paths, documented lifecycle `--port` flags, and hidden internal daemon-only flags. - Preserved JSON-mode contracts for validation errors, commander errors, TDD commands, noisy child processes, `run --json --wait`, `upload --wait --json`, review actions, and `status --json`. - Made TDD daemon `start`, `status`, `stop`, and `list` safer around explicit ports, isolated `VIZZLY_HOME`, stale files, registry entries, and wrong-port no-ops. - Removed the legacy global TDD `server.json` write/cleanup path so daemon state is no longer mirrored into the user home directory. - Validated preview/upload local paths before requiring auth so local mistakes are reported immediately. - Made `status` tolerate builds without preview metadata by returning `preview: null` instead of failing. - Updated in-repo docs, Swift SDK docs, Vizzly skill references, and the JSON output reference so the documented workflows match the CLI behavior. - Added CLI-boundary regression tests that run the real command entrypoint against local workspaces and local HTTP API servers. ## Impact This should make the CLI more predictable for both local use and automation. JSON stdout stays parseable, help/error output points users to the right command, TDD lifecycle commands avoid unrelated processes, local validation errors are surfaced before network/auth concerns, and SDK server discovery no longer depends on a stale global file.
1 parent 36a2b2c commit db168b2

39 files changed

Lines changed: 2122 additions & 468 deletions

AGENTS.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,8 +37,9 @@ Keep subjects concise; include issue/PR refs when relevant (example: `(#217)`).
3737
PRs should include:
3838
- Why the change is needed.
3939
- A clear summary of all meaningful diff areas.
40-
- A test plan with exact commands run.
40+
- A concise confidence/verification summary without listing commands run.
4141
- Screenshots or terminal output for UI/reporting changes when helpful.
42+
- Do not mention private or sibling repo work in public repo PRs.
4243

4344
## Security & Configuration Tips
4445
Use Node.js `>=22`. Keep secrets (for example `VIZZLY_TOKEN`) out of git.

README.md

Lines changed: 26 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -40,17 +40,31 @@ Use `vizzly init --agent-skill` to install only the local skill, or
4040

4141
### Start Local TDD
4242

43-
Start the TDD server, run your tests, and open the dashboard at
44-
`http://localhost:47392`.
43+
Start the TDD server, run your tests, and open the dashboard at the URL the
44+
command prints. Add `--open` when you want Vizzly to open the dashboard for
45+
you. If the default port is busy, Vizzly picks the next available port; use the
46+
printed `--port` value with `vizzly tdd status` or `vizzly tdd stop`.
4547

4648
```bash
47-
vizzly tdd start
49+
vizzly tdd start --open
4850
pnpm test -- --watch
4951
```
5052

5153
The dashboard shows screenshots, baselines, and diffs as they arrive. Accept or
5254
reject changes right from the UI.
5355

56+
For a one-off local check, wrap the test command with `tdd run`:
57+
58+
```bash
59+
vizzly tdd run "pnpm test" --no-open
60+
vizzly context build current --source local --agent
61+
```
62+
63+
That run writes review data under `.vizzly/` and prints a context command you
64+
can use for follow-up inspection. If screenshots were captured, Vizzly also
65+
generates `.vizzly/report/index.html`; omit `--no-open` when you want that
66+
report opened automatically.
67+
5468
### Run With Cloud Review
5569

5670
Use cloud builds when you want shared baselines, team review, and CI status:
@@ -206,10 +220,18 @@ export default {
206220
| Command | What it does |
207221
| --- | --- |
208222
| `vizzly tdd start` | Start the local TDD server and dashboard. |
209-
| `vizzly tdd run "cmd"` | Run tests once and generate a static visual report. |
223+
| `vizzly tdd status` | Check the local TDD server for this project. |
224+
| `vizzly tdd list` | List running local TDD servers. |
225+
| `vizzly tdd stop` | Stop the local TDD server for this project. |
226+
| `vizzly tdd run "cmd"` | Run tests once and write local review data under `.vizzly/`. |
210227
| `vizzly run "cmd"` | Run tests with cloud build and review integration. |
211228
| `vizzly context ...` | Fetch visual context for builds, comparisons, screenshots, and review queues. |
212229
| `vizzly upload <dir>` | Upload an existing folder of screenshots. |
230+
| `vizzly preview <dir>` | Upload static build output for in-context review. |
231+
| `vizzly approve <comparison-id>` | Approve a visual comparison. |
232+
| `vizzly reject <comparison-id>` | Reject a visual comparison with a reason. |
233+
| `vizzly comment <build-id>` | Add a build comment. |
234+
| `vizzly config [key]` | Inspect resolved configuration values. |
213235
| `vizzly login` | Authenticate through the browser. |
214236
| `vizzly doctor` | Validate your local setup. |
215237

clients/ember/README.md

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,15 @@ Visual testing SDK for Ember.js projects using Testem. Capture screenshots from
55
## Installation
66

77
```bash
8-
pnpm install -D @vizzly-testing/ember
8+
pnpm install -D @vizzly-testing/cli @vizzly-testing/ember
99

10-
# Install a Playwright browser
10+
# Install the Playwright browser you plan to test with
1111
pnpm exec playwright install chromium
1212
```
1313

14+
Chromium, Firefox, and WebKit are supported. Install additional browsers with
15+
`pnpm exec playwright install firefox` or `pnpm exec playwright install webkit`.
16+
1417
## Setup
1518

1619
### 1. Configure Testem
@@ -31,7 +34,9 @@ module.exports = configure({
3134
});
3235
```
3336

34-
The `configure()` function replaces standard browser launchers (Chrome, Firefox, Safari) with Playwright-powered launchers that can capture screenshots. Browsers run in **headless mode by default**.
37+
The `configure()` function maps standard Testem launchers (Chrome, Firefox,
38+
Safari) to Playwright-powered Chromium, Firefox, and WebKit launchers that can
39+
capture screenshots. Browsers run in **headless mode by default**.
3540

3641
> **Note for Ember + Vite projects**: The `cwd: 'dist'` option is required because Vite builds test files into the `dist/` directory. Without this, Testem won't find your test assets.
3742

clients/ember/package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@
5151
"test:all": "node --test --test-reporter=spec 'tests/**/*.test.js'",
5252
"test:watch": "node --test --test-reporter=spec --watch 'tests/**/*.test.js'",
5353
"test:ember": "cd test-app && pnpm install --frozen-lockfile && pnpm run build --mode development && pnpm exec testem ci --file testem.cjs",
54-
"test:ember:tdd": "../../bin/vizzly.js tdd run 'pnpm run test:ember'",
54+
"test:ember:tdd": "../../bin/vizzly.js tdd run 'pnpm run test:ember' --no-open",
5555
"test:ember:cloud": "../../bin/vizzly.js run 'pnpm run test:ember'",
5656
"lint": "biome lint src tests bin",
5757
"lint:fix": "biome lint --write src tests bin",

clients/static-site/README.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ Seamlessly integrate your static sites (Gatsby, Astro, Jekyll, Next.js static ex
55
## Installation
66

77
```bash
8-
pnpm install @vizzly-testing/static-site
8+
pnpm install -D @vizzly-testing/cli @vizzly-testing/static-site
99
```
1010

1111
The plugin is automatically discovered by the Vizzly CLI via the `@vizzly-testing/*` scope.
@@ -29,6 +29,9 @@ vizzly static-site ./dist \
2929

3030
# With concurrency control
3131
vizzly static-site ./dist --concurrency 5
32+
33+
# With comma-separated browser args
34+
vizzly static-site ./dist --browser-args "--no-sandbox,--disable-dev-shm-usage"
3235
```
3336

3437
### Programmatic Usage
@@ -151,7 +154,7 @@ Configuration is merged in this order (later overrides earlier):
151154
- `--browser <type>` - Browser engine to use: `chromium`, `firefox`, or `webkit`
152155
- `--include <pattern>` - Include page pattern (glob)
153156
- `--exclude <pattern>` - Exclude page pattern (glob)
154-
- `--browser-args <args>` - Additional Playwright browser arguments
157+
- `--browser-args <args>` - Comma-separated Playwright browser arguments
155158
- `--headless` - Run browser in headless mode (default: true)
156159
- `--no-headless` - Run browser with a visible window
157160
- `--full-page` - Capture full page screenshots (default: true)

clients/storybook/README.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ changes with position-based comments and review rules.
77
## Installation
88

99
```bash
10-
pnpm install @vizzly-testing/storybook
10+
pnpm install -D @vizzly-testing/cli @vizzly-testing/storybook
1111
```
1212

1313
The plugin is automatically discovered by the Vizzly CLI via the `@vizzly-testing/*` scope.
@@ -43,6 +43,9 @@ vizzly storybook ./storybook-static \
4343

4444
# With concurrency control
4545
vizzly storybook ./storybook-static --concurrency 5
46+
47+
# With comma-separated browser args
48+
vizzly storybook ./storybook-static --browser-args "--no-sandbox,--disable-dev-shm-usage"
4649
```
4750

4851
### Programmatic Usage
@@ -166,7 +169,7 @@ Configuration is merged in this order (later overrides earlier):
166169
- `--include <pattern>` - Include story pattern (glob)
167170
- `--exclude <pattern>` - Exclude story pattern (glob)
168171
- `--config <path>` - Path to custom config file
169-
- `--browser-args <args>` - Additional Playwright browser arguments
172+
- `--browser-args <args>` - Comma-separated Playwright browser arguments
170173
- `--headless` - Run browser in headless mode (default: true)
171174
- `--no-headless` - Run browser with a visible window
172175
- `--full-page` - Capture full page screenshots (default: true)

clients/swift/INTEGRATION.md

Lines changed: 33 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -51,28 +51,43 @@ cd /path/to/MyiOSApp
5151
Create a `vizzly.config.js` file (optional but recommended):
5252

5353
```javascript
54-
export default {
55-
// Delta E comparison threshold. Omitted screenshots use server config.
56-
threshold: 0,
57-
58-
// TDD server port
59-
port: 47392,
60-
61-
// Baseline directory
62-
baselineDir: '.vizzly/baselines'
63-
}
54+
import { defineConfig } from '@vizzly-testing/cli/config';
55+
56+
export default defineConfig({
57+
server: {
58+
port: 47392,
59+
},
60+
comparison: {
61+
// Delta E comparison threshold. Omitted screenshots use server config.
62+
threshold: 0,
63+
},
64+
});
6465
```
6566

6667
### 4. Start TDD Server
6768

6869
```bash
69-
vizzly tdd start
70+
vizzly tdd start --open
7071
```
7172

72-
This starts a local server at `http://localhost:47392` that will:
73+
This starts a local server that will:
7374
- Receive screenshots from your tests
7475
- Compare them against baselines
75-
- Serve a dashboard at `http://localhost:47392/dashboard`
76+
- Serve a dashboard at the URL printed by the command
77+
78+
Vizzly uses port `47392` by default. If that port is busy, it auto-assigns
79+
another free port and prints that URL instead.
80+
81+
For a one-off run, wrap your test command:
82+
83+
```bash
84+
vizzly tdd run \
85+
"xcodebuild test -scheme MyApp -destination 'platform=iOS Simulator,name=iPhone 15'" \
86+
--no-open
87+
```
88+
89+
That writes local review data under `.vizzly/` and creates a static report at
90+
`.vizzly/report/index.html` when screenshots are captured.
7691

7792
### 5. Write UI Tests with Vizzly
7893

@@ -443,9 +458,11 @@ vizzly tdd start
443458
**Solution**:
444459

445460
1. Ensure TDD server is running: `vizzly tdd start`
446-
2. Check `~/.vizzly/server.json` exists in your home directory
447-
3. Verify the server is reachable: `curl http://localhost:47392/health`
448-
4. Or explicitly set: `export VIZZLY_SERVER_URL=http://localhost:47392`
461+
2. Check `.vizzly/server.json` exists in your project checkout
462+
3. Verify the printed server URL is reachable, for example:
463+
`curl http://localhost:47392/health`
464+
4. Or explicitly set the printed URL:
465+
`export VIZZLY_SERVER_URL=http://localhost:47392`
449466

450467
## Best Practices
451468

clients/swift/QUICKSTART.md

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,9 +20,12 @@ pnpm install -g @vizzly-testing/cli
2020
In your iOS project directory:
2121

2222
```bash
23-
vizzly tdd start
23+
vizzly tdd start --open
2424
```
2525

26+
Vizzly uses port `47392` by default. If that port is busy, it prints the
27+
dashboard URL with the auto-assigned port.
28+
2629
## 4. Write a Visual Test
2730

2831
```swift
@@ -58,14 +61,25 @@ xcodebuild test \
5861

5962
## 6. Review Results
6063

61-
Open dashboard: **http://localhost:47392/dashboard**
64+
Open the dashboard URL printed by `vizzly tdd start`.
6265

6366
- ✅ Green = Screenshots match baselines
6467
- ⚠️ Yellow = Visual differences detected
6568
- 🆕 Blue = New screenshots (first run)
6669

6770
Click any screenshot to see side-by-side comparison and approve/reject changes.
6871

72+
For a one-off local check, wrap the test command instead:
73+
74+
```bash
75+
vizzly tdd run \
76+
"xcodebuild test -scheme MyApp -destination 'platform=iOS Simulator,name=iPhone 15'" \
77+
--no-open
78+
```
79+
80+
That writes review data under `.vizzly/` and creates `.vizzly/report/index.html`
81+
when screenshots are captured.
82+
6983
## Next Steps
7084

7185
- **More Examples**: See [Example/ExampleUITests.swift](Example/ExampleUITests.swift)

clients/swift/README.md

Lines changed: 27 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -52,10 +52,12 @@ for native app integration.
5252

5353
```bash
5454
cd /path/to/your/ios/project
55-
vizzly tdd start
55+
vizzly tdd start --open
5656
```
5757

58-
This starts a local server at `http://localhost:47392` that receives screenshots and performs visual comparisons.
58+
This starts a local server that receives screenshots and performs visual
59+
comparisons. Vizzly uses `http://localhost:47392` by default; if that port is
60+
busy, use the URL printed by the command.
5961

6062
### 2. Add Vizzly to Your UI Tests
6163

@@ -86,7 +88,20 @@ xcodebuild test -scheme MyApp -destination 'platform=iOS Simulator,name=iPhone 1
8688

8789
### 4. View Results
8890

89-
Open the dashboard at **http://localhost:47392/dashboard** to see visual comparisons, accept/reject changes, and review differences.
91+
Open the dashboard URL printed by `vizzly tdd start` to see visual comparisons,
92+
accept/reject changes, and review differences.
93+
94+
For a one-off local run, wrap your `xcodebuild` command:
95+
96+
```bash
97+
vizzly tdd run \
98+
"xcodebuild test -scheme MyApp -destination 'platform=iOS Simulator,name=iPhone 15'" \
99+
--no-open
100+
```
101+
102+
That writes local review data under `.vizzly/`. If screenshots were captured,
103+
Vizzly also creates `.vizzly/report/index.html`; omit `--no-open` when you want
104+
the report opened automatically.
90105

91106
## Usage Examples
92107

@@ -288,10 +303,11 @@ The SDK automatically discovers a running Vizzly TDD server using this priority
288303

289304
1. **VIZZLY_SERVER_URL environment variable** - Explicitly set server URL
290305
2. **Project server file** - `.vizzly/server.json` in the current directory
291-
3. **Global server file** - `~/.vizzly/server.json` written by CLI
292-
4. **Default port health check** - Tests `http://localhost:47392/health`
306+
3. **Default port health check** - Tests `http://localhost:47392/health`
293307

294-
When you run `vizzly tdd start`, the CLI automatically writes server info to `~/.vizzly/server.json` in your home directory, enabling zero-config discovery from iOS tests.
308+
When you run `vizzly tdd start`, the CLI writes server info to
309+
`.vizzly/server.json` in your project. Run UI tests from the project checkout,
310+
or set `VIZZLY_SERVER_URL` explicitly when your test process starts elsewhere.
295311

296312
### Environment Variables
297313

@@ -464,9 +480,11 @@ override func setUpWithError() throws {
464480
### Server Not Found
465481

466482
1. Ensure TDD server is running: `vizzly tdd start`
467-
2. Check `~/.vizzly/server.json` exists in your home directory
468-
3. Verify the server is reachable: `curl http://localhost:47392/health`
469-
4. Or explicitly set: `export VIZZLY_SERVER_URL=http://localhost:47392`
483+
2. Check `.vizzly/server.json` exists in your project checkout
484+
3. Verify the printed server URL is reachable, for example:
485+
`curl http://localhost:47392/health`
486+
4. Or explicitly set the printed URL:
487+
`export VIZZLY_SERVER_URL=http://localhost:47392`
470488

471489
### Visual Differences Not Showing
472490

clients/vitest/package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@
4141
"test": "pnpm run test:unit",
4242
"test:unit": "vitest run --config vitest.config.js",
4343
"test:e2e": "vitest run --config vitest.e2e.config.js",
44-
"test:e2e:tdd": "../../bin/vizzly.js tdd run 'pnpm run test:e2e'",
44+
"test:e2e:tdd": "../../bin/vizzly.js tdd run 'pnpm run test:e2e' --no-open",
4545
"test:e2e:cloud": "../../bin/vizzly.js run 'pnpm run test:e2e'",
4646
"lint": "biome lint src tests",
4747
"lint:fix": "biome lint --write src tests",

0 commit comments

Comments
 (0)