Skip to content

Commit b2ab97e

Browse files
committed
add self-hosted visual regression pipeline
Storybook scenarios were not exercised in CI, and there was no self-hosted screenshot baseline for catching unintended UI changes. This left UI regressions dependent on manual review and an external visual testing addon. Switch Storybook testing to a self-hosted model by wiring a dedicated storybook test project in Vitest and adding a Playwright visual suite that snapshots stories tagged with `visual`. The visual runner builds and serves static Storybook output, honors optional per-story `parameters.visual` settings, disables motion for deterministic output, and commits an initial baseline set for stable component stories. The CI workflow now includes an isolated Storybook/visual job that runs story execution separately from screenshot assertions and uploads Playwright artifacts when diffs fail. Story fixtures that depended on current time or randomness were made deterministic so baseline images remain stable. This adds initial coverage for the tagged Badge, TagInput, and JSON stories through committed Playwright snapshots, and broad story execution coverage through the Storybook Vitest project.
1 parent 6d9d97d commit b2ab97e

25 files changed

Lines changed: 1217 additions & 209 deletions

.github/workflows/ci.yaml

Lines changed: 60 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ name: CI
33
env:
44
# Test database.
55
TEST_DATABASE_URL: postgres://postgres:postgres@127.0.0.1:5432/river_test?sslmode=disable
6+
NODE_VERSION: &node_version "24.14.1"
67

78
on:
89
push:
@@ -133,7 +134,7 @@ jobs:
133134
strategy:
134135
matrix:
135136
node-version:
136-
- "22.18.0"
137+
- *node_version
137138
fail-fast: false
138139
timeout-minutes: 5
139140

@@ -170,7 +171,7 @@ jobs:
170171
strategy:
171172
matrix:
172173
node-version:
173-
- "22.18.0"
174+
- *node_version
174175
fail-fast: false
175176
timeout-minutes: 5
176177

@@ -209,6 +210,62 @@ jobs:
209210
# Check tsc compilation even if there were linting issues:
210211
if: always()
211212

213+
js_storybook_and_visual:
214+
name: Storybook and visual regression
215+
runs-on: ubuntu-latest
216+
strategy:
217+
matrix:
218+
node-version:
219+
- *node_version
220+
fail-fast: false
221+
timeout-minutes: 20
222+
223+
env:
224+
NODE_ENV: test
225+
226+
permissions:
227+
contents: read
228+
229+
steps:
230+
- name: Checkout repository
231+
uses: actions/checkout@v6
232+
233+
- name: Install Node.js
234+
uses: actions/setup-node@v6
235+
with:
236+
cache: "npm"
237+
cache-dependency-path: package-lock.json
238+
node-version: ${{ matrix.node-version }}
239+
240+
- name: Install dependencies
241+
run: npm ci
242+
shell: sh
243+
244+
- name: Install Playwright browser dependencies
245+
run: npx playwright install --with-deps chromium
246+
247+
- name: Storybook story tests
248+
run: npm run test-storybook
249+
250+
- name: Visual regression tests
251+
run: npm run test-visual
252+
253+
- name: Upload Playwright HTML report
254+
if: failure()
255+
uses: actions/upload-artifact@v7
256+
with:
257+
if-no-files-found: ignore
258+
name: playwright-report
259+
path: playwright-report/
260+
261+
- name: Upload Playwright test results
262+
if: failure()
263+
uses: actions/upload-artifact@v7
264+
with:
265+
if-no-files-found: ignore
266+
name: playwright-test-results
267+
path: test-results/
268+
212269
release:
213270
name: Release
214271
permissions:
@@ -231,7 +288,7 @@ jobs:
231288
with:
232289
cache: "npm"
233290
cache-dependency-path: package-lock.json
234-
node-version: "22.18.0"
291+
node-version: ${{ env.NODE_VERSION }}
235292

236293
- name: Install dependencies
237294
run: npm install

.storybook/main.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ const config: StorybookConfig = {
55
"@storybook/addon-links",
66
"@storybook/addon-onboarding",
77
"@storybook/addon-themes",
8-
"@chromatic-com/storybook",
8+
"@storybook/addon-vitest",
99
"@storybook/addon-docs",
1010
],
1111

.storybook/preview.tsx

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ import type { Features } from "../src/services/features";
1717
import { FeaturesContext } from "../src/contexts/Features";
1818
import "../src/global-type-overrides";
1919
import "../src/index.css";
20+
import "../tests/visual/visual.css";
2021
import {
2122
$userSettings,
2223
clearAllSettings,
@@ -98,6 +99,28 @@ export const withSettings: Decorator = (StoryFn, context) => {
9899
return <StoryFn />;
99100
};
100101

102+
const isVisualTestRun = () => {
103+
if (typeof window === "undefined") {
104+
return false;
105+
}
106+
107+
return (
108+
new URLSearchParams(window.location.search).get("visual-test") === "true"
109+
);
110+
};
111+
112+
export const withVisualTestingMode: Decorator = (StoryFn) => {
113+
if (typeof document !== "undefined") {
114+
if (isVisualTestRun()) {
115+
document.documentElement.setAttribute("data-visual-test", "true");
116+
} else {
117+
document.documentElement.removeAttribute("data-visual-test");
118+
}
119+
}
120+
121+
return <StoryFn />;
122+
};
123+
101124
// Define parameter types
102125
declare module "@storybook/react-vite" {
103126
interface Parameters {
@@ -108,11 +131,16 @@ declare module "@storybook/react-vite" {
108131
routes?: string[];
109132
};
110133
settings?: UserSettings;
134+
visual?: {
135+
viewport?: "desktop" | "mobile";
136+
waitFor?: string;
137+
};
111138
}
112139
}
113140

114141
const preview: Preview = {
115142
decorators: [
143+
withVisualTestingMode,
116144
withFeatures,
117145
withSettings,
118146
withRouter,

docs/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,3 +82,4 @@ By default logs are written with the [`slog.TextHandler`](https://pkg.go.dev/log
8282
## Development
8383

8484
See [developing River UI](./development.md).
85+
See [storybook visual regression](./visual_regression.md).

docs/visual_regression.md

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
# Storybook Visual Regression
2+
3+
River UI uses a self-hosted visual regression workflow based on Storybook and
4+
Playwright.
5+
6+
## What runs in CI
7+
8+
- `npm run test-storybook` runs Storybook stories with the Vitest addon so
9+
rendering and interaction failures are caught.
10+
- `npm run test-visual` runs Playwright screenshot assertions against a built
11+
static Storybook.
12+
- Visual baseline rendering is standardized on Linux Chromium.
13+
14+
## Marking stories for visual snapshots
15+
16+
Only stories tagged with `visual` are screenshot-tested.
17+
18+
```ts
19+
export const Example: Story = {
20+
tags: ["visual"],
21+
};
22+
```
23+
24+
Stories without this tag still run in `test-storybook`, but they are skipped by
25+
`test-visual`.
26+
27+
## Optional visual parameters
28+
29+
Stories can define an optional `parameters.visual` contract:
30+
31+
```ts
32+
parameters: {
33+
visual: {
34+
viewport: "mobile",
35+
waitFor: "[data-testid='ready']",
36+
},
37+
}
38+
```
39+
40+
- `viewport`: `desktop` (default) or `mobile`.
41+
- `waitFor`: CSS selector to wait for before screenshotting.
42+
43+
## Writing screenshot-safe stories
44+
45+
To keep baselines stable:
46+
47+
- Avoid `Date.now()`, `new Date()`, random values, and faker defaults unless
48+
you pass fixed values.
49+
- Avoid async timing behavior unless there is a deterministic `waitFor`
50+
selector.
51+
- Prefer static fixtures with explicit timestamps and IDs.
52+
53+
## Local workflow
54+
55+
- Run story execution tests: `npm run test-storybook`
56+
- Run visual tests: `npm run test-visual`
57+
- Update visual baselines intentionally: `npm run test-visual:update`
58+
59+
Updated snapshots are written next to the visual test spec under Playwright's
60+
snapshot directory.
61+
62+
## Debugging CI failures
63+
64+
When visual regression fails, CI uploads Playwright artifacts:
65+
66+
- `playwright-report/` (HTML report)
67+
- `test-results/` (diff images and failure diagnostics)
68+
69+
Use those artifacts to inspect unexpected changes, then either fix regressions
70+
or intentionally update snapshots.

0 commit comments

Comments
 (0)