Skip to content

Commit 439e18a

Browse files
Paul Mulliganclaude
andcommitted
docs: cross-browser baseline storage guide and reference sweep
New docs/regression-testing/cross-browser.md covers the full RFC 0002 flow: determinism rationale, provenance statuses and policies, backend matrix (commit/ci-artifact/service with the §11 exposure-and-cost tradeoffs), LFS adoption (forward-only, migration guidance), CI jobs, the blocking-flip criteria (§13 Q2: ≥20 clean containerized compares), and troubleshooting. Reference sweep: regression-testing README (shared baseline dir + manifest sync note), onboarding architecture/pipeline-configuration tables (visualBaselines key reference), scripts/README, root README script trees, error-recovery Phase 7 row, and CLAUDE.md (script list, Phase 7 diagram line, features bullet, key artifacts, honest script count 40→55). Commits the implementation plan for the branch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 8eb139b commit 439e18a

9 files changed

Lines changed: 449 additions & 13 deletions

File tree

CLAUDE.md

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -74,11 +74,19 @@ node scripts/visual-diff.js --batch <actual-dir> <expected-dir> [--output-dir di
7474
# Initialize a new React project
7575
./scripts/setup-project.sh my-app --next # or --vite
7676

77-
# Cross-browser testing (Playwright)
77+
# Cross-browser testing (Playwright, ad-hoc capture only)
7878
./scripts/cross-browser-test.sh chromium http://localhost:3000
7979
./scripts/cross-browser-test.sh firefox http://localhost:3000
8080
./scripts/cross-browser-test.sh webkit http://localhost:3000
8181

82+
# Cross-browser baselines: pinned-container capture, provenance-verified compare (RFC 0002)
83+
./scripts/cross-browser-baseline.sh capture http://localhost:3000 [--local] [--dry-run]
84+
./scripts/cross-browser-baseline.sh compare http://localhost:3000 [--json] [--blocking]
85+
./scripts/cross-browser-baseline.sh verify [--json]
86+
87+
# Route large baseline sets through Git LFS (visualBaselines.storage = "lfs")
88+
./scripts/setup-baseline-lfs.sh [--dry-run] [--force]
89+
8290
# Setup Playwright browsers (one-time)
8391
./scripts/setup-playwright.sh
8492

@@ -292,7 +300,7 @@ Autonomous 9-phase pipeline that converts a Figma design into a working, tested
292300
├─ [5] VISUAL DIFF → pixelmatch loop → max 5 iterations
293301
│ └─ [6] E2E TESTS → e2e-test-generator skill
294302
├─ [5.5] DARK MODE → check-dark-mode.sh (non-blocking)
295-
├─ [7] CROSS-BROWSER → Firefox/WebKit screenshots (non-blocking)
303+
├─ [7] CROSS-BROWSER → cross-browser-baseline.sh compare (non-blocking)
296304
├─ [7.5] REGRESSION → regression-test.sh (non-blocking)
297305
├─ [8] QUALITY GATE → [coverage|types|build|tokens|lighthouse] in parallel
298306
└─ [8.5] RESPONSIVE → check-responsive.sh (non-blocking)
@@ -306,6 +314,7 @@ Autonomous 9-phase pipeline that converts a Figma design into a working, tested
306314
- `verify-tokens.sh` — Catches hardcoded values and token drift
307315
- `verify-test-coverage.sh` — Ensures every component has tests
308316
- `visual-diff.js` — Pixel-level screenshot comparison with region analysis
317+
- `.claude/visual-qa/baselines/manifest.json` — Cross-browser baseline provenance (engine, Playwright version, pinned image, sha256, host — RFC 0002)
309318
- `sync-tokens.sh` — Detects token drift between lockfile and source
310319
- `check-dark-mode.sh` — Dark mode screenshot capture and visual comparison
311320
- `generate-stories.sh` — AST-based Storybook story + MDX generation with prop controls, variants, and action args
@@ -317,7 +326,7 @@ Autonomous 9-phase pipeline that converts a Figma design into a working, tested
317326
- **App-type awareness** — Chrome extensions, PWAs, React Native, and web apps get tailored E2E strategies
318327
- **Chrome extension E2E** — Playwright persistent context with `--load-extension`
319328
- Design token extraction with lockfile enforcement
320-
- Cross-browser verification (Firefox, WebKit) with configurable thresholds
329+
- Cross-browser verification (Firefox, WebKit) — committed per-engine baselines diffed at the 0.03 cross-engine threshold, with pinned-container capture and per-baseline provenance (RFC 0002; backends: commit | ci-artifact | service)
321330
- Quality gate: 80%+ coverage, TypeScript, Lighthouse audit
322331
- Resumable: TodoWrite tracks progress across interrupted sessions
323332
- **Dark mode verification** — automated dark theme screenshot comparison (non-blocking)
@@ -586,6 +595,8 @@ gh issue create # Create issue
586595
./scripts/audit-cross-browser-css.sh # Cross-browser CSS audit
587596
./scripts/capture-baselines.sh # Capture regression baselines
588597
./scripts/regression-test.sh # Visual regression testing
598+
./scripts/cross-browser-baseline.sh # Cross-browser baseline capture/compare (RFC 0002)
599+
./scripts/setup-baseline-lfs.sh # Git LFS opt-in for large baseline sets
589600
./scripts/export-design-system.sh # Export components + tokens as pnpm workspace
590601
./scripts/import-design-tokens.sh # Reconstruct lockfile from an export (round-trip / consumer)
591602
./scripts/validate-pipeline-config.sh # Validate pipeline.config.json against schema
@@ -623,6 +634,6 @@ node scripts/metrics-dashboard.js summary # Quick metrics summary
623634
---
624635

625636
**Last Updated:** 2026-07-01
626-
**Architecture:** 56 agents, 24 skills, 4 plugins + gh CLI, Figma + Canva + Playwright MCP, 40 scripts, 8 hooks, 5 renderers (nextjs, vite, astro, sveltekit, expo)
637+
**Architecture:** 56 agents, 24 skills, 4 plugins + gh CLI, Figma + Canva + Playwright MCP, 55 scripts, 8 hooks, 5 renderers (nextjs, vite, astro, sveltekit, expo)
627638

628639
> **Keeping counts in sync:** When adding or removing agents, skills, scripts, or hooks, update all count references across the project. Search for the old count number in `*.md` files to find all references: `CLAUDE.md`, `README.md`, `CONTRIBUTING.md`, `docs/onboarding/`, `docs/react-development/`, and `.claude/AGENT-NAMING-GUIDE.md`. The agent and skill counts are enforced automatically by `scripts/check-doc-counts.sh` (run in CI and on pre-commit), which recounts `.claude/agents/` and `.claude/skills/` and fails on any documented count that disagrees.

README.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,9 @@ project-root/
6565
│ ├── verify-tokens.sh # Design token enforcement
6666
│ ├── verify-test-coverage.sh # Component test verification
6767
│ ├── visual-diff.js # Pixel-level screenshot comparison
68-
│ ├── cross-browser-test.sh # Playwright multi-browser
68+
│ ├── cross-browser-test.sh # Playwright multi-browser (ad-hoc capture)
69+
│ ├── cross-browser-baseline.sh # Cross-browser baselines + provenance (RFC 0002)
70+
│ ├── setup-baseline-lfs.sh # Git LFS opt-in for large baseline sets
6971
│ ├── setup-playwright.sh # One-time browser setup
7072
│ ├── check-dead-code.sh # Dead code detection (knip)
7173
│ ├── check-security.sh # Dependency audit + anti-patterns
@@ -249,7 +251,9 @@ Full catalog: `.claude/skills/README.md`
249251
### Testing
250252
```bash
251253
./scripts/run-tests.sh # Vitest unit/component tests
252-
./scripts/cross-browser-test.sh # Playwright multi-browser screenshots
254+
./scripts/cross-browser-test.sh # Playwright multi-browser screenshots (ad-hoc)
255+
./scripts/cross-browser-baseline.sh # Cross-browser baseline capture/compare (RFC 0002)
256+
./scripts/setup-baseline-lfs.sh # Route large baseline sets through Git LFS
253257
./scripts/setup-playwright.sh # One-time browser engine setup
254258
```
255259

docs/guides/error-recovery.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ Every phase has characteristic failure patterns. The table below lists the most
2727
| 5: Visual Diff | Screenshot capture fails, diff threshold exceeded after 5 iterations | Check dev server is running at expected port, lower threshold or fix components |
2828
| 5.5: Dark Mode | Dark theme not configured, screenshot fails | Non-blocking — add dark mode support or skip |
2929
| 6: E2E Tests | Browser not installed, test timeout, element not found | Run `./scripts/setup-playwright.sh`, increase timeout in `pipeline.config.json` |
30-
| 7: Cross-Browser | Firefox/WebKit not installed | Run `./scripts/setup-playwright.sh` to install all browsers |
30+
| 7: Cross-Browser | No committed baselines (compare skips), provenance flags, Firefox/WebKit not installed | Capture + commit baselines via `./scripts/cross-browser-baseline.sh capture` (pinned container); `./scripts/setup-playwright.sh` for local engines; see `docs/regression-testing/cross-browser.md` |
3131
| 7.5: Regression | No baselines captured yet | Run `./scripts/capture-baselines.sh` first, then `./scripts/regression-test.sh` |
3232
| 8: Quality Gate | Coverage below 80%, TypeScript errors, Lighthouse below thresholds | Write more tests, fix type errors, optimize performance |
3333
| 8.5: Responsive | Dev server not running, screenshot timeout | Start dev server first, check ports |

docs/onboarding/architecture.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -291,7 +291,9 @@ Located in `scripts/`. These are the automation backbone that agents and pipelin
291291
| Script | Purpose |
292292
|--------|---------|
293293
| `run-tests.sh` | Vitest with coverage |
294-
| `cross-browser-test.sh` | Playwright multi-browser screenshots |
294+
| `cross-browser-test.sh` | Playwright multi-browser screenshots (ad-hoc capture, no comparison) |
295+
| `cross-browser-baseline.sh` | Cross-browser baseline capture/compare with provenance (RFC 0002) |
296+
| `setup-baseline-lfs.sh` | Route large baseline sets through Git LFS (storage="lfs") |
295297
| `setup-playwright.sh` | One-time browser engine installation |
296298
| `capture-baselines.sh` | Capture baseline screenshots for regression |
297299
| `regression-test.sh` | Visual regression testing against baselines |

docs/onboarding/pipeline-configuration.md

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -785,6 +785,71 @@ Visual regression testing against stored baselines.
785785

786786
---
787787

788+
## Cross-Browser Baselines (`visualBaselines`)
789+
790+
Cross-browser (firefox/webkit) screenshot baseline storage per
791+
[RFC 0002](../rfcs/0002-cross-browser-screenshot-baseline-storage.md):
792+
pluggable backend, deterministic pinned-container capture, and per-baseline
793+
provenance. Shares `baselineDir` with `regressionTesting` (each flow walks
794+
only its own engines). Full guide:
795+
[docs/regression-testing/cross-browser.md](../regression-testing/cross-browser.md).
796+
797+
| Field | Type | Default | Description |
798+
| ------------------------- | ------------------------------------------------ | ---------------------------------------------- | --------------------------------------------------------------------------------------------- |
799+
| `enabled` | `boolean` | `true` | Enable the cross-browser baseline flow. |
800+
| `backend` | `string` (enum: `commit`, `ci-artifact`, `service`) | `"commit"` | Where baselines live: git-tracked, last-green-main CI artifact, or a SaaS provider. |
801+
| `storage` | `string` (enum: `git`, `lfs`) | `"git"` | commit backend only — `lfs` routes PNGs through Git LFS (`setup-baseline-lfs.sh`). |
802+
| `baselineDir` | `string` | `".claude/visual-qa/baselines"` | Baseline root, shared with `regressionTesting`. |
803+
| `screenshotDir` | `string` | `".claude/visual-qa/screenshots/cross-browser"`| Transient current captures (gitignored). |
804+
| `diffDir` | `string` | `".claude/visual-qa/diffs/cross-browser"` | Transient diff images (gitignored). |
805+
| `browsers` | `browserList` | `["chromium","firefox","webkit"]` | Engines to baseline; must stay within `e2e.crossBrowserBrowsers` (structural check). |
806+
| `routes` | `string[]` | `["/"]` | Routes to capture. |
807+
| `breakpoints` | `breakpointMap` | `{ "mobile": 375, "desktop": 1440 }` | Viewport name → pixel width. |
808+
| `threshold` | `percentageRatio` (0–1) | `0.03` | Cross-engine tolerance; must equal `e2e.crossBrowserDiffThreshold` (structural check). |
809+
| `blocking` | `boolean` | `false` | When `true`, compare exits non-zero on failures (Phase B teeth). |
810+
| `reportFile` | `string` | `"cross-browser-report.md"` | Report filename under `.claude/visual-qa/`. |
811+
| `capture.mode` | `string` (enum: `container`, `local`) | `"container"` | Container is required for trustworthy firefox/webkit baselines; local is recorded + flagged. |
812+
| `capture.image` | `string` | `"mcr.microsoft.com/playwright:v1.61.1-noble"` | The determinism pin; scripts warn on drift vs the resolved Playwright version. |
813+
| `capture.waitAfterLoadMs` | `integer (≥0)` | `1500` | Wait time before capture, in ms. |
814+
| `capture.fullPage` | `boolean` | `true` | Capture full page. |
815+
| `provenance.manifest` | `string` | `".claude/visual-qa/baselines/manifest.json"` | Committed provenance manifest (sha256, engine, image, host, gitSha per baseline). |
816+
| `provenance.policy` | `string` (enum: `warn`, `enforce`) | `"warn"` | `enforce` excludes flagged baselines from diffing and counts them as failures. |
817+
| `ciArtifact.compareAgainst` | `string` (enum: `last-green-main`) | `"last-green-main"` | ci-artifact backend: which build's artifact PRs compare against. |
818+
| `ciArtifact.retentionDays` | `integer (≥1)` | `30` | Retention of the published baselines artifact. |
819+
| `service.provider` | `string` (enum: `chromatic`, `percy`) | `"chromatic"` | service backend: which provider owns capture/diff/review. |
820+
| `service.projectTokenEnv` | `string` | `"CHROMATIC_PROJECT_TOKEN"` | Env var holding the provider project token (set as a CI secret). |
821+
822+
```json
823+
"visualBaselines": {
824+
"enabled": true,
825+
"backend": "commit",
826+
"storage": "git",
827+
"baselineDir": ".claude/visual-qa/baselines",
828+
"screenshotDir": ".claude/visual-qa/screenshots/cross-browser",
829+
"diffDir": ".claude/visual-qa/diffs/cross-browser",
830+
"browsers": ["chromium", "firefox", "webkit"],
831+
"routes": ["/"],
832+
"breakpoints": { "mobile": 375, "desktop": 1440 },
833+
"threshold": 0.03,
834+
"blocking": false,
835+
"reportFile": "cross-browser-report.md",
836+
"capture": {
837+
"mode": "container",
838+
"image": "mcr.microsoft.com/playwright:v1.61.1-noble",
839+
"waitAfterLoadMs": 1500,
840+
"fullPage": true
841+
},
842+
"provenance": {
843+
"manifest": ".claude/visual-qa/baselines/manifest.json",
844+
"policy": "warn"
845+
},
846+
"ciArtifact": { "compareAgainst": "last-green-main", "retentionDays": 30 },
847+
"service": { "provider": "chromatic", "projectTokenEnv": "CHROMATIC_PROJECT_TOKEN" }
848+
}
849+
```
850+
851+
---
852+
788853
## Dead Code (`deadCode`)
789854

790855
Unused export/dependency/file detection.

0 commit comments

Comments
 (0)