Skip to content

Implement cross-browser screenshot baseline storage (RFC 0002) #111

Description

@PAMulligan

Summary

Implement the cross-browser screenshot baseline-storage design decided in
RFC 0002 — Cross-Browser Screenshot Baseline Storage.

This is the implementation follow-up for the decision issue #85 (AC #2:
"Implementation tracked as a follow-up issue"). Blocked on RFC 0002 acceptance
no code lands until the RFC Status moves Proposed → Accepted.

Decision recap

  • Default backend: commit baselines to the repo (ratify the current
    .gitignore behavior), extended to firefox/webkit; storage: "lfs" flag for
    large baseline sets.
  • Pluggable visualBaselines.backend: commit (default) | ci-artifact |
    service (Chromatic/Percy), behind a uniform adapter contract.
  • Determinism precondition: capture cross-browser baselines in a pinned
    Playwright container; record + verify per-baseline provenance.

Deliverables (phased per RFC §10)

Phase A — make the signal real (non-blocking)

  • Add the visualBaselines section to .claude/pipeline.config.json and its
    JSON Schema (strict additionalProperties: false).
  • Add the provenance manifest (baselines/manifest.json) — engine, Playwright
    version, container image, sha256, gitSha, host.
  • Capture and diff firefox/webkit from committed baselines (extend
    scripts/regression-test.sh or add scripts/cross-browser-baseline.sh),
    reusing scripts/visual-diff.js at threshold 0.03.
  • Fix the mislabeled orchestration.phases.cross-browser so the phase performs
    the comparison it advertises (replaces capture-only
    scripts/cross-browser-test.sh behavior).
  • Docs: update CLAUDE.md + a docs/ page for the cross-browser baseline flow.

Phase B — determinism, then teeth

  • Pin the Playwright container for capture in CI and locally (docker run).
  • Enforce provenance in the comparator (flag stale/foreign baselines instead of
    emitting false pixel diffs).
  • Once the measured false-positive rate is acceptable, allow blocking: true
    and flip qualityGate.crossBrowserScreenshotsRequired.

Phase C — the other adapters

  • storage: "lfs" flag + .gitattributes automation + migration docs.
  • ci-artifact adapter (compare against last-green main).
  • service adapter (Chromatic/Percy) — token via CI secret, opt-in.

Open questions to resolve during implementation

See RFC §13: unify regressionTesting vs. keep separate; blocking threshold;
image-pin strategy; local firefox/webkit capture policy (warn vs. fail); LFS
history migration.

References

Milestone: v2.0.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestpipelineFigma/Canva-to-React conversion pipeline

    Projects

    Status
    Done

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions