ci: fix visual regression tests — provision a pinned Chrome for Puppeteer + Percy (#3255)

* ci: cache Puppeteer's Chrome instead of re-installing it every run

The build_lint_and_test job cached only the pnpm store. Puppeteer downloads
Chrome to ~/.cache/puppeteer (outside node_modules / the store), and pnpm's
side-effects cache records the postinstall as already-run on a store cache hit,
so the browser binary was never preserved — the explicit `puppeteer browsers
install chrome` step re-fetched it every run. When the runner image shipped a
stale/partial ~/.cache/puppeteer for the pinned Chrome build, @puppeteer/browsers
refused to overwrite it and failed with "the browser folder exists but the
executable is missing", blocking the job.

Relocate the browser to a workspace-local PUPPETEER_CACHE_DIR and cache that
directory (keyed on the lockfile). The install step is now a no-op on a cache
hit and a clean download into an empty dir on a miss, so it can't collide with a
partially-populated system cache. No `rm -rf`, and the binary is actually
preserved across runs.

* ci: key Puppeteer browser cache on the resolved puppeteer version

Resolve the installed puppeteer version and use it as the cache key, instead of
the lockfile hash + restore-keys. The Chrome build is pinned transitively by the
puppeteer dependency, so a Puppeteer bump now changes the key and forces a clean
re-download, while same-version runs reuse the cached binary. Dropping
restore-keys avoids ever falling back to a stale-version browser, and avoids the
cache churning on unrelated lockfile changes.

* ci: skip Puppeteer's postinstall download; install Chrome explicitly

Root cause of the install failure: with PUPPETEER_CACHE_DIR set, Puppeteer's
postinstall (run during `pnpm install` under the restored pnpm store) leaves a
partial Chrome — the version folder without the executable. @puppeteer/browsers
2.3.0 then refuses to repair it and fails the explicit install with "the browser
folder exists but the executable is missing".

Set PUPPETEER_SKIP_DOWNLOAD=true for the install step so downloadBrowser() returns
before touching the cache dir (verified against puppeteer 22.15.0's
node/install.js). The explicit `puppeteer browsers install chrome` step — which
does not consult that flag — remains the sole, authoritative installer into the
version-keyed cache dir.

* ci: re-trigger CI (Percy build did not finalize on prior run)

* ci: clear stale Puppeteer cache before install

Reverts the cache/relocation/skip-download experiments back to main's working
flow. Adds a single step to remove a partial ~/.cache/puppeteer (browser folder
without the executable) that the runner image can ship — @puppeteer/browsers
refuses to repair an incomplete install (same check in 2.3.0 through 2.13.2), so
a version bump can't fix it. Clearing it lets the postinstall download a clean
Chrome.

* ci: pin build_lint_and_test to ubuntu-22.04 to fix VRT snapshots

The ubuntu-24.04 runner image (~20260525) tightened AppArmor unprivileged-userns
restrictions, preventing Percy/Chromium from launching — VRT ran green but
captured 0 snapshots (Percy builds 'manually failed'). Last green VRT (81
snapshots, May 27) was on image 20260518.

Pin the job to ubuntu-22.04, which has no such restriction, and drop the now-
obsolete 24.04-specific workarounds: the aa-exec --profile=chrome wrapper (added
for 24.04 AppArmor) and the ~/.cache/puppeteer clear (the husk was a 24.04 image
artifact). This restores the pre-24.04 VRT configuration on a compatible OS.

* ci: point Percy at runner's Chrome to skip pruned Chromium download

Root cause of the 0-snapshot VRT runs: @percy/core 1.31.13 pins Chromium
snapshot revision 1300309 and auto-downloads it for asset discovery. That
revision has been pruned from the public Chromium snapshots bucket, so the
download (which took 4s on May 27) now hangs indefinitely and Percy captures no
snapshots — independent of the runner OS. Set PERCY_BROWSER_EXECUTABLE to the
runner's pre-installed Chrome so @percy/core uses it and skips the download
(verified in @percy/core dist/browser.js: executable || install.chromium()).

* ci: use runner's Chrome for both visual tests and Percy

Both pinned browser downloads now fail in CI, yielding 0 snapshots:
- puppeteer's Chrome-for-Testing 127.0.6533.88 no longer installs (jest-puppeteer
  then can't find Chrome -> globalSetup error)
- @percy/core's Chromium snapshot 1300309 was pruned from the public bucket, so
  its auto-download hangs

Point both at the runner's pre-installed Chrome via PUPPETEER_EXECUTABLE_PATH and
PERCY_BROWSER_EXECUTABLE, and drop the now-redundant 'Install Puppeteer's Chrome'
step. jest-puppeteer.config.js already reads PUPPETEER_EXECUTABLE_PATH and runs
with --no-sandbox. Fails fast if no Chrome is found rather than silently falling
back to a broken download.

* ci: provision Chrome via setup-chrome on ubuntu-24.04 for VRT + Percy

Revert the ubuntu-22.04 pin (unnecessary: deleting the puppeteer install step
removes the ~/.cache/puppeteer husk issue, and --no-sandbox in both launchers
makes the 24.04 AppArmor workaround moot — so 24.04 works and avoids the 22.04
deprecation track).

Provision Chrome explicitly with browser-actions/setup-chrome (SHA-pinned) and
point PUPPETEER_EXECUTABLE_PATH + PERCY_BROWSER_EXECUTABLE at it, instead of
relying on the runner's rolling-latest Chrome. Using chrome-version: stable for
now; will pin a specific build for reproducible Percy snapshots once green.

* fix(vrt): add --no-sandbox to self-launching rich-text specs; pin Chrome 127

The rich-text-input and localized-rich-text-input visualspecs launch their OWN
puppeteer browser (not the jest-puppeteer global one) and did not pass
--no-sandbox. On ubuntu-24.04 (AppArmor restricts unprivileged user namespaces)
that crashes with 'No usable sandbox', failing those 2 suites and dropping a
Percy snapshot. They previously only passed because the removed aa-exec wrapper
covered them. Add --no-sandbox/--disable-setuid-sandbox to both launches, matching
jest-puppeteer.config.js used by the other 68 specs.

Also pin setup-chrome to 127.0.6533.88 (puppeteer 22.15.0's target build and the
Percy baseline's browser) instead of 'stable' — 'stable' pulled Chrome 149.

Author: misama-ct

.github/workflows/main.yml

@@ -67,25 +67,31 @@ jobs:
       - name: Building Visual Regression Tests application for UI components
         run: pnpm visual-testing-app:build
 
-      # Puppeteer's postinstall hook downloads Chrome to ~/.cache/puppeteer.
-      # Under pnpm, the postinstall script's side-effects are cached in the pnpm
-      # store (which is restored from the GitHub Actions cache), so pnpm skips
-      # re-running it. But Chrome lives outside node_modules / the pnpm store,
-      # so the binary is missing on the runner. Install it explicitly here so
-      # the visual regression tests can find it.
-      - name: Install Puppeteer's Chrome
-        run: pnpm exec puppeteer browsers install chrome
+      # The visual-test browser (jest-puppeteer) and Percy's asset-discovery
+      # browser both need a Chrome. Puppeteer's and Percy's own pinned downloads
+      # (Chrome-for-Testing 127.0.6533.88 / Percy Chromium snapshot 1300309) stopped
+      # working in CI and yielded 0 snapshots, so we provision Chrome explicitly
+      # here and point both tools at it. jest-puppeteer.config.js reads
+      # PUPPETEER_EXECUTABLE_PATH and launches with --no-sandbox.
+      #
+      # Pinned to 127.0.6533.88 — the Chrome build puppeteer 22.15.0 targets and
+      # the one the Percy baseline was captured with. `chrome-version: stable`
+      # pulled Chrome 149 (~22 majors newer), which broke 2 visual test suites
+      # via CDP/runtime drift. Bump this in lockstep with puppeteer.
+      - name: Set up Chrome
+        id: setup-chrome
+        uses: browser-actions/setup-chrome@2e1d749697dd1612b833dba4a722266286fbefcd # v2.1.2
+        with:
+          chrome-version: 127.0.6533.88
 
       - name: Running Visual Regression Tests for UI components
-      # aa-exec:
-      # ubuntu-24.04, the image used for `ubuntu-latest` as of Oct 14 2024(https://github.com/actions/runner-images/issues/10636),
-      # introduced security measures in its app-armor security  (https://github.com/puppeteer/puppeteer/issues/12818#issuecomment-2247844464)
-      # that made it so that puppeteers chromium installation is not whitelisted,
-      # which made it so that the chromium sandbox is not available and puppeteer errors out,
-      # so we need to specify that we are using app armor's chrome profile when running puppeteer (https://github.com/mermaid-js/mermaid-cli/issues/730#issuecomment-2408615110)
-        run: aa-exec --profile=chrome -- pnpm vrt:components
+        run: |
+          echo "Using Chrome ${{ steps.setup-chrome.outputs.chrome-version }} at $PUPPETEER_EXECUTABLE_PATH"
+          pnpm vrt:components
         timeout-minutes: 20
         env:
+          PUPPETEER_EXECUTABLE_PATH: ${{ steps.setup-chrome.outputs.chrome-path }}
+          PERCY_BROWSER_EXECUTABLE: ${{ steps.setup-chrome.outputs.chrome-path }}
           PERCY_TOKEN: ${{ secrets.PERCY_TOKEN }}
 
 

packages/components/inputs/localized-rich-text-input/src/localized-rich-text-input.visualspec.js

@@ -9,6 +9,11 @@ jest.setTimeout(20000);
 beforeEach(async () => {
   browser = await puppeteer.launch({
     headless: 'new',
+    // This spec launches its own browser (not the jest-puppeteer global one), so
+    // it must opt out of the Chrome sandbox itself. CI runners (Ubuntu 24.04+)
+    // restrict unprivileged user namespaces via AppArmor, so the sandbox can't
+    // start otherwise. Matches jest-puppeteer.config.js used by other specs.
+    args: ['--no-sandbox', '--disable-setuid-sandbox'],
     slowMo: 10, // Launching the browser in slow motion is necessary due to race conditions. Otherwise browser closes prematurely and tests fail.
   });
   page = await browser.newPage();

packages/components/inputs/rich-text-input/src/rich-text-input.visualspec.js

@@ -9,6 +9,11 @@ jest.setTimeout(20000);
 beforeEach(async () => {
   browser = await puppeteer.launch({
     headless: 'new',
+    // This spec launches its own browser (not the jest-puppeteer global one), so
+    // it must opt out of the Chrome sandbox itself. CI runners (Ubuntu 24.04+)
+    // restrict unprivileged user namespaces via AppArmor, so the sandbox can't
+    // start otherwise. Matches jest-puppeteer.config.js used by other specs.
+    args: ['--no-sandbox', '--disable-setuid-sandbox'],
     slowMo: 10, // Launching the browser in slow motion is necessary due to race conditions. Otherwise browser closes prematurely and tests fail.
   });
   page = await browser.newPage();