feat(ui-scripts): show a11y violations on the screenshot itself

The a11y section of the visual-diff report was a nested list of axe rule ids, CSS
selectors, and raw failureSummary prose, repeated per theme. Nothing was anchored to
the image, so the artifact designers actually read - the screenshot - carried no
accessibility information at all.

Capture: while the page is still rendered, record each violating element's bounding
box, a human-readable name, and the contrast numbers axe already computed. a11y.json
gains a per-screenshot page size so boxes can be positioned as percentages and hold at
any rendered scale.

Report: numbered boxes over each offending element on the "Actual" image, colored and
ordered by impact; a card per finding leading with plain language, the element's name,
a suggested fix, and - for contrast - swatches with the measured and required ratios;
a run-level overview grouped by rule that collapses the three themes into one line;
impact and rule filters that narrow boxes and cards together; and a lightbox A11y mode
pairing the full-size screenshot with its findings. Raw axe output moves behind a
Technical details disclosure.

The PR comment now names the failing rules in a table instead of pointing at the run
logs.

Reports generated from a11y.json captured before this change still render: the boxes
are omitted, and contrast is recovered by parsing axe's prose.

Author: balzss

.github/workflows/visual-regression.yml

@@ -125,7 +125,7 @@ jobs:
         run: |
           node -e "
             const fs = require('fs');
-            const { summary: s, results } = JSON.parse(fs.readFileSync('regression-test/.report/summary.json', 'utf8'));
+            const { summary: s, results, a11y = [] } = JSON.parse(fs.readFileSync('regression-test/.report/summary.json', 'utf8'));
             const base = 'https://instructure.design/visual-regression/pr-' + process.env.PR_NUMBER;
             const interesting = results.filter(r => r.status === 'changed' || r.status === 'removed');
             let md = '';
@@ -141,6 +141,17 @@ jobs:
               }
               md += '</details>';
             }
+            // Name the a11y rules in the comment. Numbers are per offending
+            // element; the same rule across themes is one row, matching the
+            // report's overview panel.
+            let a11yMd = '✅ No violations.';
+            if (a11y.length) {
+              a11yMd = '⚠️ **' + s.a11yViolations + ' violation' + (s.a11yViolations === 1 ? '' : 's') +
+                '** on ' + s.a11yPages + ' screenshot' + (s.a11yPages === 1 ? '' : 's') + '.\n\n' +
+                '| Impact | Issue | Rule | Elements | Screenshots |\n|---|---|---|---:|---:|\n' +
+                a11y.map(r => '| ' + r.impact + ' | ' + r.title + ' | [`' + r.rule + '`](' + r.helpUrl + ') | ' +
+                  r.nodes + ' | ' + r.screens + ' |').join('\n');
+            }
             const lines = [
               'total=' + s.total,
               'unchanged=' + s.unchanged,
@@ -151,6 +162,7 @@ jobs:
             ];
             let out = lines.join('\n') + '\n';
             out += 'diff_images<<DIFFIMAGES_EOF\n' + md + '\nDIFFIMAGES_EOF\n';
+            out += 'a11y<<A11Y_EOF\n' + a11yMd + '\nA11Y_EOF\n';
             fs.appendFileSync(process.env.GITHUB_OUTPUT, out);
           "
 
@@ -161,7 +173,7 @@ jobs:
           message: |
             ## Visual regression report
 
-            **Cypress assertions (a11y + console errors):** ${{ steps.cypress.outcome == 'failure' && format('❌ **Failing** — see the [run logs]({0}/{1}/actions/runs/{2}).', github.server_url, github.repository, github.run_id) || '✅ Passing' }}
+            **Cypress suite:** ${{ steps.cypress.outcome == 'failure' && format('❌ **Failing** — see the [run logs]({0}/{1}/actions/runs/{2}).', github.server_url, github.repository, github.run_id) || '✅ Passing' }}
 
             **Visual diff:** ${{ steps.summary.outputs.status }}
 
@@ -172,11 +184,13 @@ jobs:
             | New | ${{ steps.summary.outputs.added }} |
             | Removed | ${{ steps.summary.outputs.removed }} |
 
-            📊 **[View full report](https://instructure.design/visual-regression/pr-${{ github.event.pull_request.number }}/index.html)**
+            **Accessibility (axe):** ${{ steps.summary.outputs.a11y }}
+
+            📊 **[View full report](https://instructure.design/visual-regression/pr-${{ github.event.pull_request.number }}/index.html)** — every violation is boxed and numbered on its screenshot, with the offending element named and contrast failures shown as color swatches.
 
             ${{ steps.summary.outputs.diff_images }}
 
-            <sub>Baselines come from the `visual-baselines` branch. They refresh on every merge to `master`. The <b>Cypress assertions</b> line above covers a11y (axe) and console-error checks — a ❌ there means the suite found real issues even if the visual diff is clean.</sub>
+            <sub>Baselines come from the `visual-baselines` branch. They refresh on every merge to `master`. The <b>Cypress suite</b> line covers the a11y and console-error assertions — a ❌ there means the suite found real issues even if the visual diff is clean.</sub>
 
       - name: Fail if regressions or Cypress failed
         if: steps.diff.outcome == 'failure' || steps.cypress.outcome == 'failure'

docs/contributing/testing/visual-regression.md

@@ -59,35 +59,58 @@ open .report/index.html
 
 `npm run visual-diff` invokes the same `ui-scripts visual-diff` command CI uses. Pass extra flags to override the defaults:
 
-| Flag                                                           | Default               | Description                                                                                                                                                                |
-| -------------------------------------------------------------- | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `--actual-dir <dir>`                                           | `cypress/screenshots` | Newly captured screenshots.                                                                                                                                                |
-| `--baseline-dir <dir>`                                         | `.baselines`          | Baselines to compare against.                                                                                                                                              |
-| `--output-dir <dir>`                                           | `visual-report`       | Where the HTML report + diff PNGs are written.                                                                                                                             |
-| `--threshold <0..1>`                                           | `0.1`                 | Per-pixel YIQ color threshold. Lower is stricter.                                                                                                                          |
-| `--fail-on-missing-baseline` / `--no-fail-on-missing-baseline` | on                    | Whether to exit 1 when actual screenshots have no baseline.                                                                                                                |
-| `--pr-number <n>`                                              | —                     | Rendered in the report header as a link to the PR.                                                                                                                         |
-| `--pr-url <url>`                                               | —                     | Target of the PR link.                                                                                                                                                     |
-| `--meta <file>`                                                | —                     | JSON mapping screenshot slug → visited URL path. Enables per-row source-file links in the report. Produced automatically by `spec.cy.ts` via `cy.task('recordMeta', ...)`. |
-| `--source-base-url <url>`                                      | —                     | GitHub blob URL of `regression-test/src/app` on the branch being reviewed. Combined with `--meta` to build links like `treebrowser/page.tsx`.                              |
-| `--facets <list>`                                              | —                     | Comma-separated facet suffixes (e.g. `canvas,light,dark`) rendered as one-click filter chips that match screenshots named `<name>-<facet>`.                                |
-| `--app-path <path>`                                            | —                     | Path, relative to the report root, where a static export of the app is published (CI uses `app`). Enables the lightbox **HTML** view, which iframes the live rendered page. |
+| Flag                                                           | Default               | Description                                                                                                                                                                                              |
+| -------------------------------------------------------------- | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `--actual-dir <dir>`                                           | `cypress/screenshots` | Newly captured screenshots.                                                                                                                                                                              |
+| `--baseline-dir <dir>`                                         | `.baselines`          | Baselines to compare against.                                                                                                                                                                            |
+| `--output-dir <dir>`                                           | `visual-report`       | Where the HTML report + diff PNGs are written.                                                                                                                                                           |
+| `--threshold <0..1>`                                           | `0.1`                 | Per-pixel YIQ color threshold. Lower is stricter.                                                                                                                                                        |
+| `--fail-on-missing-baseline` / `--no-fail-on-missing-baseline` | on                    | Whether to exit 1 when actual screenshots have no baseline.                                                                                                                                              |
+| `--pr-number <n>`                                              | —                     | Rendered in the report header as a link to the PR.                                                                                                                                                       |
+| `--pr-url <url>`                                               | —                     | Target of the PR link.                                                                                                                                                                                   |
+| `--meta <file>`                                                | —                     | JSON mapping screenshot slug → visited URL path. Enables per-row source-file links in the report. Produced automatically by `spec.cy.ts` via `cy.task('recordMeta', ...)`.                               |
+| `--source-base-url <url>`                                      | —                     | GitHub blob URL of `regression-test/src/app` on the branch being reviewed. Combined with `--meta` to build links like `treebrowser/page.tsx`.                                                            |
+| `--facets <list>`                                              | —                     | Comma-separated facet suffixes (e.g. `canvas,light,dark`) rendered as one-click filter chips that match screenshots named `<name>-<facet>`.                                                              |
+| `--app-path <path>`                                            | —                     | Path, relative to the report root, where a static export of the app is published (CI uses `app`). Enables the lightbox **HTML** view, which iframes the live rendered page.                              |
+| `--a11y <file>`                                                | —                     | JSON of axe violations keyed by screenshot slug. Enables everything in [Accessibility findings](#accessibility-findings) below. Produced automatically by `spec.cy.ts` via `cy.task('recordA11y', ...)`. |
 
 ### Interpreting the report
 
 The HTML report is a single self-contained page showing one section per screenshot. Each row has a status badge (`ok`, `changed`, `new`, `removed`), a three-up grid of baseline / actual / diff images, and — when metadata is available — a link to the source `page.tsx`.
 
 Top-bar controls:
 
-- **Filter buttons** (`All`, `Changed`, `New`, `Removed`, `Unchanged`) — the active one is highlighted.
+- **Filter buttons** (`All`, `Changed`, `New`, `Removed`, `Unchanged`, `⚠ A11y`) — the active one is highlighted.
 - **Search input** — live, debounced substring filter on the screenshot name.
+- **Theme chips** — narrow to one theme's screenshots.
+- **Impact chips** and **Show markers on screenshots** — see below.
 - **Lightbox** — click any thumbnail to open a fullscreen viewer. Inside:
   - `Baseline`, `Actual`, `Diff` buttons switch between the three images.
   - `Slider` mode overlays baseline and actual with a drag handle so you can scrub the boundary back and forth.
+  - `⚠ N a11y` mode shows the screenshot at full size with every violating element boxed, and the findings docked alongside. Shown only for screenshots with findings.
   - `HTML` mode iframes the live rendered page (a static export of the app published next to the report) so you can inspect the real DOM, computed styles, and text alongside the pixels. Shown only when the page still exists and the app was published (`--app-path`).
   - `1:1` / `Fit` toggles between native pixel size (scrollable) and fit-to-window.
   - `‹` / `›` step through visible rows (respects the current filter).
 
+### Accessibility findings
+
+axe runs against every page and theme, and the report shows what it found **on the picture**, so a designer can act on it without reading a selector.
+
+While the page is still on screen, `spec.cy.ts` records each violating element's bounding box, a human-readable name for it, and — for contrast failures — the foreground/background colors and ratios axe computed (see `cypress/support/a11y-capture.ts`). The report then renders:
+
+- **A run-level overview** at the top of the page: one line per rule with its impact, how many elements it affects, and how many screenshots it appears on. The same rule tripping in `canvas`, `light`, and `dark` collapses to a single line. Click a rule to narrow the list to just the screenshots it affects.
+- **Numbered boxes on the “Actual” image**, colored by impact. The numbers match the cards below, so “#3” is the same finding everywhere it appears. Uncheck **Show markers on screenshots** when they get in the way of comparing pixels.
+- **One card per offending element**, leading with a plain-language headline ("Text contrast is too low") rather than the rule id, naming the element (`button “Archive this item”`), and suggesting a fix. Contrast failures show both colors as swatches with the measured and required ratios. The raw axe output — rule id, selector, and `failureSummary` — sits behind **Technical details**.
+- **Impact chips** (`critical` / `serious` / `moderate` / `minor`) that narrow both the list and the boxes on each image.
+
+Clicking a card's number badge opens the lightbox in **A11y** mode, scrolled to that finding. In that view, clicking a card highlights its box and vice versa.
+
+Findings are ordered worst-impact-first, so `#1` on a screenshot is always the most serious thing on it.
+
+Plain-language copy comes from a rule dictionary in `packages/ui-scripts/lib/commands/visual-diff.ts` (`RULE_COPY`). A rule that isn't in it still renders correctly — it just falls back to axe's own `help` text. Add an entry when a new rule starts showing up.
+
+A page with a known, tracked a11y issue can opt out of the axe check entirely with `a11y: false` and an `a11ySkipReason` in the `PAGES` array in `spec.cy.ts`.
+
 ## Adding a new component page
 
 1. Create `regression-test/src/app/<component-name>/page.tsx`. The page component must start with `'use client'` and render meaningful variations of the component you're covering.