Skip to content

Commit cc05b04

Browse files
mmckyclaude
andauthored
test: wait for stable page height before visual snapshots (#402)
* test: wait for stable page height before visual snapshots After the Playwright 1.57 -> 1.60 (Chromium) bump in #400, math-heavy fixture pages on mobile-chrome intermittently settled ~60px shorter than their baselines (2888-2891px vs 2950px, varying between retries of the same run). toHaveScreenshot fails on any full-page dimension mismatch before pixel tolerances apply, so these flaked regardless of the maxDiffPixelRatio already granted to math pages. waitForReady now waits for document.fonts.ready and then polls until the document height is unchanged for three consecutive 250ms polls, replacing the fixed 500ms sleep after MathJax typesetting. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: address Copilot review — font-wait timeout + misplaced waitForFunction options - Update the waitForReady doc comment: pages without MathJax no longer return immediately; the font and height-stability waits apply to all pages. - Give the document.fonts.ready wait the same explicit 15s timeout as the other gates, via waitForFunction instead of evaluate. - Fix a latent bug the second comment surfaced: waitForFunction's second positional parameter is arg, not options, so the existing timeout (and the new polling interval) were never applied. Options now go in the third position on all three waits, making polling: 250 effective. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent 0258896 commit cc05b04

2 files changed

Lines changed: 41 additions & 2 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
88
## [Unreleased]
99

1010
### CI
11+
- **Stabilized flaky mobile-chrome visual tests on math-heavy pages** — after the Playwright 1.57→1.60 (Chromium) bump in #400, `math`/`proofs`/`cross-references` full-page screenshots intermittently rendered ~60px shorter than baseline, failing Playwright's dimension check before pixel tolerances apply. `waitForReady` now waits for `document.fonts.ready` and polls until the document height holds steady for 750ms, instead of a fixed 500ms sleep after MathJax typesetting.
1112
- **Cleared open Dependabot security alerts via `npm audit fix`**`webpack-dev-server` 5.2.2→5.2.4 (GHSA-79cf-xcqc-c78w), `shell-quote` 1.8.1→1.8.4 (GHSA-w7jw-789q-3m8p, critical), `qs` 6.14.2→6.15.2 (GHSA-q8mj-m7cp-5q26), `ws` 8.18.0→8.21.0 (GHSA-58qx-3vcg-4xpx). All transitive dev/build-time only; built theme assets unchanged. The remaining `uuid` alert (GHSA-w5hq-g745-h8pq) was dismissed as not exploitable — `sockjs` only calls `uuid.v4()`, the advisory affects `v3`/`v5`/`v6` with a caller-provided `buf`.
1213
- **Dependabot now watches npm** — added an `npm` ecosystem entry to `dependabot.yml` (monthly, grouped into a single PR) alongside the existing `github-actions` config.
1314

tests/visual/theme.spec.ts

Lines changed: 40 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,20 @@ import { test, expect, Page } from "@playwright/test";
1313
*/
1414

1515
// Wait for the page to fully settle, including MathJax typesetting when the
16-
// page contains math. Pages without MathJax resolve immediately.
16+
// page contains math (pages without MathJax pass the typeset gate
17+
// immediately; the font and height-stability waits below apply to every
18+
// page).
19+
//
20+
// After MathJax's startup promise resolves, font loading and late reflow
21+
// can still shift the total page height for a short window. Full-page
22+
// screenshots fail outright on any dimension mismatch (before pixel
23+
// tolerances apply), so a fixed post-typeset delay isn't enough: math-heavy
24+
// pages were observed settling ~60px apart between runs on mobile-chrome.
25+
// Instead of sleeping, wait for web fonts and then require the document
26+
// height to hold steady across consecutive polls.
27+
//
28+
// NB: waitForFunction's second positional parameter is `arg` (forwarded to
29+
// the page function) — `timeout`/`polling` options must go third.
1730
async function waitForReady(page: Page) {
1831
await page.waitForLoadState("networkidle");
1932
await page.waitForFunction(
@@ -22,9 +35,34 @@ async function waitForReady(page: Page) {
2235
if (!mj || !mj.startup || !mj.startup.promise) return true;
2336
return mj.startup.promise.then(() => true);
2437
},
38+
undefined,
39+
{ timeout: 15000 }
40+
);
41+
await page.waitForFunction(
42+
() => document.fonts.ready.then(() => true),
43+
undefined,
2544
{ timeout: 15000 }
2645
);
27-
await page.waitForTimeout(500);
46+
// Stable means three consecutive 250ms polls (750ms) without the
47+
// document height changing.
48+
await page.waitForFunction(
49+
() => {
50+
const w = window as any;
51+
const height = Math.max(
52+
document.body.scrollHeight,
53+
document.documentElement.scrollHeight
54+
);
55+
if (w.__qeLastHeight === height) {
56+
w.__qeStablePolls = (w.__qeStablePolls ?? 0) + 1;
57+
} else {
58+
w.__qeLastHeight = height;
59+
w.__qeStablePolls = 0;
60+
}
61+
return w.__qeStablePolls >= 3;
62+
},
63+
undefined,
64+
{ timeout: 15000, polling: 250 }
65+
);
2866
}
2967

3068
// hasMath: true loosens the snapshot tolerance for that page. MathJax font

0 commit comments

Comments
 (0)