fix(engine): make software-renderer screenshot guard opt-in (CI regression)

PR #822's software-renderer guard auto-flipped `captureMode` to "screenshot"
whenever `resolveBrowserGpuMode` returned "software" — but the WebGL probe
classifies GitHub Actions runners (SwiftShader) as "software" too, even
though `HeadlessExperimental.beginFrame` works on them. As a result the
guard pessimized mainstream CI, exceeding `ffmpegStreamingTimeout` (10 min
default) on the largest fixtures and producing this regression on three
shards of run 25841853063:

  - fast → sub-composition-video (390 frames, 1080×1920, video + sub-comp)
  - styles-e → style-13-prod (12 MB baseline, slow tag)
  - styles-g → overlay-montage-prod (42.36 s duration, 1080×1920)

All three failed with `Streaming encode failed: FFmpeg exited with code 255
... received signal 15`. Wall-clock for overlay-montage-prod jumped from
9 m 13 s (main, beginframe) to 10 m 38 s (PR-822, screenshot) — just past
the streaming timeout. ffmpeg was alive and consuming frames the whole
time; the producer side simply couldn't sustain the rate in screenshot
mode for these fixtures.

Fix: gate the auto-flip behind an opt-in env var
`HYPERFRAMES_FORCE_SCREENSHOT_ON_SOFTWARE_GPU`. When unset (default), the
GPU probe is skipped entirely and the original (pre-PR-822) capture-mode
selection runs — Linux + headless-shell → beginframe, with the existing
`probeBeginFrameSupport` fallback handling actual BeginFrame unavailability
at runtime. Operators on hosts where BeginFrame is technically present but
stalls under shader load (the CPU-only Linux sandbox where hf#677 was
reproduced) set the env var to short-circuit BeginFrame entirely.

Mirrored the same gate in `frameCapture.ts`'s preMode resolution so the
expensive probe Chrome is also skipped on the chromeArgs side when the
guard is off.

Tests: kept all eight software-renderer tests; flipped the block-scoped
default to "guard ON" (set in `beforeEach`) so the pre-existing assertions
exercise the guard path. Added a positive-control test that pins the new
default behavior: `Linux + software + headless-shell + guard OFF →
beginframe` (the regression case).

— Vai

Author: vanceingalls

packages/engine/src/services/browserManager.test.ts

@@ -128,14 +128,18 @@ describe("resolveBrowserGpuMode", () => {
   });
 });
 
-describe("acquireBrowser — software-renderer guard", () => {
-  // Tests for the defense-in-depth rule that a "software"-resolved GPU mode
-  // must force `captureMode = "screenshot"` regardless of platform/binary.
-  // Without this guard, BeginFrame attempts on a software-rendered compositor
-  // hang at the CDP protocolTimeout (the hf#677 spike repro). Tests below
-  // mock puppeteer at the module-import level so no real Chrome is launched.
+describe("acquireBrowser — software-renderer guard (opt-in)", () => {
+  // Tests for the opt-in rule that, when
+  // `HYPERFRAMES_FORCE_SCREENSHOT_ON_SOFTWARE_GPU=1` is set, a
+  // "software"-resolved GPU mode must force `captureMode = "screenshot"`
+  // regardless of platform/binary. Without the env var the guard is dormant —
+  // mainstream CI hosts (SwiftShader on GitHub runners) report "software" but
+  // BeginFrame works on them, and pessimizing those hosts caused regressions
+  // on the largest fixtures (hf#822). Tests below mock puppeteer at the
+  // module-import level so no real Chrome is launched.
 
   const origPlatform = process.platform;
+  const origGuardEnv = process.env.HYPERFRAMES_FORCE_SCREENSHOT_ON_SOFTWARE_GPU;
 
   // Captures the most recent puppeteer.launch call so tests can assert on the
   // args/executablePath actually passed to Chrome.
@@ -173,16 +177,25 @@ describe("acquireBrowser — software-renderer guard", () => {
     // `vi.resetModules()` already gives each test a fresh `browserManager.js`
     // module — so `_softwareGuardWarned` starts at false per test. No
     // additional reset needed.
+    // Default: guard ON for the bulk of tests in this block (they exercise
+    // the guard's behavior). Individual tests opt back to "guard OFF" by
+    // deleting the env var.
+    process.env.HYPERFRAMES_FORCE_SCREENSHOT_ON_SOFTWARE_GPU = "1";
   });
 
   afterEach(() => {
     Object.defineProperty(process, "platform", { value: origPlatform, configurable: true });
+    if (origGuardEnv === undefined) {
+      delete process.env.HYPERFRAMES_FORCE_SCREENSHOT_ON_SOFTWARE_GPU;
+    } else {
+      process.env.HYPERFRAMES_FORCE_SCREENSHOT_ON_SOFTWARE_GPU = origGuardEnv;
+    }
     vi.restoreAllMocks();
     vi.doUnmock("puppeteer");
     vi.doUnmock("puppeteer-core");
   });
 
-  it("forces screenshot capture mode when browserGpuMode resolves to 'software' on Linux", async () => {
+  it("forces screenshot capture mode when browserGpuMode resolves to 'software' on Linux (guard enabled)", async () => {
     installPuppeteerMock();
     const { acquireBrowser: acquire } = await import("./browserManager.js");
 
@@ -210,6 +223,27 @@ describe("acquireBrowser — software-renderer guard", () => {
     expect(lastLaunchArgs).toContain("--no-sandbox");
   });
 
+  it("KEEPS beginframe capture mode on Linux+software when the guard is NOT enabled (default)", async () => {
+    // Mirror of the "forces screenshot" test above with the guard OFF — this
+    // is the production default that mainstream CI hits. Without this, GH
+    // Actions runners (SwiftShader, reports "software") regress to screenshot
+    // mode and the streaming encoder times out on shader-heavy fixtures
+    // (hf#822 regression on overlay-montage-prod, sub-composition-video,
+    // style-13-prod).
+    delete process.env.HYPERFRAMES_FORCE_SCREENSHOT_ON_SOFTWARE_GPU;
+    installPuppeteerMock();
+    const { acquireBrowser: acquire } = await import("./browserManager.js");
+
+    const chromeArgs = ["--no-sandbox", "--enable-begin-frame-control"];
+    const result = await acquire(chromeArgs, {
+      chromePath: "/fake/chrome-headless-shell",
+      browserGpuMode: "software",
+    });
+
+    expect(result.captureMode).toBe("beginframe");
+    expect(lastLaunchArgs).toContain("--enable-begin-frame-control");
+  });
+
   it("keeps beginframe capture mode when browserGpuMode is 'hardware' on Linux with headless-shell", async () => {
     installPuppeteerMock();
     const { acquireBrowser: acquire } = await import("./browserManager.js");

packages/engine/src/services/browserManager.ts

@@ -260,6 +260,33 @@ export function _resetSoftwareGuardWarnedForTests(): void {
   _softwareGuardWarned = false;
 }
 
+/**
+ * Opt-in: force `captureMode = "screenshot"` whenever the GPU resolves to
+ * "software" (no hardware WebGL).
+ *
+ * Why opt-in rather than always-on: GitHub Actions runners and other common
+ * SwiftShader hosts report "software" from the WebGL probe but
+ * `HeadlessExperimental.beginFrame` works reliably on them — so a blanket
+ * software → screenshot flip pessimizes mainstream CI by ~1.5 min on
+ * shader/composition-heavy fixtures and exceeds `ffmpegStreamingTimeout` on
+ * the largest renders.
+ *
+ * The runtime probe (`probeBeginFrameSupport` below) already catches actual
+ * BeginFrame *unavailability*. This env var exists for hosts where BeginFrame
+ * is technically present but the compositor stalls under shader load (the
+ * CPU-only Linux sandbox where hf#677 was reproduced). Operators on such
+ * hosts set `HYPERFRAMES_FORCE_SCREENSHOT_ON_SOFTWARE_GPU=1` to short-circuit
+ * BeginFrame entirely.
+ *
+ * Accepts the same truthy values as other engine env flags (`"1"`, `"true"`).
+ */
+export function isSoftwareScreenshotGuardEnabled(): boolean {
+  const raw = process.env.HYPERFRAMES_FORCE_SCREENSHOT_ON_SOFTWARE_GPU;
+  if (!raw) return false;
+  const v = raw.trim().toLowerCase();
+  return v === "1" || v === "true" || v === "yes" || v === "on";
+}
+
 export interface AcquireBrowserOptions {
   /**
    * If the caller already resolved `browserGpuMode` (e.g. `frameCapture.ts`
@@ -301,31 +328,32 @@ export async function acquireBrowser(
   const isLinux = process.platform === "linux";
   const forceScreenshot = config?.forceScreenshot ?? DEFAULT_CONFIG.forceScreenshot;
 
-  // Resolve browserGpuMode. On software-renderer hosts (no hardware GPU /
-  // SwiftShader) HeadlessExperimental.beginFrame is unreliable — the
-  // compositor stalls indefinitely on shader-heavy frames and the CDP call
-  // times out at protocolTimeout (default 5 min). Force screenshot mode
-  // whenever resolveBrowserGpuMode resolves to "software", independent of
-  // platform/binary. This is a separate defense from the producer-side
-  // `captureHdrStage` guard (which unconditionally forces screenshot for the
-  // HDR layered-composite path); this guard covers the SDR-render-on-software-
-  // host case that captureHdrStage doesn't touch.
+  // The software-renderer screenshot guard is opt-in (see
+  // `isSoftwareScreenshotGuardEnabled` above). Skip the GPU probe entirely
+  // unless the guard is enabled — the probe launches a throwaway Chrome to
+  // run a WebGL availability check, so paying for it on every render when
+  // the result is unused is wasteful. Operators on stall-prone hosts set
+  // `HYPERFRAMES_FORCE_SCREENSHOT_ON_SOFTWARE_GPU=1` and accept the probe
+  // cost as part of that contract.
   //
   // If the caller has already resolved the mode (frameCapture.ts does) it
   // hands us the value via options.resolvedBrowserGpuMode to avoid a second
   // resolution from raw config. The probe Promise is cached for the process
-  // lifetime so the fallback path is still cheap.
-  let resolvedGpuMode: "software" | "hardware";
-  if (options.resolvedBrowserGpuMode) {
-    resolvedGpuMode = options.resolvedBrowserGpuMode;
-  } else {
-    const browserGpuMode = config?.browserGpuMode ?? DEFAULT_CONFIG.browserGpuMode;
-    resolvedGpuMode = await resolveBrowserGpuMode(browserGpuMode, {
-      chromePath: headlessShell ?? undefined,
-      browserTimeout: config?.browserTimeout,
-    });
+  // lifetime so even when both paths fire, only one Chrome launch happens.
+  const guardEnabled = isSoftwareScreenshotGuardEnabled();
+  let resolvedGpuMode: "software" | "hardware" | undefined;
+  if (guardEnabled) {
+    if (options.resolvedBrowserGpuMode) {
+      resolvedGpuMode = options.resolvedBrowserGpuMode;
+    } else {
+      const browserGpuMode = config?.browserGpuMode ?? DEFAULT_CONFIG.browserGpuMode;
+      resolvedGpuMode = await resolveBrowserGpuMode(browserGpuMode, {
+        chromePath: headlessShell ?? undefined,
+        browserTimeout: config?.browserTimeout,
+      });
+    }
   }
-  const isSoftwareRenderer = resolvedGpuMode === "software";
+  const isSoftwareRenderer = guardEnabled && resolvedGpuMode === "software";
 
   let captureMode: CaptureMode;
   let executablePath: string | undefined;

packages/engine/src/services/frameCapture.ts

@@ -18,6 +18,7 @@ import {
   releaseBrowser,
   forceReleaseBrowser,
   buildChromeArgs,
+  isSoftwareScreenshotGuardEnabled,
   resolveBrowserGpuMode,
   resolveHeadlessShellPath,
   type CaptureMode,
@@ -195,27 +196,35 @@ export async function createCaptureSession(
   // need explicit clip+scale on `Page.captureScreenshot`, so fall back to
   // the screenshot path for any DPR > 1.
   const supersampling = (options.deviceScaleFactor ?? 1) > 1;
+  // Mirror the software-renderer guard in `acquireBrowser`. The guard is
+  // opt-in (env var `HYPERFRAMES_FORCE_SCREENSHOT_ON_SOFTWARE_GPU`); when it
+  // is off we skip the GPU probe entirely to avoid the extra Chrome launch.
+  // When on, the probe Promise is cached for the process lifetime, so
+  // resolving here and threading the result to acquireBrowser collapses to
+  // a single probe.
+  const guardEnabled = isSoftwareScreenshotGuardEnabled();
   const requestedGpuMode = config?.browserGpuMode ?? DEFAULT_CONFIG.browserGpuMode;
-  const resolvedGpuMode = await resolveBrowserGpuMode(requestedGpuMode, {
-    chromePath: headlessShell ?? undefined,
-    browserTimeout: config?.browserTimeout,
-  });
-  // Mirror the software-renderer guard in `acquireBrowser`: on a software
-  // host the BeginFrame compositor stalls, so build the chromeArgs without
-  // BeginFrame flags up-front. acquireBrowser will also strip them as a
-  // belt-and-braces measure, but doing it here keeps the launch args clean.
+  const resolvedGpuMode = guardEnabled
+    ? await resolveBrowserGpuMode(requestedGpuMode, {
+        chromePath: headlessShell ?? undefined,
+        browserTimeout: config?.browserTimeout,
+      })
+    : undefined;
+  const isSoftwareRenderer = guardEnabled && resolvedGpuMode === "software";
   const preMode: CaptureMode =
-    headlessShell && isLinux && !forceScreenshot && !supersampling && resolvedGpuMode !== "software"
+    headlessShell && isLinux && !forceScreenshot && !supersampling && !isSoftwareRenderer
       ? "beginframe"
       : "screenshot";
   const chromeArgs = buildChromeArgs(
     { width: options.width, height: options.height, captureMode: preMode },
-    { ...config, browserGpuMode: resolvedGpuMode },
+    resolvedGpuMode ? { ...config, browserGpuMode: resolvedGpuMode } : config,
   );
 
-  // Thread the already-resolved GPU mode into acquireBrowser so it doesn't
-  // re-resolve from raw config. Promise-cached anyway, but removes the smell
-  // of two parallel resolutions that future refactors could let diverge.
+  // Thread the already-resolved GPU mode into acquireBrowser when available
+  // so it doesn't re-resolve from raw config. (Both sides agree on whether
+  // the guard is enabled because they read the same env var; if the env var
+  // flips between these two calls — basically only possible in tests — the
+  // guard semantics fall back to acquireBrowser's own resolution.)
   const { browser, captureMode } = await acquireBrowser(chromeArgs, config, {
     resolvedBrowserGpuMode: resolvedGpuMode,
   });