Skip to content

Commit f89c17f

Browse files
liriansu-opusmiguel-heygen
authored andcommitted
fix(engine): harden ancestor-hidden video skip against mask + caller cache
Two follow-ups to the ancestor-visibility skip in `injectVideoFramesBatch` and `syncVideoFrameVisibility`. 1. **Mask defence.** Both ancestor-hidden branches previously wrote a plain `img.style.visibility = "hidden"`. `applyDomLayerMask` writes the stylesheet rule `#${showId} *{visibility:visible !important}`, and CSS cascade puts important stylesheet author above non-important inline author — so a sub-comp host landing in the active layer's `show` set would revive a stale `__render_frame__` and let it bleed onto the layer composite. Write the hide via `style.setProperty("visibility", "hidden", "important")` instead; important inline beats important stylesheet. 2. **Caller cache hygiene.** `createVideoFrameInjector` unconditionally wrote `lastInjectedFrameByVideo.set(id, frameIndex)` after calling `injectVideoFramesBatch`, even for videos the page silently skipped due to a hidden visual ancestor. On the next call at the same frameIndex — common with source-fps < output-fps, paused source frames, or non-frame-aligned host starts — the cache short-circuited the second inject and the host's first visible frame painted blank because the replacement `<img>` was never created. Make `injectVideoFramesBatch` return `string[]` (the subset of ids it actually painted) and have the caller cache only those. The cli-side `snapshot.ts` consumer is unaffected: its local `InjectFn` types the return as `Promise<void>`, which is structurally compatible with `Promise<string[]>` under TS void-return assignment rules. Tests: linkedom doesn't preserve `!important` in cssText, so the two new mask-defence cases spy on the live `<img>`'s `style.setProperty` and assert the 3-arg call shape. The cache-hygiene case stubs the page-side primitives via `vi.mock`, drives the hook twice at the same frameIndex with a stubbed "injected nothing" first response, and verifies the second call still issues an inject. A counter-test pins the happy-path cache hit so a future refactor can't trade the skip bug for a never-cache regression.
1 parent f3bb6dc commit f89c17f

4 files changed

Lines changed: 206 additions & 11 deletions

File tree

packages/engine/src/services/screenshotService.test.ts

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -453,4 +453,58 @@ describe("video-frame injection respects ancestor visibility", () => {
453453
// <img> to `visibility: visible` so it overrides the ancestor.
454454
expect(seededImg.style.visibility).toBe("visible");
455455
});
456+
457+
// Regression for the layered/HDR mask path: `applyDomLayerMask` writes an
458+
// `!important` stylesheet rule `#${showId} *{visibility:visible !important}`
459+
// which, if a sub-comp host id appears in the show set, would revive a
460+
// plain (non-important) inline `visibility: hidden` on a descendant
461+
// `__render_frame__` — the cascade rule is "important stylesheet author
462+
// beats non-important inline author". To stay safe regardless of which
463+
// layer ends up in `show`, the ancestor-hidden hide must be written with
464+
// `!important` so inline `!important` beats stylesheet `!important`.
465+
//
466+
// linkedom strips `!important` from `cssText`/`getPropertyPriority`, so we
467+
// pin the contract on the API call site instead: a `setProperty(name,
468+
// value, "important")` invocation on the live `<img>`'s style.
469+
it("injectVideoFramesBatch hides a stale <img> with !important so the layer mask cannot revive it", async () => {
470+
const { teardown, setup } = withGlobals(setupHostHiddenScenario({ visibility: "hidden" }));
471+
const seededImg = setup.document.createElement("img");
472+
seededImg.classList.add("__render_frame__");
473+
seededImg.style.visibility = "visible";
474+
setup.video.parentNode?.insertBefore(seededImg, setup.video.nextSibling);
475+
const setPropertySpy = vi.spyOn(seededImg.style, "setProperty");
476+
477+
try {
478+
await injectVideoFramesBatch(passthroughPage(), [
479+
{
480+
videoId: "pip",
481+
dataUri:
482+
"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkAAIAAAoAAv/lxKUAAAAASUVORK5CYII=",
483+
},
484+
]);
485+
} finally {
486+
teardown();
487+
}
488+
489+
expect(seededImg.style.visibility).toBe("hidden");
490+
expect(setPropertySpy).toHaveBeenCalledWith("visibility", "hidden", "important");
491+
});
492+
493+
it("syncVideoFrameVisibility hides an existing <img> with !important so the layer mask cannot revive it", async () => {
494+
const { teardown, setup } = withGlobals(setupHostHiddenScenario({ visibility: "hidden" }));
495+
const seededImg = setup.document.createElement("img");
496+
seededImg.classList.add("__render_frame__");
497+
seededImg.style.visibility = "visible";
498+
setup.video.parentNode?.insertBefore(seededImg, setup.video.nextSibling);
499+
const setPropertySpy = vi.spyOn(seededImg.style, "setProperty");
500+
501+
try {
502+
await syncVideoFrameVisibility(passthroughPage(), ["pip"]);
503+
} finally {
504+
teardown();
505+
}
506+
507+
expect(seededImg.style.visibility).toBe("hidden");
508+
expect(setPropertySpy).toHaveBeenCalledWith("visibility", "hidden", "important");
509+
});
456510
});

packages/engine/src/services/screenshotService.ts

Lines changed: 27 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -369,13 +369,22 @@ export async function removeDomLayerMask(page: Page, extraHideIds: string[]): Pr
369369
);
370370
}
371371

372+
/**
373+
* Returns the subset of `updates.videoId`s that were actually painted in
374+
* this call. Videos skipped because of a hidden visual ancestor are NOT
375+
* included — the caller relies on this to avoid recording a `lastInjected`
376+
* cache entry for a frame that never reached the page, which would otherwise
377+
* short-circuit the next inject at the same frameIndex and leave the host's
378+
* first visible frame blank.
379+
*/
372380
export async function injectVideoFramesBatch(
373381
page: Page,
374382
updates: Array<{ videoId: string; dataUri: string }>,
375-
): Promise<void> {
376-
if (updates.length === 0) return;
377-
await page.evaluate(
383+
): Promise<string[]> {
384+
if (updates.length === 0) return [];
385+
return await page.evaluate(
378386
async (items: Array<{ videoId: string; dataUri: string }>, visualProperties: string[]) => {
387+
const injectedIds: string[] = [];
379388
const pendingDecodes: Array<Promise<void>> = [];
380389
const replacementLayoutProperties = new Set([
381390
"width",
@@ -435,7 +444,13 @@ export async function injectVideoFramesBatch(
435444
// Don't paint a frame over a hidden host — if an existing replacement
436445
// <img> is still around from when the host was visible, hide it so it
437446
// doesn't bleed through a sibling host that *is* visible on this seek.
438-
if (hasImg && img) img.style.visibility = "hidden";
447+
//
448+
// Use `!important` so the inline hide survives `applyDomLayerMask`'s
449+
// stylesheet `#${showId} *{visibility:visible !important}` when the
450+
// sub-comp host happens to land in the active layer's `show` set —
451+
// important stylesheet beats non-important inline, but important
452+
// inline beats important stylesheet.
453+
if (hasImg && img) img.style.setProperty("visibility", "hidden", "important");
439454
continue;
440455
}
441456

@@ -522,10 +537,12 @@ export async function injectVideoFramesBatch(
522537
// GSAP-controlled value.
523538
video.style.setProperty("visibility", "hidden", "important");
524539
video.style.setProperty("pointer-events", "none", "important");
540+
injectedIds.push(item.videoId);
525541
}
526542
if (pendingDecodes.length > 0) {
527543
await Promise.all(pendingDecodes);
528544
}
545+
return injectedIds;
529546
},
530547
updates,
531548
[...MEDIA_VISUAL_STYLE_PROPERTIES],
@@ -577,11 +594,16 @@ export async function syncVideoFrameVisibility(
577594
} else {
578595
// Inactive (or ancestor-hidden) video: hide both. Use visibility only
579596
// (never opacity) so we never clobber GSAP-controlled inline opacity.
597+
// Use `!important` on the <img> hide so `applyDomLayerMask`'s
598+
// important stylesheet rule (`#${showId} *{visibility:visible !important}`)
599+
// cannot revive a stale frame when the sub-comp host lands in the
600+
// active layer's `show` set — same mask-defense reasoning as the
601+
// `isVisualAncestorHidden` branch in `injectVideoFramesBatch`.
580602
video.style.removeProperty("display");
581603
video.style.setProperty("visibility", "hidden", "important");
582604
video.style.setProperty("pointer-events", "none", "important");
583605
if (hasImg) {
584-
img.style.visibility = "hidden";
606+
img.style.setProperty("visibility", "hidden", "important");
585607
}
586608
}
587609
}

packages/engine/src/services/videoFrameInjector.test.ts

Lines changed: 111 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,30 @@
11
// @vitest-environment node
2-
import { describe, it, expect, beforeEach, afterEach } from "vitest";
2+
import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
33
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
44
import { tmpdir } from "node:os";
55
import { join } from "node:path";
6-
import { __testing } from "./videoFrameInjector.js";
6+
import { type Page } from "puppeteer-core";
7+
8+
// Hoist mocks before importing the module under test so the mock factory wins.
9+
// The cache-hygiene block exercises createVideoFrameInjector against stubbed
10+
// page-side primitives so we can assert on Node-side state (cache poisoning)
11+
// without standing up a real browser.
12+
const { injectVideoFramesBatchMock, syncVideoFrameVisibilityMock } = vi.hoisted(() => ({
13+
injectVideoFramesBatchMock: vi.fn<
14+
(page: Page, updates: Array<{ videoId: string; dataUri: string }>) => Promise<string[]>
15+
>(async (_page, updates) => updates.map((u) => u.videoId)),
16+
syncVideoFrameVisibilityMock: vi.fn<(page: Page, ids: string[]) => Promise<void>>(
17+
async () => undefined,
18+
),
19+
}));
20+
21+
vi.mock("./screenshotService.js", () => ({
22+
injectVideoFramesBatch: injectVideoFramesBatchMock,
23+
syncVideoFrameVisibility: syncVideoFrameVisibilityMock,
24+
}));
25+
26+
import { __testing, createVideoFrameInjector } from "./videoFrameInjector.js";
27+
import { type FrameLookupTable } from "./videoFrameExtractor.js";
728
import { DEFAULT_CONFIG } from "../config.js";
829

930
const { createFrameSourceCache } = __testing;
@@ -143,3 +164,91 @@ describe("frame source cache eviction", () => {
143164
expect(cache.stats()).toMatchObject({ ...SHARED_STATS, entries: 0, bytes: 0 });
144165
});
145166
});
167+
168+
describe("createVideoFrameInjector cache hygiene against page-side skips", () => {
169+
// Build a minimal FrameLookupTable stand-in that returns one fixed payload
170+
// for every time so we can drive the hook deterministically. The real
171+
// table is exercised exhaustively in videoFrameExtractor.test.ts.
172+
function fakeTable(payload: { videoId: string; framePath: string; frameIndex: number }) {
173+
return {
174+
getActiveFramePayloads: () =>
175+
new Map([
176+
[payload.videoId, { framePath: payload.framePath, frameIndex: payload.frameIndex }],
177+
]),
178+
} as unknown as FrameLookupTable;
179+
}
180+
181+
// Bypass the on-disk frame cache by handing back a synthetic data URI.
182+
function inlineResolver(framePath: string): string {
183+
return `data:image/png;base64,fake-${framePath}`;
184+
}
185+
186+
beforeEach(() => {
187+
injectVideoFramesBatchMock.mockReset();
188+
syncVideoFrameVisibilityMock.mockReset();
189+
syncVideoFrameVisibilityMock.mockResolvedValue(undefined);
190+
});
191+
192+
it("does not poison the lastInjected cache when the page reports zero ids injected", async () => {
193+
// Regression for the agentic-finecut scenario after PR #1028's ancestor
194+
// skip: when injectVideoFramesBatch silently drops a video (its sub-comp
195+
// host is hidden), the caller used to record `lastInjectedFrame[v] = N`
196+
// anyway. On the next frame, if the source frameIndex is unchanged
197+
// (low-fps source, multiple output frames per source frame, or
198+
// non-frame-aligned host start), the cache short-circuits the second
199+
// call and the host's first visible frame paints blank because the
200+
// replacement <img> was never created.
201+
//
202+
// Pin the contract: when the page returns `[]` (no ids actually
203+
// injected), the cache must not record those frameIndexes, so a follow-
204+
// up call at the same frameIndex still issues an inject.
205+
const fakePage = {} as Page;
206+
const hook = createVideoFrameInjector(
207+
fakeTable({ videoId: "pip", framePath: "/p", frameIndex: 5 }),
208+
{
209+
frameSrcResolver: inlineResolver,
210+
},
211+
);
212+
expect(hook).not.toBeNull();
213+
214+
// First call: simulate the ancestor-hidden skip — page-side reports it
215+
// injected nothing.
216+
injectVideoFramesBatchMock.mockResolvedValueOnce([]);
217+
await hook!(fakePage, 0);
218+
expect(injectVideoFramesBatchMock).toHaveBeenCalledTimes(1);
219+
expect(injectVideoFramesBatchMock).toHaveBeenLastCalledWith(fakePage, [
220+
{ videoId: "pip", dataUri: "data:image/png;base64,fake-/p" },
221+
]);
222+
223+
// Second call: same frameIndex, but the previous call did not really
224+
// paint. The cache must NOT short-circuit; the inject must run again.
225+
injectVideoFramesBatchMock.mockResolvedValueOnce(["pip"]);
226+
await hook!(fakePage, 0);
227+
expect(injectVideoFramesBatchMock).toHaveBeenCalledTimes(2);
228+
expect(injectVideoFramesBatchMock).toHaveBeenLastCalledWith(fakePage, [
229+
{ videoId: "pip", dataUri: "data:image/png;base64,fake-/p" },
230+
]);
231+
});
232+
233+
it("does cache normally when the page reports the id as injected", async () => {
234+
// Counter-test: when injection succeeds for a videoId, the cache must
235+
// record it and a second call at the same frameIndex must short-circuit.
236+
// This pins the happy path so a future refactor can't trade the skip
237+
// bug for a never-cache regression.
238+
const fakePage = {} as Page;
239+
const hook = createVideoFrameInjector(
240+
fakeTable({ videoId: "pip", framePath: "/p", frameIndex: 5 }),
241+
{
242+
frameSrcResolver: inlineResolver,
243+
},
244+
);
245+
246+
injectVideoFramesBatchMock.mockResolvedValueOnce(["pip"]);
247+
await hook!(fakePage, 0);
248+
expect(injectVideoFramesBatchMock).toHaveBeenCalledTimes(1);
249+
250+
await hook!(fakePage, 0);
251+
// Cache hit — no second inject for the same frameIndex.
252+
expect(injectVideoFramesBatchMock).toHaveBeenCalledTimes(1);
253+
});
254+
});

packages/engine/src/services/videoFrameInjector.ts

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -197,12 +197,22 @@ export function createVideoFrameInjector(
197197

198198
await syncVideoFrameVisibility(page, Array.from(activeIds));
199199
if (updates.length > 0) {
200-
await injectVideoFramesBatch(
201-
page,
202-
updates.map((u) => ({ videoId: u.videoId, dataUri: u.dataUri })),
200+
// Only record cache entries for videos the page actually painted.
201+
// injectVideoFramesBatch skips any video whose visual ancestor is
202+
// hidden (sub-comp host out-of-window) and returns the subset of ids
203+
// it really wrote — recording the rest would short-circuit the next
204+
// call at the same frameIndex and leave the host's first visible
205+
// frame blank.
206+
const injectedIds = new Set(
207+
await injectVideoFramesBatch(
208+
page,
209+
updates.map((u) => ({ videoId: u.videoId, dataUri: u.dataUri })),
210+
),
203211
);
204212
for (const update of updates) {
205-
lastInjectedFrameByVideo.set(update.videoId, update.frameIndex);
213+
if (injectedIds.has(update.videoId)) {
214+
lastInjectedFrameByVideo.set(update.videoId, update.frameIndex);
215+
}
206216
}
207217
}
208218
};

0 commit comments

Comments
 (0)