Skip to content

Commit 4573e4e

Browse files
jrusso1020claude
andcommitted
feat(engine): closed-GOP VP9 encoder args + concat-copy smoke test
PR 8.1 of the WebM distributed-rendering plan (v1.5 backlog #1; see DISTRIBUTED-RENDERING-PLAN.md §7.2). Gates whether PR 8.2 ships Path A (concat-copy) or Path B (re-encode-in-assemble) for webm distributed mode. Two changes: 1. Extend buildEncoderArgs to lay closed-GOP args on libvpx-vp9 when lockGopForChunkConcat=true: -g <chunkSize>, -keyint_min <chunkSize>, -auto-alt-ref 0, -cpu-used 2. Mirrors the existing libx264/libx265 branches. Alt-ref disabling is the critical bit — libvpx-vp9's default non-displayable alt-ref frames can reach across chunk seams and break concat-copy. cpu-used=2 pins the speed/quality tradeoff so chunks encoded on workers with different libvpx-vp9 defaults produce visually consistent output across seams. 2. Smoke test at packages/producer/tests/distributed/_smoke/webm-concat-copy.test.ts. Generates 60 PNGs via lavfi testsrc2, encodes them as 4 VP9 chunks of 15 frames using buildEncoderArgs with lockGopForChunkConcat=true, concat-copies via ffmpeg -f concat -c copy, then runs three independent verifications: ffprobe -show_streams, ffmpeg -f null - decode test, and ffprobe -count_frames. Each verification surfaces its failure fingerprint in the error message so PR 8.2 has the data it needs to pick the right architectural path. Smoke test result locally: ALL 6 tests pass. Path A (concat-copy) works for libvpx-vp9 with the new closed-GOP args. PR 8.2 will ship the plan-time-error-removal path. Also exports buildEncoderArgs from @hyperframes/engine so adapters / tests can construct args without re-implementing the contract. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 3eec777 commit 4573e4e

5 files changed

Lines changed: 446 additions & 5 deletions

File tree

packages/engine/src/index.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,7 @@ export {
9393

9494
// ── Encoding ───────────────────────────────────────────────────────────────────
9595
export {
96+
buildEncoderArgs,
9697
encodeFramesFromDir,
9798
encodeFramesChunkedConcat,
9899
muxVideoWithAudio,

packages/engine/src/services/chunkEncoder.test.ts

Lines changed: 79 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -551,7 +551,11 @@ describe("buildEncoderArgs lockGopForChunkConcat", () => {
551551
expect(args.indexOf("-x264-params")).toBe(-1);
552552
});
553553

554-
it("true is a no-op on VP9", () => {
554+
// Closed-GOP for libvpx-vp9 is required to make `ffmpeg -f concat -c copy`
555+
// stitch VP9 chunks losslessly: every chunk's first frame must be an
556+
// independently-decodable keyframe with no alt-ref references reaching
557+
// back across the seam.
558+
it("true appends closed-GOP args for libvpx-vp9", () => {
555559
const args = buildEncoderArgs(
556560
{
557561
...baseOptions,
@@ -564,9 +568,82 @@ describe("buildEncoderArgs lockGopForChunkConcat", () => {
564568
inputArgs,
565569
"out.webm",
566570
);
571+
expect(args[args.indexOf("-g") + 1]).toBe("240");
572+
expect(args[args.indexOf("-keyint_min") + 1]).toBe("240");
573+
// Alt-ref frames are non-displayable references that break concat-copy
574+
// at chunk seams; closed-GOP must disable them.
575+
expect(args[args.indexOf("-auto-alt-ref") + 1]).toBe("0");
576+
// cpu-used is locked so workers with different libvpx-vp9 defaults
577+
// produce visually consistent output across chunk boundaries.
578+
expect(args[args.indexOf("-cpu-used") + 1]).toBe("2");
579+
// libvpx-vp9 uses `-deadline good` for non-ultrafast presets — the
580+
// closed-GOP path doesn't change that.
581+
expect(args[args.indexOf("-deadline") + 1]).toBe("good");
582+
// x264/x265-only params must not leak into the VP9 branch.
583+
expect(args.indexOf("-x264-params")).toBe(-1);
584+
expect(args.indexOf("-x265-params")).toBe(-1);
585+
expect(args.indexOf("-sc_threshold")).toBe(-1);
586+
expect(args.indexOf("-force_key_frames")).toBe(-1);
587+
});
588+
589+
it("default (false) omits closed-GOP args for libvpx-vp9", () => {
590+
const args = buildEncoderArgs(
591+
{ ...baseOptions, codec: "vp9", preset: "good", quality: 23 },
592+
inputArgs,
593+
"out.webm",
594+
);
567595
expect(args).not.toContain("-g");
568596
expect(args).not.toContain("-keyint_min");
569-
expect(args).not.toContain("-force_key_frames");
597+
expect(args).not.toContain("-cpu-used");
598+
// The non-locked, non-alpha VP9 path leaves `-auto-alt-ref` at the
599+
// libvpx default. Alpha branches still emit `-auto-alt-ref 0` for an
600+
// unrelated reason (alpha + alt-ref is unsupported), but that's a
601+
// separate test below.
602+
expect(args).not.toContain("-auto-alt-ref");
603+
});
604+
605+
it("true with alpha pixel format keeps alpha metadata and emits -auto-alt-ref once", () => {
606+
// Regression: alpha + closed-GOP must NOT double-push `-auto-alt-ref 0`.
607+
// Both paths want it disabled; the encoder branch emits it exactly once.
608+
const args = buildEncoderArgs(
609+
{
610+
...baseOptions,
611+
codec: "vp9",
612+
preset: "good",
613+
quality: 23,
614+
pixelFormat: "yuva420p",
615+
lockGopForChunkConcat: true,
616+
gopSize: 240,
617+
},
618+
inputArgs,
619+
"out.webm",
620+
);
621+
const autoAltRefIndices = args.reduce<number[]>((acc, a, i) => {
622+
if (a === "-auto-alt-ref") acc.push(i);
623+
return acc;
624+
}, []);
625+
expect(autoAltRefIndices.length).toBe(1);
626+
expect(args[autoAltRefIndices[0] + 1]).toBe("0");
627+
expect(args[args.indexOf("-metadata:s:v:0") + 1]).toBe("alpha_mode=1");
628+
expect(args[args.indexOf("-g") + 1]).toBe("240");
629+
});
630+
631+
it("vp9 + lockGopForChunkConcat=true throws on missing gopSize", () => {
632+
// Mirrors the libx264/libx265 branch: closed-GOP without a GOP size
633+
// makes no sense — surface the caller error eagerly.
634+
expect(() =>
635+
buildEncoderArgs(
636+
{
637+
...baseOptions,
638+
codec: "vp9",
639+
preset: "good",
640+
quality: 23,
641+
lockGopForChunkConcat: true,
642+
},
643+
inputArgs,
644+
"out.webm",
645+
),
646+
).toThrow(/lockGopForChunkConcat=true requires a positive integer gopSize/);
570647
});
571648

572649
it("true is a no-op on ProRes (intra-only — no GOP forcing needed)", () => {

packages/engine/src/services/chunkEncoder.ts

Lines changed: 51 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -254,8 +254,58 @@ export function buildEncoderArgs(
254254
args.push("-c:v", "libvpx-vp9", "-b:v", bitrate || "0", "-crf", String(quality));
255255
args.push("-deadline", preset === "ultrafast" ? "realtime" : "good");
256256
args.push("-row-mt", "1");
257+
258+
// Closed-GOP args for distributed chunk concat-copy. Mirrors the
259+
// libx264/libx265 branch above: `lockGopForChunkConcat=true` lays a
260+
// keyframe at every chunk boundary so `ffmpeg -f concat -c copy` can
261+
// stitch sibling chunks losslessly.
262+
//
263+
// VP9-specific: `-auto-alt-ref 0` is mandatory. Alt-ref (a.k.a.
264+
// "ARNR") frames are non-displayable references libvpx-vp9 inserts
265+
// anywhere in the GOP for compression; they break concat-copy at
266+
// chunk seams because the boundary frame is no longer the first
267+
// displayable reference. The alpha branch below already disables
268+
// alt-ref for an unrelated reason (alpha + alt-ref is unsupported);
269+
// closed-GOP extends that to every pixel format.
270+
//
271+
// `-cpu-used 2` pins the libvpx-vp9 speed/quality tradeoff so chunks
272+
// encoded on workers with different default cpu-used values still
273+
// produce visually consistent output across seams. libvpx-vp9's
274+
// default with `-deadline good` has drifted across versions
275+
// historically — locking it makes the planHash round-trip
276+
// deterministic.
277+
const lockGopVp9 = options.lockGopForChunkConcat === true;
278+
if (lockGopVp9) {
279+
if (
280+
typeof options.gopSize !== "number" ||
281+
!Number.isFinite(options.gopSize) ||
282+
options.gopSize <= 0
283+
) {
284+
throw new Error(
285+
`[chunkEncoder] lockGopForChunkConcat=true requires a positive integer gopSize (received ${String(options.gopSize)})`,
286+
);
287+
}
288+
const gop = Math.floor(options.gopSize);
289+
args.push(
290+
"-g",
291+
String(gop),
292+
"-keyint_min",
293+
String(gop),
294+
"-auto-alt-ref",
295+
"0",
296+
"-cpu-used",
297+
"2",
298+
);
299+
}
257300
if (pixelFormat === "yuva420p") {
258-
args.push("-auto-alt-ref", "0");
301+
// Alpha + alt-ref is unsupported by libvpx-vp9. The closed-GOP
302+
// branch above already disables alt-ref; only push the flag for
303+
// the non-locked alpha case to keep the args list clean (a second
304+
// `-auto-alt-ref 0` is harmless but noisier in `ffmpeg -loglevel`
305+
// diagnostics).
306+
if (!lockGopVp9) {
307+
args.push("-auto-alt-ref", "0");
308+
}
259309
args.push("-metadata:s:v:0", "alpha_mode=1");
260310
}
261311
} else if (codec === "prores") {

packages/engine/src/services/chunkEncoder.types.ts

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,15 @@ export interface EncoderOptions {
2222
* (open-GOP, scenecut-driven keyframes), preserving the in-process
2323
* renderer's byte-identical output.
2424
*
25-
* Only honored by the SW libx264 / libx265 paths. GPU encoders, vp9, and
26-
* prores ignore the flag (their concat-copy story is separate).
25+
* Honored by the SW libx264 / libx265 / libvpx-vp9 paths. GPU encoders
26+
* and ProRes ignore the flag — GPU concat-copy is a separate story and
27+
* ProRes is intra-only (every frame is already a keyframe, so no
28+
* closed-GOP forcing is needed).
29+
*
30+
* For libvpx-vp9, closed-GOP also forces `-auto-alt-ref 0` so the
31+
* boundary frame between chunks remains independently decodable —
32+
* libvpx-vp9's default alt-ref frames can land anywhere in the GOP
33+
* for compression and break concat-copy seams.
2734
*/
2835
lockGopForChunkConcat?: boolean;
2936
/**

0 commit comments

Comments
 (0)