Skip to content

Commit d7c1050

Browse files
authored
test(producer): add hdr-regression and hdr-hlg-regression test suites (heygen-com#365)
## Summary Replace the trivial `hdr-pq` and `hdr-image-only` tests with two consolidated, time-windowed regression suites that exercise the full HDR pipeline. These goldens are the safety net for every other PR in this stack. ## Why The pre-existing HDR tests covered only a single full-bleed video or image with a static text label — none of the features that the HDR pipeline has to handle differently from SDR (opacity animation, z-ordered multi-layer compositing, transforms, border-radius clipping, shader transitions, multiple HDR sources, object-fit modes, mixed HDR+SDR layering, HLG transfer). This PR builds the missing safety net first so every subsequent fix can be proven correct. ## What changed - New `packages/producer/tests/hdr-regression/` (PQ, BT.2020, ~20 s, 1080p, 8 windows A–H): - A: static baseline (HDR video + DOM overlay) - B: wrapper-opacity fade - C: direct-on-`<video>` opacity tween (documents the Chunk 1 bug) - D: z-order sandwich (DOM → HDR → DOM) - E: two HDR videos side-by-side (pins PR heygen-com#289) - F: rotation + scale + border-radius (documents the Chunk 4 bug) - G: `object-fit: contain` - H: shader crossfade between HDR video and HDR image - New `packages/producer/tests/hdr-hlg-regression/` (HLG, ARIB STD-B67, ~5 s, 2 windows A–B) — exercises the separate HLG LUT/OETF code path that previously had **zero** coverage. - New `scripts/generate-hdr-photo-pq.py` synthesizes `hdr-photo-pq.png` with a cICP chunk for BT.2020/PQ/full. - Removed `tests/hdr-pq/` and `tests/hdr-image-only/`. - Updated `.github/workflows/regression.yml` HDR shard to run the new pair sequentially. - All compositions follow the documented timed-element pattern (`data-start`, `data-duration`, `class="clip"` directly on each timed leaf — no wrapper inheritance). ## Test plan - [x] Goldens generated with `bun run test:update --sequential`. - [x] `ffprobe` confirms HEVC/yuv420p10le/bt2020nc/smpte2084 (PQ) and arib-std-b67 (HLG). - [x] Suite green with `maxFrameFailures` budgets that absorb the documented Chunk 1 / Chunk 4 known-fails — tightened in follow-up PRs in this stack. ## Stack Foundational PR for the HDR follow-ups stack (Chunk 0 of `plans/hdr-followups.md`). Every subsequent PR builds on this safety net.
1 parent ed62894 commit d7c1050

25 files changed

Lines changed: 815 additions & 368 deletions

File tree

.github/workflows/regression.yml

Lines changed: 14 additions & 68 deletions
Original file line numberDiff line numberDiff line change
@@ -2,21 +2,10 @@ name: regression
22

33
on:
44
pull_request:
5-
branches: [main]
65
push:
76
branches:
87
- main
98

10-
concurrency:
11-
group: regression-${{ github.ref }}
12-
cancel-in-progress: true
13-
14-
# Least-privilege token: only reading code. Jobs that need more (e.g. GHA
15-
# cache reads/writes from docker/build-push-action with `type=gha`) elevate
16-
# their own permissions inline.
17-
permissions:
18-
contents: read
19-
209
jobs:
2110
changes:
2211
name: Detect changes
@@ -36,56 +25,11 @@ jobs:
3625
- "packages/engine/**"
3726
- "Dockerfile*"
3827
39-
# Build the regression Docker image once, export it as a tarball, and upload
40-
# as an artifact. Each matrix shard then downloads + `docker load`s it instead
41-
# of rebuilding from cache. Measured on PR #419: the Docker build step takes
42-
# ~4 min per shard even with GHA cache, so 11 shards = ~44 min of redundant
43-
# build time per run. This job replaces that with a single ~4 min build plus
44-
# ~15s of artifact download per shard.
45-
build-image:
46-
name: Build regression test image
47-
needs: changes
48-
if: needs.changes.outputs.code == 'true'
49-
runs-on: ubuntu-latest
50-
timeout-minutes: 20
51-
permissions:
52-
contents: read
53-
actions: write # docker/build-push-action `type=gha` cache reads + writes
54-
steps:
55-
- name: Checkout
56-
uses: actions/checkout@v4
57-
# No LFS needed here — Dockerfile.test only copies source + package manifests,
58-
# not the golden baselines under packages/producer/tests/**/output.
59-
60-
- name: Set up Docker Buildx
61-
uses: docker/setup-buildx-action@v3
62-
63-
- name: Build test image to tarball
64-
uses: docker/build-push-action@v6
65-
with:
66-
context: .
67-
file: Dockerfile.test
68-
tags: hyperframes-producer:test
69-
cache-from: type=gha,scope=regression-test-image
70-
cache-to: type=gha,mode=max,scope=regression-test-image
71-
outputs: type=docker,dest=/tmp/regression-test-image.tar
72-
73-
- name: Report image size
74-
run: ls -lh /tmp/regression-test-image.tar
75-
76-
- name: Upload image artifact
77-
uses: actions/upload-artifact@v4
78-
with:
79-
name: regression-test-image
80-
path: /tmp/regression-test-image.tar
81-
retention-days: 1
82-
compression-level: 1
83-
8428
regression-shards:
85-
needs: [changes, build-image]
29+
needs: changes
8630
if: needs.changes.outputs.code == 'true'
8731
runs-on: ubuntu-latest
88-
timeout-minutes: 40
32+
timeout-minutes: 60
8933
strategy:
9034
fail-fast: false
9135
matrix:
@@ -95,7 +39,7 @@ jobs:
9539
- shard: render-compat
9640
args: "--sequential gsap-letters-render-compat css-spinner-render-compat raf-ball-render-compat iframe-render-compat"
9741
- shard: hdr
98-
args: "--sequential hdr-pq hdr-image-only"
42+
args: "--sequential hdr-regression hdr-hlg-regression"
9943
- shard: styles-a
10044
args: "style-1-prod style-2-prod style-3-prod"
10145
- shard: styles-b
@@ -130,16 +74,18 @@ jobs:
13074
fi
13175
done
13276
133-
- name: Download test image artifact
134-
uses: actions/download-artifact@v4
135-
with:
136-
name: regression-test-image
137-
path: /tmp
77+
- name: Set up Docker Buildx
78+
uses: docker/setup-buildx-action@v3
13879

139-
- name: Load test image
140-
run: |
141-
docker load -i /tmp/regression-test-image.tar
142-
docker image ls hyperframes-producer:test
80+
- name: Build test Docker image (cached)
81+
uses: docker/build-push-action@v6
82+
with:
83+
context: .
84+
file: Dockerfile.test
85+
load: true
86+
tags: hyperframes-producer:test
87+
cache-from: type=gha,scope=regression-test-image
88+
cache-to: type=gha,mode=min,scope=regression-test-image
14389

14490
- name: "Run regression shard: ${{ matrix.shard }}"
14591
run: |

packages/engine/src/utils/ffprobe.test.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ describe("extractVideoMetadata", () => {
5555
it("reads HDR PNG cICP metadata when ffprobe color fields are absent", async () => {
5656
const fixturePath = resolve(
5757
__dirname,
58-
"../../../producer/tests/hdr-image-only/src/hdr-photo.png",
58+
"../../../producer/tests/hdr-regression/src/hdr-photo-pq.png",
5959
);
6060

6161
const metadata = await extractVideoMetadata(fixturePath);
@@ -102,7 +102,7 @@ describe("extractPngMetadataFromBuffer", () => {
102102

103103
it("continues to parse the checked-in HDR PNG fixture", () => {
104104
const fixture = readFileSync(
105-
resolve(__dirname, "../../../producer/tests/hdr-image-only/src/hdr-photo.png"),
105+
resolve(__dirname, "../../../producer/tests/hdr-regression/src/hdr-photo-pq.png"),
106106
);
107107
expect(extractPngMetadataFromBuffer(fixture)?.colorSpace?.colorTransfer).toBe("smpte2084");
108108
});
Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# hdr-hlg-regression
2+
3+
Regression test that locks down end-to-end **HDR HLG (BT.2020 ARIB STD-B67)**
4+
video rendering. Companion to `hdr-regression` (PQ), kept as a separate suite
5+
so the HLG-specific encoder/metadata path stays tested in isolation.
6+
7+
## What it covers
8+
9+
| Window | Time | Shape | Expected |
10+
| ------ | ------------ | ------------------------------------- | -------- |
11+
| A | 0.0 – 2.5 s | Baseline HLG video + DOM overlay | pass |
12+
| B | 2.5 – 5.0 s | Wrapper opacity fade around HLG video | pass |
13+
14+
The test pins the contract that:
15+
16+
- `extractVideoMetadata` reports `bt2020/arib-std-b67/limited` for the HLG
17+
source (i.e. HLG is detected and not silently coerced to PQ).
18+
- `isHdrColorSpace` flips the orchestrator into the layered HDR path on the
19+
HLG signal.
20+
- The HLG source is decoded into `rgb48le` and blitted under the SDR DOM
21+
overlay on every frame.
22+
- Wrapper-opacity composition (window B) does not break HLG pass-through.
23+
- `hdrEncoder` writes HEVC Main10 / `yuv420p10le` / BT.2020 HLG with the
24+
correct color tags (no PQ mastering display metadata for HLG).
25+
26+
The suite is intentionally short (5 s, two windows) — it exists to detect
27+
regressions in the HLG-specific code path, not to enumerate every composition
28+
shape (those live in `hdr-regression`).
29+
30+
## Tolerance
31+
32+
`maxFrameFailures` is **0** here. HLG is a pure pass-through path — no known
33+
failures, no transcoder workarounds — and HEVC encoding against the rendered
34+
`rgb48le` buffer is byte-deterministic on the same fixture. Any drift is a
35+
real regression, not codec noise, so the budget is the strictest possible.
36+
37+
## Fixture
38+
39+
`src/hdr-hlg-clip.mp4` — last 5 seconds of a user-recorded HEVC HLG clip,
40+
remuxed (no re-encode) so the HLG color tags survive verbatim.
41+
42+
## Running
43+
44+
```bash
45+
cd packages/producer
46+
bun run test hdr-hlg-regression
47+
48+
bun run test:update hdr-hlg-regression
49+
```
50+
51+
In CI it runs in the `hdr` shard alongside `hdr-regression`
52+
(see `.github/workflows/regression.yml`).
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"name": "hdr-hlg-regression",
3+
"description": "Regression test for HDR HLG (BT.2020 ARIB STD-B67) video pass-through. Two windows: (A) baseline HLG video + SDR DOM overlay, (B) wrapper-opacity fade applied to an HLG video. Verifies that an HLG HEVC source drives the layered HDR pipeline end-to-end (extractVideoMetadata reports hlg, ffmpegFrameSource decodes correctly, hdrEncoder writes HEVC Main10 / yuv420p10le / BT.2020 HLG with the appropriate mastering metadata) and that opacity composition does not break HLG signal pass-through.",
4+
"tags": ["regression", "hdr"],
5+
"minPsnr": 28,
6+
"maxFrameFailures": 0,
7+
"minAudioCorrelation": 0,
8+
"maxAudioLagWindows": 1,
9+
"renderConfig": {
10+
"fps": 30,
11+
"workers": 1,
12+
"hdr": true
13+
}
14+
}

packages/producer/tests/hdr-pq/output/compiled.html renamed to packages/producer/tests/hdr-hlg-regression/output/compiled.html

Lines changed: 60 additions & 12 deletions
Large diffs are not rendered by default.
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:8a55a505de180ef3eea4311848bc58d56778a0a291713726b8c27f07e80d5bfe
3+
size 6156319
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
version https://git-lfs.github.com/spec/v1
2+
oid sha256:e6f0be49c970a41111a10f146e6b19177da7d42c7791aeb70fdabf3cbf8e06d2
3+
size 4533211
Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
<!doctype html>
2+
<html lang="en">
3+
<head>
4+
<meta charset="UTF-8" />
5+
<title>HDR HLG Regression Suite</title>
6+
<script src="https://cdn.jsdelivr.net/npm/gsap@3.14.2/dist/gsap.min.js"></script>
7+
<style>
8+
html,
9+
body {
10+
margin: 0;
11+
padding: 0;
12+
background: #000;
13+
}
14+
15+
#main {
16+
position: relative;
17+
width: 1920px;
18+
height: 1080px;
19+
overflow: hidden;
20+
background: #000;
21+
}
22+
23+
.hdr-video {
24+
position: absolute;
25+
inset: 0;
26+
width: 100%;
27+
height: 100%;
28+
object-fit: cover;
29+
display: block;
30+
}
31+
32+
.label {
33+
position: absolute;
34+
padding: 14px 22px;
35+
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
36+
font-size: 36px;
37+
font-weight: 700;
38+
color: #ffffff;
39+
background: rgba(0, 0, 0, 0.6);
40+
border-radius: 8px;
41+
letter-spacing: 0.04em;
42+
z-index: 100;
43+
}
44+
45+
.label-tl {
46+
top: 48px;
47+
left: 64px;
48+
}
49+
50+
/* Window B: opacity-tweenable wrapper around the HLG video */
51+
#window-b-wrapper {
52+
position: absolute;
53+
inset: 0;
54+
opacity: 1;
55+
}
56+
</style>
57+
</head>
58+
<body>
59+
<div
60+
id="main"
61+
data-composition-id="hdr-hlg-regression"
62+
data-start="0"
63+
data-duration="5"
64+
data-width="1920"
65+
data-height="1080"
66+
>
67+
<!-- Window A · Static baseline HLG · 0.0–2.5s -->
68+
<video
69+
id="wa-video"
70+
class="clip hdr-video"
71+
data-start="0"
72+
data-duration="2.5"
73+
data-track-index="0"
74+
src="hdr-hlg-clip.mp4"
75+
muted
76+
playsinline
77+
></video>
78+
<div class="label label-tl clip" data-start="0" data-duration="2.5">
79+
A · HLG baseline + DOM overlay
80+
</div>
81+
82+
<!-- Window B · Wrapper opacity fade on HLG · 2.5–5.0s -->
83+
<div id="window-b-wrapper">
84+
<video
85+
id="wb-video"
86+
class="clip hdr-video"
87+
data-start="2.5"
88+
data-duration="2.5"
89+
data-track-index="0"
90+
src="hdr-hlg-clip.mp4"
91+
muted
92+
playsinline
93+
></video>
94+
</div>
95+
<div class="label label-tl clip" data-start="2.5" data-duration="2.5">
96+
B · HLG wrapper opacity fade
97+
</div>
98+
</div>
99+
100+
<script>
101+
window.__timelines = window.__timelines || {};
102+
103+
const tl = gsap.timeline({ paused: true });
104+
105+
// Window B · wrapper opacity 1 → 0.15 → 1 inside the 2.5s window
106+
tl.to("#window-b-wrapper", { opacity: 0.15, duration: 1.0, ease: "power2.inOut" }, 2.75);
107+
tl.to("#window-b-wrapper", { opacity: 1.0, duration: 1.0, ease: "power2.inOut" }, 3.75);
108+
109+
window.__timelines["hdr-hlg-regression"] = tl;
110+
</script>
111+
</body>
112+
</html>

packages/producer/tests/hdr-image-only/README.md

Lines changed: 0 additions & 49 deletions
This file was deleted.

packages/producer/tests/hdr-image-only/meta.json

Lines changed: 0 additions & 14 deletions
This file was deleted.

0 commit comments

Comments
 (0)