Skip to content

Commit b2bf830

Browse files
committed
feat(lambda): add Lambda handler, ZIP bundling, and BeginFrame probe
Phase 6 of the distributed rendering plan: AWS Lambda turnkey adoption (see DISTRIBUTED-RENDERING-PLAN.md §11 Phase 6 + §15). This PR adds the new packages/aws-lambda/ workspace package that wraps the OSS plan/renderChunk/assemble primitives in an AWS Lambda handler, plus a build pipeline that bundles the handler + Chromium runtime + ffmpeg into a deployable ZIP. Architecture: ZIP deploy (not Docker image), Chrome via @sparticuz/chromium with chrome-headless-shell fallback, dispatch on event.Action ∈ {plan, renderChunk, assemble}. The load-bearing concern — does @sparticuz/chromium's chrome-headless-shell build honour CDP HeadlessExperimental.beginFrame? — is pinned by the new scripts/probe-beginframe.ts regression guard. Probe boots the runtime inside public.ecr.aws/lambda/nodejs:22, navigates to a static page, and asserts beginFrame returns a PNG buffer. Verified locally + inside the Docker container; both pass with hasDamage=true. Sizes (sparticuz source): unzipped 157 MiB, zipped 99 MiB. Well under the 240 MiB / 150 MiB in-house gates and the Lambda 250 MiB hard ceiling. This is part of a stack of 8 PRs (3 in Phase 6a, 5 in Phase 6b); this is PR 6.1.
1 parent f84cc49 commit b2bf830

21 files changed

Lines changed: 2816 additions & 26 deletions

.github/workflows/windows-render.yml

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -353,6 +353,11 @@ jobs:
353353
# invocations, CRLF, file URLs, etc.) in existing vitest suites.
354354
# The producer package is skipped because its tests require Docker /
355355
# Linux-only tooling (Dockerfile.test, LFS golden MP4 baselines).
356+
# The aws-lambda package is skipped because it targets the AWS Lambda
357+
# Linux runtime exclusively (`@sparticuz/chromium` is Linux-only; the
358+
# ZIP layout is for `/var/task` on AL2023). Its handler.test.ts also
359+
# trips a bun-on-Windows workspace-symlink quirk reading the
360+
# producer's transitive `hono` dep.
356361
# -------------------------------------------------------------------
357362
test-windows:
358363
name: Tests on windows-latest
@@ -407,9 +412,9 @@ jobs:
407412
shell: pwsh
408413
run: bun run build
409414

410-
- name: Run tests (all packages except producer)
415+
- name: Run tests (all packages except producer + aws-lambda)
411416
shell: pwsh
412-
run: bun run --filter "!@hyperframes/producer" test
417+
run: bun run --filter "!@hyperframes/producer" --filter "!@hyperframes/aws-lambda" test
413418

414419
- name: Run runtime contract test
415420
shell: pwsh

Dockerfile.test

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,7 @@ COPY packages/producer/package.json packages/producer/package.json
7373
COPY packages/cli/package.json packages/cli/package.json
7474
COPY packages/studio/package.json packages/studio/package.json
7575
COPY packages/shader-transitions/package.json packages/shader-transitions/package.json
76+
COPY packages/aws-lambda/package.json packages/aws-lambda/package.json
7677
RUN bun install --frozen-lockfile
7778

7879
# Copy source

bun.lock

Lines changed: 196 additions & 11 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

packages/aws-lambda/README.md

Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,119 @@
1+
# @hyperframes/aws-lambda
2+
3+
AWS Lambda adapter for HyperFrames distributed rendering. Wraps the OSS
4+
`plan` / `renderChunk` / `assemble` primitives into a single Lambda handler
5+
that Step Functions can dispatch on, plus a build pipeline that bundles
6+
the handler + Chrome runtime + ffmpeg into a deployable ZIP.
7+
8+
This is part of [Phase 6 of the distributed rendering
9+
plan](../../DISTRIBUTED-RENDERING-PLAN.md#15-aws-lambda-turnkey-deployment).
10+
Phase 6a (this PR) validates the architecture on real AWS; Phase 6b ships
11+
the user-facing CLI and CDK construct.
12+
13+
## Status
14+
15+
- **6.1 (current)** Lambda handler + ZIP bundling + Chromium runtime probe.
16+
- 6.2 SAM template (`examples/aws-lambda/`).
17+
- 6.3 Real-AWS benchmark workflow.
18+
19+
## Architecture
20+
21+
```
22+
┌──────────────────────────────────────────────────────────────────┐
23+
│ Step Functions state machine │
24+
│ Plan → Map(N) RenderChunk → Assemble │
25+
└──────────────────────────────────────────────────────────────────┘
26+
│ dispatches by event.Action
27+
28+
┌──────────────────────────────────────────────────────────────────┐
29+
│ One Lambda function (this package's `dist/handler.zip`) │
30+
│ handler.mjs │
31+
│ ├─ Action="plan" → @hyperframes/producer/distributed │
32+
│ ├─ Action="renderChunk" → @hyperframes/producer/distributed │
33+
│ └─ Action="assemble" → @hyperframes/producer/distributed │
34+
│ bin/ffmpeg — ffmpeg-static │
35+
│ node_modules/@sparticuz/chromium/ — Lambda-optimised Chromium │
36+
└──────────────────────────────────────────────────────────────────┘
37+
│ pure functions over local paths
38+
39+
┌──────────────────────────────────────────────────────────────────┐
40+
│ S3 bucket — plan tarball + per-chunk outputs + final mp4 │
41+
└──────────────────────────────────────────────────────────────────┘
42+
```
43+
44+
The handler downloads inputs from S3 into `/tmp`, calls the OSS primitive,
45+
uploads outputs back to S3, and returns a small JSON result that fits
46+
inside Step Functions' history budget (under 200 bytes per chunk).
47+
48+
## Chrome runtime
49+
50+
The package supports two Chromium sources:
51+
52+
| Source | Default | Size | When to pick it |
53+
| ------------------------------- | ------- | ----------------- | --------------------------------------------------------------------------------------------------------------------- |
54+
| `@sparticuz/chromium` | yes | ~70 MB compressed | Lambda. Decompresses into `/tmp` at runtime; the rest of the ecosystem already uses it for headless-Chrome-in-Lambda. |
55+
| Bundled `chrome-headless-shell` | no | ~140 MB | Fallback. Used if `@sparticuz/chromium` ever drops `HeadlessExperimental.beginFrame` support. |
56+
57+
Pick the source at build time:
58+
59+
```bash
60+
bun run --cwd packages/aws-lambda build:zip
61+
bun run --cwd packages/aws-lambda build:zip -- --source=chrome-headless-shell
62+
```
63+
64+
The handler reads `HYPERFRAMES_LAMBDA_CHROME_SOURCE` at boot. The build
65+
script sets that env var via Lambda function configuration in
66+
`examples/aws-lambda/template.yaml`.
67+
68+
## BeginFrame regression guard
69+
70+
HyperFrames' renderer drives Chrome via the CDP
71+
`HeadlessExperimental.beginFrame` command — same path the K8s deploy uses.
72+
The Lambda adapter assumes that `@sparticuz/chromium`'s
73+
chrome-headless-shell build honours BeginFrame. To prove it (and re-prove
74+
it on every release), the package ships a Docker probe:
75+
76+
```bash
77+
# Build the Lambda-like container and run the probe.
78+
bun run --cwd packages/aws-lambda probe:beginframe:docker
79+
```
80+
81+
The probe boots `@sparticuz/chromium` inside
82+
`public.ecr.aws/lambda/nodejs:22` and asserts CDP `beginFrame` with
83+
`screenshot: true` returns a PNG buffer. Exit code 0 = green; non-zero =
84+
fall back to bundling chrome-headless-shell directly via `--source=chrome-headless-shell`.
85+
86+
## Building the ZIP
87+
88+
```bash
89+
bun install # at the monorepo root
90+
bun run --cwd packages/aws-lambda build:zip # → packages/aws-lambda/dist/handler.zip
91+
bun run --cwd packages/aws-lambda verify:zip-size # CI gate
92+
```
93+
94+
The build script bundles `src/handler.ts` via esbuild, stages
95+
`@sparticuz/chromium` and `puppeteer-core` under `node_modules/`, copies
96+
ffmpeg-static into `bin/`, and zips the result. The unzipped layout is
97+
designed to extract cleanly into Lambda's `/var/task/`.
98+
99+
`verify:zip-size` enforces:
100+
101+
- Unzipped ≤ 248 MB (in-house budget; Lambda hard ceiling is 250 MB unzipped)
102+
- Zipped ≤ 150 MB (in-house budget; Lambda has no hard zipped cap for S3-deployed functions)
103+
104+
CI fails the PR if either is exceeded.
105+
106+
## Running tests
107+
108+
```bash
109+
bun run --cwd packages/aws-lambda test # unit tests (no Chrome)
110+
bun run --cwd packages/aws-lambda probe:beginframe # local probe (Linux only)
111+
```
112+
113+
## What's NOT in this PR
114+
115+
- `examples/aws-lambda/template.yaml` (SAM template — PR 6.2).
116+
- Real-AWS deploy workflow (PR 6.3).
117+
- `npx hyperframes lambda deploy` CLI (Phase 6b, PR 6.5).
118+
- CDK construct (Phase 6b, PR 6.4).
119+
- Migration guide (Phase 6b, PR 6.8).

packages/aws-lambda/package.json

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
{
2+
"name": "@hyperframes/aws-lambda",
3+
"version": "0.0.1",
4+
"description": "AWS Lambda adapter for HyperFrames distributed rendering — Plan/RenderChunk/Assemble handler + ZIP bundling.",
5+
"repository": {
6+
"type": "git",
7+
"url": "https://github.com/heygen-com/hyperframes",
8+
"directory": "packages/aws-lambda"
9+
},
10+
"files": [
11+
"src/",
12+
"scripts/",
13+
"README.md"
14+
],
15+
"type": "module",
16+
"main": "./src/index.ts",
17+
"types": "./src/index.ts",
18+
"exports": {
19+
".": "./src/index.ts",
20+
"./handler": "./src/handler.ts"
21+
},
22+
"publishConfig": {
23+
"access": "public",
24+
"registry": "https://registry.npmjs.org/"
25+
},
26+
"scripts": {
27+
"build": "tsc --noEmit",
28+
"build:zip": "tsx scripts/build-zip.ts",
29+
"probe:beginframe": "tsx scripts/probe-beginframe.ts",
30+
"probe:beginframe:docker": "docker build -f scripts/probe-beginframe.dockerfile -t hyperframes-lambda-probe:local ../.. && docker run --rm hyperframes-lambda-probe:local",
31+
"test": "bun test",
32+
"typecheck": "tsc --noEmit",
33+
"verify:zip-size": "tsx scripts/verify-zip-size.ts"
34+
},
35+
"dependencies": {
36+
"@aws-sdk/client-s3": "^3.700.0",
37+
"@hyperframes/producer": "workspace:^",
38+
"@sparticuz/chromium": "148.0.0",
39+
"ffmpeg-static": "^5.2.0",
40+
"ffprobe-static": "^3.1.0",
41+
"puppeteer-core": "^24.39.1",
42+
"tar": "^7.4.3"
43+
},
44+
"devDependencies": {
45+
"@types/aws-lambda": "^8.10.146",
46+
"@types/node": "^25.0.10",
47+
"@types/tar": "^6.1.13",
48+
"esbuild": "^0.25.12",
49+
"tsx": "^4.21.0",
50+
"typescript": "^5.7.2"
51+
},
52+
"engines": {
53+
"node": ">=22"
54+
}
55+
}
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
/**
2+
* Shared binary-unit byte formatter for the build/verify scripts.
3+
*
4+
* The Lambda ZIP-size budget is in mebibytes (Lambda's 250 MB / 248 MiB
5+
* gate is binary, not decimal), so logs and CI failure messages use
6+
* KiB / MiB / GiB. This is intentionally a different unit system from
7+
* `packages/cli/src/ui/format.ts`'s `formatBytes` (KB / MB, decimal) —
8+
* don't conflate them.
9+
*/
10+
export function formatBytes(bytes: number): string {
11+
if (bytes < 1024) return `${bytes} B`;
12+
if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)} KiB`;
13+
if (bytes < 1024 * 1024 * 1024) return `${(bytes / (1024 * 1024)).toFixed(1)} MiB`;
14+
return `${(bytes / (1024 * 1024 * 1024)).toFixed(2)} GiB`;
15+
}

0 commit comments

Comments
 (0)