docs(lambda): document webm support + simplify-review fixes#953
docs(lambda): document webm support + simplify-review fixes#953jrusso1020 wants to merge 7 commits into
Conversation
miguel-heygen
left a comment
There was a problem hiding this comment.
APPROVE — clean stack-finisher with real bug fixes.
Verified:
- resolveEncoderTriple preset fix: previously hardcoded
preset: "good"for webm, silently overridingquality: "draft"mapping. Now delegates togetEncoderPresetso draft actually gets-deadline realtime. Genuine correctness fix. - Lambda font cache fix:
$HOME/.cache/hyperframes/fontsis read-only on Lambda (/home/sbx_*). Routes totmpdir()when$AWS_LAMBDA_FUNCTION_NAMEis set. Supports$HYPERFRAMES_FONT_CACHE_DIRoverride. Not webm-specific — affects all distributed renders with Google Fonts. DistributedFormatconsolidation: single source of truth inshared.ts, replaces 12+ inline union literals.satisfies readonly DistributedFormat[]ensures compile-time alignment.- CLI format allowlist:
lambda rendernow accepts webm (was hardcoded to mp4/mov/png-sequence). - Docs: format table, migration checklist, perf tradeoffs (10-25% larger) all accurate.
- Test cleanup: 4-block yuva420p smoke consolidated to single
it()with try/finally. PR-track narrative removed from source. - Net -48 lines (235 added, 283 removed) — genuine simplification.
All CI green.
vanceingalls
left a comment
There was a problem hiding this comment.
Docs + cleanup PR closes out the webm-distributed series cleanly. The DistributedFormat type centralization, the resolveEncoderTriple correctness fix, and the Lambda font-cache routing are all good. Doc claims cross-reference cleanly to the code.
Strengths
- Single source of truth for the format union —
packages/producer/src/services/distributed/shared.ts:21definesDistributedFormatand the lambda + CLI + harness + planHash + plan/renderChunk/assemble all derive from it. Theas const satisfies readonly DistributedFormat[]pattern atvalidateConfig.ts:35andlambda.ts:328is exactly right — the runtime allowlist is provably aligned with the compile-time union, so a future format add starts at one type definition and the TS compiler enumerates every callsite that needs updating. resolveEncoderTriplebug fix is a real correctness improvement. Before, the webm branch returnedpreset: "good"unconditionally; that silently overrodegetEncoderPreset's draft-tier mapping to-deadline realtime. Nowplan.ts:559-568defers to the engine helper, andgetEncoderPreset("draft", "webm")atchunkEncoder.ts:50-56correctly returnsrealtime. The inline comment captures the "why hardcoding was wrong" in one paragraph.- Font-cache fix is environment-correct.
AWS_LAMBDA_FUNCTION_NAMEis set by the Lambda runtime before module init, so the module-levelGOOGLE_FONTS_CACHE_DIRconst atdeterministicFonts.ts:345resolves totmpdir()/hyperframes/fontson Lambda.HYPERFRAMES_FONT_CACHE_DIRas an explicit override is good defensive design for adopters running in other write-restricted environments. - Doc claims verified against code. VP9 closed-GOP args (
-g,-keyint_min,-auto-alt-ref 0,-cpu-used 2) matchchunkEncoder.ts:265-280. "Audio is muxed as Opus" matcheschunkEncoder.ts:541(libopus,128k). The cost-transparency paragraph (10-25% file-size penalty +-cpu-used 2slowdown vs. in-process default) is exactly the kind of trade-off info adopters need to know before migrating short webm renders — surfacing the cost up front beats a wiki page nobody reads.
Findings
nit — webm-with-audio coverage gap. The doc states "Audio is muxed as Opus" (migrating-to-hyperframes-lambda.mdx:71), and the engine path exists at chunkEncoder.ts:541. But the webm-vp9 fixture explicitly omits audio (tests/distributed/webm-vp9/src/index.html:93-95) and the trimmed comment now drops the "Other webm-with-audio fixtures cover the mux path separately when added" note. That removes the only in-tree breadcrumb that the audio path is uncovered by distributed-mode regression. Not a blocker — the engine-level mux is exercised by other tests — but a one-line // TODO: distributed webm-with-audio fixture in the fixture comment would let future contributors find the gap without grepping. (Or a follow-up ticket.)
nit — formatExtension.ts re-export shape. Lines 9-11 import then re-export the same type from the same package. export type { DistributedFormat } from "@hyperframes/producer/distributed" alone would do it; the local import type is dead unless something inside the file uses it. (Doesn't — formatExtension(format: DistributedFormat) resolves through the re-export.) Cosmetic.
nit — stacked-PR CI gap. baseRefName is test/producer-webm-vp9-distributed-fixture, so the main CI workflow's branches: [main] trigger filter means Test / Typecheck / Build / CLI smoke did NOT run on this head SHA. Author's PR body lists local bunx vitest + bun test + docker:test passes, which is fine, but they aren't CI-verified on this commit. The three workflows that DID run (regression, preview-regression, Player perf) all pass. Per the stack plan, this closes once #952 lands and #953 rebases onto main. Flagging per discipline; not a blocker since this PR's diff is dominated by type-alias substitution + docs + doc-comment trims, all of which are visually verifiable in review.
CI
Verbatim status: regression SUCCESS (8 shards + parent), preview-regression SUCCESS, Player perf SUCCESS, Preflight SUCCESS on regression+preview-regression workflows. Graphite / mergeability_check IN_PROGRESS (non-blocking). Mintlify Deployment SKIPPED. WIP SUCCESS. No failures. Missing from rollup (stacked-PR gap): Build, Typecheck, Test, CLI smoke.
Verdict
Verdict: APPROVE
Reasoning: Doc claims cross-reference cleanly to the code; the bundled bug fixes (encoder-preset hardcode, font-cache write path, CLI format allowlist) are real correctness improvements that the real-AWS verification caught. The type consolidation reduces format-add blast radius from ~10 callsites to 1. Findings are nits — none block merge.
— Vai
5726d1e to
df2a67e
Compare
9ded0e6 to
8b73ca9
Compare
|
Thanks for the careful read both of you.
|
8b73ca9 to
a338eba
Compare
vanceingalls
left a comment
There was a problem hiding this comment.
Re-review against head a338ebaa (prior approve was on 9ded0e69).
What changed since prior approve
One new commit on this PR's branch: a338ebaa docs(lambda): drop plan-doc reference from migration checklist. The PR's base (test/producer-webm-vp9-distributed-fixture) also advanced via a rebase pulling in main's recent merges (#944, #947, #956, #948, #949, #954) but those land through the parent stack, not this PR.
The only diff this PR introduces vs. prior approve:
-1. **Inventory** the compositions you want to migrate. ... webm renders distributed-mode under v1.5 via closed-GOP VP9.
+1. **Inventory** the compositions you want to migrate. ... webm renders distributed via closed-GOP VP9 + concat-copy (see the webm section above).Tightening the migration checklist sentence to point at the doc's own webm section rather than the version label. No code touched, no regression surface. Good edit.
Prior nits status
- webm-with-audio coverage gap — not addressed. Cosmetic; the engine-level mux path is still exercised by other tests. Not worth holding the PR.
formatExtension.tsre-export shape — not addressed. Theimport type { DistributedFormat }at line 9 is still dead (file's only consumer is the re-export on line 11). Cosmetic.- Stacked-PR CI gap — unchanged.
baseRefNameis stilltest/producer-webm-vp9-distributed-fixture, soBuild/Typecheck/Test/CLI smokefrom the mainCIworkflow still don't run on this head SHA (the workflow'sbranches: [main]filter excludes it). Will resolve once the parent PR (#952) lands and this rebases onto main.
None of these are blockers; all three were nits in the prior review and remain nits.
CI
At time of re-review: Preflight PASS, preview-regression PASS, player-perf PASS, Preview parity PASS, regression-shards (all 8 shards) PENDING (running on the new head). Graphite / mergeability_check pending (non-blocking). No failures. Same stacked-PR gap as before.
Verdict
Verdict: APPROVE
Reasoning: Single new commit is a doc-only tightening that addresses upstream review feedback cleanly. No new code surface, no regression risk. Prior nits remain cosmetic. Stacked-PR CI gap resolves on rebase to main.
Review by Vai (re-review)
d8190d7 to
94b2a6c
Compare
a338eba to
32c5a76
Compare
94b2a6c to
ced2454
Compare
32c5a76 to
f5fbae5
Compare
vanceingalls
left a comment
There was a problem hiding this comment.
Round 3 re-review.
Delta since round 2 approval at a338eba: the tip commit's patch on docs/deploy/migrating-to-hyperframes-lambda.mdx is byte-identical (verified via gh api .../commits/<sha> .files[].patch on both SHAs). The new HEAD f5fbae58 differs only because Graphite restacked the parent chain underneath — the trailing 12 commits in compare/a338eba...f5fbae58 are previously-reviewed stack commits whose parent pointers got rewritten, not new content on this PR.
No new functional changes to evaluate. Round 2 verdict carries.
Verdict: APPROVE
Reasoning: Tip patch byte-identical; restack-only delta; re-stamping per the branch-protection rule that invalidates prior approval on SHA change.
Review by Vai (re-review round 3)
ced2454 to
1795568
Compare
140d6b5 to
ff7f976
Compare
1795568 to
6f6dc30
Compare
vanceingalls
left a comment
There was a problem hiding this comment.
Re-stamp at ff7f9767 — restacked. Prior approve (f5fbae5) still applies content-wise; this is the post-restack head. No new substantive changes.
Re-review by Vai (post-restack re-stamp)
The base branch was changed.
PR 8.4 of the WebM distributed-rendering plan (v1.5 backlog #1; see DISTRIBUTED-RENDERING-PLAN.md §7.2). User-facing docs catch up with the shipped capability. Updates docs/deploy/migrating-to-hyperframes-lambda.mdx: - "Output format" row in the migration table now lists `webm` alongside mp4 / mov / png-sequence with a note that webm uses libvpx-vp9 + closed-GOP concat-copy. HDR mp4 remains the only refused format. - "No webm distributed" caveat replaced with "webm uses closed-GOP VP9" explainer covering the encoder args (`-g <chunkSize>`, `-keyint_min <chunkSize>`, `-auto-alt-ref 0`, `-cpu-used 2`), why alt-ref disable is load-bearing, and that the output preserves alpha via yuva420p with Opus audio. - Migration checklist no longer asks adopters to filter out webm compositions; only HDR-dependent renders need to stay on the previous framework. aws-lambda.mdx doesn't currently call out webm as unsupported (only HDR in the v1 surface list), so it gets no copy edits beyond the migration guide. The internal planning doc (DISTRIBUTED-RENDERING-PLAN.md §7.2, §8, §12 — kept outside the repo) gets matching updates: format support matrix flipped ✓, v1.5 backlog #1 marked shipped, HDR promoted to the new top item, and the rev-12 → rev-13 status line. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Folds in cleanups identified by a multi-agent code-review pass over the 4-PR webm-distributed stack: - plan.ts: `resolveEncoderTriple()` webm case now calls `getEncoderPreset(quality, "webm")` for its preset string instead of hardcoding "good". The hardcode was wrong for `quality: "draft"` (`getEncoderPreset` returns "realtime" for that tier) — would have silently overridden the draft → realtime mapping for distributed webm renders. - chunkEncoder.ts: trim the new VP9 closed-GOP comment block from ~18 lines of WHY narration down to the 6 lines that actually explain why (alt-ref + cpu-used drift). Match the alpha branch's idempotent-push comment to the same standard. - chunkEncoder.test.ts: drop the duplicate WHY comment that restated the implementation comment in plain words. - webm-concat-copy.test.ts: rewrite the file-header docstring to describe the contract being tested instead of the PR-8.1-gating history; strip "PR 8.2 / Path A / Path B" references from error messages (they belong in PR bodies, not in test output). Consolidate the yuva420p alpha smoke into a single `it()` block (was a full 4-test describe with duplicated setup) — the yuv420p block already covers the probe/decode/frame-count contract; the alpha smoke only needs to prove the alpha args don't break concat-copy. - plan.test.ts: drop the "PR 8.1 proved the contract" comment. - webm-vp9 fixture: drop the aspirational "Other webm-with-audio fixtures cover the mux path separately when added" sentence (no other fixtures exist). Regenerated the baseline via `docker:test:update webm-vp9` to reflect the updated comment. - migrating-to-hyperframes-lambda.mdx: add a paragraph about distributed webm's perf cost — ~10-25% larger files at constant CRF due to forced keyframes, and slower per-chunk encode due to `-cpu-used 2` being more conservative than the libvpx default. All unit tests + the webm-vp9 distributed-simulated regression still pass after these changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The CLI's `lambda render` subcommand's FORMATS allowlist and the `RenderArgs.format` type still narrowed to `mp4 | mov | png-sequence`, so even though the producer + aws-lambda packages now support webm end-to-end, the CLI surface rejected it with `--format must be mp4|mov| png-sequence`. Add webm to both spots and update the --help description. Surfaced during real-AWS deploy prep — the local lambda-local / distributed-simulated tests didn't go through the CLI so the gap went unnoticed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The deterministic Google Fonts cache was rooted at
`\$HOME/.cache/hyperframes/fonts`, which fails on AWS Lambda — the
runtime's `\$HOME` resolves to a `/home/sbx_*` directory tree that's
read-only. `mkdirSync(..., { recursive: true })` can't create that
path and the plan stage trips with `ENOENT: no such file or directory,
mkdir '/home/sbx_user1051/.cache/hyperframes/fonts/space-mono'` on
every Lambda render that pulls a Google Font (i.e. every distributed
fixture using `@import url("https://fonts.googleapis.com/...")`).
Detect Lambda via `\$AWS_LAMBDA_FUNCTION_NAME` and route the cache to
`tmpdir()/hyperframes/fonts` in that case. Lambda's `/tmp` survives
across invocations on a warm container, so cache hit rate is the same
as non-Lambda runs. Also honor an explicit
`\$HYPERFRAMES_FONT_CACHE_DIR` override for adopters who want a
different location regardless of the runtime.
Surfaced while verifying webm distributed end-to-end on real AWS — the
same bug affects mp4 fixtures using Google Fonts; webm just happened to
be the one I tried first.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Second simplify-review pass on the webm stack flagged two cleanups:
1. **`DistributedFormat` type duplicated 10 times.** Every file in the
distributed pipeline carried its own copy of
`"mp4" | "mov" | "png-sequence" | "webm"` — adding a new format
meant a 10-place edit with no compile-time guarantee they stayed in
sync. Extract a single source of truth in
`packages/producer/src/services/distributed/shared.ts`, re-export
from `@hyperframes/producer/distributed` and
`@hyperframes/aws-lambda/sdk`, and have all callers pull from
there. The aws-lambda `ALLOWED_FORMATS` runtime tuple and the CLI's
`FORMATS` tuple now both use `satisfies readonly DistributedFormat[]`
so the compiler enforces the runtime allowlist stays in sync with
the type.
2. **`deterministicFonts.ts` font-cache resolver was over-commented.**
Trim the 7-line block to 4 lines (drop the aspirational
"and other read-only-FS execution environments" — only Lambda is
detected — and the warm-container `/tmp` persistence narration —
anyone reading already knows Lambda /tmp semantics). Collapse the
two-step `if (explicit && explicit.length > 0)` into a single
nullish-coalesce expression now that the empty-string defensive
check is gone (`process.env.X` is `string | undefined`, no third
shape to guard against).
Out-of-scope skips (called out by the agents, deferred):
- In-process `RenderConfig.format` and the in-process CLI's
`render.ts` format union still carry their own inline copies. The
union happens to coincide today but they're separate concerns —
leaving them alone limits this PR's blast radius.
- `fontCacheDir(slug)` / `resolveFontCacheRoot()` naming asymmetry
flagged as taste; skipping.
- Pre-existing redundant `existsSync` before `mkdirSync({ recursive:
true })` in `fontCacheDir` — out of scope.
All tests + typecheck still pass. Lambda render still works
end-to-end (no functional changes).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR review feedback: source/docs should not mention the distributed-rendering planning doc. Tighten the migration checklist sentence to describe the webm path directly rather than referencing the doc's version label. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ff7f976 to
f2c22be
Compare
…pers CI Fallow audit on PR #953 flagged `resolveEncoderTriple` at CRAP 31.6 — the function interleaved (a) mp4 codec validation + dispatch, (b) the non-mp4 codec-rejection throw, and (c) per-format dispatch. Splitting into `resolveMp4EncoderTriple` + `resolveNonMp4EncoderTriple` drops the top-level function's cyclomatic complexity below the threshold while preserving every error message and code path. Behavior unchanged. Also extracts an `EncoderTriple` type alias so the three functions share the return shape declaratively rather than repeating it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
miguel-heygen
left a comment
There was a problem hiding this comment.
Re-approving at 1cb7ef3 — resolveEncoderTriple split into resolveMp4EncoderTriple + resolveNonMp4EncoderTriple to clear fallow CRAP threshold. Behavior unchanged, clean decomposition.
vanceingalls
left a comment
There was a problem hiding this comment.
Re-stamp at 1cb7ef3d — pure refactor split of resolveEncoderTriple → resolveMp4EncoderTriple + resolveNonMp4EncoderTriple to clear fallow's CRAP 31.6 finding. Single-file diff (+34/-21 in plan.ts); behavior-preserving per the PR body. Same content end-state as prior approves; same review verdict applies.
CI still in_progress on this head (Build/Typecheck/Test/CLI smoke/regression-shards/Windows pending) but were all green on the prior head and the diff is pure-refactor.
Re-stamp by Vai (post-refactor)

Description
PR 4 of 4 in the WebM distributed-rendering series. Originally
docs-only; expanded to fold in simplify-review findings + two bugs
uncovered while verifying webm distributed end-to-end against a real
AWS Lambda stack.
Real-AWS verification
Deployed
hyperframes-webm-teststack tous-east-1, ran a webmrender through the full plan → renderChunk × 4 → assemble pipeline on
real Lambda, and verified the output:
ALPHA_MODE=1tagffmpeg -f null -decodeStack + S3 bucket torn down cleanly post-verification.
Commits
docs(lambda): document webm support in distributed mode—updates
docs/deploy/migrating-to-hyperframes-lambda.mdx: formattable now lists webm, "No webm distributed" caveat replaced with
"webm uses closed-GOP VP9" explainer, migration checklist no longer
filters out webm.
refactor: address simplify-review findings on webm stack—bundle of cleanups identified by a multi-agent code-review pass:
plan.tsresolveEncoderTriple()webm preset hardcode (waswrong for
quality: "draft").chunkEncoder.tsand its mirrorin
chunkEncoder.test.ts.PR bodies, not in test output or source comments.
duplicated setup into a single focused
it()block.regenerated.
constant CRF, slower per-chunk encode) in the migration doc.
fix(cli): accept --format=webm in hyperframes lambda render—surfaced during real-AWS deploy prep. The CLI's
lambda renderFORMATS allowlist and
RenderArgs.formattype still narrowed tomp4 | mov | png-sequence.fix(producer): font cache writes to /tmp on Lambda— surfacedby the first real-AWS render attempt failing at plan stage with
ENOENT: no such file or directory, mkdir '/home/sbx_user1051/.cache/hyperframes/fonts/space-mono'. Thedeterministic Google Fonts cache was rooted at
$HOME/.cache/...,which fails on Lambda —
$HOMEresolves to/home/sbx_*which isread-only. Not webm-specific: this bug affected every existing
distributed fixture using Google Fonts. Fix detects Lambda via
$AWS_LAMBDA_FUNCTION_NAMEand routes totmpdir()/hyperframes/fonts.refactor: extract DistributedFormat type + trim font-cache resolver— second simplify-review pass identified that the
"mp4" | "mov" | "png-sequence" | "webm"union was duplicated12+ times across producer, aws-lambda, CLI, and the harness.
Extracted to a single source of truth in
packages/producer/src/services/distributed/shared.tsandre-exported through
@hyperframes/producer/distributedand@hyperframes/aws-lambda/sdk. The runtimeALLOWED_FORMATStupleand CLI
FORMATStuple now useas const satisfies readonly DistributedFormat[]so the compiler enforces tuple ↔ type sync.Also trimmed the over-commented font-cache resolver and collapsed
the empty-string defensive check into a nullish-coalesce.
docs(lambda): drop plan-doc reference from migration checklist— review feedback: source/docs should not reference the internal
planning doc. Tightened the migration checklist sentence to
describe the webm path directly.
Testing
Local:
bunx vitest run --root packages/engine src/services/chunkEncoder.test.ts— passesbun test packages/producer/tests/distributed/_smoke/webm-concat-copy.test.ts— passes (now with real alpha checks)bun test packages/producer/src/services/distributed/planFormatBanlist.test.ts— passesdocker:test webm-vp9(in-process) — passesdocker:test --mode=distributed-simulatedfor all 5 distributed fixtures (mp4-h264-sdr, mp4-h265-sdr, mov-prores, png-sequence, webm-vp9) — 5/5 passReal AWS:
hyperframes lambda deploy --stack-name=hyperframes-webm-test --region=us-east-1 --concurrency=4— stack createdhyperframes lambda render webm-vp9 --format=webm --chunk-size=15 --max-parallel-chunks=4 --wait— succeeded, PSNR 58–63 dB vs local baselinehyperframes lambda destroy+ S3 cleanup — stack and bucket fully reclaimed🤖 Generated with Claude Code