HDR export: 10/12-bit AVIF & JPEG XL (BT.2020 PQ/HLG)#1288
HDR export: 10/12-bit AVIF & JPEG XL (BT.2020 PQ/HLG)#1288KaiserRuben wants to merge 21 commits into
Conversation
- PLAN.md / HDR_WORKLOG.md: confirmed architecture against checkout. The 8-bit ceiling is shader.wgsl's rgba8unorm storage output (L198); headroom is clipped upstream at the clamping linear_to_srgb encode (L1668/L1675), not just the final store clamp (L1734). - New pub module src-tauri/src/hdr.rs: pinned ST.2084 PQ inverse-EOTF + EOTF, BT.2100 HLG OETF, extended-sRGB<->linear (exact inverse of the shader's transfer), BT.2087 Rec.709->Rec.2020 matrix, CICP tag mapping, full-range quantization, and a synthetic linear test image (black / diffuse-white / 2x / 4x / ramp patches). - 6 unit tests pin the math: 203-nit ref white -> PQ code 594 (within 2 of the brief's 593), strictly-increasing headroom codes, headroom-preserving sRGB roundtrip, neutral-preserving primaries matrix.
…green) - hdr::encode_avif_hdr: pure-Rust rav1e + avif-serialize. CICP primaries=9 (BT.2020), transfer=16 (PQ) / 18 (HLG), matrix=0 (Identity/RGB, 4:4:4 full-range), 10 & 12 bit. Identity matrix feeds transfer-encoded R'G'B' codes directly (G,B,R plane order) — no YCbCr/subsampling, so the round-trip is lossless and exact for all colors. - hdr::encode_jxl_hdr: jpegxl-rs + system libjxl, PQ/HLG Rec.2020 f32, uses_original_profile so the custom color encoding survives. Behind optional `hdr_jxl` feature. - Cargo.toml: rav1e 0.8.1 + avif-serialize 0.8.9 (always-on, no system deps); jpegxl-rs 0.14 + jpegxl-sys 0.12 (optional `hdr_jxl`, needs libjxl). - tests/hdr_export.rs: the gate. Parses nclx + av1C from bytes (encoder-independent), decodes via avifdec (libavif, independent of rav1e). All AVIF assertions pass with EXACT codes: 10-bit white=594 rel2=669 rel4=746; 12-bit white=2378 rel2=2679 rel4=2986; black=0; monotonic ramp; identity plane-order preserved (R<G<B).
JxlPrimaries::SRGB -> SRgb. With --features hdr_jxl (PKG_CONFIG_PATH at brew jpeg-xl), all 5 harness tests pass: JXL PQ tagged correctly, reference-white PQ code and headroom verified via jxl-oxide decode.
- Extract build_main_bgl(device, storage_format) shared by SDR (Rgba8Unorm) and HDR (Rgba16Float) so the two layouts cannot drift. - hdr_shader_source(): derive the HDR shader from shader.wgsl at runtime via 4 targeted substitutions: rgba8unorm->rgba16float storage; clamping linear_to_srgb -> headroom- preserving linear_to_srgb_extended (the real headroom killer, upstream of the store); drop the [0,1] store clamp (keep max(.,0)); disable the 8-bit dither. - GpuProcessor: lazily-built (OnceLock) HDR pipeline + rgba16float tile texture, so SDR-only sessions allocate nothing extra. run() takes output_hdr and selects pipeline/bgl/tile/ bytes-per-pixel (8 for f16 vs 4). read_texture_data_roi parameterized by bytes_per_pixel. - process_and_get_dynamic_image_hdr(): new entry point. On HDR readback, decode f16 -> invert extended sRGB (hdr::srgb_extended_to_linear) -> DynamicImage::ImageRgba32F in LINEAR scene-referred light (diffuse white = 1.0, headroom > 1.0 preserved) for the HDR encoders. The existing rgba8unorm pipeline, textures, and readback are byte-for-byte unchanged.
…line - ExportSettings: add bit_depth, transfer_function, primaries (serde camelCase defaults); hdr_enabled(ext) gates true-HDR to AVIF (always) / JXL (only with hdr_jxl feature) at >=10 bit with a PQ/HLG transfer. export_bit_depth() clamps to 10/12. - encode_image_to_bytes takes &ExportSettings; avif/jxl arms route to hdr::encode_avif_hdr / encode_jxl_hdr on the linear Rgba32F (image.to_rgba32f()). SDR arms unchanged. - process_image_for_export[_pipeline] take output_hdr and call process_and_get_dynamic_image_hdr (rgba16float GPU path) when HDR is requested. - Metadata writer leaves avif/jxl bytes untouched (its match returns Ok for non jpeg/png/ tiff), so CICP tags survive. resize() preserves the f32 variant. - Add gpu_processing test guarding the 4 HDR shader-substitution anchors so a future shader.wgsl edit can't silently no-op them.
…larize asserts Audit (negative-control) revealed two real gaps: 1. The tone-curve stage was STILL clipping headroom to [0,1] even after the sRGB-encode and store-clamp fixes — the deeper half of the bug the brief warned about. A new GPU end-to-end test (headless wgpu device, real pipeline, white at +2 stops) caught it: recovered linear was 1.0 instead of 4.0. Three more targeted hdr_shader_source() substitutions fix it: treat empty curves as default (skip the normalize branch); pass curve-domain headroom through instead of snapping to the top control point; and don't renormalize >1.0 when per-channel RGB curves are active. With these, the GPU now recovers linear 3.997 (~4.0) end-to-end through the full look stage; SDR still clamps to 255. 2. The integration reference-white/headroom assertions were CIRCULAR (expected values derived from the same encoder functions under test). Replaced with independent ST.2084 ground-truth literals (10-bit 594/669/746, 12-bit 2378/2679/2986; JXL 0.5807/0.6542/0.7291). Also: GPU E2E test (exercises run() + f16 readback + srgb_extended_to_linear inversion, skips gracefully without a GPU) and extended the shader-substitution guard to the 3 new curve anchors. Full suite: 8 lib + 5 integration (with hdr_jxl) green.
- useExportSettings: add bitDepth (8/10/12), transferFunction ('srgb'|'pq'|'hlg'),
primaries ('srgb'|'bt2020') state, threaded through preset apply + currentSettingsObject.
- ExportSettings / ExportPreset TS interfaces gain the three optional fields.
- ExportPanel: for AVIF/JXL, a Bit Depth dropdown; when >8-bit, Transfer + Primaries
dropdowns appear (picking an HDR depth defaults to PQ/Rec.2020) plus an HDR-display hint.
The three fields are spread into both the estimate and export settings objects via one
memoized hdrSettings (camelCase -> backend serde bit_depth/transfer_function/primaries).
- en.json: new export.file.* strings.
Vite build passes; no new tsc errors (repo has 117 pre-existing).
HDR_REPORT.md: files changed, dependencies + system-lib install, how to run the harness, observed CICP/codes, known limitations (headroom-through-curves choice, matrix=0 vs 9, HLG anchoring, watermark/EXIF), human HDR-display sign-off list, and build commands.
…e/gamut/metadata) hdr.rs introduces HdrEncodeConfig and parameterizes the color science: - matrix: Identity (RGB, exact, 4:4:4) or YCbCr (BT.2020-NCL for Rec.2020, BT.709-NCL for sRGB/P3) with correct non-constant-luminance conversion. - chroma subsampling 4:4:4 / 4:2:2 / 4:2:0 (box-averaged); identity forces 4:4:4. - full vs limited signal range (proper 219/224 + 16/128 scaling per bit depth). - primaries adds Display-P3 (CICP 12; 709->P3 matrix, rav1e SMPTE432, JXL P3). - reference-white nits (default 203) and HLG peak ratio (default 12) parameterized. - mastering metadata: per-frame MaxCLL/MaxFALL (+ BT.2020/D65 MDCV) via avif-serialize. - JXL quality/lossless threaded from jpegQuality. HdrEncodeConfig::default() reproduces the prior identity/4:4:4/full/203 output exactly, so the existing verified path is unchanged. export_processing: ExportSettings gains matrix/chromaSubsampling/range/referenceWhiteNits/ hlgPeakRatio/masteringMetadata (serde camelCase, defaulted); to_hdr_config() builds the config; guardrails normalize invalid inputs (identity->4:4:4, ref-white<=0->203, hlg<=0->12). Tests: colored YCbCr 4:4:4 round-trip (R/G/B 0.1/0.4/0.9 -> 367/499/583, channels not swapped), 4:2:0 uniform round-trip, limited(941) vs full(1023) range via y4m, primaries tagged+applied, mastering clli box present, bad-bit-depth rejected. 13 lib + 11 integration pass (default 13+10). Co-Authored-By: KaiserRuben <ruben@kaiser.fyi>
…anguage help Export panel (AVIF/JXL) gets a two-tier color UI for non-experts: - Tier 1: a "Color profile" dropdown - Standard (SDR), HDR10 (best compatibility), HLG (broadcast), Maximum quality (12-bit 4:4:4), Archival/exact (RGB lossless), and Custom. Each preset sets the whole field bundle; editing Advanced flips the picker to Custom. - Tier 2: Advanced controls (bit depth, color encoding/matrix, chroma detail, signal range, color gamut incl Display-P3, reference white, HLG peak, mastering metadata) - each with a one-line jargon-free explanation underneath, plus guardrails that hide/disable invalid combos (chroma only for YCbCr, HDR only >8-bit, HLG peak only for HLG, etc.). useExportSettings/ExportSettings/ExportPreset gain matrix/chromaSubsampling/range/ referenceWhiteNits/hlgPeakRatio/masteringMetadata; all spread into the estimate and export settings objects via the shared hdrSettings memo (camelCase keys match the backend serde). en.json gains the new labels + help strings (other locales fall back to English, consistent with the rest of the export panel). Vite build passes; no new type errors.
Merge the new export.file HDR keys (color profile presets + advanced controls + help) into de/fr/es/it/pt/pl/ru/ja/ko/zh-CN. Technical tokens (PQ, HLG, Rec.2020, Display-P3, 4:2:0, nits, MaxCLL, ...) kept verbatim; existing keys untouched; en.json unchanged. Machine translation - may warrant native-speaker review.
Large 4:2:2 AVIF exports (e.g. 6177x4118) produced a non-conformant AV1
bitstream that strict decoders reject ("Decoding of color planes failed").
rav1e only encodes 4:2:2 conformantly as a single AV1 tile, but AV1's
per-tile caps (4096 px wide, 4096*2304 px area) force a multi-tile layout
on larger frames, which its 4:2:2 path mis-codes. 4:2:0 and 4:4:4 tile
correctly at any size and now round-trip at full resolution.
- hdr: bound 4:2:2 AVIF to the single-tile envelope and return a clear
error (never a silent chroma downgrade) when a frame exceeds it; expose
the limit as AVIF_422_MAX_WIDTH / AVIF_422_MAX_PIXELS.
- export: add an avif_422_limits command exposing the 4:2:2 limit (single
source of truth) for the UI to derive its resolution cap from.
- export panel: rework the HDR control wording for non-experts; selecting
4:2:2 auto-applies the resolution cap (visible and overridable) and warns
if it is removed.
- DRY: one quality->JXL-distance mapping (SDR output byte-identical);
to_hdr_config is a raw projection of the user's settings with all
coercion owned by HdrEncodeConfig::sanitized / clamp_anchors; the JXL
anchor-clamp duplicate is removed.
- tests: large-frame conformance (4:2:0/4:4:4 at 6177x4118), 4:2:2
single-tile round-trip, oversize-errors, and a derivable-cap tripwire.
- frontend: derive the 4:2:2 AVIF resolution cap from the avif_422_limits
command instead of hardcoding it, and interpolate the limit into the
warning/note ({{maxLongEdge}}). The backend is the single source of truth.
- export panel: truncate dropdown trigger labels so they no longer wrap,
shorten the color-profile names (the detail lives in the profile
descriptions), and raise the 4:2:2 warning text contrast.
- i18n: sync the HDR export strings (profile names + descriptions, advanced
color controls, and the 4:2:2 cap warning/note) into de/es/fr/it/ja/ko/
pl/pt/ru/zh-CN; the {{maxLongEdge}} placeholder is preserved in each.
HDR AVIF/JXL export held more full-resolution buffers than necessary: - encode_image_to_bytes called image.to_rgba32f() on an image that is already Rgba32F, allocating a full 16-byte/px duplicate held for the entire encode (~0.4 GB at 25 MP, ~1 GB at 60 MP). Borrow the existing buffer instead. - encode_avif_hdr copied every plane into a fresh little-endian byte Vec before handing it to rav1e; on little-endian targets the plane bytes are already in that layout, so pass them zero-copy via bytemuck::cast_slice. Output is byte-for-byte unchanged (the exact-value decode tests still pass); this only lowers peak RAM.
The HDR compute shader was derived from the SDR shader by seven runtime string .replace() calls — fragile against any edit to shader.wgsl. Move the headroom-preserving logic into the shared shader behind a pipeline-overridable `IS_HDR` constant (compiler-checked), leaving only the one thing WGSL can't express as an override: the storage texture format (rgba8unorm -> rgba16float). - shader.wgsl: add `override IS_HDR: i32 = 0;` and gate the sRGB encode, the store clamp, the dither, and the three tone-curve clamp points on it via select(). Default 0 = SDR. - build_hdr_resources: set IS_HDR=1 via PipelineCompilationOptions. Every other pipeline leaves it at the default, so their behavior is unchanged. - hdr_shader_source: now a single format swap. - Rewrote the shader guard test for the new approach. Behavior is unchanged: the GPU end-to-end test confirms SDR still clamps to 255 and HDR still recovers linear ~4.0 (headroom preserved). Seven substitutions -> one.
…the PR description
The 'Embed brightness metadata' toggle shipped off while its help text says 'Recommended — safe to leave on'. The values (MaxCLL/MaxFALL) are computed from the actual pixels and the flag is read only by the HDR encode path (SDR ignores it), so default it on to match the copy. An explicit false in a saved preset is still honored.
3f3aede to
32d79e0
Compare
The HDR encoders added build-time native requirements the CI matrix couldn't satisfy, breaking clippy and the x86_64 / Windows-arm build jobs (ARM Linux and Apple Silicon happened to pass). rav1e asm / nasm rav1e's "asm" fast paths need nasm at build time on x86_64 (missing on the Ubuntu and macOS-intel runners) and fail to link on aarch64-pc-windows-msvc (LNK1181). Keep asm everywhere it works via a per-target dependency (all but aarch64-pc-windows-msvc) and install nasm on the x86_64 build/clippy runners; Windows x86_64 already ships it. asm is a build-time-only speedup with no runtime requirement (rav1e dispatches per-CPU), so this keeps fast AVIF encode for ~all users; Windows-on-ARM falls back to the pure-Rust encode. clippy --all-features enabled the optional hdr_jxl feature, which links system libjxl (>= 0.11.2); the Ubuntu runners ship far older libjxl, so the job died compiling jpegxl-sys. Build with `--features hdr_jxl,jpegxl-rs/docs` instead: the `docs` flag flips jpegxl-sys's build script to its no-op branch (no link), so clippy still type-checks the full surface including the hdr_jxl module without any system libjxl. Windows MSI version error The matrix sets `builds-args: --bundles nsis`, but the reusable workflow read `matrix.args`, so the flag was never passed and tauri also built an MSI bundle — which rejects the non-numeric `1.5.7-hdr` pre-release. Read `matrix.builds-args` (falling back to `matrix.args` for Android) so Windows builds NSIS only, as intended; the MSI was never uploaded anyway (the artifact step only collects *.exe). Clippy findings, surfaced once clippy could reach the crate: - removed a dead `MAX_MASK_BINDINGS` const added to GpuProcessor::new - #[allow(too_many_arguments)] on GpuProcessor::run (8 args w/ output_hdr) - #[allow(field_reassign_with_default)] on encode_avif_hdr (rav1e idiom) - rewrote a match_like_matches_macro hit in ExportSettings::hdr_enabled
32d79e0 to
3266ebf
Compare
|
Can you resolve the conflict and merge it please? |
Conflicts resolved: - src-tauri/src/lib.rs: keep both module decls — `pub mod hdr` (HDR export) and upstream `mod hdr_deghosting`. - src-tauri/tauri.conf.json: adopt upstream version 1.5.8, dropping the 1.5.7-hdr build marker. Verified: cargo check clean (warnings only); tsc --noEmit adds no new errors over the upstream baseline.
|
@haico1992 merge possible now, waiting for maintainer to review. |
|
I cannot merge this pull request in its current form. At over 3,000 lines, it introduces too many high-risk changes at once, including modifications to core logic, altered build requirements, and extensive AI-generated comments. Additionally, regarding the new tests: since our codebase does not currently have a testing framework, we need to either set up a general test suite first or remove these specific tests for now. To move forward, please:
Let me know if you would like to discuss how to structure these smaller PRs. |
|
@haico1992 You can clone my Fork and build it yourself, as I did. That way, you can utilize the improvements without waiting for the merge. However, you will loose the ability to automatically update. @CyberTimon I do understand the concern. I will investigate the options to move forward as time allows. Thanks for your feedback. |
Motivation
Modern full-frame sensors capture far more dynamic range than 8-bit sRGB can show. After editing a Sony A7C II raw, the highlight detail is in the file — but the export step throws it away, clamping everything to 8-bit sRGB. Meanwhile HDR stills now render correctly across the ecosystem: Google Photos, iOS/Android, Chrome/Safari, and the major social platforms all decode HDR AVIF.
RapidRAW already grades in a wide-gamut, scene-linear pipeline that carries real headroom above diffuse white. This PR adds an export path that keeps that headroom instead of discarding it: 10/12-bit AVIF and JPEG XL, tagged BT.2020 + PQ (or HLG).
Summary
UI
The SDR default is untouched — same controls, same output:
A "Color profile" dropdown picks the intent in one step; each preset configures everything below it:
"Custom" reveals the Advanced section — bit depth, transfer, gamut, encoding/matrix, chroma, range, reference white — each with non-expert help text:
Scope & compatibility
hdr.rs+ tests; the SDR-affecting surface is small.bitDepth ?? 8, etc.), so existing saved presets behave exactly as before.What's included
src-tauri/src/hdr.rs): pinned SMPTE ST.2084 (PQ) / ITU-R BT.2100 (HLG) transfer functions with the ISO 22028-5 203-nit reference-white anchor, BT.2087 Rec.709→Rec.2020 and Rec.709→Display-P3 matrices, CICP/nclx tag mapping, full/limited-range quantization, and the AVIF + JXL encoders. One source of truth, shared by the encoders and the test harness.rav1e+avif-serialize(no runtime system deps).jpegxl-rs+ system libjxl, behind an optionalhdr_jxlcargo feature (default builds need no libjxl).rgba16floaton a pipeline that shares the SDR shader; anIS_HDRpipeline-overridable constant switches on headroom-preserving behavior so values above diffuse white survive the look stage instead of clipping to[0,1].ExportPanel.tsx): the presets dropdown (SDR / HDR10 / HLG / Max quality / Archival / Custom) plus the Advanced section, all with non-expert help text. Strings localized into 10 languages.How it works (data flow)
The HDR shader is the same WGSL as the SDR path, gated by the
IS_HDRoverride constant — so the headroom-preserving differences (extended-sRGB encode, dropped store clamp, disabled dither, no tone-curve snapping at the top) are compiler-checked rather than patched in by string replacement. The one thing WGSL can't express as an override — the storage texture format — is the single remaining textual swap (rgba8unorm→rgba16float).4:2:2 handling (the conformance fix)
rav1e only encodes AV1 4:2:2 ("Professional" profile) conformantly as a single AV1 tile. AV1's per-tile caps (4096 px wide, 4096·2304 = 9,437,184 px area) force a multi-tile layout on larger frames, which its 4:2:2 path mis-codes. 4:2:0 and 4:4:4 tile correctly at any size. So:
hdr::AVIF_422_MAX_WIDTH/AVIF_422_MAX_PIXELS), exposed to the frontend via anavif_422_limitscommand so the UI derives its cap rather than hardcoding it.Verification
src-tauri/tests/hdr_export.rsdecodes output with independent decoders (libavifavifdec, jxl-oxide) and parses the rawnclx/av1Cboxes from the file bytes. Expected values are pinned ST.2084 ground-truth literals (not derived from the encoder under test):The headroom codes are strictly increasing, and the GPU end-to-end case proves both that headroom survives the full look stage and that SDR is byte-identical with the override.
Build (cross-platform)
The AVIF path is pure Rust with no platform-specific code;
nasmis the only added build-time requirement (rav1e's assembly), available viaapt/choco/brewon all three platforms. JPEG XL is opt-in and needs system libjxl.Built and run locally on macOS arm64. There is no platform-specific code in the HDR path, so Windows/Linux builds should be unaffected — but I have not built them; worth a CI pass.
Known limitations
HLG_PEAK_RATIO = 12is a reasonable default, not tuned.Dependencies & licensing
rav1e+avif-serialize. Build-timenasm.hdr_jxl):jpegxl-rs+ system libjxl (GPL-3.0 bindings — compatible with the project's AGPL-3.0). Default builds do not require libjxl.License unchanged (AGPL-3.0).