Add hardware-accelerated decoding and encoding support#445
Merged
Conversation
Decoding (openvideo / load): - Add hwaccel kwarg to VideoReader/openvideo for GPU-accelerated decoding (VideoToolbox, CUDA, VAAPI, QSV) - Frames auto-transferred from HW surface to CPU; identical to SW decoding - Lazy SwsTransform rebuild when actual HW download format differs from initial guess - Store swscale/sws_color options on VideoReader so rebuild preserves them - Add available_hw_devices() public API Encoding (open_video_out / save): - Add hwaccel kwarg to VideoWriter/open_video_out/save for HW encoding - Auto-select HW encoder when hwaccel set and codec_name is nil - Create hw_device_ctx for codecs needing HW_DEVICE_CTX (NVENC, QSV) - Add available_hw_encoders() public API Tests: - test/hwaccel.jl: decode + encode tests - util/bench_hwaccel.jl: SW vs HW throughput benchmark Docs: - docs/src/reading.md: HW decoding section - docs/src/writing.md: HW encoding section Co-Authored-By: Claude <81847+Claude@users.noreply.github.com>
7894883 to
c327bf4
Compare
…devices in tests - Add `hwaccel_available(sym::Symbol) -> Bool` to src/avio.jl: probes whether a device type is actually usable at runtime (not just compile-time supported) by attempting av_hwdevice_ctx_create and releasing the context on success - Export hwaccel_available - Improve error messages in VideoReader and VideoWriter hw device ctx creation to hint at hwaccel_available - Rewrite test/hwaccel.jl: compute working_hw_devs = filter(hwaccel_available, hw_devs) at the top; remove try/catch from all decode tests (device is confirmed operational); use working_hw_devs in all loops; add standalone hwaccel_available testset - Fixes CI failures on Ubuntu where :vaapi/:drm appear in available_hw_devices() (compile-time support) but no GPU hardware is present at runtime - Docs: add hwaccel_available @docs block and usage examples to reading.md and writing.md alongside available_hw_devices / available_hw_encoders"
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #445 +/- ##
==========================================
+ Coverage 77.87% 79.12% +1.24%
==========================================
Files 10 10
Lines 1329 1466 +137
==========================================
+ Hits 1035 1160 +125
- Misses 294 306 +12 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
- available_hw_encoders: iterate all hw config indices instead of only index 0 - _vio_rebuild_sws_for_hw!: read dst colorspace from dstframe fields instead of re-deriving defaults; persist color_range/color_primaries on dstframe at SwsTransform construction so rebuild has accurate values - decode: update r.input_pix_fmt for non-SwsTransform hw paths (AVFramePtr, GrayTransform) not just SwsTransform, fixing stale format after lazy HW format correction - retrieve_raw / retrieve_raw!: call fill_graph_input! before buffer allocation / size check so lazy HW format correction fires first and the buffer is sized for the real pixel format - encoding: add _vio_is_rgb_pix_fmt helper; when hwaccel is set and avcodec_find_best_pix_fmt_of_list auto-selects a packed-RGB format (e.g. BGRA), re-select using NV12 as source so VideoToolbox HEVC/ProRes encoders don't reject the stream at avcodec_open2 - test/hwaccel.jl: add transcode=false raw interface test, explicit target_colorspace_details test, bump explicit-codec test to 128x128, use .mov for ProRes, pass hwaccel=dev to explicit-codec test, remove misused @test_broken annotations
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Made possible now that we have JuliaPackaging/Yggdrasil#13288
Developed with Claude
Decoding (openvideo / load):
Encoding (open_video_out / save):
Tests:
Docs:
On my M2 mac