Fix H.264 HW decode for AVC3 in-band streams (V4L2/m2m)#2035
Fix H.264 HW decode for AVC3 in-band streams (V4L2/m2m)#2035scottrix wants to merge 128 commits into
Conversation
[backport][Bento4] Version bump (upstream sync)
[backport][DashTree] Always ensure period duration
Will be introduced on next Kodi versions
[backport][KodiProps] New drm_legacy, removed drm property
[backport] Replaced deprecated sprintf with snprintf
[backport][AdaptiveTree] IsLastSegment return false if period is the last one
[backport][Deps] Reverted recents bento4 commits
[backport] Fix wrong Dash KID data format and cleanups
src/decrypters/Helpers.h:55:13: error: 'uint8_t' was not declared in this scope 55 | std::vector<uint8_t> ConvertKidStrToBytes(std::string_view kidStr);
[Omega/backport] Fix build error with gcc 14
CADTSSampleReader::ReadSample call ADTSReader::ReadPacket each time kodi request buffer data but when the stream buffer reach the end the m_id3TagParser.parse cannot read data from m_stream by returning ID3TAG::PARSE_FAIL this was breaking the while loop and return true since return true was causing that CADTSSampleReader::ReadSample to return AP4_SUCCESS with bad data where instead was needed to check for waitingForSegment and mark the m_eos as true, this to allow stop the playback and so avoid flooding kodi demuxer of bad package data
[backport][ADTSReader] Return false when id3tagparser fails due to eos
This variable is modified within the for loop so place it outside will cause that sideeffects because the variable is assigned by some conditions
[Session] Restored licenseData variable on for loop
This should allow play Widevine streams by using ClearKey DRM untested since i havent found a sample
The previous code was assuming that the startnumber was the result of (period start / duration) now take in account that startNumber of segmenttemplate can be different
[backport][DashTree] Fix segmentNumber based on tsb
[backport][widevine] Updated CDM interface
[backport][UrlUtils] Fix Join to keep dots in a path
[backport][UrlUtils] Preserve url port number
If crypto info has been set previously, and the current package does not use it, it must be reset to prevent the decryptor from attempting to decrypt unencrypted content
[backport][FragmentedSampleReader] Reset crypto info on change
[backport][FragmentedSampleReader] Fix switching between unencrypted/crypted content
On platforms using V4L2 mem2mem H.264 decoding (e.g. Raspberry Pi with OSMC's custom FFmpeg), the decoder's h264_xd_copy() function rejects avcC extradata with 0 SPS/0 PPS entries, causing a fallback to software decode. This format is used by AVC3 in-band streams (e.g. BBC iPlayer DASH) where SPS/PPS are carried in-band rather than in the codec private data. Fix by: - Detecting avcC with 0 SPS in Session::UpdateStream() and FragmentedSampleReader::UpdateSampleDescription(), and clearing extradata so the V4L2 decoder skips h264_xd_copy() - Adding AVCCodecHandler::Transform() to convert NALU length-prefixed packets (avcC framing) to Annex B start-code framing at decode time - Setting the AnnexB transform flag for all H.264 AVC1-4 formats (with and without SPS), ensuring consistent packet-level conversion
|
Decision: APPROVE kodiai responseDecision: APPROVE Evidence:
Review Details
Files not fully reviewed (sorted by risk score)
|
8 participants
Summary
On platforms using V4L2 mem2mem H.264 decoding (e.g. Raspberry Pi with OSMC's custom FFmpeg), the decoder's
h264_xd_copy()function rejects avcC extradata with 0 SPS/0 PPS entries, causing a fallback to software decode. This format is used by AVC3 in-band streams (e.g. BBC iPlayer DASH) where SPS/PPS are carried in-band rather than in the codec private data.Problem
AVC3 streams provide an avcC record like
01 64 00 1f ff e0 00— 7 bytes with 0 SPS and 0 PPS. The V4L2 mem2mem decoder's init function (h264_xd_copy) tries to parse SPS/PPS from this record and fails withAVERROR(EINVAL), causing the decoder to fall back to software decode.Fix
Three-part approach:
Clear extradata for 0-SPS avcC — In both
Session::UpdateStream()andFragmentedSampleReader::UpdateSampleDescription(), detect avcC records with 0 SPS and set extradata to null. This preventsh264_xd_copyfrom being called, allowing the V4L2 decoder to open successfully (it receives SPS/PPS from in-band NALUs instead).Convert avcC extradata to Annex B — For avcC records that do contain SPS, convert to Annex B format. This is already done for DRM streams (
SSD_ANNEXB_REQUIRED), but non-DRM AVC1-4 streams now also get this treatment, which is required for V4L2/m2mem compatibility.Packet-level Annex B transform — Add
AVCCodecHandler::Transform()override that replaces NALU length prefixes (1/2/4 byte avcC framing) with Annex B start codes (00 00 00 01). This is needed because when extradata is cleared or converted, the downstream decoder expects Annex B framed packets. TheTransform()virtual already exists inCodecHandlerand is called byFragmentedSampleReader::ReadSample().Changes
src/Session.cpp— RestructureUpdateStream()H.264 extradata handling: DRM+SPS → Annex B conversion; avcC with 0 SPS → clear extradata; avcC with SPS → Annex B conversion; non-avcC → pass through. Added error handling for failedAvcToAnnexb()calls.src/codechandler/AVCCodecHandler.h— AddTransform()override andSetAnnexBTransformNeeded()/m_needAnnexBTransformmembers.src/codechandler/AVCCodecHandler.cpp— ImplementTransform(): iterates buffer replacing NALU length prefixes with00 00 00 01start codes.src/samplereader/FragmentedSampleReader.cpp— Add AVC1-4 handling inUpdateSampleDescription(): 0-SPS → clear extradata + enable transform; with SPS →ExtraDataToAnnexB()+ enable transform; failed conversion → clear extradata + enable transform.Testing
Tested on Raspberry Pi 4 (OSMC 2026.05-1, kernel 5.15.92, Kodi 21.3 Omega) with BBC iPlayer DASH streams: