Skip to content

Fix H.264 HW decode for AVC3 in-band streams (V4L2/m2m)#2034

Closed
scottrix wants to merge 1 commit into
xbmc:Omegafrom
scottrix:fix/avc3-v4l2-hw-decode
Closed

Fix H.264 HW decode for AVC3 in-band streams (V4L2/m2m)#2034
scottrix wants to merge 1 commit into
xbmc:Omegafrom
scottrix:fix/avc3-v4l2-hw-decode

Conversation

@scottrix

@scottrix scottrix commented May 19, 2026

Copy link
Copy Markdown

Summary

On platforms using V4L2 mem2mem H.264 decoding (e.g. Raspberry Pi with OSMC's custom FFmpeg), the decoder's h264_xd_copy() function rejects avcC extradata with 0 SPS/0 PPS entries, causing a fallback to software decode. This format is used by AVC3 in-band streams (e.g. BBC iPlayer DASH) where SPS/PPS are carried in-band rather than in the codec private data.

Problem

AVC3 streams provide an avcC record like 01 64 00 1f ff e0 00 — 7 bytes with 0 SPS and 0 PPS. The V4L2 mem2mem decoder's init function (h264_xd_copy) tries to parse SPS/PPS from this record and fails with AVERROR(EINVAL), causing the decoder to fall back to software decode.

Fix

Three-part approach:

  1. Clear extradata for 0-SPS avcC — In both Session::UpdateStream() and FragmentedSampleReader::UpdateSampleDescription(), detect avcC records with 0 SPS and set extradata to null. This prevents h264_xd_copy from being called, allowing the V4L2 decoder to open successfully (it receives SPS/PPS from in-band NALUs instead).

  2. Convert avcC extradata to Annex B — For avcC records that do contain SPS, convert to Annex B format. This is already done for DRM streams (SSD_ANNEXB_REQUIRED), but non-DRM AVC1-4 streams now also get this treatment, which is required for V4L2/m2mem compatibility.

  3. Packet-level Annex B transform — Add AVCCodecHandler::Transform() override that replaces NALU length prefixes (1/2/4 byte avcC framing) with Annex B start codes (00 00 00 01). This is needed because when extradata is cleared or converted, the downstream decoder expects Annex B framed packets. The Transform() virtual already exists in CodecHandler and is called by FragmentedSampleReader::ReadSample().

Changes

  • src/Session.cpp — Restructure UpdateStream() H.264 extradata handling: DRM+SPS → Annex B conversion; avcC with 0 SPS → clear extradata; avcC with SPS → Annex B conversion; non-avcC → pass through. Added error handling for failed AvcToAnnexb() calls.
  • src/codechandler/AVCCodecHandler.h — Add Transform() override and SetAnnexBTransformNeeded()/m_needAnnexBTransform members.
  • src/codechandler/AVCCodecHandler.cpp — Implement Transform(): iterates buffer replacing NALU length prefixes with 00 00 00 01 start codes.
  • src/samplereader/FragmentedSampleReader.cpp — Add AVC1-4 handling in UpdateSampleDescription(): 0-SPS → clear extradata + enable transform; with SPS → ExtraDataToAnnexB() + enable transform; failed conversion → clear extradata + enable transform.

Testing

Tested on Raspberry Pi 4 (OSMC 2026.05-1, kernel 5.15.92, Kodi 21.3 Omega) with BBC iPlayer DASH streams:

  • V4L2 mem2mem H.264 decoder opens successfully
  • Video+audio playback works correctly
  • Stream adaptive bitrate switching works (960x540 → 1280x720)
  • No regression observed with non-AVC3 streams

Fix H.264 HW decode for AVC3 in-band streams (V4L2/m2m)
ed23ebc 
On platforms using V4L2 mem2mem H.264 decoding (e.g. Raspberry Pi with
OSMC's custom FFmpeg), the decoder's h264_xd_copy() function rejects
avcC extradata with 0 SPS/0 PPS entries, causing a fallback to software
decode. This format is used by AVC3 in-band streams (e.g. BBC iPlayer
DASH) where SPS/PPS are carried in-band rather than in the codec
private data.

Fix by:
- Detecting avcC with 0 SPS in Session::UpdateStream() and
  FragmentedSampleReader::UpdateSampleDescription(), and clearing
  extradata so the V4L2 decoder skips h264_xd_copy()
- Adding AVCCodecHandler::Transform() to convert NALU length-prefixed
  packets (avcC framing) to Annex B start-code framing at decode time
- Setting the AnnexB transform flag for all H.264 AVC1-4 formats
  (with and without SPS), ensuring consistent packet-level conversion
@kodiai

kodiai Bot commented May 19, 2026 •
edited
Loading

Copy link
Copy Markdown

Decision: APPROVE

kodiai response

Decision: APPROVE
Issues: none

Evidence:

  • Review prompt covered 4 changed files.
  • Review scope note: output was scoped by prompt budget limits; Review Details include bounded counts only.
Review Details

  • Review plan: ready hash=8ddc0302d7d1 route=standard task=review.full files=4 lines=133(local-diff) budget=25t/766s gates=3/3 publish=canonical-visible-surface graph=skipped candidates=preferred doctrine=disabled/0/0/0 reasons=disabled

  • Review reducer: ready input=3 kept=3 suppressed=0 rewritten=0 deprioritized=0 lowConfidence=0 auditEvents=0 severityDemoted=0 graphValidated=0 graphUncertain=0 doctrine=disabled/0/0/0 reasons=disabled

  • Review candidates: shadow recorded=3 rejected=0 errors=0 artifact=present repo=xbmc-inputstream.adaptive pr=2034 key=kodiai-review-output:v1:inst-109141824:xbmc-inputstream.adaptive:pr-2034:action- delivery=38cbfcd0-53d8-11f1-8f88-1e9bb78d7519

  • Review candidate publication: mode=blocked approved=3 rewritten=0 published=0 directFallback=0 reasons=none

  • Files reviewed: 4

  • Findings: 0 critical, 0 major, 0 medium, 0 minor

  • Lines changed: +123 -10

  • Profile: balanced (auto, lines changed: 133)

  • Contributor experience: coarse-fallback (using coarse fallback signals only)

  • Shadow specialist: lane=docs-config-truth status=skipped reason=no-operator-truth-paths candidateCount=0 decisionCount=0 decisionCounts=candidate:0,duplicate:0,disagreement:0,dismissed:0,unclassifiable:0 duplicateCount=0 disagreementCount=0 dismissedCount=0 unclassifiableCount=0 truncatedCandidateCount=0 metricAvailability=token:n,cost:n,latency:n visiblePublicationDenied=true approvalPublicationDenied=true privateOnly=true shadowOnly=true redacted=raw:n,publication:n,approval:n,unsafe:0 correlationKey=e612cfc3e3c6d030 deliveryId=38cbfcd0-53d8-11f1-8f88-1e9bb78… reviewOutputKey=kodiai-review-output:v1:inst-10…

  • M072 candidate publication bridge: status=denied; bridgeVersion=candidate-publication-bridge.v1; bridgeId=candidate-publication-record:4c276440c85cdb1011e0c5e183e4c47c; recordKey=candidate-publication-record:4c276440c85cdb1011e0c5e183e4c47c; correlationKey=candidate-publication-bridge:a1767f97abf5367385490e8f2cd5e0b7; source=review-handler-publication; candidateRef=candidate-publication-summary-eb14196e; verification=none; counts=candidateCount:0,evidenceCount:0,verifiedCount:0,partiallyVerifiedCount:0,unverifiedCount:0,disprovenCount:0,publicationEligibleCount:0,malformedRecordCount:0,unsafeInputFieldCount:0; reasons=no-evidence,publication-ineligible; malformed=none; presence=deliveryId:y,reviewOutputKey:y,upstreamCorrelationKey:y,policyCorrelationKey:y; handoffOwner=available; redaction=privateOnly:y,rawPayloads:n,publicationFields:n,evidencePayloads:n,githubCommentBody:n,reducerRawPayload:n,discardedRawPayload:n,discardedPublicationFields:n,discardedEvidencePayloads:n

  • Review finding lifecycle: status=normalized; counts=input:3,recorded:3,rejected:0,unsafeInputFields:0; correlation=repo:y,pull:y,reviewOutputKey:y,deliveryId:y,commit:y; statuses=detected:3,open:3,suggested:0,validated:0,revalidated:0,resolved:0,blocked:0,degraded:0; severity=critical:0,major:2,medium:1,minor:0; actionability=actionable:0,needs-human-review:3,needs-reproduction:0,blocked:0,not-actionable:0; reasons=automatic-detected,automatic-open,automatic-review; rejected=none; redaction=privateOnly:y,rawPrompts:n,rawModelOutput:n,candidateBodies:n,toolPayloads:n,secretLike:n,diffs:n,unboundedArrays:n,unsafeFields:0

  • Review validation truth: status=normalized; counts=detected:3,suggested:0,validated:0,revalidated:0,resolved:0,blocked:0,degraded:0,open:3,uncertain:0,inputFindings:3,unsafeInputFields:0; evidence=fresh:0,stale:0,missingValidation:3,missingRevalidation:3; reasons=validation-missing:3; refs=rfl-2c6db55ba3459aec:open:validation-missing:fix:n:validation:n:revalidation:n,rfl-141379ea2d667fbc:open:validation-missing:fix:n:validation:n:revalidation:n,rfl-488d9a9345b70a2c:open:validation-missing:fix:n:validation:n:revalidation:n; correlation=reviewOutputKey:y,deliveryId:y; redaction=privateOnly:y,rawPrompts:n,rawModelOutput:n,candidateBodies:n,replacementText:n,toolPayloads:n,secretLike:n,diffs:n,unboundedArrays:n,unsafeFields:0

  • Review completed: 2026-05-19T23:21:17.587Z

  • Total wall-clock: 8m 41s

  • Phase timings:

    • queue wait: 1ms
    • workspace preparation: 1.3s
    • retrieval/context assembly: 3.5s
    • executor handoff: 36s
    • remote runtime: 7m 54s
    • publication: 2.8s

  • Tokens: 94 in / 24,313 out | 0.6129

  • Keyword parsing: No keywords detected

  • Budget behavior: scoped (prompt-budget-limited).

  • Prompt budget: 5 sections, 1 trimmed, 0 bypassed, 626 trimmed tokens.

  • Cache behavior: 2 observations, 1 hits, 1 misses, 0 degraded, 0 bypassed.

  • Continuation behavior: 0 observations, 0 compacted, 0 fallback, 0 degraded, 0 bypassed.

@CastagnaIT

CastagnaIT commented May 20, 2026

Copy link
Copy Markdown
Collaborator

Development work should be done first on the main branch, and then, if accepted, in the older branches
so i expect PR for Kodi 22 Piers
you haven't even tested Kodi 22 branches?

the way you explain it seems more like an issue that needs to be addressed at the higher level rather than here
Have you tried ask to OSMC devs about this problem?

Also, what happens with same stream on FFmpeg direct add-on? should have same problem?

@CastagnaIT

CastagnaIT commented May 20, 2026

Copy link
Copy Markdown
Collaborator

sorry if i ping you @samnazarko but i dont know exactly who ping
Is this an issue that you can solve on OSMC instead of customizing the extradata from addons?

@CastagnaIT CastagnaIT added the Don't merge PR that should not be merged (yet) label May 21, 2026
@samnazarko

samnazarko commented May 21, 2026

Copy link
Copy Markdown

Honestly, I’d advise getting access to a stream.
It could also be tested against the latest version of LE to see if this is resolved in a more modern kernel / firmware.

@CastagnaIT

CastagnaIT commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator

Talking to some LE devs, if this is not working also with last firmware/OS build (more likely best on v22)
the fix should be proposed on ffmpeg side, and probably add a temporary patch in your case for the OSMC build

Using workarounds like this that must be applied to every binary add-on is not the most appropriate way
especially since the problem is focused on a specific platform

Given the nature of the issue, i don't think it's appropriate to accept this PR, especially from a code maintenance perspective, since this is focused to a third-party Kodi OS and not the official Kodi platforms

@CastagnaIT CastagnaIT closed this Jun 15, 2026
@samnazarko

samnazarko commented Jun 15, 2026 via email

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

Don't merge PR that should not be merged (yet)

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@scottrix @CastagnaIT @samnazarko