Skip to content

Add support for VP8 and VP9 video encoders#6769

Open
pngocthach wants to merge 1 commit into
Genymobile:masterfrom
pngocthach:feat-vp8-vp9-support
Open

Add support for VP8 and VP9 video encoders#6769
pngocthach wants to merge 1 commit into
Genymobile:masterfrom
pngocthach:feat-vp8-vp9-support

Conversation

@pngocthach
Copy link
Copy Markdown

@pngocthach pngocthach commented Apr 13, 2026

Summary

This PR adds support for VP8 and VP9 video codecs. This is primarily intended as a fallback for devices where the manufacturer has intentionally disabled H.264 (AVC) and H.265 (HEVC) encoders in the firmware.

The Problem (Onyx Boox Note X case)

On recent Onyx Boox devices (e.g., Note X with current firmware), the vendor has disabled both hardware and software H.264/H.265 encoders. This makes scrcpy unusable as it cannot find any compatible video encoder.

Evidence

In /vendor/etc/media_codecs.xml, all H.264/H.265 encoders are commented out:

/vendor/etc/media_codecs.xml 
<MediaCodecs>
    <Include href="media_codecs_google_audio.xml" />
    <Include href="media_codecs_google_telephony.xml" />
    <Settings>
        <Setting name="max-video-encoder-input-buffers" value="11" />
    </Settings>
    <Encoders>
        <!-- Video Hardware  -->
        <!-- Disabled: H.264/AVC encoder -->
        <!--
        <MediaCodec name="OMX.qcom.video.encoder.avc" type="video/avc" >
            <Quirk name="requires-allocate-on-input-ports" />
            <Quirk name="requires-allocate-on-output-ports" />
            <Quirk name="requires-loaded-to-idle-after-allocation" />
            <Limit name="size" min="128x128" max="1920x1088" />
            <Limit name="alignment" value="2x2" />
            <Limit name="block-size" value="16x16" />
            <Limit name="blocks-per-second" min="64" max="244800" />
            <Limit name="bitrate" range="1-60000000" />
            <Limit name="frame-rate" range="1-120" />
            <Limit name="concurrent-instances" max="16" />
            <Limit name="performance-point-1920x1080" value="30" />
            <Limit name="performance-point-1280x720" value="60" />
            <Limit name="performance-point-720x480" value="120" />
        </MediaCodec>
        -->
        <!-- Disabled: HEVC encoder -->
        <!--
        <MediaCodec name="OMX.qcom.video.encoder.hevc" type="video/hevc" >
            <Quirk name="requires-allocate-on-input-ports" />
            <Quirk name="requires-allocate-on-output-ports" />
            <Quirk name="requires-loaded-to-idle-after-allocation" />
            <Limit name="size" min="128x128" max="1920x1088" />
            <Limit name="alignment" value="2x2" />
            <Limit name="block-size" value="16x16" />
            <Limit name="blocks-per-second" min="64" max="244800" />
            <Limit name="bitrate" range="1-60000000" />
            <Limit name="frame-rate" range="1-120" />
            <Limit name="concurrent-instances" max="16" />
            <Limit name="quality" range="0-100" default="80" />
            <Feature name="bitrate-modes" value="VBR,CBR" />
            <Limit name="performance-point-1920x1080" value="30" />
            <Limit name="performance-point-1280x720" value="60" />
            <Limit name="performance-point-720x480" value="120" />
        </MediaCodec>
        -->
        <!-- Disabled: HEVC CQ encoder -->
        <!--
        <MediaCodec name="OMX.qcom.video.encoder.hevc.cq" type="video/hevc" >
                <Quirk name="requires-allocate-on-input-ports" />
                <Quirk name="requires-allocate-on-input-ports" />
                <Quirk name="requires-allocate-on-output-ports" />
                <Quirk name="requires-loaded-to-idle-after-allocation" />
                <Limit name="size" min="128x128" max="512x512" />
                <Limit name="frame-rate" range="1-20" />
                <Limit name="concurrent-instances" max="16" />
                <Limit name="quality" range="0-100" default="80" />
                <Feature name="bitrate-modes" value="CQ" />
                <Limit name="performance-point-512x512" value="2025" />
        </MediaCodec>
        -->
        <!-- Disabled: HEIC encoder -->
        <!--
        <MediaCodec name="OMX.qcom.video.encoder.heic" type="image/vnd.android.heic" >
            <Quirk name="requires-allocate-on-input-ports" />
            <Quirk name="requires-allocate-on-input-ports" />
            <Quirk name="requires-allocate-on-output-ports" />
            <Quirk name="requires-loaded-to-idle-after-allocation" />
            <Limit name="size" min="512x512" max="8192x8192" />
            <Limit name="frame-rate" range="1-20" />
            <Limit name="concurrent-instances" max="6" />
            <Limit name="quality" range="0-100" default="80" />
            <Feature name="bitrate-modes" value="CQ" />
            <Limit name="performance-point-8192x4320" value="3" />
            <Limit name="performance-point-1920x1080" value="6" />
        </MediaCodec>
        -->
        <!-- Video Software -->
        <!-- Disabled: H.263 encoder -->
        <!--
        <MediaCodec name="OMX.qcom.video.encoder.h263sw" type="video/3gpp" >
            <Quirk name="requires-allocate-on-input-ports" />
            <Quirk name="requires-allocate-on-output-ports" />
            <Quirk name="requires-loaded-to-idle-after-allocation" />
            <Limit name="size" min="96x96" max="864x480" />
            <Limit name="alignment" value="4x4" />
            <Limit name="block-size" value="16x16" />
            <Limit name="blocks-per-second" min="36" max="48600" />
            <Limit name="bitrate" range="1-2000000" />
            <Limit name="frame-rate" range="1-30" />
            <Limit name="concurrent-instances" max="3" />
            <Limit name="performance-point-720x480" value="30" />
        </MediaCodec>
        -->
        <!-- Disabled: MPEG-4 encoder -->
        <!--
        <MediaCodec name="OMX.qcom.video.encoder.mpeg4sw" type="video/mp4v-es" >
             <Quirk name="requires-allocate-on-input-ports" />
             <Quirk name="requires-allocate-on-output-ports" />
             <Quirk name="requires-loaded-to-idle-after-allocation" />
             <Limit name="size" min="96x96" max="864x480" />
             <Limit name="alignment" value="2x2" />
             <Limit name="block-size" value="16x16" />
             <Limit name="blocks-per-second" min="36" max="48600" />
             <Limit name="bitrate" range="1-8000000" />
             <Limit name="frame-rate" range="1-30" />
             <Limit name="concurrent-instances" max="3" />
             <Limit name="performance-point-720x480" value="30" />
        </MediaCodec>
        -->
    </Encoders>
    <Decoders>
       <!-- Video Hardware  -->
        <MediaCodec name="OMX.qcom.video.decoder.avc" type="video/avc" >
            <Quirk name="requires-allocate-on-input-ports" />
            <Quirk name="requires-allocate-on-output-ports" />
            <Limit name="size" min="128x128" max="1920x1088" />
            <Limit name="alignment" value="2x2" />
            <Limit name="block-size" value="16x16" />
            <Limit name="blocks-per-second" min="64" max="244800" />
            <Limit name="bitrate" range="1-60000000" />
            <Limit name="frame-rate" range="1-120" />
            <Feature name="adaptive-playback" />
            <Limit name="concurrent-instances" max="16" />
            <Limit name="performance-point-1920x1080" value="30" />
            <Limit name="performance-point-1280x720" value="60" />
            <Limit name="performance-point-720x480" value="120" />
        </MediaCodec>
        <MediaCodec name="OMX.qcom.video.decoder.avc.secure" type="video/avc" >
            <Quirk name="requires-allocate-on-input-ports" />
            <Quirk name="requires-allocate-on-output-ports" />
            <Limit name="size" min="128x128" max="1920x1088" />
            <Limit name="alignment" value="2x2" />
            <Limit name="block-size" value="16x16" />
            <Limit name="blocks-per-second" min="64" max="244800" />
            <Limit name="bitrate" range="1-35000000" />
            <Limit name="frame-rate" range="1-60" />
            <Feature name="adaptive-playback" />
            <Feature name="secure-playback" required="true" />
            <Limit name="concurrent-instances" max="3" />
            <Limit name="performance-point-1920x1080" value="30" />
        </MediaCodec>
        <MediaCodec name="OMX.qcom.video.decoder.vp9" type="video/x-vnd.on2.vp9" >
            <Quirk name="requires-allocate-on-input-ports" />
            <Quirk name="requires-allocate-on-output-ports" />
            <Limit name="size" min="128x128" max="1920x1088" />
            <Limit name="alignment" value="2x2" />
            <Limit name="block-size" value="16x16" />
            <Limit name="blocks-per-second" min="64" max="244800" />
            <Limit name="bitrate" range="1-60000000" />
            <Limit name="frame-rate" range="1-120" />
            <Feature name="adaptive-playback" />
            <Limit name="concurrent-instances" max="6" />
            <Limit name="performance-point-1920x1080" value="30" />
            <Limit name="performance-point-1280x720" value="60" />
            <Limit name="performance-point-720x480" value="120" />
        </MediaCodec>
        <MediaCodec name="OMX.qcom.video.decoder.vp9.secure" type="video/x-vnd.on2.vp9" >
            <Quirk name="requires-allocate-on-input-ports" />
            <Quirk name="requires-allocate-on-output-ports" />
            <Limit name="size" min="128x128" max="1920x1088" />
            <Limit name="alignment" value="2x2" />
            <Limit name="block-size" value="16x16" />
            <Limit name="blocks-per-second" min="64" max="244800" />
            <Limit name="bitrate" range="1-35000000" />
            <Limit name="frame-rate" range="1-60" />
            <Feature name="adaptive-playback" />
            <Feature name="secure-playback" required="true" />
            <Limit name="concurrent-instances" max="3" />
            <Limit name="performance-point-1920x1080" value="30" />
        </MediaCodec>
        <!-- Disabled: HEVC decoder -->
        <!--
        <MediaCodec name="OMX.qcom.video.decoder.hevc" type="video/hevc" >
            <Quirk name="requires-allocate-on-input-ports" />
            <Quirk name="requires-allocate-on-output-ports" />
            <Limit name="size" min="128x128" max="1920x1088" />
            <Limit name="alignment" value="2x2" />
            <Limit name="block-size" value="16x16" />
            <Limit name="blocks-per-second" min="64" max="244800" />
            <Limit name="bitrate" range="1-60000000" />
            <Limit name="frame-rate" range="1-120" />
            <Feature name="adaptive-playback" />
            <Limit name="concurrent-instances" max="16" />
            <Limit name="performance-point-1920x1080" value="30" />
            <Limit name="performance-point-1280x720" value="60" />
            <Limit name="performance-point-720x480" value="120" />
        </MediaCodec>
        -->
        <!-- Disabled: HEVC secure decoder -->
        <!--
        <MediaCodec name="OMX.qcom.video.decoder.hevc.secure" type="video/hevc" >
            <Quirk name="requires-allocate-on-input-ports" />
            <Quirk name="requires-allocate-on-output-ports" />
            <Limit name="size" min="128x128" max="1920x1088" />
            <Limit name="alignment" value="2x2" />
            <Limit name="block-size" value="16x16" />
            <Limit name="blocks-per-second" min="64" max="244800" />
            <Limit name="bitrate" range="1-35000000" />
            <Limit name="frame-rate" range="1-60" />
            <Feature name="adaptive-playback" />
            <Feature name="secure-playback" required="true" />
            <Limit name="concurrent-instances" max="3" />
            <Limit name="performance-point-1920x1080" value="30" />
        </MediaCodec>
        -->
        <!-- Video Software -->
        <!-- Disabled: H.263 decoder -->
        <!--
        <MediaCodec name="OMX.qti.video.decoder.h263sw" type="video/3gpp" >
             <Quirk name="requires-allocate-on-input-ports" />
             <Quirk name="requires-allocate-on-output-ports" />
             <Limit name="size" min="96x96" max="864x480" />
             <Limit name="alignment" value="4x4" />
             <Limit name="block-size" value="16x16" />
             <Limit name="blocks-per-second" min="36" max="48600" />
             <Limit name="bitrate" range="1-16000000" />
             <Limit name="frame-rate" range="1-30" />
             <Feature name="adaptive-playback" />
             <Limit name="concurrent-instances" max="4" />
             <Limit name="performance-point-720x480" value="30" />
        </MediaCodec>
        -->
        <!-- Disabled: MPEG-4 decoder -->
        <!--
        <MediaCodec name="OMX.qti.video.decoder.mpeg4sw" type="video/mp4v-es">
             <Quirk name="requires-allocate-on-input-ports" />
             <Quirk name="requires-allocate-on-output-ports" />
             <Limit name="size" min="96x96" max="1920x1088" />
             <Limit name="alignment" value="2x2" />
             <Limit name="block-size" value="16x16" />
             <Limit name="blocks-per-second" min="36" max="244800" />
             <Limit name="bitrate" range="1-40000000" />
             <Limit name="frame-rate" range="1-30" />
             <Limit name="concurrent-instances" max="4" />
             <Limit name="performance-point-1920x1080" value="30" />
        </MediaCodec>
	-->
    </Decoders>
    <Include href="media_codecs_google_video.xml" />
</MediaCodecs>

Even the default Android software H.264 encoder in media_codecs_google_video.xml is disabled. Only VP8 and VP9 software encoders remain available:

media_codecs_google_video.xml 
<Included>
    <Decoders>
        <!-- Disabled: MPEG-4 decoder -->
        <!--
        <MediaCodec name="OMX.google.mpeg4.decoder" type="video/mp4v-es">
            <Limit name="size" min="2x2" max="352x288" />
            <Limit name="alignment" value="2x2" />
            <Limit name="block-size" value="16x16" />
            <Limit name="blocks-per-second" range="12-11880" />
            <Limit name="bitrate" range="1-384000" />
            <Feature name="adaptive-playback" />
        </MediaCodec>
        -->
        <!-- Disabled: H.263 decoder -->
        <!--
        <MediaCodec name="OMX.google.h263.decoder" type="video/3gpp">
            <Limit name="size" min="2x2" max="352x288" />
            <Limit name="alignment" value="2x2" />
            <Limit name="bitrate" range="1-384000" />
            <Feature name="adaptive-playback" />
        </MediaCodec>
        -->
        <MediaCodec name="OMX.google.h264.decoder" type="video/avc">
            <Limit name="size" min="2x2" max="4080x4080" />
            <Limit name="alignment" value="2x2" />
            <Limit name="block-size" value="16x16" />
            <Limit name="block-count" range="1-32768" />
            <Limit name="blocks-per-second" range="1-1966080" />
            <Limit name="bitrate" range="1-48000000" />
            <Feature name="adaptive-playback" />
        </MediaCodec>
        <!-- Disabled: HEVC decoder -->
        <!--
        <MediaCodec name="OMX.google.hevc.decoder" type="video/hevc">
            <Limit name="size" min="2x2" max="4096x4096" />
            <Limit name="alignment" value="2x2" />
            <Limit name="block-size" value="8x8" />
            <Limit name="block-count" range="1-196608" />
            <Limit name="blocks-per-second" range="1-2000000" />
            <Limit name="bitrate" range="1-10000000" />
            <Feature name="adaptive-playback" />
        </MediaCodec>
        -->
        <MediaCodec name="OMX.google.vp8.decoder" type="video/x-vnd.on2.vp8">
            <Limit name="size" min="2x2" max="2048x2048" />
            <Limit name="alignment" value="2x2" />
            <Limit name="block-size" value="16x16" />
            <Limit name="block-count" range="1-16384" />
            <Limit name="blocks-per-second" range="1-1000000" />
            <Limit name="bitrate" range="1-40000000" />
            <Feature name="adaptive-playback" />
        </MediaCodec>
        <MediaCodec name="OMX.google.vp9.decoder" type="video/x-vnd.on2.vp9">
            <Limit name="size" min="2x2" max="2048x2048" />
            <Limit name="alignment" value="2x2" />
            <Limit name="block-size" value="16x16" />
            <Limit name="block-count" range="1-16384" />
            <Limit name="blocks-per-second" range="1-500000" />
            <Limit name="bitrate" range="1-40000000" />
            <Feature name="adaptive-playback" />
        </MediaCodec>
    </Decoders>

    <Encoders>
        <!-- Disabled: H.263 encoder -->
        <!--
        <MediaCodec name="OMX.google.h263.encoder" type="video/3gpp">
            <Limit name="size" min="176x144" max="176x144" />
            <Limit name="alignment" value="16x16" />
            <Limit name="bitrate" range="1-128000" />
        </MediaCodec>
        -->
        <!-- Disabled: H.264/AVC encoder -->
        <!--
        <MediaCodec name="OMX.google.h264.encoder" type="video/avc">
            <Limit name="size" min="16x16" max="2048x2048" />
            <Limit name="alignment" value="2x2" />
            <Limit name="block-size" value="16x16" />
            <Limit name="block-count" range="1-8192" />
            <Limit name="blocks-per-second" range="1-245760" />
            <Limit name="bitrate" range="1-12000000" />
            <Feature name="intra-refresh" />
        </MediaCodec>
        -->
        <!-- Disabled: MPEG-4 encoder -->
        <!--
        <MediaCodec name="OMX.google.mpeg4.encoder" type="video/mp4v-es">
            <Limit name="size" min="16x16" max="176x144" />
            <Limit name="alignment" value="16x16" />
            <Limit name="block-size" value="16x16" />
            <Limit name="blocks-per-second" range="12-1485" />
            <Limit name="bitrate" range="1-64000" />
        </MediaCodec>
        -->
        <MediaCodec name="OMX.google.vp8.encoder" type="video/x-vnd.on2.vp8">
            <!-- profiles and levels:  ProfileMain : Level_Version0-3 -->
            <Limit name="size" min="2x2" max="2048x2048" />
            <Limit name="alignment" value="2x2" />
            <Limit name="block-size" value="16x16" />
            <!-- 2016 devices can encode at about 10fps at this block count -->
            <Limit name="block-count" range="1-16384" />
            <Limit name="bitrate" range="1-40000000" />
            <Feature name="bitrate-modes" value="VBR,CBR" />
        </MediaCodec>
        <MediaCodec name="OMX.google.vp9.encoder" type="video/x-vnd.on2.vp9">
            <!-- profiles and levels:  ProfileMain : Level_Version0-3 -->
            <Limit name="size" min="2x2" max="2048x2048" />
            <Limit name="alignment" value="2x2" />
            <Limit name="block-size" value="16x16" />
            <!-- 2016 devices can encode at about 8fps at this block count -->
            <Limit name="block-count" range="1-3600" /> <!-- max 1280x720 -->
            <Limit name="bitrate" range="1-40000000" />
            <Feature name="bitrate-modes" value="VBR,CBR" />
        </MediaCodec>
    </Encoders>
</Included>

Known Limitations (Latency)

Since these encoders are software-based on the affected devices, there is a noticeable increase in latency compared to hardware-accelerated H.264.

However, this latency can be significantly mitigated by reducing the stream resolution and frame rate (e.g., using --max-size=1024 --max-fps=15). In my testing on the Onyx Boox Note X, these optimizations make the device much more responsive while maintaining a stable and clear stream. Given that the alternative is a total lack of functionality, this fallback is a necessary and usable compromise.

Testing Environment

  • Host: CachyOS (Kernel 6.19.11)
  • Desktop: GNOME 50.0 (Wayland)
  • Target Device: Onyx Boox Note X (Android 11, Firmware version: 2026-02-11_13-07_4.1.1-rel_0210_8841760b67)

Fixes #6763

@pngocthach pngocthach mentioned this pull request Apr 13, 2026
Open
1 task
@danielemegna
Copy link
Copy Markdown

danielemegna commented May 19, 2026 •
edited
Loading

Please merge this, scrcpy is still currently unusable on recent Onyx Boox devices 🙏

@rom1v
Copy link
Copy Markdown
Collaborator

rom1v commented May 19, 2026 •
edited
Loading

Thank you for the reminder.

LGTM, I'll merge this probably for scrcpy 4.1.

@rom1v
Copy link
Copy Markdown
Collaborator

rom1v commented May 19, 2026

I made the following changes:

  • enable VP8 and VP9 decoders in FFmpeg (in your commit directly)
  • update develop.md to mention VP8 and VP9 (in your commit directly)
  • fix VP8 recording (contrary to the other codecs, it does not produce config packets)
  • reject recording VP8 in MP4 (it's not supported)

Please test/review the vp9 branch.

@pngocthach
Copy link
Copy Markdown
Author

pngocthach commented May 20, 2026

Thanks for the updates. I tested the vp9 branch on Onyx Boox Note X (Android 11), and found two issues.

Available encoders:

--video-codec=vp8 --video-encoder=c2.android.vp8.encoder          (sw)
--video-codec=vp8 --video-encoder=OMX.google.vp8.encoder          (sw) (alias for c2.android.vp8.encoder)
--video-codec=vp9 --video-encoder=c2.android.vp9.encoder          (sw)
--video-codec=vp9 --video-encoder=OMX.google.vp9.encoder          (sw) (alias for c2.android.vp9.encoder)

VP8 works and is usable:

./run build-auto --video-codec=vp8 --max-size=1600 --video-bit-rate=32M --max-fps=5
INFO: Texture: 1600x1200

VP8 recording to MKV also works:

./run build-auto --video-codec=vp8 --record=file-vp8.mkv

VP9 is constrained to a very low resolution

VP9 streaming starts, but on the vp9 branch it is constrained to a very low resolution:

./run build-auto --video-codec=vp9 --max-size=1600 --video-bit-rate=32M --max-fps=5
INFO: Texture: 596x448

This makes the stream very blurry.

I added temporary debug logs around VideoCapabilities. The runtime reports:

[server] DEBUG: Using video encoder: 'c2.android.vp9.encoder'
[server] DEBUG: Video codec supported widths: [2, 2048]
[server] DEBUG: Video codec supported heights: [2, 2048]
[server] DEBUG: Video codec max landscape size: 2048x448
[server] DEBUG: Video codec max portrait size: 448x2048

For comparison, my original branch without the new encoder maximum size constraint streams VP9 at 1600x1200 with the same device, encoder and options:

./run build-auto-oldcheck --video-codec=vp9 --max-size=1600 --video-bit-rate=32M --max-fps=5
INFO: Texture: 1600x1200

So this looks like a regression related to the new encoder size constraint logic from #6766. For this VP9 encoder, it computes a max landscape size of 2048x448, which is very restrictive for this 4:3 display.

VP9 recording to MKV fails

VP9 recording to MKV fails:

./run build-auto --video-codec=vp9 --record=file.mkv
ERROR: The first video packet is not a config packet
ERROR: Recording failed to file.mkv
ERROR: Recorder error

This looks similar to the VP8 config-packet issue fixed in c0fe9d8. I tested this local change:

recorder->video_expects_config_packet =
    ctx->codec_id != AV_CODEC_ID_VP8
    && ctx->codec_id != AV_CODEC_ID_VP9;

With this change, VP9 recording works.

@rom1v
Copy link
Copy Markdown
Collaborator

rom1v commented May 20, 2026 •
edited
Loading

So this looks like a regression related to the new encoder size constraint logic from #6766.

Some encoders (especially software encoders) report incorrect capabilities. #6849 adds an option --ignore-video-encoder-constraints.

ERROR: The first video packet is not a config packet

Oh, that does not happen with my device (Pixel 8) when recording to VP9. But not expecting a config packet also works, so I will make that change.

Thank you.

@anotheruserofgithub anotheruserofgithub mentioned this pull request May 20, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Encoder support for new Onyx Devices

3 participants

@pngocthach @danielemegna @rom1v