[Core] Reference implementation for RGBtoNV12 and BGRtoNV12 operations by mostafafaheem · Pull Request #34437 · openvinotoolkit/openvino

mostafafaheem · 2026-03-02T14:57:05Z

Details:

Added 2 new operations to template plugin, RGBtoNV12 and BGRtoNV12, supporting single and two-plane conversion to NV12 format
Integrated new operation into PostProcessing pipeline
Added tests for operation and PostProcessing

Tickets:

[Good First Issue] PostProcess supports RGB->NV12 color convert function #22985

AI Assistance:

AI assistance used: yes
AI was used to give in-depth guidance about operation enabling flow and help validating code changes. All code was thoroughly reviewed and tested

mostafafaheem · 2026-03-02T15:05:22Z

Hello @riverlijunjie and @praasz, sorry for the large PR, I thought that splitting it into smaller PRs would not be suitable for the granularity of this task, however, once this PR is merged, I will submit another PR containing CPU plugin reference implementation, and hopefully extend the operation to the GPU plugin.

Copilot

Pull request overview

Adds core ops and template-plugin reference support for RGB/BGR → NV12 conversion, and wires the conversion into the postprocessing pipeline (NV12 single-plane), with accompanying functional/type/visitor tests.

Changes:

Introduce new opset16 operations RGBtoNV12 and BGRtoNV12 (single-plane and two-plane outputs) with shape inference and reference implementation.
Enable evaluation/registration in the template plugin backend and add reference/functional tests.
Extend postprocessing convert_color() to support RGB/BGR → NV12_SINGLE_PLANE, plus new preprocess/postprocess tests.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/tests/test_utils/common_test_utils/include/common_test_utils/subgraph_builders/preprocess_builders.hpp	Adds postprocess subgraph builders for RGB/BGR → NV12 single-plane.
src/plugins/template/tests/functional/subgraph_reference/preprocess.cpp	Adds template-plugin reference preprocess tests for postprocess RGB/BGR → NV12.
src/plugins/template/tests/functional/op_reference/convert_color_to_nv12.cpp	Adds op-reference tests for `RGBtoNV12`/`BGRtoNV12` (single/two-plane).
src/plugins/template/backend/opset_int_tbl.hpp	Registers new ops in the template plugin opset table.
src/plugins/template/backend/ops/ops_evaluates.hpp	Declares template-backend evaluate specializations for the new ops.
src/plugins/template/backend/ops/convert_color_to_nv12.cpp	Implements template-backend evaluation via the core reference function.
src/core/tests/visitors/op/convert_color_to_nv12.cpp	Adds visitor attribute tests for both ops and plane modes.
src/core/tests/type_prop/convert_color_to_nv12_base.hpp	Adds typed shape/type propagation tests for the new NV12 base op.
src/core/tests/type_prop/convert_color_to_nv12.cpp	Instantiates the new type-prop test suite for both ops.
src/core/tests/preprocess.cpp	Adds postprocess pipeline tests for RGB/BGR → NV12 single-plane and rejects two-plane.
src/core/src/preprocess/preprocess_steps_impl.cpp	Wires postprocess `convert_color()` to emit `RGBtoNV12`/`BGRtoNV12` for `NV12_SINGLE_PLANE`.
src/core/src/op/util/convert_color_to_nv12_base.cpp	Implements validation/type inference for the NV12 base op.
src/core/src/op/rgb_to_nv12.cpp	Implements `RGBtoNV12` op shell + cloning.
src/core/src/op/bgr_to_nv12.cpp	Implements `BGRtoNV12` op shell + cloning.
src/core/shape_inference/include/rgb_bgr_to_nv12_shape_inference.hpp	Adds shape inference for RGB/BGR → NV12 (single/two-plane).
src/core/reference/include/openvino/reference/convert_color_to_nv12.hpp	Adds reference kernel for RGB/BGR → NV12 conversion.
src/core/include/openvino/opsets/opset16_tbl.hpp	Registers new ops in core opset16 table.
src/core/include/openvino/op/util/convert_color_to_nv12_base.hpp	Introduces the new base op API/attributes for RGB/BGR → NV12.
src/core/include/openvino/op/rgb_to_nv12.hpp	Adds public op header for `RGBtoNV12`.
src/core/include/openvino/op/ops.hpp	Exposes new op headers via umbrella include.
src/core/include/openvino/op/bgr_to_nv12.hpp	Adds public op header for `BGRtoNV12`.
src/core/docs/operation_enabling_flow.md	Updates documentation path for op implementation locations.

Copilot

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated 2 comments.

p-durandin · 2026-07-13T15:15:42Z

build_jenkins

praasz · 2026-07-14T09:16:38Z

+    constructor_validate_and_infer_types();
+}
+ov::op::v17::BGRtoNV12::BGRtoNV12(const Output<Node>& arg, bool single_plane)
+    : util::ConvertColorToNV12Base(arg, util::ConvertColorToNV12Base::ColorConversion::BGR_TO_NV12, single_plane) {


There is concept of base class which defined source color format. It will more clean to keep this patter instead add ConvertColorToNV12Base

For class BGRtoNV12 the format can be set RGB_TO_NV12 which is wrong and should be not possible to set it as source is not RGB

I think this ConvertColorToNV12Base::ColorConversion can be removed entirely, since it is not accessed anywhere in the code anymore and contains no useful information. However, this will diverge from other color conversion ops.

Is it OK to remove it? @praasz

If there is no need for this enum to be set by user it should be removed from public interface.

Done f563812

praasz · 2026-07-14T10:07:30Z

    return std::make_shared<ov::Model>(results, ParameterVector{params}, "ConvertColorGraph");
 }

+std::shared_ptr<ov::Model> generateConvertColorToNV12(const std::shared_ptr<ov::op::Op> &node) {


Why generateConvertColor cannot be expanded?

Because RGB input has a channel dim, unlike I420 and NV12, so params in generateConvertColor wouldn't work with RGBtoNV12 and BGRtoNV12.

Co-authored-by: Pawel Raasz <pawel.raasz@intel.com>

mostafafaheem · 2026-07-14T23:22:06Z

@praasz Thanks for the diligent review. All comments were addressed. I provided commit links under each comment for easier review (except namespace related changes, they are in 320f11d) and asked clarifying questions if needed. Please have a look whenever you have the time :)

cc @mitruska

p-durandin · 2026-07-15T05:18:50Z

build_jenkins

p-wysocki

Thank you for your PR, it's very well done. I only had minor comments, mostly about tests. For other reviewers, the scope of my changes was shape infer, spec, type prop tests, reference tests, reference implementation.

p-wysocki · 2026-07-15T08:15:40Z

@@ -0,0 +1,129 @@
+// Copyright (C) 2018-2026 Intel Corporation


[random spot]

Can we get a multibatch test? There's some logic in reference implementation which is not tested when N=1

The inputs are monochrome, can we switch at least one of the tests to have different colors in the 2x2 pixel blocks? While we're at it, also all tests assume 2x2 blocks, can we test bigger ones as well?

Since we already have NV12toRGB and NV12toBGR operators, can we get a round-trip tests, such as NV12 -> RGB -> NV12? It would explicitly assert that the operation is correct.

Thank you for these suggestions, 0ba5bfa added the following:

Multi-batch RGB to single-plane NV12

Multi-batch BGR to two-plane NV12

Mixed primary colors RGB to single-plane NV12

Mixed primary colors BGR to two-plane NV12

Mixed RGB values to single-plane NV12

Mixed RGB values to two-plane NV12

4x4 RGB to single-plane NV12

4x4 BGR to two-plane NV12

Round-trip NV12 to RGB to NV12

Round-trip NV12 to BGR to NV12

All pass locally. I used a python script to calculate the expected outputs for each using the RGB -> YUV formula, I can share if you want.

Co-authored-by: Przemyslaw Wysocki <przemyslaw.wysocki@intel.com>

mostafafaheem · 2026-07-15T12:24:14Z

@p-wysocki I really appreciate it, and thanks for the review😄 Always glad to be contributing

p-wysocki · 2026-07-16T08:40:02Z

Some builds are failing due to

/__w/openvino/openvino/openvino/src/core/reference/include/openvino/reference/convert_color_to_nv12.hpp:18:36: error: no member named 'min' in namespace 'std'; did you mean 'fmin'?
   18 |         return static_cast<T>(std::min(std::max(std::round(a), 0.0), 255.0));

Overall LGTM, thank you for your PR. Let's wait for more reviews, since this PR is quite big, I'm only covering a part of it. My approve is for the scope of shape infer, spec, type prop tests, reference tests and reference implementation.

@riverlijunjie do you know if there's a model we can use to test the changes?

p-wysocki · 2026-07-16T09:36:27Z

build_jenkins

mostafafaheem requested review from a team as code owners March 2, 2026 14:57

github-actions Bot added category: Core OpenVINO Core (aka ngraph) category: IE Tests OpenVINO Test: plugins and common category: TEMPLATE OpenVINO Template plugin category: CPP API OpenVINO CPP API bindings labels Mar 2, 2026

sys-openvino-ci added the ExternalPR External contributor label Mar 2, 2026

mostafafaheem changed the title ~~Reference implementation for NV12toRGB and NV12toBGR operations~~ [Core] Reference implementation for NV12toRGB and NV12toBGR operations Mar 2, 2026

mostafafaheem changed the title ~~[Core] Reference implementation for NV12toRGB and NV12toBGR operations~~ [Core] Reference implementation for RGBtoNV12 and BGRtoNV12 operations Mar 2, 2026

mostafafaheem requested a review from a team as a code owner March 2, 2026 15:10

mostafafaheem requested review from zKulesza and removed request for a team March 2, 2026 15:10

github-actions Bot added the category: docs OpenVINO documentation label Mar 2, 2026

mlukasze requested review from Copilot, mitruska and mmikolajcz March 3, 2026 05:46

Copilot started reviewing on behalf of mlukasze March 3, 2026 05:47 View session

Copilot AI reviewed Mar 3, 2026

View reviewed changes

Comment thread src/core/include/openvino/op/util/convert_color_to_nv12_base.hpp Outdated

mlukasze requested a review from Copilot March 4, 2026 05:52

Copilot started reviewing on behalf of mlukasze March 4, 2026 05:53 View session

Copilot AI reviewed Mar 4, 2026

View reviewed changes

Comment thread src/core/src/preprocess/preprocess_steps_impl.cpp Outdated

Comment thread src/plugins/template/tests/functional/op_reference/convert_color_to_nv12.cpp

mostafafaheem requested review from a team as code owners March 4, 2026 21:17

Merge branch 'master' into rgb_bgr_to_nv12_template_ref

3dc6dd6

praasz reviewed Jul 14, 2026

View reviewed changes

mostafafaheem and others added 12 commits July 14, 2026 13:35

Merge branch 'master' into rgb_bgr_to_nv12_template_ref

688bdab

Apply suggestions from code review

5a33d4b

Co-authored-by: Pawel Raasz <pawel.raasz@intel.com>

Apply namespace-related changes

320f11d

Remove using namespace reference_tests

a500d10

Use recommended utilities

a0b2f71

Avoid move in shape_infer

e8f96ec

Make is_type_supported local helper in op file

d24ba15

Remove unnecessary parameters in reference

868c780

Use template property to avoid passing op type

20e3f40

Use make_tuple

2931ba7

Generalize rgb_pixel_to_yuv helper

7e42f96

Merge branch 'master' into rgb_bgr_to_nv12_template_ref

5ff0b64

p-wysocki reviewed Jul 15, 2026

View reviewed changes

mostafafaheem and others added 4 commits July 15, 2026 13:46

Update docs

e78b880

Co-authored-by: Przemyslaw Wysocki <przemyslaw.wysocki@intel.com>

Remove redundant casts

66ad1de

Add tests

0ba5bfa

Fix code style

f3b01d2

p-wysocki approved these changes Jul 16, 2026

View reviewed changes

mostafafaheem added 2 commits July 16, 2026 12:30

Avoid using transitive include for <algorithm> in reference header

b052f2c

Merge branch 'master' into rgb_bgr_to_nv12_template_ref

871b08f

mostafafaheem added 2 commits July 17, 2026 16:12

Remove ColorConversion enum

f563812

Merge branch 'master' into rgb_bgr_to_nv12_template_ref

dba2914

		@@ -0,0 +1,129 @@
		// Copyright (C) 2018-2026 Intel Corporation

Uh oh!

Conversation

mostafafaheem commented Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Details:

Tickets:

AI Assistance:

Uh oh!

mostafafaheem commented Mar 2, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

p-durandin commented Jul 13, 2026

Uh oh!

Uh oh!

praasz Jul 14, 2026

Choose a reason for hiding this comment

Uh oh!

mostafafaheem Jul 14, 2026

Choose a reason for hiding this comment

Uh oh!

praasz Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

mostafafaheem Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

praasz Jul 14, 2026

Choose a reason for hiding this comment

Uh oh!

mostafafaheem Jul 14, 2026

Choose a reason for hiding this comment

Uh oh!

mostafafaheem commented Jul 14, 2026

Uh oh!

p-durandin commented Jul 15, 2026

Uh oh!

p-wysocki left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

p-wysocki Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mostafafaheem Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mostafafaheem commented Jul 15, 2026

Uh oh!

p-wysocki commented Jul 16, 2026

Uh oh!

p-wysocki commented Jul 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

mostafafaheem commented Mar 2, 2026 •

edited

Loading

p-wysocki Jul 15, 2026 •

edited

Loading

mostafafaheem Jul 15, 2026 •

edited

Loading