Skip to content

[GPU] Add Level Zero - OpenCL interoperability#36014

Merged
isanghao merged 42 commits into
openvinotoolkit:masterfrom
jkasprza:add_ocl_ze_interop_prod
Jun 30, 2026
Merged

[GPU] Add Level Zero - OpenCL interoperability#36014
isanghao merged 42 commits into
openvinotoolkit:masterfrom
jkasprza:add_ocl_ze_interop_prod

Conversation

@jkasprza

@jkasprza jkasprza commented May 20, 2026

Copy link
Copy Markdown
Contributor

Details:

  • Detect if current system supports interoperability.
    • If check is passed, then create OCL remote context over L0 GPU runtime.
    • If check fails, then create L0 remote context over L0 GPU runtime.
  • Remote context and remote tensors query internal GPU plugin objects fur runtime specific handles.
    • OCL remote context will query for OCL handles.
    • L0 remote context will query for L0 handles.
  • Enable L0 internal GPU plugin objects to return converted OCL handles.
  • Add resource_owner template for gpu resource management.
  • Add ze_resource template that will hold Level Zero objects and optionally corresponding OpenCL objects.
  • Add operators for conversion between Level Zero objects and OpenCL objects.

Benchmarks:

Peak memory usage measured on benchmark_app running resnet-50 on BMG Linux

VmPeak (kB) MaxRSS (kB)
This PR (runtime=L0, interop=ON) 3,262,536 190,924
This PR (runtime=L0, interop=OFF) 3,263,688 193,200
Master (runtime=L0) 3,293,048 192,188
[REF] Master (runtime=OCL) 3,281,412 193,196

Tickets:

AI Assistance:

  • AI assistance used: yes
  • AI help was used to create tests

@github-actions github-actions Bot added category: inference OpenVINO Runtime library - Inference category: GPU OpenVINO GPU plugin category: CPP API OpenVINO CPP API bindings labels May 20, 2026
@jkasprza jkasprza marked this pull request as ready for review May 22, 2026 15:46
@jkasprza jkasprza requested review from a team as code owners May 22, 2026 15:46
@jkasprza jkasprza requested a review from a team as a code owner June 23, 2026 17:38
@jkasprza jkasprza requested review from zKulesza and removed request for a team June 23, 2026 17:38
@github-actions github-actions Bot added the category: docs OpenVINO documentation label Jun 23, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds Level Zero–OpenCL interoperability support to the Intel GPU plugin, enabling OpenCL-style RemoteContext/RemoteTensor usage while running the plugin on the Level Zero runtime. It introduces shared/RAII resource wrappers for Level Zero + optional OpenCL handles, implements import/export conversions, and updates runtime interfaces (engine/memory/stream) plus tests and documentation accordingly.

Changes:

  • Introduce resource_owner/ze_resource abstractions and refactor ZE runtime objects to use shared resource holders (command lists, events/pools, kernels/modules, images/USM).
  • Add ZE↔OCL handle conversion utilities (ze_ocl_interop, ze_resource_interop) and integrate them into context/stream/memory export/import paths.
  • Update engine and memory interfaces to return runtime-specific internal handles, and adjust remote context/tensor logic + test coverage.

Reviewed changes

Copilot reviewed 55 out of 55 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/plugins/intel_gpu/tests/unit/test_utils/test_utils.cpp Update test engine creation to use new default engine/runtime helpers.
src/plugins/intel_gpu/tests/unit/module_tests/ze/ze_resource_test.cpp Add unit tests for ze_resource behavior (empty/ownership/OCL attachments).
src/plugins/intel_gpu/tests/unit/module_tests/ze/ze_device_test.cpp Update ZE device tests to use resource wrappers (handle()/is_empty()).
src/plugins/intel_gpu/tests/functional/remote_tensor_tests/ocl_remote_tensor_tests.cpp Remove local OCL runtime guard to run remote tensor tests under the functional test target (which links OpenCL).
src/plugins/intel_gpu/src/runtime/ze/ze_stream.hpp Store command list as ze_command_list_resource; add ctor for external command list.
src/plugins/intel_gpu/src/runtime/ze/ze_stream.cpp Support stream creation from imported OCL queue; queue-type detection via OCL queue properties; use resource wrappers.
src/plugins/intel_gpu/src/runtime/ze/ze_resource.hpp Add ze_resource<T> shared wrapper for ZE handles with optional attached OCL handles.
src/plugins/intel_gpu/src/runtime/ze/ze_resource_interop.hpp Declare import/export helpers for converting between ZE resources and OCL handles.
src/plugins/intel_gpu/src/runtime/ze/ze_resource_interop.cpp Implement resource-level import/export conversions using ze_ocl_interop.
src/plugins/intel_gpu/src/runtime/ze/ze_owner.hpp Add resource_owner template + ZE resource type definitions/deleters.
src/plugins/intel_gpu/src/runtime/ze/ze_ocl_owner.hpp Add combined ZE+optional OCL owner model (ze_ocl_owner_impl) and OCL resource type definitions.
src/plugins/intel_gpu/src/runtime/ze/ze_ocl_interop.hpp Add singleton providing direct ZE↔OCL handle conversion APIs + support checks.
src/plugins/intel_gpu/src/runtime/ze/ze_ocl_interop.cpp Implement ZE↔OCL conversions and device mapping initialization.
src/plugins/intel_gpu/src/runtime/ze/ze_ocl_common.hpp Add OV_OCL_WARN helper macro for OpenCL error logging in ZE interop code.
src/plugins/intel_gpu/src/runtime/ze/ze_module_holder.hpp Remove legacy ZE module RAII holder (replaced by resource wrappers).
src/plugins/intel_gpu/src/runtime/ze/ze_memory.hpp Refactor ZE memory objects to use ze_usm_resource / ze_image_resource and runtime-aware internal params.
src/plugins/intel_gpu/src/runtime/ze/ze_memory.cpp Implement interop-aware internal param export; add allocation-range validation; refactor USM/image code to resource wrappers.
src/plugins/intel_gpu/src/runtime/ze/ze_kernel.hpp Refactor kernel/module lifetime management to use ze_*_resource holders and optional build logs.
src/plugins/intel_gpu/src/runtime/ze/ze_kernel_holder.hpp Remove legacy ZE kernel RAII holder (replaced by resource wrappers).
src/plugins/intel_gpu/src/runtime/ze/ze_kernel_builder.hpp Refactor builder API to build kernels directly (no module-holder return).
src/plugins/intel_gpu/src/runtime/ze/ze_kernel_builder.cpp Update kernel build flow; cache build support per device; use resource wrappers.
src/plugins/intel_gpu/src/runtime/ze/ze_event.hpp Refactor event to hold ze_event_resource and keep pool alive via ze_event_pool_resource.
src/plugins/intel_gpu/src/runtime/ze/ze_event.cpp Update ZE event operations to use resource wrappers; remove manual destructor destroy.
src/plugins/intel_gpu/src/runtime/ze/ze_event_pool.hpp Remove legacy event pool RAII type (replaced by ze_event_pool_resource).
src/plugins/intel_gpu/src/runtime/ze/ze_event_pool.cpp Remove legacy event pool implementation (now created in factory).
src/plugins/intel_gpu/src/runtime/ze/ze_event_factory.hpp Store current pool as ze_event_pool_resource; fix <mutex> include style.
src/plugins/intel_gpu/src/runtime/ze/ze_event_factory.cpp Create/destroy event pools via resource wrapper; create events as ze_event_resource.
src/plugins/intel_gpu/src/runtime/ze/ze_engine.hpp Make user context retrieval runtime-specific; return ZE context/device/driver as resources.
src/plugins/intel_gpu/src/runtime/ze/ze_engine.cpp Add OCL-handle export for contexts; support creating ZE stream from OCL queue; refactor memory reinterpret/import paths.
src/plugins/intel_gpu/src/runtime/ze/ze_device.hpp Refactor ZE device to store driver/device/context resources; default destructor.
src/plugins/intel_gpu/src/runtime/ze/ze_device.cpp Add supports_leo detection; add cl_mem capability when supported; refactor init/lifetime.
src/plugins/intel_gpu/src/runtime/ze/ze_device_detector.hpp Add helper to create ZE device from OCL device.
src/plugins/intel_gpu/src/runtime/ze/ze_device_detector.cpp Convert OCL devices to ZE devices when querying under ZE runtime; use resource wrappers for subdevices.
src/plugins/intel_gpu/src/runtime/ze/ze_counter_based_event.hpp Refactor counter-based events to use ze_event_resource.
src/plugins/intel_gpu/src/runtime/ze/ze_counter_based_event.cpp Update counter-based event API calls to use resource wrapper handles; remove manual destructor destroy.
src/plugins/intel_gpu/src/runtime/ze/ze_counter_based_event_factory.cpp Update factory to use resource wrapper handles and pass driver handle for init.
src/plugins/intel_gpu/src/runtime/ze/ze_common.hpp Adjust ZeroApi wrapper behavior for missing symbols; keep symbols-list macro control local.
src/plugins/intel_gpu/src/runtime/ocl/ocl_memory.hpp Make internal params query runtime-specific (assert OCL runtime).
src/plugins/intel_gpu/src/runtime/ocl/ocl_memory.cpp Enforce OCL-only internal param queries in OCL memory implementations.
src/plugins/intel_gpu/src/runtime/ocl/ocl_engine.hpp Make user context retrieval runtime-specific (assert OCL runtime).
src/plugins/intel_gpu/src/runtime/ocl/ocl_engine.cpp Update context retrieval calls to pass explicit runtime type.
src/plugins/intel_gpu/src/runtime/ocl/ocl_device.cpp Initialize supports_leo=false for pure OCL devices.
src/plugins/intel_gpu/src/runtime/engine_configuration.cpp Add global helpers for default engine/runtime selection.
src/plugins/intel_gpu/src/runtime/device_query.cpp Use OCL query path for OCL user contexts and convert devices to ZE when running ZE runtime.
src/plugins/intel_gpu/src/plugin/remote_tensor.cpp Query runtime-specific internal handles based on the remote context type.
src/plugins/intel_gpu/src/plugin/remote_context.cpp Default context type selection; auto-enable OCL remote context when LEO is supported; make properties runtime-specific.
src/plugins/intel_gpu/include/intel_gpu/runtime/memory.hpp Change internal-handle query API to be runtime-specific, with a default helper.
src/plugins/intel_gpu/include/intel_gpu/runtime/memory_caps.hpp Clarify that shared context handle can be OCL or ZE.
src/plugins/intel_gpu/include/intel_gpu/runtime/engine.hpp Change engine user context API to be runtime-specific.
src/plugins/intel_gpu/include/intel_gpu/runtime/engine_configuration.hpp Declare global get_default_engine_type() / get_default_runtime_type().
src/plugins/intel_gpu/include/intel_gpu/runtime/device_query.hpp Remove default engine/runtime selection from device_query (now global helpers).
src/plugins/intel_gpu/include/intel_gpu/runtime/device_info.hpp Add supports_leo device capability flag.
src/plugins/intel_gpu/include/intel_gpu/plugin/remote_context.hpp Remove preprocessor-initialized context type; store computed context type.
src/plugins/intel_gpu/docs/blog/ze_ocl_interop_2026_3.md Add design/usage documentation for Level Zero–OpenCL interop and new resource model.
src/inference/include/openvino/runtime/intel_gpu/remote_properties.hpp Minor wording/order tweak for ContextType::ZE parsing/comment.

Comment thread src/plugins/intel_gpu/src/runtime/ze/ze_memory.cpp Outdated
Comment thread src/plugins/intel_gpu/src/runtime/ze/ze_engine.cpp Outdated
jkasprza and others added 2 commits June 26, 2026 14:41
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

@isanghao isanghao left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@isanghao isanghao added this pull request to the merge queue Jun 30, 2026
Merged via the queue into openvinotoolkit:master with commit 41cc96b Jun 30, 2026
197 checks passed
@isanghao isanghao deleted the add_ocl_ze_interop_prod branch June 30, 2026 08:11
praasz pushed a commit to praasz/openvino that referenced this pull request Jul 1, 2026
### Details:
 - Detect if current system supports interoperability. 
- If check is passed, then create OCL remote context over L0 GPU
runtime.
   - If check fails, then create L0 remote context over L0 GPU runtime.
- Remote context and remote tensors query internal GPU plugin objects
fur runtime specific handles.
   - OCL remote context will query for OCL handles.
   - L0 remote context will query for L0 handles.
- Enable L0 internal GPU plugin objects to return converted OCL handles.
 - Add `resource_owner` template for gpu resource management.
- Add `ze_resource` template that will hold Level Zero objects and
optionally corresponding OpenCL objects.
- Add operators for conversion between Level Zero objects and OpenCL
objects.

### Benchmarks:
Peak memory usage measured on benchmark_app running resnet-50 on BMG
Linux
|  | VmPeak (kB) | MaxRSS (kB) |
| --- | --- | --- |
| This PR (runtime=L0, interop=ON) | 3,262,536 | 190,924 |
| This PR (runtime=L0, interop=OFF) | 3,263,688 | 193,200 |
| Master (runtime=L0) | 3,293,048 | 192,188 |
| [REF] Master (runtime=OCL) | 3,281,412 | 193,196 |

### Tickets:
 - [CVS-185378](https://jira.devtools.intel.com/browse/CVS-185378)

### AI Assistance:
 - AI assistance used: yes
 - AI help was used to create tests

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: CPP API OpenVINO CPP API bindings category: docs OpenVINO documentation category: GPU OpenVINO GPU plugin category: inference OpenVINO Runtime library - Inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants