Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
25ae74b
Add ze_holder header
jkasprza May 5, 2026
a297057
Fix reference and unused method
jkasprza May 5, 2026
4b7e0bb
Move context to holder
jkasprza May 5, 2026
878d2b1
Move counter based events to holder
jkasprza May 5, 2026
be1dc84
Move regular events to holder
jkasprza May 5, 2026
ff6f022
Fix and improve ze_holder
jkasprza May 6, 2026
b12ab6c
Fix ze_event_factory
jkasprza May 6, 2026
3f41632
Adjust cldnn events
jkasprza May 6, 2026
5e6ccd5
Move kernel and module to holder
jkasprza May 6, 2026
3002a61
Fix module build log parent
jkasprza May 6, 2026
4cc638d
Move usm memory to holder
jkasprza May 6, 2026
c365047
Move image to holder
jkasprza May 6, 2026
11559fa
Move stream to holder
jkasprza May 6, 2026
adc1374
Change ze_resource to ze_owner
jkasprza May 7, 2026
b0dd0d4
Merge remote-tracking branch 'origin/master' into add_ocl_ze_interop_…
jkasprza May 18, 2026
95f418f
Implement ze_resource
jkasprza May 18, 2026
715cdf1
Fix issues
jkasprza May 20, 2026
2831019
Fix test issues
jkasprza May 21, 2026
0e833d2
Merge remote-tracking branch 'origin/master' into add_ocl_ze_interop_…
jkasprza May 21, 2026
72157af
Check usm and cl_mem range
jkasprza May 21, 2026
c1370fc
More explicit interop usage
jkasprza May 21, 2026
f91a998
Only enable interop when supported
jkasprza May 22, 2026
d0afd19
Fix copyright
jkasprza May 22, 2026
4ff7dab
Add unit tests for ze_resource
jkasprza May 22, 2026
85ae300
Merge remote-tracking branch 'origin/master' into add_ocl_ze_interop_…
jkasprza Jun 18, 2026
66d3147
Fix queue type detection
jkasprza Jun 18, 2026
ac97c2c
Rename is_shared to is_borrowed
jkasprza Jun 19, 2026
c78d1a3
Replace import/export functors with functions
jkasprza Jun 22, 2026
832bf29
Shorten get_handle method names
jkasprza Jun 23, 2026
a6bb5b7
Fix lifetime bug
jkasprza Jun 23, 2026
596018d
Add ze_ocl_interop doc in blog format
jkasprza Jun 23, 2026
4a9db5a
Fix cl_mem support check
jkasprza Jun 23, 2026
51127c9
Return device resources by value
jkasprza Jun 24, 2026
0f4b531
Return resources by value
jkasprza Jun 24, 2026
8cfbd1b
Remove more resource references
jkasprza Jun 24, 2026
6dd1bb3
Merge remote-tracking branch 'origin/master' into add_ocl_ze_interop_…
jkasprza Jun 24, 2026
3c22480
Merge remote-tracking branch 'origin/master' into add_ocl_ze_interop_…
jkasprza Jun 24, 2026
9e0bf8a
Add usm fallback
jkasprza Jun 24, 2026
fe4ea93
Fix compilation warnings
jkasprza Jun 25, 2026
2b38333
Fix OpenCL header warnings
jkasprza Jun 25, 2026
04c3289
Apply suggestions from AI code review
jkasprza Jun 26, 2026
b082a4a
Merge branch 'master' into add_ocl_ze_interop_prod
jkasprza Jun 29, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ using os_handle_param = void*;
enum class ContextType {
OCL = 0, //!< Pure OpenCL context
VA_SHARED = 1, //!< Context shared with a video decoding device
ZE = 2, //!< Pure Level0 context
ZE = 2, //!< Pure Level Zero context
};

/** @cond INTERNAL */
Expand All @@ -52,10 +52,10 @@ inline std::istream& operator>>(std::istream& is, ContextType& context_type) {
is >> str;
if (str == "OCL") {
context_type = ContextType::OCL;
} else if (str == "ZE") {
context_type = ContextType::ZE;
} else if (str == "VA_SHARED") {
context_type = ContextType::VA_SHARED;
} else if (str == "ZE") {
context_type = ContextType::ZE;
} else {
OPENVINO_THROW("Unsupported context type: ", str);
}
Expand Down
50 changes: 50 additions & 0 deletions src/plugins/intel_gpu/docs/blog/ze_ocl_interop_2026_3.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Support for Level Zero - OpenCL interoperability: New in OpenVINO 2026.3

*By Jakub Kasprzak | June 23, 2026*

Some projects maintain their own OpenCL GPU pipeline and require features such as memory, context or queue sharing. For such users OpenVINO GPU plugin implements Remote Context and Remote Tensor interface. However, this OpenCL interoperability works only if OpenVINO GPU runtime is set to OpenCL and prevents OpenVINO from switching to Level Zero runtime.
OpenVINO 2026.3 introduces Level Zero - OpenCL interoperability enabling users that depend on OpenVINO Remote Context/Tensor OpenCL API to swap OpenVINO GPU runtime to Level Zero without changing their OpenCL based integration logic. This post explains: how to enable this feature and provides OpenVINO maintainers with design description.

## How to enable Level Zero - OpenCL interoperability

OpenVINO GPU plugin with Level Zero runtime detects if interoperability is supported on current system and enables it automatically. User must ensure that currently installed Intel GPU driver supports and enables "Level Zero executing OpenCL" (LEO) feature.

## Level Zero - OpenCL interoperability design

*This section describes internal design of the feature and is intended for OpenVINO maintainers*

### Object lifetimes

This subsection describes part of the design responsible for managing resources. Main goal was to provide modular API that will minimize chances of memory leaks and other memory bugs.

Logic for managing all (Level Zero and OpenCL) GPU resources is provided by `resource_owner` template class. It contains `is_borrowed` flag to distinguish between resources that need to be released and resources that should not be released (borrowed resources, for example: external memory buffers provided by the user). Once `resource_owner` is destroyed it will release the resource or do nothing if resource was borrowed. Basically `resource_owner` with `is_borrowed=false` is a generic RAII wrapper which means that only one such object can exist for a given resource and that it can not be copied (only move semantics are allowed).

In order to work correctly `resource_owner` template must be provided with information about the type of resource it manages and how to release it. This information is defined for Level Zero and OpenCL resources in `ze_resource_info` and `ocl_resource_info` class template specializations respectively. For example `ocl_resource_info<ocl_resource_type::command_queue>` defines `handle_t` as `cl_command_queue` and `deleter_t` as a functor that calls `clReleaseCommandQueue`. Finally `ze_owner` and `ocl_owner` aliases are defined by combining `resource_owner` with `ze_resource_info` and `ocl_resource_info` respectively.

### Binding corresponding Level Zero - OpenCL handles

It is important to recognize that with interoperability some Level Zero handles will have corresponding OpenCL handles and vice versa. For example, after converting Level Zero immediate command list to OpenCL command queue we end up with two different handles referring to the same entity. `ze_ocl_owner_impl` is a template that is responsible for combining Level Zero handle with OpenCL handle and managing them as a single resource. Internally it contains `ze_owner` and tuple of optional `ocl_owner`s. By design `ze_ocl_owner_impl` always contains Level Zero resource and OpenCL resources can be attached to it. Once OpenCL resource of some given type is attached to `ze_ocl_owner_impl` attempt to attach another OpenCL resource of the same type will result in **exception** (this design is to prevent exporting single Level Zero resource multiple times).

Not all handle combinations are valid. For example in current context it does not make sense to attach/bind `cl_command_queue` handle to `ze_ocl_owner_impl` that holds `ze_image_handle_t`. To prevent this `ze_ocl_owner` template inherits only valid `ze_ocl_owner_impl` specializations. For example `ze_ocl_owner<ze_resource_type::command_list>` is `ze_ocl_owner_impl<ze_resource_type::command_list, ocl_resource_type::command_queue>` meaning it holds `ze_command_list_handle_t` and optionally `cl_command_queue` handles. If specialization is not defined explicitly it fall backs to default implementation where only Level Zero handle is allowed, for example `ze_ocl_owner<ze_resource_type::kernel>` holds only `ze_kernel_handle_t` and attempt to attach or access any type of OpenCL handle will result in **compilation error**.

### Shared resource ownership

Finally `ze_resource` template combines `std::shared_ptr` and `ze_ocl_owner` to implement shared ownership model for GPU resources. This is especially useful for handling lifetime of Level Zero events and event pools where `ze_resource<ze_resource_type::event_pool>` can be copied to all event objects created from given pool ensuring that the pool resource will be destroyed only after last event from the pool was destroyed (same scheme is used to manage lifetime of Level Zero modules and kernels).

Additionaly aliases in form of `ze_*_resource` such as `ze_event_pool_resource` are defined for developer convenience.

### Interoperability integration

Basic interoperability is implemented by a `ze_ocl_interop` singleton that detects interoperability support and provides direct (no abstraction) Level Zero - OpenCL handle to handle conversion interface. When `ze_ocl_interop` is accessed for the first time it will initialize GPU runtimes and map corresponding Level Zero and OpenCL devices.

Conversion interface provided by `ze_ocl_interop` is wrapped by `ze_import_*` and `ze_export_ocl` functions from `ze_resource_interop.hpp` in order to simplify conversion logic when used with `ze_resource` objects. For example calling `ze_export_ocl_command_queue` for given `ze_command_list_resource` will:
1. Return early if `ze_command_list_resource` already has attached OpenCL command queue handle
2. Create new OpenCL command queue based on present Level Zero handle
3. Attach new OpenCL handle to `ze_command_list_resource`
4. Guarantee strong exception safety

### Remote context

Type of remote context decides what kind of objects are exposed by the GPU plugin. During model compilation OpenVINO creates remote context and internal engine that by default match currently selected GPU runtime. For example, when GPU runtime is set to Level Zero this causes the creation of Level Zero remote context and `ze_engine`. Remote context will query internal GPU plugin objects for handles based on the remote context type. So if Level Zero remote context is used only Level Zero handles can be requested from internal objects and OpenCL interoperability is not required.

If user provides OpenCL context or queue or OpenVINO detects that current system supports Level Zero - OpenCL interoperability then remote context type will be changed to OpenCL but engine will still match currently selected GPU runtime. In result OpenCL remote context will request OpenCL handles from internal Level Zero GPU plugin objects like `ze_engine` forcing usage of interoperability features.
Original file line number Diff line number Diff line change
Expand Up @@ -95,11 +95,8 @@ class RemoteContextImpl : public ov::IRemoteContext {
ov::intel_gpu::gpu_handle_param m_va_display = nullptr;
ov::intel_gpu::gpu_handle_param m_external_queue = nullptr;

#ifdef OV_GPU_WITH_ZE_RT
ContextType m_type = ContextType::ZE;
#else
ContextType m_type = ContextType::OCL;
#endif

ContextType m_type;
std::string m_device_name = "";
static const size_t cache_capacity = 100;
cldnn::LruCache<size_t, cldnn::memory::ptr> m_memory_cache = cldnn::LruCache<size_t, cldnn::memory::ptr>(cache_capacity);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -122,8 +122,9 @@ struct device_info {
bool supports_usm; ///< Does engine support unified shared memory.
bool has_separate_cache; ///< Does the target hardware has separate cache for usm_device and usm_host

bool supports_cp_offload; ///< [ZE] Does the command queue support copy offload
bool supports_counter_based_events; ///< [ZE] Does the target runtime support counter based events
bool supports_cp_offload; ///< [L0] Does the command queue support copy offload
bool supports_counter_based_events; ///< [L0] Does the target runtime support counter based events
bool supports_leo; ///< [L0] Does the device support Level Zero - OpenCL interoperability (LEO)

std::vector<size_t> supported_simd_sizes; ///< List of SIMD sizes supported by current device and compiler

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,6 @@ namespace cldnn {
struct device_query {
public:
static int device_id;
/// @brief Get default engine type
static engine_types get_default_engine_type();

/// @brief Get default runtime type
static runtime_types get_default_runtime_type();

explicit device_query(engine_types engine_type,
runtime_types runtime_type,
Expand Down
4 changes: 2 additions & 2 deletions src/plugins/intel_gpu/include/intel_gpu/runtime/engine.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -113,8 +113,8 @@ class engine {
/// Returns device object associated with the engine
const device::ptr get_device() const;

/// Returns user context handle which was used to create the engine
virtual void* get_user_context() const = 0;
/// Returns L0 or OpenCL user context handle which was used to create the engine.
virtual void* get_user_context(runtime_types rt_type) const = 0;

/// Returns the total maximum amount of GPU memory allocated by engine in current process for all allocation types
uint64_t get_max_used_device_memory() const;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -64,4 +64,10 @@ inline std::ostream& operator<<(std::ostream& os, const backend_types& type) {
return os;
}

/// @brief Get default engine type
engine_types get_default_engine_type();

/// @brief Get default runtime type
runtime_types get_default_runtime_type();

} // namespace cldnn
8 changes: 6 additions & 2 deletions src/plugins/intel_gpu/include/intel_gpu/runtime/memory.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,11 @@ struct memory {

size_t size() const { return _bytes_count; }
size_t count() const { return _layout.count(); }
virtual shared_mem_params get_internal_params() const = 0;
/// @brief Return internal params for specified runtime type
virtual shared_mem_params get_internal_params(runtime_types rt_type) const = 0;
shared_mem_params get_internal_params() const {
return get_internal_params(get_default_runtime_type());
}
virtual bool is_allocated_by(const engine& engine) const { return &engine == _engine && _type != allocation_type::unknown; }
engine* get_engine() const { return _engine; }
const layout& get_layout() const { return _layout; }
Expand Down Expand Up @@ -156,7 +160,7 @@ struct simple_attached_memory : memory {
void unlock(const stream& /* stream */) override {}
event::ptr fill(stream& /* stream */, unsigned char, const std::vector<event::ptr>&, bool) override { return nullptr; }
event::ptr fill(stream& /* stream */, const std::vector<event::ptr>&, bool) override { return nullptr; }
shared_mem_params get_internal_params() const override { return { shared_mem_type::shared_mem_empty, nullptr, nullptr, nullptr,
shared_mem_params get_internal_params(runtime_types rt_type) const override { return { shared_mem_type::shared_mem_empty, nullptr, nullptr, nullptr,
#ifdef _WIN32
nullptr,
#else
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ using shared_surface = uint32_t;
/// @brief Low-level API handles required for using cldnn memory objects in external API calls.
struct shared_mem_params {
shared_mem_type mem_type; ///< shared buffer type
shared_handle context; ///< OpenCL context for external operations
shared_handle context; ///< OpenCL or Level Zero context for external operations
shared_handle user_device; ///< DX/VA device for external operations
shared_handle mem; ///< memory object handle
#ifdef _WIN32
Expand Down
36 changes: 29 additions & 7 deletions src/plugins/intel_gpu/src/plugin/remote_context.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,29 @@ Type extract_object(const ov::AnyMap& params, const ov::Property<Type>& p) {
return res.as<Type>();
}

ContextType get_default_context_type() {
#ifdef OV_GPU_WITH_ZE_RT
return ContextType::ZE;
#elif defined(OV_GPU_WITH_OCL_RT)
return ContextType::OCL;
#else
#error "Expected OpenVINO GPU runtime macros to be defined"
#endif
}

} // namespace

RemoteContextImpl::RemoteContextImpl(const std::string& device_name, std::vector<cldnn::device::ptr> devices, bool initialize_ctx)
: m_device_name(device_name) {
OPENVINO_ASSERT(devices.size() == 1, "[GPU] Currently context can be created for single device only");
m_device = devices.front();
m_type = get_default_context_type();
const auto &info = m_device->get_info();
if (m_type == ContextType::ZE && info.supports_leo) {
m_type = ContextType::OCL;
GPU_DEBUG_INFO << "Enabled Level Zero - OpenCL interoperability for "
<< m_device_name << " (" << info.dev_name << ")" << std::endl;
}

if (initialize_ctx) {
initialize();
Expand All @@ -39,6 +56,7 @@ RemoteContextImpl::RemoteContextImpl(const std::map<std::string, RemoteContextIm
gpu_handle_param context_id = nullptr;
int ctx_device_id = 0;
int target_tile_id = -1;
m_type = get_default_context_type();

if (params.size()) {
auto ctx_type = extract_object(params, ov::intel_gpu::context_type);
Expand All @@ -56,20 +74,23 @@ RemoteContextImpl::RemoteContextImpl(const std::map<std::string, RemoteContextIm
} else if (ctx_type == ov::intel_gpu::ContextType::VA_SHARED) {
m_va_display = extract_object(params, ov::intel_gpu::va_device);
OPENVINO_ASSERT(m_va_display != nullptr, "[GPU] Can't create shared VA/DX context as user handle is nullptr! Params:\n", params);
m_type = ContextType::VA_SHARED;
} else if (ctx_type == ov::intel_gpu::ContextType::ZE) {
OPENVINO_THROW("Level Zero interoperability is not supported");
} else {
OPENVINO_THROW("Invalid execution context type", ctx_type);
}
m_type = ctx_type;
if (params.find(ov::intel_gpu::tile_id.name()) != params.end()) {
target_tile_id = extract_object(params, ov::intel_gpu::tile_id);
}
}

const auto initialize_devices = true;

cldnn::device_query device_query(context_id, m_va_display, ctx_device_id, target_tile_id, initialize_devices);
auto device_map = device_query.get_available_devices();

// Always use OCL for device query
// Query will return devices compatible with default runtime
cldnn::device_query ocl_device_query(cldnn::engine_types::ocl, cldnn::runtime_types::ocl, context_id, m_va_display, ctx_device_id, target_tile_id, initialize_devices);
auto device_map = ocl_device_query.get_available_devices();
OPENVINO_ASSERT(device_map.size() == 1, "[GPU] Exactly one device expected in case of context sharing, but ", device_map.size(), " found");

m_device = device_map.begin()->second;
Expand All @@ -89,19 +110,20 @@ const cldnn::engine& RemoteContextImpl::get_engine() const {
}

void RemoteContextImpl::init_properties() {
properties = { ov::intel_gpu::ocl_context(m_engine->get_user_context()) };

switch (m_type) {
case ContextType::OCL:
properties.insert(ov::intel_gpu::context_type(ov::intel_gpu::ContextType::OCL));
properties.insert(ov::intel_gpu::ocl_context(m_engine->get_user_context(cldnn::runtime_types::ocl)));
properties.insert(ov::intel_gpu::ocl_queue(m_external_queue));
break;
case ContextType::VA_SHARED:
properties.insert(ov::intel_gpu::context_type(ov::intel_gpu::ContextType::VA_SHARED));
properties.insert(ov::intel_gpu::ocl_context(m_engine->get_user_context(cldnn::runtime_types::ocl)));
properties.insert(ov::intel_gpu::va_device(m_va_display));
break;
case ContextType::ZE:
properties.insert(ov::intel_gpu::context_type(ov::intel_gpu::ContextType::ZE));
properties.insert(ov::intel_gpu::ocl_context(m_engine->get_user_context(cldnn::runtime_types::ze)));
break;
default:
OPENVINO_THROW("[GPU] Unsupported shared context type ", m_type);
Expand Down Expand Up @@ -256,7 +278,7 @@ void RemoteContextImpl::initialize() {

m_device->initialize(); // Initialize associated device before use
m_engine = cldnn::engine::create(
cldnn::device_query::get_default_engine_type(), cldnn::device_query::get_default_runtime_type(), m_device);
cldnn::get_default_engine_type(), cldnn::get_default_runtime_type(), m_device);

init_properties();

Expand Down
15 changes: 13 additions & 2 deletions src/plugins/intel_gpu/src/plugin/remote_tensor.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -319,7 +319,6 @@ void RemoteTensorImpl::allocate() {

switch (m_mem_type) {
case TensorType::BT_BUF_INTERNAL: {
// BT_BUF_INTERNAL should map to cl_mem however ZE engine can not allocate cl_mem
if (engine.supports_allocation(cldnn::allocation_type::cl_mem)) {
m_memory_object = engine.allocate_memory(m_layout, cldnn::allocation_type::cl_mem, reset);
} else if (engine.supports_allocation(cldnn::allocation_type::sycl_buffer)) {
Expand Down Expand Up @@ -450,7 +449,19 @@ std::shared_ptr<RemoteContextImpl> RemoteTensorImpl::get_context() const {

void RemoteTensorImpl::update_properties() {
OPENVINO_ASSERT(is_allocated(), "[GPU] Can't initialize RemoteTensorImpl parameters as memory was not allocated");
auto params = m_memory_object->get_internal_params();
const auto &ctx_props = m_context->get_property();
const auto it = ctx_props.find(ov::intel_gpu::context_type.name());
OPENVINO_ASSERT(it != ctx_props.end(), "[GPU] Could not find context type in RemoteContext properties");
const auto ctx_type = it->second.as<ContextType>();

cldnn::shared_mem_params params;
if (ctx_type == ContextType::OCL || ctx_type == ContextType::VA_SHARED) {
params = m_memory_object->get_internal_params(cldnn::runtime_types::ocl);
} else if (ctx_type == ContextType::ZE) {
params = m_memory_object->get_internal_params(cldnn::runtime_types::ze);
} else {
OPENVINO_THROW("[GPU] Can't update RemoteTensorImpl properties for unsupported context type (", ctx_type, ")");
}

switch (m_mem_type) {
case TensorType::BT_BUF_INTERNAL:
Expand Down
Loading
Loading