[GPU] Enable zero-copy subbuffers for usm_device on iGPUs by Prithviraj-R · Pull Request #36645 · openvinotoolkit/openvino

Prithviraj-R · 2026-06-30T13:41:28Z

[GPU] Enable zero-copy subbuffers for usm_device on iGPUs

Details:

Description of the issue (symptom, root-cause, how it was resolved)

Zero-copy handling for usm_host and usm_shared was already implemented in
earlier PRs ([GPU] Enable mmap for loading model weights on iGPUs #34494, [GPU] Support CPU pointers on remote tensor #36539); this PR focuses on closing the usm_device gap.
Previously, for usm_device allocations, a separate GPU usm_device buffer
was allocated and filled from mapped host buffer data during cache load,
causing duplicate allocation/copy overhead and increasing inference memory
footprint.
This change enables zero-copy subbuffer usage for supported XE2+ iGPUs in the
usm_device flow, removing extra staging/copy in applicable cases.
Blob offset alignment handling in deserialization is kept correct for
subbuffer creation.

Implementation changes:

Extend zero-copy eligibility in data.hpp to include usm_device while
preserving existing usm_host/usm_shared behavior.
Add/use helper check for zero-copy-capable device (XE2+ integrated GPU).
Use seek_current_ptr() for padding skip during deserialization alignment.
Refine zero-copy condition naming/readability in load path.

Changed files:
src/plugins/intel_gpu/include/intel_gpu/graph/serialization/binary_buffer.hpp
src/plugins/intel_gpu/include/intel_gpu/primitives/data.hpp
src/plugins/intel_gpu/include/intel_gpu/runtime/device_info.hpp
src/plugins/intel_gpu/src/graph/program.cpp
src/plugins/intel_gpu/src/runtime/ocl/ocl_device.cpp
src/plugins/intel_gpu/src/runtime/ocl/ocl_engine.cpp
src/plugins/intel_gpu/tests/unit/test_cases/cache_serialization_test.cpp

Reproduction step and snapshot (if applicable. Do not attach for customer model)
NA

Problematic graph
NA

Checklist

Is it a proper fix? Yes (removes duplicate usm_device staging/copy in
supported zero-copy cases)
Did you include test case for this fix, if necessary? NA
Did you review existing test that can be extended to cover this scenario?
NA

Tickets:

CVS-176160

AI Assistance:

AI assistance used: yes
AI was used for wording refinement and commit message cleanup.

Copilot

Pull request overview

This PR updates Intel GPU cache serialization/deserialization for cldnn::data weights to broaden zero-copy subbuffer usage (xe2+ iGPU) and to apply sub-buffer alignment padding more consistently.

Changes:

Move sub-buffer padding logic to be applied outside the host-accessible branch in save/load paths.
Relax zero-copy enablement conditions (xe2+ integrated GPU + mmap alignment + non-weightless).
Simplify branching in load_weights().

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

javier-intel

Looks good to me

maxnick · 2026-07-13T13:55:06Z

@Prithviraj-R , please update the PR description. The PR body lists 12 changed files (incl. ze_engine.* , sycl_engine.* , ocl_engine.hpp ) and claims CACHE_PAGE_SIZE was moved to memory_caps.hpp , but the diff touches 7 files and the constant is simply deleted (no references remain, so no build break)

p-durandin · 2026-07-15T09:31:32Z

@Prithviraj-R please take a look on L0 related errors. They look not a random
FAILED TESTS (2/447):
2058 ms: /__w/openvino/openvino/install/tests/ov_gpu_unit_tests smoke/conv_dyn_3d_test.convolution_gpu_b_fs_zyx_fsv16_imad_quantized/1
2079 ms: /__w/openvino/openvino/install/tests/ov_gpu_unit_tests smoke/conv_dyn_3d_test.convolution_gpu_b_fs_zyx_fsv16_imad_quantized/3

#### Details: Description of the issue (symptom, root-cause, how it was resolved) - Zero-copy handling for `usm_host` and `usm_shared` was already implemented in earlier PRs (openvinotoolkit#34494, openvinotoolkit#36539); this PR focuses on closing the `usm_device` gap. - Previously, for `usm_device` allocations, a separate GPU `usm_device` buffer was allocated and filled from mapped host buffer data during cache load, causing duplicate allocation/copy overhead and increasing inference memory footprint. - This change enables zero-copy subbuffer usage for supported XE2+ iGPUs in the `usm_device` flow, removing extra staging/copy in applicable cases. - Blob offset alignment handling in deserialization is kept correct for subbuffer creation. Implementation changes: - Extend zero-copy eligibility in `data.hpp` to include `usm_device` while preserving existing `usm_host`/`usm_shared` behavior. - Add/use helper check for zero-copy-capable device (XE2+ integrated GPU). - Use `seek_current_ptr()` for padding skip during deserialization alignment. - Refine zero-copy condition naming/readability in load path. Changed files: src/plugins/intel_gpu/include/intel_gpu/graph/serialization/binary_buffer.hpp src/plugins/intel_gpu/include/intel_gpu/primitives/data.hpp src/plugins/intel_gpu/include/intel_gpu/runtime/device_info.hpp src/plugins/intel_gpu/src/graph/program.cpp src/plugins/intel_gpu/src/runtime/ocl/ocl_device.cpp src/plugins/intel_gpu/src/runtime/ocl/ocl_engine.cpp src/plugins/intel_gpu/tests/unit/test_cases/cache_serialization_test.cpp Reproduction step and snapshot (if applicable. Do not attach for customer model) NA Problematic graph NA #### Checklist - [x] Is it a proper fix? Yes (removes duplicate `usm_device` staging/copy in supported zero-copy cases) - [x] Did you include test case for this fix, if necessary? NA - [x] Did you review existing test that can be extended to cover this scenario? NA #### Tickets: - CVS-176160 #### AI Assistance: - AI assistance used: yes - AI was used for wording refinement and commit message cleanup.

Prithviraj-R · 2026-07-16T09:55:48Z

@Prithviraj-R please take a look on L0 related errors. They look not a random FAILED TESTS (2/447): 2058 ms: /__w/openvino/openvino/install/tests/ov_gpu_unit_tests smoke/conv_dyn_3d_test.convolution_gpu_b_fs_zyx_fsv16_imad_quantized/1 2079 ms: /__w/openvino/openvino/install/tests/ov_gpu_unit_tests smoke/conv_dyn_3d_test.convolution_gpu_b_fs_zyx_fsv16_imad_quantized/3

@p-durandin fixed.

Prithviraj-R · 2026-07-16T09:56:13Z

@Prithviraj-R , please update the PR description. The PR body lists 12 changed files (incl. ze_engine.* , sycl_engine.* , ocl_engine.hpp ) and claims CACHE_PAGE_SIZE was moved to memory_caps.hpp , but the diff touches 7 files and the constant is simply deleted (no references remain, so no build break)

@maxnick fixed

Prithviraj-R requested review from a team as code owners June 30, 2026 13:41

Prithviraj-R requested a review from Copilot June 30, 2026 13:41

github-actions Bot added the category: GPU OpenVINO GPU plugin label Jun 30, 2026

Copilot started reviewing on behalf of Prithviraj-R June 30, 2026 13:42 View session

Copilot AI reviewed Jun 30, 2026

View reviewed changes

Comment thread src/plugins/intel_gpu/include/intel_gpu/primitives/data.hpp

Comment thread src/plugins/intel_gpu/include/intel_gpu/primitives/data.hpp Outdated

Prithviraj-R force-pushed the zero_copy_bug_fix branch from 0ecb60b to 05d98be Compare June 30, 2026 14:15

Prithviraj-R requested a review from Copilot June 30, 2026 14:18

Copilot started reviewing on behalf of Prithviraj-R June 30, 2026 14:19 View session

Copilot AI reviewed Jun 30, 2026

View reviewed changes

Comment thread src/plugins/intel_gpu/include/intel_gpu/graph/serialization/binary_buffer.hpp Outdated

Comment thread src/plugins/intel_gpu/include/intel_gpu/primitives/data.hpp Outdated

Prithviraj-R force-pushed the zero_copy_bug_fix branch from 05d98be to 86d28cc Compare June 30, 2026 14:27

Prithviraj-R changed the title ~~[GPU] Enable zero-copy subbuffers for all allocation types and fix pa…~~ [GPU] Enable zero-copy subbuffers across all alloc types for iGPUs Jun 30, 2026

Prithviraj-R requested a review from Copilot June 30, 2026 14:28

Copilot started reviewing on behalf of Prithviraj-R June 30, 2026 14:29 View session

Copilot AI reviewed Jun 30, 2026

View reviewed changes

Comment thread src/plugins/intel_gpu/include/intel_gpu/primitives/data.hpp Outdated

Prithviraj-R requested a review from Copilot June 30, 2026 14:33

Copilot started reviewing on behalf of Prithviraj-R June 30, 2026 14:34 View session

Copilot AI reviewed Jun 30, 2026

View reviewed changes

Comment thread src/plugins/intel_gpu/include/intel_gpu/primitives/data.hpp Outdated

Comment thread src/plugins/intel_gpu/include/intel_gpu/primitives/data.hpp Outdated

javier-intel reviewed Jul 1, 2026

View reviewed changes

Comment thread src/plugins/intel_gpu/include/intel_gpu/primitives/data.hpp Outdated

Prithviraj-R force-pushed the zero_copy_bug_fix branch 3 times, most recently from 1dd8ab4 to 556df4f Compare July 1, 2026 10:33

Prithviraj-R requested a review from javier-intel July 1, 2026 10:36

Prithviraj-R force-pushed the zero_copy_bug_fix branch 2 times, most recently from 4a4a190 to ae5b220 Compare July 1, 2026 13:15

javier-intel reviewed Jul 1, 2026

View reviewed changes

p-durandin assigned maxnick Jul 2, 2026

p-durandin added this to the 2026.3 milestone Jul 2, 2026

maxnick requested changes Jul 2, 2026

View reviewed changes

Comment thread src/plugins/intel_gpu/include/intel_gpu/runtime/engine.hpp Outdated

Comment thread src/plugins/intel_gpu/include/intel_gpu/primitives/data.hpp Outdated

Comment thread src/plugins/intel_gpu/src/runtime/ocl/ocl_engine.cpp Outdated

maxnick reviewed Jul 2, 2026

View reviewed changes

Comment thread src/plugins/intel_gpu/src/runtime/ocl/ocl_engine.cpp Outdated

Comment thread src/plugins/intel_gpu/src/runtime/engine.cpp Outdated

Prithviraj-R force-pushed the zero_copy_bug_fix branch 6 times, most recently from 8835f47 to c6d708d Compare July 13, 2026 11:52

maxnick requested changes Jul 13, 2026

View reviewed changes

maxnick modified the milestones: 2026.3, 2026.4 Jul 13, 2026

Prithviraj-R force-pushed the zero_copy_bug_fix branch 7 times, most recently from c2fec9f to c122efa Compare July 14, 2026 14:12

Prithviraj-R force-pushed the zero_copy_bug_fix branch 2 times, most recently from 5e1790e to e086125 Compare July 15, 2026 15:42

Prithviraj-R changed the title ~~[GPU] Enable zero-copy subbuffers across all alloc types for iGPUs~~ [GPU] Enable zero-copy subbuffers for usm_device on iGPUs Jul 15, 2026

Prithviraj-R force-pushed the zero_copy_bug_fix branch 4 times, most recently from acf8db0 to e0a1663 Compare July 15, 2026 18:01

Prithviraj-R force-pushed the zero_copy_bug_fix branch from e0a1663 to aba5074 Compare July 16, 2026 06:14

Prithviraj-R requested a review from maxnick July 16, 2026 07:55

Uh oh!

Conversation

Prithviraj-R commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Details:

Checklist

Tickets:

AI Assistance:

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

javier-intel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

maxnick commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

p-durandin commented Jul 15, 2026

Uh oh!

Prithviraj-R commented Jul 16, 2026

Uh oh!

Prithviraj-R commented Jul 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Prithviraj-R commented Jun 30, 2026 •

edited

Loading

maxnick commented Jul 13, 2026 •

edited

Loading