Skip to content

[DRAFT: DONT COMMIT] Enable zero-copy subbuffers across all alloc typ…#36862

Open
Prithviraj-R wants to merge 1 commit into
openvinotoolkit:masterfrom
Prithviraj-R:zero_copy_bug_fix_cleanup
Open

[DRAFT: DONT COMMIT] Enable zero-copy subbuffers across all alloc typ…#36862
Prithviraj-R wants to merge 1 commit into
openvinotoolkit:masterfrom
Prithviraj-R:zero_copy_bug_fix_cleanup

Conversation

@Prithviraj-R

Copy link
Copy Markdown
Contributor

…es for iGPUs

Details:

Description of the issue (symptom, root-cause, how it was resolved)

  • Eliminates redundant weight copies during model loading for integrated GPUs by binding mmap'd cache file memory directly to GPU without intermediate usm_device allocation
  • Reduces memory footprint during inference by reusing host memory as GPU-accessible buffers
  • Zero-copy enabled when: iGPU, XE2+ architecture, USM host support available
  • Falls back gracefully to copy-based path when zero-copy requirements not met

Implementation changes:

  • Add ocl_engine::supports_hostbuffer() to validate zero-copy capability (checks for integrated GPU, XE2+ arch, USM host allocation support)
  • Add ocl_engine::create_hostbuffer_impl() to bind host memory with CL_MEM_USE_HOST_PTR
  • Simplify zero-copy detection in data.hpp load path (remove redundant capability checks)
  • Use ib.seek_current_ptr() instead of dummy reads for padding alignment
  • Remove is_mmap_tensor_4K_aligned() helpers

Changed files:
src/plugins/intel_gpu/include/intel_gpu/graph/serialization/binary_buffer.hpp src/plugins/intel_gpu/include/intel_gpu/primitives/data.hpp src/plugins/intel_gpu/include/intel_gpu/runtime/device_info.hpp src/plugins/intel_gpu/src/graph/program.cpp
src/plugins/intel_gpu/src/runtime/ocl/ocl_device.cpp src/plugins/intel_gpu/src/runtime/ocl/ocl_engine.cpp src/plugins/intel_gpu/tests/unit/test_cases/cache_serialization_test.cpp

Reproduction step and snapshot (if applicable. Do not attach for customer model) NA

Problematic graph
NA

Checklist

  • Is it a proper fix? Yes (memory optimization, not a workaround)
  • Did you include test case for this fix, if necessary? NA
  • Did you review existing test that can be extended to cover this scenario? NA

Tickets:

AI Assistance:

  • AI assistance used: yes
  • AI was used for code implementation, naming suggestions, and comment refinement.

Details:

  • item1
  • ...

Tickets:

  • ticket-id

AI Assistance:

  • AI assistance used: no / yes
  • If yes, summarize how AI was used and what human validation was performed (build/tests/manual checks).

@Prithviraj-R Prithviraj-R requested review from a team as code owners July 14, 2026 04:36
@Prithviraj-R Prithviraj-R marked this pull request as draft July 14, 2026 04:36
@github-actions github-actions Bot added the category: GPU OpenVINO GPU plugin label Jul 14, 2026
@Prithviraj-R Prithviraj-R force-pushed the zero_copy_bug_fix_cleanup branch from 5f3a5e3 to 5451093 Compare July 14, 2026 05:02
@Prithviraj-R Prithviraj-R marked this pull request as ready for review July 14, 2026 05:04
@Prithviraj-R Prithviraj-R force-pushed the zero_copy_bug_fix_cleanup branch 8 times, most recently from 02023e2 to b9eea12 Compare July 15, 2026 09:49
…es for iGPUs

Description of the issue (symptom, root-cause, how it was resolved)
- Eliminates redundant weight copies during model loading for integrated GPUs by binding
  mmap'd cache file memory directly to GPU without intermediate usm_device allocation
- Reduces memory footprint during inference by reusing host memory as GPU-accessible buffers
- Zero-copy enabled when: iGPU, XE2+ architecture, USM host support available
- Falls back gracefully to copy-based path when zero-copy requirements not met

Implementation changes:
- Add ocl_engine::supports_hostbuffer() to validate zero-copy capability
  (checks for integrated GPU, XE2+ arch, USM host allocation support)
- Add ocl_engine::create_hostbuffer_impl() to bind host memory with CL_MEM_USE_HOST_PTR
- Simplify zero-copy detection in data.hpp load path (remove redundant capability checks)
- Use ib.seek_current_ptr() instead of dummy reads for padding alignment
- Remove is_mmap_tensor_4K_aligned() helpers

Changed files:
src/plugins/intel_gpu/include/intel_gpu/graph/serialization/binary_buffer.hpp
src/plugins/intel_gpu/include/intel_gpu/primitives/data.hpp
src/plugins/intel_gpu/include/intel_gpu/runtime/device_info.hpp
src/plugins/intel_gpu/src/graph/program.cpp
src/plugins/intel_gpu/src/runtime/ocl/ocl_device.cpp
src/plugins/intel_gpu/src/runtime/ocl/ocl_engine.cpp
src/plugins/intel_gpu/tests/unit/test_cases/cache_serialization_test.cpp

Reproduction step and snapshot (if applicable. Do not attach for customer model)
NA

Problematic graph
NA

- [x] Is it a proper fix? Yes (memory optimization, not a workaround)
- [x] Did you include test case for this fix, if necessary? NA
- [x] Did you review existing test that can be extended to cover this scenario? NA

 - CVS-176160

 - AI assistance used: yes
 - AI was used for code implementation, naming suggestions, and comment refinement.
@Prithviraj-R Prithviraj-R force-pushed the zero_copy_bug_fix_cleanup branch from b9eea12 to 96cd85d Compare July 15, 2026 12:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: GPU OpenVINO GPU plugin

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant