[DRAFT: DONT COMMIT] Enable zero-copy subbuffers across all alloc typ…#36862
Open
Prithviraj-R wants to merge 1 commit into
Open
[DRAFT: DONT COMMIT] Enable zero-copy subbuffers across all alloc typ…#36862Prithviraj-R wants to merge 1 commit into
Prithviraj-R wants to merge 1 commit into
Conversation
5f3a5e3 to
5451093
Compare
02023e2 to
b9eea12
Compare
…es for iGPUs Description of the issue (symptom, root-cause, how it was resolved) - Eliminates redundant weight copies during model loading for integrated GPUs by binding mmap'd cache file memory directly to GPU without intermediate usm_device allocation - Reduces memory footprint during inference by reusing host memory as GPU-accessible buffers - Zero-copy enabled when: iGPU, XE2+ architecture, USM host support available - Falls back gracefully to copy-based path when zero-copy requirements not met Implementation changes: - Add ocl_engine::supports_hostbuffer() to validate zero-copy capability (checks for integrated GPU, XE2+ arch, USM host allocation support) - Add ocl_engine::create_hostbuffer_impl() to bind host memory with CL_MEM_USE_HOST_PTR - Simplify zero-copy detection in data.hpp load path (remove redundant capability checks) - Use ib.seek_current_ptr() instead of dummy reads for padding alignment - Remove is_mmap_tensor_4K_aligned() helpers Changed files: src/plugins/intel_gpu/include/intel_gpu/graph/serialization/binary_buffer.hpp src/plugins/intel_gpu/include/intel_gpu/primitives/data.hpp src/plugins/intel_gpu/include/intel_gpu/runtime/device_info.hpp src/plugins/intel_gpu/src/graph/program.cpp src/plugins/intel_gpu/src/runtime/ocl/ocl_device.cpp src/plugins/intel_gpu/src/runtime/ocl/ocl_engine.cpp src/plugins/intel_gpu/tests/unit/test_cases/cache_serialization_test.cpp Reproduction step and snapshot (if applicable. Do not attach for customer model) NA Problematic graph NA - [x] Is it a proper fix? Yes (memory optimization, not a workaround) - [x] Did you include test case for this fix, if necessary? NA - [x] Did you review existing test that can be extended to cover this scenario? NA - CVS-176160 - AI assistance used: yes - AI was used for code implementation, naming suggestions, and comment refinement.
b9eea12 to
96cd85d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…es for iGPUs
Details:
Description of the issue (symptom, root-cause, how it was resolved)
Implementation changes:
Changed files:
src/plugins/intel_gpu/include/intel_gpu/graph/serialization/binary_buffer.hpp src/plugins/intel_gpu/include/intel_gpu/primitives/data.hpp src/plugins/intel_gpu/include/intel_gpu/runtime/device_info.hpp src/plugins/intel_gpu/src/graph/program.cpp
src/plugins/intel_gpu/src/runtime/ocl/ocl_device.cpp src/plugins/intel_gpu/src/runtime/ocl/ocl_engine.cpp src/plugins/intel_gpu/tests/unit/test_cases/cache_serialization_test.cpp
Reproduction step and snapshot (if applicable. Do not attach for customer model) NA
Problematic graph
NA
Checklist
Tickets:
AI Assistance:
Details:
Tickets:
AI Assistance: