Commit 28f8cc5
committed
[DRAFT: DONT COMMIT] Enable zero-copy subbuffers across all alloc types for iGPUs
#### Details:
Description of the issue (symptom, root-cause, how it was resolved)
- Eliminates redundant weight copies during model loading for integrated GPUs by binding
mmap'd cache file memory directly to GPU without intermediate usm_device allocation
- Reduces memory footprint during inference by reusing host memory as GPU-accessible buffers
- Zero-copy enabled when: iGPU, XE2+ architecture, USM host support available
- Falls back gracefully to copy-based path when zero-copy requirements not met
Implementation changes:
- Add ocl_engine::supports_hostbuffer() to validate zero-copy capability
(checks for integrated GPU, XE2+ arch, USM host allocation support)
- Add ocl_engine::create_hostbuffer_impl() to bind host memory with CL_MEM_USE_HOST_PTR
- Simplify zero-copy detection in data.hpp load path (remove redundant capability checks)
- Use ib.seek_current_ptr() instead of dummy reads for padding alignment
- Remove is_mmap_tensor_4K_aligned() helpers
Changed files:
src/plugins/intel_gpu/include/intel_gpu/graph/serialization/binary_buffer.hpp
src/plugins/intel_gpu/include/intel_gpu/primitives/data.hpp
src/plugins/intel_gpu/include/intel_gpu/runtime/device_info.hpp
src/plugins/intel_gpu/src/graph/program.cpp
src/plugins/intel_gpu/src/runtime/ocl/ocl_device.cpp
src/plugins/intel_gpu/src/runtime/ocl/ocl_engine.cpp
src/plugins/intel_gpu/tests/unit/test_cases/cache_serialization_test.cpp
Reproduction step and snapshot (if applicable. Do not attach for customer model)
NA
Problematic graph
NA
#### Checklist
- [x] Is it a proper fix? Yes (memory optimization, not a workaround)
- [x] Did you include test case for this fix, if necessary? NA
- [x] Did you review existing test that can be extended to cover this scenario? NA
#### Tickets:
- CVS-176160
#### AI Assistance:
- AI assistance used: yes
- AI was used for code implementation, naming suggestions, and comment refinement.1 parent bbb6398 commit 28f8cc5
7 files changed
Lines changed: 40 additions & 51 deletions
File tree
- src/plugins/intel_gpu
- include/intel_gpu
- graph/serialization
- primitives
- runtime
- src
- graph
- runtime/ocl
- tests/unit/test_cases
Lines changed: 12 additions & 36 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
17 | | - | |
18 | | - | |
19 | | - | |
20 | | - | |
21 | | - | |
22 | | - | |
23 | | - | |
24 | 17 | | |
25 | 18 | | |
26 | 19 | | |
| |||
61 | 54 | | |
62 | 55 | | |
63 | 56 | | |
64 | | - | |
65 | | - | |
66 | | - | |
67 | | - | |
68 | | - | |
69 | | - | |
70 | | - | |
71 | | - | |
72 | | - | |
73 | | - | |
| 57 | + | |
| 58 | + | |
74 | 59 | | |
75 | 60 | | |
76 | | - | |
77 | | - | |
| 61 | + | |
| 62 | + | |
78 | 63 | | |
79 | 64 | | |
80 | 65 | | |
| |||
117 | 102 | | |
118 | 103 | | |
119 | 104 | | |
120 | | - | |
| 105 | + | |
121 | 106 | | |
122 | 107 | | |
123 | 108 | | |
124 | | - | |
125 | | - | |
| 109 | + | |
| 110 | + | |
126 | 111 | | |
127 | 112 | | |
128 | | - | |
| 113 | + | |
129 | 114 | | |
130 | 115 | | |
131 | 116 | | |
| |||
150 | 135 | | |
151 | 136 | | |
152 | 137 | | |
153 | | - | |
154 | | - | |
| 138 | + | |
| 139 | + | |
155 | 140 | | |
156 | | - | |
157 | | - | |
158 | | - | |
159 | | - | |
160 | | - | |
161 | | - | |
162 | | - | |
163 | | - | |
164 | | - | |
165 | | - | |
166 | | - | |
| 141 | + | |
| 142 | + | |
167 | 143 | | |
168 | 144 | | |
169 | 145 | | |
| |||
Lines changed: 6 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
395 | 395 | | |
396 | 396 | | |
397 | 397 | | |
398 | | - | |
399 | | - | |
| 398 | + | |
| 399 | + | |
400 | 400 | | |
401 | 401 | | |
402 | 402 | | |
| |||
431 | 431 | | |
432 | 432 | | |
433 | 433 | | |
434 | | - | |
| 434 | + | |
| 435 | + | |
435 | 436 | | |
436 | 437 | | |
437 | 438 | | |
| |||
443 | 444 | | |
444 | 445 | | |
445 | 446 | | |
446 | | - | |
447 | | - | |
| 447 | + | |
| 448 | + | |
448 | 449 | | |
449 | 450 | | |
450 | 451 | | |
| |||
Lines changed: 1 addition & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
145 | 145 | | |
146 | 146 | | |
147 | 147 | | |
| 148 | + | |
148 | 149 | | |
149 | 150 | | |
150 | 151 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1985 | 1985 | | |
1986 | 1986 | | |
1987 | 1987 | | |
1988 | | - | |
1989 | | - | |
1990 | | - | |
| 1988 | + | |
| 1989 | + | |
1991 | 1990 | | |
1992 | 1991 | | |
1993 | 1992 | | |
| |||
2017 | 2016 | | |
2018 | 2017 | | |
2019 | 2018 | | |
2020 | | - | |
| 2019 | + | |
| 2020 | + | |
2021 | 2021 | | |
2022 | 2022 | | |
2023 | 2023 | | |
2024 | 2024 | | |
2025 | | - | |
| 2025 | + | |
2026 | 2026 | | |
2027 | 2027 | | |
2028 | 2028 | | |
| |||
2215 | 2215 | | |
2216 | 2216 | | |
2217 | 2217 | | |
2218 | | - | |
2219 | | - | |
| 2218 | + | |
| 2219 | + | |
2220 | 2220 | | |
2221 | 2221 | | |
2222 | 2222 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
209 | 209 | | |
210 | 210 | | |
211 | 211 | | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
212 | 215 | | |
213 | 216 | | |
214 | 217 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
7 | | - | |
8 | 7 | | |
9 | 8 | | |
10 | 9 | | |
| |||
Lines changed: 11 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
329 | 329 | | |
330 | 330 | | |
331 | 331 | | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
332 | 340 | | |
333 | | - | |
334 | | - | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
335 | 344 | | |
336 | 345 | | |
337 | 346 | | |
| |||
0 commit comments