Commit c2fec9f
committed
[GPU] Enable zero-copy subbuffers across all alloc types for iGPUs
#### Details:
Description of the issue (symptom, root-cause, how it was resolved)
- Eliminates redundant weight copies during model loading for integrated GPUs by binding
mmap'd cache file memory directly to GPU without intermediate usm_device allocation
- Reduces memory footprint during inference by reusing host memory as GPU-accessible buffers
- Zero-copy enabled when: iGPU, XE2+ architecture, USM host support available
- Falls back gracefully to copy-based path when zero-copy requirements not met
Implementation changes:
- Add ocl_engine::supports_hostbuffer() to validate zero-copy capability
(checks for integrated GPU, XE2+ arch, USM host allocation support)
- Add ocl_engine::create_hostbuffer_impl() to bind host memory with CL_MEM_USE_HOST_PTR
- Simplify zero-copy detection in data.hpp load path (remove redundant capability checks)
- Use ib.seek_current_ptr() instead of dummy reads for padding alignment
- Remove is_mmap_tensor_4K_aligned() helpers
Changed files:
src/plugins/intel_gpu/include/intel_gpu/graph/serialization/binary_buffer.hpp
src/plugins/intel_gpu/include/intel_gpu/primitives/data.hpp
src/plugins/intel_gpu/include/intel_gpu/runtime/device_info.hpp
src/plugins/intel_gpu/src/graph/program.cpp
src/plugins/intel_gpu/src/runtime/ocl/ocl_device.cpp
src/plugins/intel_gpu/src/runtime/ocl/ocl_engine.cpp
src/plugins/intel_gpu/tests/unit/test_cases/cache_serialization_test.cpp
Reproduction step and snapshot (if applicable. Do not attach for customer model)
NA
Problematic graph
NA
#### Checklist
- [x] Is it a proper fix? Yes (memory optimization, not a workaround)
- [x] Did you include test case for this fix, if necessary? NA
- [x] Did you review existing test that can be extended to cover this scenario? NA
#### Tickets:
- CVS-176160
#### AI Assistance:
- AI assistance used: yes
- AI was used for code implementation, naming suggestions, and comment refinement.1 parent 654ba6f commit c2fec9f
7 files changed
Lines changed: 55 additions & 76 deletions
File tree
- src/plugins/intel_gpu
- include/intel_gpu
- graph/serialization
- primitives
- runtime
- src
- graph
- runtime/ocl
- tests/unit/test_cases
Lines changed: 15 additions & 40 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
17 | | - | |
18 | | - | |
19 | | - | |
20 | | - | |
21 | | - | |
22 | | - | |
23 | | - | |
24 | 17 | | |
25 | 18 | | |
26 | 19 | | |
| |||
61 | 54 | | |
62 | 55 | | |
63 | 56 | | |
64 | | - | |
65 | | - | |
66 | | - | |
67 | 57 | | |
68 | | - | |
69 | | - | |
| 58 | + | |
| 59 | + | |
70 | 60 | | |
71 | | - | |
72 | | - | |
73 | | - | |
74 | | - | |
75 | | - | |
76 | | - | |
77 | | - | |
| 61 | + | |
| 62 | + | |
78 | 63 | | |
79 | 64 | | |
80 | 65 | | |
| |||
117 | 102 | | |
118 | 103 | | |
119 | 104 | | |
120 | | - | |
121 | | - | |
122 | | - | |
123 | | - | |
124 | | - | |
125 | | - | |
126 | | - | |
127 | 105 | | |
128 | | - | |
| 106 | + | |
129 | 107 | | |
130 | 108 | | |
131 | 109 | | |
| |||
142 | 120 | | |
143 | 121 | | |
144 | 122 | | |
145 | | - | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
146 | 129 | | |
147 | 130 | | |
148 | 131 | | |
149 | 132 | | |
150 | 133 | | |
151 | 134 | | |
152 | 135 | | |
153 | | - | |
154 | | - | |
155 | | - | |
156 | | - | |
157 | | - | |
158 | | - | |
159 | | - | |
160 | 136 | | |
161 | | - | |
162 | | - | |
| 137 | + | |
| 138 | + | |
163 | 139 | | |
164 | | - | |
165 | | - | |
166 | | - | |
| 140 | + | |
| 141 | + | |
167 | 142 | | |
168 | 143 | | |
169 | 144 | | |
| |||
Lines changed: 15 additions & 21 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
394 | 394 | | |
395 | 395 | | |
396 | 396 | | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
397 | 401 | | |
398 | | - | |
399 | | - | |
400 | | - | |
401 | | - | |
402 | 402 | | |
403 | 403 | | |
404 | 404 | | |
| |||
430 | 430 | | |
431 | 431 | | |
432 | 432 | | |
433 | | - | |
434 | | - | |
435 | | - | |
436 | | - | |
437 | | - | |
438 | | - | |
| 433 | + | |
| 434 | + | |
439 | 435 | | |
440 | 436 | | |
441 | 437 | | |
442 | 438 | | |
443 | 439 | | |
444 | 440 | | |
445 | | - | |
446 | | - | |
447 | | - | |
448 | | - | |
449 | | - | |
450 | | - | |
451 | | - | |
452 | | - | |
453 | | - | |
454 | | - | |
455 | | - | |
| 441 | + | |
| 442 | + | |
| 443 | + | |
| 444 | + | |
| 445 | + | |
| 446 | + | |
| 447 | + | |
| 448 | + | |
| 449 | + | |
456 | 450 | | |
457 | 451 | | |
458 | 452 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
145 | 145 | | |
146 | 146 | | |
147 | 147 | | |
148 | | - | |
| 148 | + | |
149 | 149 | | |
150 | 150 | | |
151 | 151 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1985 | 1985 | | |
1986 | 1986 | | |
1987 | 1987 | | |
1988 | | - | |
1989 | | - | |
1990 | | - | |
| 1988 | + | |
| 1989 | + | |
| 1990 | + | |
1991 | 1991 | | |
1992 | 1992 | | |
1993 | 1993 | | |
| |||
2017 | 2017 | | |
2018 | 2018 | | |
2019 | 2019 | | |
2020 | | - | |
2021 | | - | |
2022 | 2020 | | |
2023 | | - | |
| 2021 | + | |
| 2022 | + | |
| 2023 | + | |
| 2024 | + | |
| 2025 | + | |
| 2026 | + | |
| 2027 | + | |
| 2028 | + | |
| 2029 | + | |
| 2030 | + | |
2024 | 2031 | | |
2025 | | - | |
| 2032 | + | |
2026 | 2033 | | |
2027 | 2034 | | |
2028 | 2035 | | |
| |||
2214 | 2221 | | |
2215 | 2222 | | |
2216 | 2223 | | |
2217 | | - | |
2218 | | - | |
2219 | | - | |
2220 | | - | |
| 2224 | + | |
| 2225 | + | |
2221 | 2226 | | |
2222 | 2227 | | |
2223 | 2228 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
209 | 209 | | |
210 | 210 | | |
211 | 211 | | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
212 | 215 | | |
213 | 216 | | |
214 | 217 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
7 | | - | |
8 | 7 | | |
9 | 8 | | |
10 | 9 | | |
| |||
381 | 380 | | |
382 | 381 | | |
383 | 382 | | |
| 383 | + | |
384 | 384 | | |
385 | 385 | | |
386 | 386 | | |
| |||
Lines changed: 4 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
330 | 330 | | |
331 | 331 | | |
332 | 332 | | |
333 | | - | |
334 | | - | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
335 | 337 | | |
336 | 338 | | |
337 | 339 | | |
| |||
0 commit comments