Skip to content

Commit 89af464

Browse files
authored
Documentation: Update docs on IPC/P2P Usage of Reserved Memory (#474)
Signed-off-by: Neil R. Spruit <neil.r.spruit@intel.com>
1 parent 16fc96c commit 89af464

4 files changed

Lines changed: 561 additions & 7 deletions

File tree

scripts/core/EXT_IPC_MEM_HANDLE_TYPE.rst

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,17 @@ When users are requesting IPC handles for L0 memory, there may be requirements f
3939

4040
To facilitate this, the IPC Memory Handle Type Extension provides a mechanism to specify the type of IPC handle to be created during memory allocation or when obtaining an IPC handle for existing memory.
4141

42+
Physical memory handles (``${x}_physical_mem_handle_t``) may be used as a source for IPC handles via
43+
${x}MemGetIpcHandleWithProperties. The ``ptr`` argument may be either:
44+
45+
- A virtual address that was mapped to a physical memory object via ${x}VirtualMemMap, or
46+
- A value returned directly from ${x}PhysicalMemCreate.
47+
48+
Only one physical memory object may be associated with a single IPC handle at a time. When the
49+
resulting IPC handle is opened in the receiving process via ${x}MemOpenIpcHandle, the driver
50+
assigns a new virtual address in the importing process without requiring a separate
51+
${x}VirtualMemReserve or ${x}VirtualMemMap call.
52+
4253
Checking IPC Support
4354
~~~~~~~~~~~~~~~~~~~~~
4455

@@ -77,6 +88,65 @@ To request an IPC handle for L0 memory that is fabric accessible, the user would
7788

7889
If the driver does not support ``ZE_IPC_MEM_HANDLE_TYPE_FLAG_FABRIC_ACCESSIBLE``, the function will return ``ZE_RESULT_ERROR_UNSUPPORTED_FEATURE``. Applications can proactively check for support by querying ``${x}_driver_ipc_properties_t`` as shown in the "Checking IPC Support" section above.
7990

91+
Get IPC Handle from Physical Memory Example
92+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
93+
94+
To obtain an IPC handle from a physical memory object, the ``ptr`` argument passed to
95+
${x}MemGetIpcHandleWithProperties may be either the physical memory handle cast to a pointer,
96+
or a virtual address previously mapped to the physical memory object via ${x}VirtualMemMap.
97+
The following example demonstrates the mapped VA case:
98+
99+
**Sending process:**
100+
101+
.. parsed-literal::
102+
103+
// Create a physical memory object
104+
ze_physical_mem_desc_t pmemDesc = {
105+
.stype = ZE_STRUCTURE_TYPE_PHYSICAL_MEM_DESC,
106+
.pNext = NULL,
107+
.flags = 0,
108+
.size = physicalSize
109+
};
110+
ze_physical_mem_handle_t hPhysicalMem;
111+
zePhysicalMemCreate(hContext, hDevice, &pmemDesc, &hPhysicalMem);
112+
113+
// Reserve VA and map the physical memory
114+
void* ptr = NULL;
115+
zeVirtualMemReserve(hContext, NULL, physicalSize, &ptr);
116+
zeVirtualMemMap(hContext, ptr, physicalSize, hPhysicalMem, 0,
117+
ZE_MEMORY_ACCESS_ATTRIBUTE_READWRITE);
118+
119+
// Request a fabric-accessible IPC handle for the mapped physical memory
120+
ze_ipc_mem_handle_type_ext_desc_t desc = {
121+
.stype = ZE_STRUCTURE_TYPE_IPC_MEM_HANDLE_TYPE_EXT_DESC,
122+
.pNext = NULL,
123+
.typeFlags = ZE_IPC_MEM_HANDLE_TYPE_FLAG_FABRIC_ACCESSIBLE
124+
};
125+
ze_ipc_mem_handle_t ipcHandle;
126+
ze_result_t result = zeMemGetIpcHandleWithProperties(hContext, ptr, &ipcHandle, &desc);
127+
128+
// NOTE: Transferring IPC handles is not a Level Zero API. The application is responsible
129+
// for sending the handle to the receiving process using its preferred out of band method
130+
// (e.g., Shared memory, TCP/IP, fabric, etc.)
131+
132+
**Receiving process:**
133+
134+
.. parsed-literal::
135+
136+
// NOTE: Receiving IPC handles is not a Level Zero API. The application is responsible
137+
// for receiving the handle from the sending process using its preferred out of band method
138+
// (e.g., Shared memory, TCP/IP, fabric, etc.)
139+
ze_ipc_mem_handle_t ipcHandle;
140+
141+
// Opening the handle assigns a new VA in this process for the physical memory.
142+
void* remotePtr = NULL;
143+
zeMemOpenIpcHandle(hContext, hDevice, ipcHandle, 0, &remotePtr);
144+
145+
.. note::
146+
147+
Only one physical memory object may be mapped per IPC handle. Passing a ``ptr`` that spans
148+
multiple physical memory mappings will result in an error.
149+
80150

81151
Memory Allocation Usage Example
82152
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

0 commit comments

Comments
 (0)