clarify CB Events Device assignment (#476)

Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>

Author: BartoszDunajski

scripts/core/PROG.rst

@@ -1612,6 +1612,7 @@ Key features
 - ${x}EventHostSignal is not allowed. Can be signaled only from in-order command list
 - No need to wait for completion before reusing/destroying
 - CB Event doesn't own any memory allocations. Can be reused/destroyed with low cost. Timestamp allocation is also handled internally by the Driver
+- A CB Event's device association is not fixed at creation. It re-associates with the signaling device on each signal operation. See `Device association`_.
 - IPC sharing is one-directional. IPC CB Event opened in different process can be used only for waiting. If original Event state is changed (for example by next append call) and second process needs to see that update, IPC handle must be opened again.
 - Regular command list (known as recorded or non-immediate) is a special use case for CB Events. Will be described in separate section
 - When Event is reset (assigned as signal event to new append call), new timestamp data storage is provided implicitly. User can immediately query new data, without handling the completion
@@ -1634,6 +1635,21 @@ Regular Event rely on memory state controlled by the user (explicit Reset calls)
 
        ${x}EventHostSynchronize(event1, UINT32_MAX); // wait for counter=Y on memory CL3_alloc
 
+Device association
+^^^^^^^^^^^^^^^^^^^
+Unlike Regular Events, a CB Event's device association is not fixed by the device passed to ${x}EventCounterBasedCreate. Its association follows the latest signaling point:
+
+- After creation, the Event is associated with the device passed to ${x}EventCounterBasedCreate. This initial association does not restrict on which device the Event may later be signaled.
+- When the Event is passed as a signal event to an append call, it drops its previous tracking point and re-associates with the device on which that command list was created. The Event may be signaled on any device, regardless of the device used to create it or the device that signaled it previously. No peer-to-peer access between the previous and the new signaling device is required.
+- Waiting on the Event (for example ${x}CommandListAppendWaitOnEvents) does not change its association. The waiting device must have peer-to-peer access to the device that last had the Event enqueued for signaling (or, if the Event has not been enqueued for signaling yet, the device it is currently associated with).
+
+.. parsed-literal::
+       ${x}EventCounterBasedCreate(context, deviceA, &desc, &event); // associated with deviceA
+       ${x}CommandListAppendSignalEvent(cmdListDeviceB, event);       // re-associates with deviceB (no P2P deviceA<->deviceB required)
+       ${x}CommandListAppendWaitOnEvents(cmdListDeviceC, 1, &event);  // deviceC must have P2P access to deviceB
+
+This is also the basis for IPC sharing: an opened IPC CB Event tracks the signaling device and can be waited on from any device that has peer-to-peer access to it.
+
 IPC sharing
 ^^^^^^^^^^^
 As mentioned previously, signaling CB Event replaces its state. This is why IPC sharing is one-directional. Opened event can be used only for waiting/querying (on host and GPU).

scripts/core/event.yml

@@ -492,7 +492,8 @@ analogue:
     - "**clSetUserEventStatus**"
     - vkCmdSetEvent
 details:
-    - "The application must ensure the events are accessible by the device on which the command list was created."
+    - "The application must ensure the events are accessible by the device on which the command list was created. This requirement does not apply to counter-based events."
+    - "For counter-based events, the event may be signaled on any device, regardless of the device used to create the event. When a counter-based event is passed for signaling, it drops its previous tracking point and re-associates with the device on which the command list was created, tracking the new signaling point. No peer-to-peer access between the previously associated device and the new signaling device is required."
     - "The duration of an event created from an event pool that was created using $X_EVENT_POOL_FLAG_KERNEL_TIMESTAMP or $X_EVENT_POOL_FLAG_KERNEL_MAPPED_TIMESTAMP flags is undefined. However, for consistency and orthogonality the event will report correctly as signaled when used by other event API functionality."
     - "The application must ensure the command list and events were created on the same context."
     - "The application must **not** call this function from simultaneous threads with the same command list handle."
@@ -513,6 +514,7 @@ class: $xCommandList
 name: AppendWaitOnEvents
 details:
     - "The application must ensure the events are accessible by the device on which the command list was created."
+    - "For counter-based events, the device on which the command list was created must have peer-to-peer access to the device that last signaled the event. Unlike signaling, waiting on a counter-based event does not re-associate it with another device."
     - "The application must ensure the command list and events were created on the same context."
     - "The application must **not** call this function from simultaneous threads with the same command list handle."
     - "The implementation of this function should be lock-free."