You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
<1> The name `dpdk1` in this example is a user-created `SriovNetworkNodePolicy` resource. You can substitute this name for that of a resource that you create.
55
-
<2> If your performance profile is not named `cnf-performance profile`, replace that string with the correct performance profile name.
55
+
56
+
--
57
+
* The name `dpdk1` in this example is a user-created `SriovNetworkNodePolicy` resource. You can substitute this name for that of a resource that you create.
58
+
* If your performance profile is not named `cnf-performance profile`, replace that string with the correct performance profile name.
Copy file name to clipboardExpand all lines: modules/nw-running-dpdk-rootless-tap.adoc
+57-51Lines changed: 57 additions & 51 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,9 +6,10 @@
6
6
[id="nw-running-dpdk-rootless-tap_{context}"]
7
7
= Using the TAP CNI to run a rootless DPDK workload with kernel access
8
8
9
+
[role="_abstract"]
9
10
DPDK applications can use `virtio-user` as an exception path to inject certain types of packets, such as log messages, into the kernel for processing. For more information about this feature, see link:https://doc.dpdk.org/guides/howto/virtio_user_as_exception_path.html[Virtio_user as Exception Path].
10
11
11
-
In OpenShift Container Platform version 4.14 and later, you can use non-privileged pods to run DPDK applications alongside the tap CNI plugin. To enable this functionality, you need to mount the `vhost-net` device by setting the `needVhostNet` parameter to `true` within the `SriovNetworkNodePolicy` object.
12
+
In {product-title} version 4.14 and later, you can use non-privileged pods to run DPDK applications alongside the tap CNI plugin. To enable this functionality, you need to mount the `vhost-net` device by setting the `needVhostNet` parameter to `true` within the `SriovNetworkNodePolicy` object.
12
13
13
14
.DPDK and TAP example configuration
14
15
image::348_OpenShift_rootless_DPDK_0923.png[DPDK and TAP plugin]
@@ -27,7 +28,7 @@ Use the Machine Config Operator to set this SELinux boolean.
27
28
28
29
.Procedure
29
30
30
-
. Create a file, such as `test-namespace.yaml`, with content like the following example:
31
+
. Create a file, such as `test-namespace.yaml`, with content such as the following example:
31
32
+
32
33
[source,yaml]
33
34
----
@@ -49,7 +50,7 @@ metadata:
49
50
$ oc apply -f test-namespace.yaml
50
51
----
51
52
52
-
. Create a file, such as `sriov-node-network-policy.yaml`, with content like the following example::
53
+
. Create a file, such as `sriov-node-network-policy.yaml`, with content such as the following example:
<1> This indicates that the profile is tailored specifically for Mellanox Network Interface Controllers (NICs).
76
-
<2> Setting `isRdma` to `true` is only required for a Mellanox NIC.
77
-
<3> This mounts the `/dev/net/tun` and `/dev/vhost-net` devices into the container so the application can create a tap device and connect the tap device to the DPDK workload.
78
-
<4> The vendor hexadecimal code of the SR-IOV network device. The value 15b3 is associated with a Mellanox NIC.
79
-
<5> The device hexadecimal code of the SR-IOV network device.
76
+
+
77
+
where:
78
+
+
79
+
`spec.deviceType`:: Specifies that the profile is tailored specifically for Mellanox Network Interface Controllers (NICs). Set to `netdevice`.
80
+
`spec.isRdma`:: Setting to `true` is only required for a Mellanox NIC.
81
+
`spec.needVhostNet`:: Setting to `true` mounts the `/dev/net/tun` and `/dev/vhost-net` devices into the container so the application can create a tap device and connect the tap device to the DPDK workload.
82
+
`spec.nicSelector.vendor`:: Specifies the vendor hexadecimal code of the SR-IOV network device. The value `"15b3"` is associated with a Mellanox NIC.
83
+
`spec.nicSelector.deviceID`:: Specifies the device hexadecimal code of the SR-IOV network device.
80
84
81
85
. Create the `SriovNetworkNodePolicy` object by running the following command:
82
86
+
@@ -115,15 +119,15 @@ An optional library, `app-netutil`, provides several API methods for gathering n
115
119
$ oc create -f sriov-network-attachment.yaml
116
120
----
117
121
118
-
. Create a file, such as `tap-example.yaml`, that defines a network attachment definition, with content like the following example:
122
+
. Create a file, such as `tap-example.yaml`, that defines a network attachment definition, with content such as the following example:
119
123
+
120
124
[source,yaml]
121
125
----
122
126
apiVersion: "k8s.cni.cncf.io/v1"
123
127
kind: NetworkAttachmentDefinition
124
128
metadata:
125
129
name: tap-one
126
-
namespace: test-namespace <1>
130
+
namespace: test-namespace
127
131
spec:
128
132
config: '{
129
133
"cniVersion": "0.4.0",
@@ -143,7 +147,10 @@ spec:
143
147
]
144
148
}'
145
149
----
146
-
<1> Specify the same `target_namespace` where the `SriovNetwork` object is created.
150
+
+
151
+
where:
152
+
+
153
+
`metadata.namespace`:: Specifies the same `target_namespace` where the `SriovNetwork` object is created.
147
154
148
155
. Create the `NetworkAttachmentDefinition` object by running the following command:
149
156
+
@@ -152,15 +159,15 @@ spec:
152
159
$ oc apply -f tap-example.yaml
153
160
----
154
161
155
-
. Create a file, such as `dpdk-pod-rootless.yaml`, with content like the following example:
162
+
. Create a file, such as `dpdk-pod-rootless.yaml`, with content such as the following example:
<1> Specify the same `target_namespace` in which the `SriovNetwork` object is created. If you want to create the pod in a different namespace, change `target_namespace` in both the `Pod` spec and the `SriovNetwork` object.
213
-
<2> Sets the group ownership of volume-mounted directories and files created in those volumes.
214
-
<3> Specify the primary group ID used for running the container.
215
-
<4> Specify the DPDK image that contains your application and the DPDK library used by application.
216
-
<5> Removing all capabilities (`ALL`) from the container's securityContext means that the container has no special privileges beyond what is necessary for normal operation.
217
-
<6> Specify additional capabilities required by the application inside the container for hugepage allocation, system resource allocation, and network interface access. These capabilities must also be set in the binary file by using the `setcap` command.
218
-
<7> Mellanox network interface controller (NIC) requires the `NET_RAW` capability.
219
-
<8> Specify the user ID used for running the container.
220
-
<9> This setting indicates that the container or containers within the pod should not be granted privileged access to the host system.
221
-
<10> This setting allows a container to escalate its privileges beyond the initial non-root privileges it might have been assigned.
222
-
<11> This setting ensures that the container runs with a non-root user. This helps enforce the principle of least privilege, limiting the potential impact of compromising the container and reducing the attack surface.
223
-
<12> Mount a hugepage volume to the DPDK pod under `/mnt/huge`. The hugepage volume is backed by the emptyDir volume type with the medium being `Hugepages`.
224
-
<13> Optional: Specify the number of DPDK devices allocated for the DPDK pod. If not explicitly specified, this resource request and limit is automatically added by the SR-IOV network resource injector. The SR-IOV network resource injector is an admission controller component managed by SR-IOV Operator. It is enabled by default and can be disabled by setting the `enableInjector` option to `false` in the default `SriovOperatorConfig` CR.
225
-
<14> Specify the number of CPUs. The DPDK pod usually requires exclusive CPUs to be allocated from the kubelet. This is achieved by setting CPU Manager policy to `static` and creating a pod with `Guaranteed` QoS.
226
-
<15> Specify hugepage size `hugepages-1Gi` or `hugepages-2Mi` and the quantity of hugepages that will be allocated to the DPDK pod. Configure `2Mi` and `1Gi` hugepages separately. Configuring `1Gi` hugepage requires adding kernel arguments to Nodes. For example, adding kernel arguments `default_hugepagesz=1GB`, `hugepagesz=1G` and `hugepages=16` will result in `16*1Gi` hugepages be allocated during system boot.
227
-
<16> If your performance profile is not named `cnf-performance profile`, replace that string with the correct performance profile name.
228
-
--
218
+
where:
229
219
+
220
+
`metadata.namespace`:: Specifies the same `target_namespace` in which the `SriovNetwork` object is created. If you want to create the pod in a different namespace, change `target_namespace` in both the `Pod` spec and the `SriovNetwork` object.
221
+
`spec.securityContext.fsGroup`:: Sets the group ownership of volume-mounted directories and files created in those volumes.
222
+
`spec.securityContext.runAsGroup`:: Specifies the primary group ID used for running the container.
223
+
`spec.containers.image`:: Specifies the DPDK image that contains your application and the DPDK library used by application.
224
+
`spec.containers.securityContext.capabilities.drop`:: Removing all capabilities (`ALL`) from the container's `securityContext` means that the container has no special privileges beyond what is necessary for normal operation.
225
+
`spec.containers.securityContext.capabilities.add`:: Specifies additional capabilities required by the application inside the container for hugepage allocation, system resource allocation, and network interface access. These capabilities must also be set in the binary file by using the `setcap` command. Mellanox network interface controller (NIC) requires the `NET_RAW` capability.
226
+
`spec.containers.securityContext.runAsUser`:: Specifies the user ID used for running the container.
227
+
`spec.containers.securityContext.privileged`:: Setting to `false` indicates that the container or containers within the pod should not be granted privileged access to the host system.
228
+
`spec.containers.securityContext.allowPrivilegeEscalation`:: Setting to `true` allows a container to escalate its privileges beyond the initial non-root privileges it might have been assigned.
229
+
`spec.containers.securityContext.runAsNonRoot`:: Setting to `true` ensures that the container runs with a non-root user. This helps enforce the principle of least privilege, limiting the potential impact of compromising the container and reducing the attack surface.
230
+
`spec.containers.volumeMounts.mountPath`:: Specifies the path where a hugepage volume is mounted in the DPDK pod. The hugepage volume is backed by the `emptyDir` volume type with the medium being `Hugepages`.
231
+
`spec.containers.resources.limits.openshift.io/sriovnic`:: Optional: Specifies the number of DPDK devices allocated for the DPDK pod. If not explicitly specified, this resource request and limit is automatically added by the SR-IOV network resource injector. The SR-IOV network resource injector is an admission controller component managed by SR-IOV Operator. It is enabled by default and can be disabled by setting the `enableInjector` option to `false` in the default `SriovOperatorConfig` CR.
232
+
`spec.containers.resources.limits.cpu`:: Specifies the number of CPUs. The DPDK pod usually requires exclusive CPUs to be allocated from the kubelet. This is achieved by setting CPU Manager policy to `static` and creating a pod with `Guaranteed` QoS.
233
+
`spec.containers.resources.limits.hugepages-1Gi`:: Specifies the hugepage size `hugepages-1Gi` or `hugepages-2Mi` and the quantity of hugepages that will be allocated to the DPDK pod. Configure `2Mi` and `1Gi` hugepages separately. Configuring `1Gi` hugepage requires adding kernel arguments to Nodes. For example, adding kernel arguments `default_hugepagesz=1GB`, `hugepagesz=1G` and `hugepages=16` will result in `16*1Gi` hugepages be allocated during system boot.
234
+
`spec.runtimeClassName`:: Specifies the performance profile runtime class. If your performance profile is not named `cnf-performance profile`, replace that string with the correct performance profile name.
235
+
230
236
. Create the DPDK pod by running the following command:
Copy file name to clipboardExpand all lines: modules/nw-sriov-app-netutil.adoc
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,6 +6,7 @@
6
6
[id="nw-sriov-app-netutil_{context}"]
7
7
= DPDK library for use with container applications
8
8
9
+
[role="_abstract"]
9
10
An link:https://github.com/openshift/app-netutil[optional library], `app-netutil`, provides several API methods for gathering network information about a pod from within a container running within that pod.
10
11
11
12
This library can assist with integrating SR-IOV virtual functions (VFs) in Data Plane Development Kit (DPDK) mode into the container.
Copy file name to clipboardExpand all lines: modules/nw-sriov-concept-dpdk-line-rate.adoc
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,6 +6,7 @@
6
6
[id="nw-sriov-example-dpdk-line-rate_{context}"]
7
7
= Overview of achieving a specific DPDK line rate
8
8
9
+
[role="_abstract"]
9
10
To achieve a specific Data Plane Development Kit (DPDK) line rate, deploy a Node Tuning Operator and configure Single Root I/O Virtualization (SR-IOV). You must also tune the DPDK settings for the following resources:
10
11
11
12
- Isolated CPUs
@@ -17,13 +18,12 @@ To achieve a specific Data Plane Development Kit (DPDK) line rate, deploy a Node
17
18
In previous versions of {product-title}, the Performance Addon Operator was used to implement automatic tuning to achieve low latency performance for {product-title} applications. In {product-title} 4.11 and later, this functionality is part of the Node Tuning Operator.
18
19
====
19
20
20
-
.DPDK test environment
21
-
The following diagram shows the components of a traffic-testing environment:
21
+
The following diagram shows the components of a DPDK test environment:
22
22
23
23
image::261_OpenShift_DPDK_0722.png[DPDK test environment]
24
24
25
25
- **Traffic generator**: An application that can generate high-volume packet traffic.
26
-
- **SR-IOV-supporting NIC**: A network interface card compatible with SR-IOV. The card runs a number of virtual functions on a physical interface.
26
+
- **SR-IOV-supporting NIC**: A network interface controller (NIC) compatible with SR-IOV. The card runs several virtual functions on a physical interface.
27
27
- **Physical Function (PF)**: A PCI Express (PCIe) function of a network adapter that supports the SR-IOV interface.
28
28
- **Virtual Function (VF)**: A lightweight PCIe function on a network adapter that supports SR-IOV. The VF is associated with the PCIe PF on the network adapter. The VF represents a virtualized instance of the network adapter.
29
29
- **Switch**: A network switch. Nodes can also be connected back-to-back.
<1> You can use a different IP Address Management (IPAM) implementation, such as Whereabouts. For more information, see _Dynamic IP address assignment configuration with Whereabouts_.
39
-
<2> You must request the `networkNamespace` where the network attachment definition will be created. You must create the `sriovNetwork` CR under the `openshift-sriov-network-operator` namespace.
40
-
<3> The `resourceName` value must match that of the `resourceName` created under the `sriovNetworkNodePolicy`.
39
+
40
+
--
41
+
* You can use a different IP Address Management (IPAM) implementation, such as Whereabouts. For more information, see _Dynamic IP address assignment configuration with Whereabouts_.
42
+
* You must request the `networkNamespace` where the network attachment definition will be created. You must create the `sriovNetwork` CR under the `openshift-sriov-network-operator` namespace.
43
+
* The `resourceName` value must match that of the `resourceName` created under the `sriovNetworkNodePolicy`.
0 commit comments