Content dev

Author: madeline-underwood

content/learning-paths/servers-and-cloud-computing/tune-network-workloads-on-bare-metal/3_nic-queue.md

@@ -1,23 +1,22 @@
 ---
-title: Tuning via NIC queue count
+title: Tune performance with NIC queue counts
 weight: 4
 
 ### FIXED, DO NOT MODIFY
 layout: learningpathall
 ---
 
-## Tuning via NIC queue count
-To further optmize your settings, you can set the NIC queue count and observe the performance uplift:
+## Overview
 
-Typically, the number of transmit/receive queues for network cards in bare-metal environments is relatively large, reaching 63 on Arm Neoverse. Each transmit/receive queue corresponds to one interrupt number. Before CPU cores are taken offline, there are sufficient cores to handle these interrupt numbers. However, when only 8 cores are retained, it results in a single core having to handle multiple interrupt numbers, thereby triggering more context switches.
+After establishing a baseline, you can further improve throughput and latency by tuning your network interface controller (NIC) queue count. Many bare‑metal NICs expose a relatively large number of transmit/receive queues (multi‑queue). Each queue maps to an interrupt; when you limit the server to a small number of online CPUs (for example, 8 cores), too many interrupts can land on the same core and increase context switching. Reducing the number of queues to match available CPUs can stabilize performance on Arm Neoverse systems.
 
-### Setting NIC queue count
+## Set the NIC queue count
 
-1. Use the following command to find the NIC name corresponding to he IP address.
+Use the following command to find the NIC name corresponding to the IP address:
 ```bash
 ip addr
 ```
-From the output you can see that the NIC name `enp1s0f0np0` corresponds to the IP address `10.169.226.181`.
+From the output you can see that the NIC name `enp1s0f0np0` corresponds to the IP address `10.169.226.181`:
 ```output
 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
@@ -33,12 +32,12 @@ From the output you can see that the NIC name `enp1s0f0np0` corresponds to the I
        valid_lft forever preferred_lft forever
 ```
 
-2. Set the network interface name variable
+Set the network interface name variable:
 ```bash
 net=enp1s0f0np0
 ```
 
-3. Use the following command to check the current transmit/receive queues of the ${net} network interface
+Use the following command to check the current transmit/receive queues of the ${net} network interface:
 ```bash
 sudo ethtool -l ${net}
 ```
@@ -57,48 +56,56 @@ Other:		n/a
 Combined:	32
 ```
 
-4. Use the following command to reset the number of transmit/receive queues for the ${net} to match the number of CPUs, which is 8.
-```bash
-sudo ethtool -L ${net} combined 8
-```
-5. Verify whether the settings have been successfully applied.
-```bash
-sudo ethtool -l ${net}
-```
-You should see that the number of combined Rx/Tx queues has been updated to 8.
-```output
-Channel parameters for enP11p4s0:
-Pre-set maximums:
-RX:		n/a
-TX:		n/a
-Other:		n/a
-Combined:	32
-Current hardware settings:
-RX:		n/a
-TX:		n/a
-Other:		n/a
-Combined:	8
-```
+Set the number of combined queues to match your online CPUs (example: 8):
+   ```bash
+   sudo ethtool -L ${NET_IFACE} combined 8
+   ```
 
-### The performance uplift after tuning NIC queue count
+Verify the change:
+   ```bash
+   sudo ethtool -l ${NET_IFACE}
+   ```
 
-1. Shutdown and restart `Tomcat` on your Arm Neoverse bare-metal instance as shown:
-```bash
-~/apache-tomcat-11.0.10/bin/shutdown.sh 2>/dev/null
-ulimit -n 65535 && ~/apache-tomcat-11.0.10/bin/startup.sh
-```
+   You should see the `Combined` value updated to `8` under **Current hardware settings**:
+   ```output
+   Channel parameters for enP11p4s0:
+   Pre-set maximums:
+   RX:             n/a
+   TX:             n/a
+   Other:          n/a
+   Combined:       32
+   Current hardware settings:
+   RX:             n/a
+   TX:             n/a
+   Other:          n/a
+   Combined:       8
+   ```
 
-2. Run `wrk2` on your `x86_64` bare-metal instance:
-```bash
-ulimit -n 65535 && wrk -c1280 -t128 -R500000 -d60 http://${tomcat_ip}:8080/examples/servlets/servlet/HelloWorldExample
-```
+{{% notice Note %}}
+Queue settings applied with `ethtool -L` are not persistent across reboots. If needed, make the change persistent using a systemd unit or your distro’s NIC configuration mechanism.
+{{% /notice %}}
 
-Notice the performance uplift after tuning the NIC queue count:
-```output
-  Thread Stats   Avg      Stdev     Max   +/- Stdev
-    Latency     8.35s     4.14s   16.33s    61.16%
-    Req/Sec     2.96k    73.02     3.24k    89.16%
-  22712999 requests in 1.00m, 11.78GB read
-Requests/sec: 378782.37
-Transfer/sec:    201.21MB
-```
+## Measure performance after tuning
+
+Restart Tomcat on your Arm Neoverse bare‑metal instance:
+   ```bash
+   ~/apache-tomcat-11.0.10/bin/shutdown.sh 2>/dev/null
+   ulimit -n 65535 && ~/apache-tomcat-11.0.10/bin/startup.sh
+   ```
+
+Run `wrk2` from your `x86_64` bare‑metal client (replace `${tomcat_ip}` with your server IP):
+   ```bash
+   ulimit -n 65535 && wrk -c1280 -t128 -R500000 -d60 http://${tomcat_ip}:8080/examples/servlets/servlet/HelloWorldExample
+   ```
+
+   Example results after tuning NIC queues:
+   ```output
+     Thread Stats   Avg      Stdev     Max   +/- Stdev
+       Latency     8.35s     4.14s   16.33s    61.16%
+       Req/Sec     2.96k    73.02     3.24k    89.16%
+     22712999 requests in 1.00m, 11.78GB read
+   Requests/sec: 378782.37
+   Transfer/sec:    201.21MB
+   ```
+
+If performance improves and latency stabilizes, keep the adjusted queue count for subsequent tuning steps (RSS/RPS/XPS, IRQ affinity, and NUMA locality).

content/learning-paths/servers-and-cloud-computing/tune-network-workloads-on-bare-metal/4_local-numa.md

@@ -6,19 +6,20 @@ weight: 5
 layout: learningpathall
 ---
 
-## Tuning via local (Non-Uniform Memory Access) NUMA
-In this section you will learn how to configure local NUMA and assess performance uplift acheieved through tuning.
+## Tune with local NUMA (Non-Uniform Memory Access)
 
-Typically, cross-NUMA data transfers within the CPU incur higher latency than intra-NUMA transfers. Therefore, Tomcat should be deployed on the NUMA node where the network interface resides.
+In this section, you configure local NUMA and assess the performance uplift achieved through tuning. Cross‑NUMA data transfers generally incur higher latency than intra‑NUMA transfers, so Tomcat should be deployed on the NUMA node where the network interface resides to reduce cross‑node memory traffic and improve throughput and latency.
 
-### Setting local NUMA
+### Configure local NUMA
+
+Check NUMA topology and relative latencies:
 
-1. Use the following command to check that the latency for cross-NUMA transfers is greater than the latency for intra-NUMA transfers:
 ```bash
 numactl -H
 ```
 
-It can be observed that the cross-NUMA latency to intra-NUMA latency ratio is 10:100.
+You should see that cross‑NUMA latency is higher than intra‑NUMA latency (for example, 10:100):
+
 ```output
 available: 2 nodes (0-1)
 node 0 cpus: 0 1 2 3 4 5 6 7
@@ -33,28 +34,34 @@ node   0   1
   1:  100  10
 ```
 
-2. Use the following command to check the NUMA node where the ${net} network interface resides.
+Identify the NUMA node for your network interface (using the `${net}` variable defined earlier):
+
 ```bash
 cat /sys/class/net/${net}/device/numa_node
 ```
-You will notice that the NUMA node where the ${net} network interface resides is 1.
+
+Example output (NIC on NUMA node 1):
+
 ```output
 1
 ```
 
-3. As a result, allocate the reserved 8 cores to NUMA node 1.
+Allocate the eight reserved cores to the NIC’s NUMA node (the example below keeps CPUs 96–103 online on node 1 and offlines the rest):
+
 ```bash
 for no in {96..103}; do sudo bash -c "echo 1 > /sys/devices/system/cpu/cpu${no}/online"; done
 for no in {0..95} {104..191}; do sudo bash -c "echo 0 > /sys/devices/system/cpu/cpu${no}/online"; done
 ```
 
-4. Verify whether the settings have been successfully applied.
+Verify CPU online status and NUMA placement:
+
 ```bash
 lscpu
 ```
 
-From the output you will see that the only online CPUs are 72-79 on NUMA node 1.
-```bash
+You should see only CPUs 96–103 online on NUMA node 1:
+
+```output
 Architecture:                aarch64
   CPU op-mode(s):            64-bit
   Byte Order:                Little Endian
@@ -71,20 +78,23 @@ NUMA:
 ...
 ```
 
-### The result after tuning local NUMA
+### Validate performance after NUMA tuning
+
+Restart Tomcat on the Arm Neoverse bare‑metal instance:
 
-1. Shutdown and start Tomcat on the Arm Neoverse bare-metal instance:
 ```bash
 ~/apache-tomcat-11.0.10/bin/shutdown.sh 2>/dev/null
 ulimit -n 65535 && ~/apache-tomcat-11.0.10/bin/startup.sh
 ```
 
-2. Run `wrk2` on the `x86_64` bare-metal instance:
+Run `wrk2` on the `x86_64` bare‑metal client to measure throughput and latency:
+
 ```bash
 ulimit -n 65535 && wrk -c1280 -t128 -R500000 -d60 http://${tomcat_ip}:8080/examples/servlets/servlet/HelloWorldExample
 ```
 
-The result after configuring NUMA node should look like:
+Sample results after placing Tomcat on the NIC’s local NUMA node:
+
 ```output
   Thread Stats   Avg      Stdev     Max   +/- Stdev
     Latency     9.41s     4.71s   18.02s    61.07%