You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-1.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,7 +6,7 @@ weight: 2
6
6
layout: learningpathall
7
7
---
8
8
9
-
## What the Arm Performix Instruction Mix recipe is
9
+
## Why to use the Arm Performix Instruction Mix recipe
10
10
11
11
The Arm Performix Instruction Mix recipe shows the types and proportions of machine instructions your workload executes at runtime and in static analysis, so you can see how efficiently your code uses Arm CPU hardware resources.
12
12
@@ -23,7 +23,7 @@ The Instruction Mix result gives you two complementary views:
23
23
- static analysis, which inspects compiled machine code without running it
24
24
- dynamic analysis, which measures instruction usage during real execution
25
25
26
-
Together, these views help you verify whether architecture-specific features are actually active in hot code paths.
26
+
Together, these views help you verify whether architecture-specific features are active in hot code paths.
27
27
28
28
Instruction Mix is useful when you need to confirm that performance-critical code uses Arm CPU features effectively. This is especially helpful when you are, for example, validating the effectiveness of compiler autovectorization.
29
29
@@ -39,6 +39,6 @@ You'll also try implementing your own `matmul` kernels that target Neon and SVE,
39
39
40
40
## What you've learned and what's next
41
41
42
-
You now know what instruction mix represents and why it matters for LLM inference optimization on Arm.
42
+
You now know what the Arm Instruction Mix recipe represents and why it matters for LLM inference optimization on Arm.
43
43
44
44
Next, you'll set up the GPT-2 example, build the binaries, and run a baseline test.
Copy file name to clipboardExpand all lines: content/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-2.md
+9-8Lines changed: 9 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@ layout: learningpathall
8
8
9
9
## Prepare the environment
10
10
11
-
Use an Arm Linux target, such as an Arm Neoverse-based cloud instance. The results in this Learning Path were collected on a Graviton 3-based instance based on Neoverse V1 running Ubuntu 24.04 LTS.
11
+
Use an Arm Linux target, such as an Arm Neoverse-based cloud instance. The results in this Learning Path were collected on a Graviton 3-based Amazon EC2 instance running Ubuntu 24.04 LTS.
12
12
13
13
If you've not configured Arm Performix yet, complete setup and target connection using the [Arm Performix install guide](/install-guides/performix/).
14
14
@@ -26,8 +26,6 @@ git checkout tags/v0.0.2
26
26
27
27
The C++ runtime expects exported model binaries. Create a Python virtual environment, install dependencies, and export GPT-2 Medium weights and vocabulary:
28
28
29
-
The workload uses [openai-community/gpt2-medium on Hugging Face](https://huggingface.co/openai-community/gpt2-medium), which corresponds to the GPT-2 Medium model from the original OpenAI GPT-2 release in 2019. The model has 355 million parameters, and in this workflow it runs with unquantized FP32 (32-bit floating-point) weights.
30
-
31
29
```bash
32
30
python3 -m venv venv
33
31
source venv/bin/activate
@@ -40,9 +38,11 @@ This creates:
40
38
-`models/gpt2-medium/weights.bin`
41
39
-`models/gpt2-medium/vocab.bin`
42
40
41
+
The workload uses [openai-community/gpt2-medium on Hugging Face](https://huggingface.co/openai-community/gpt2-medium), which corresponds to the GPT-2 Medium model from the original OpenAI GPT-2 release in 2019. The model has 355 million parameters and runs with unquantized FP32 (32-bit floating-point) weights.
42
+
43
43
## Review the source code
44
44
45
-
The `src/gpt2.cpp` file implements the end-to-end GPT-2 inference loop. Each generated token triggers a forward pass over all 24 transformer layers. Inside each layer, `matmul` is called multiple times: for the query/key/value projection, the attention output projection, and both feed-forward layers. It is called once more at the end for logits projection over the vocabulary:
45
+
The `src/gpt2.cpp` file implements the end-to-end GPT-2 inference loop. Each generated token triggers a forward pass over all 24 transformer layers. Inside each layer, `matmul` is called multiple times for the query, key, and value projection, the attention output projection, and both feed-forward layers. It's called once more at the end for `logits` projection over the vocabulary:
The baseline kernel (`src/kernels/matmul_ref.cpp`) is a straightforward scalar nested for loop: for each output row, it walks the weight matrix row and accumulates a dot product with the input vector:
79
+
The baseline kernel (`src/kernels/matmul_ref.cpp`) is a scalar nested for loop: for each output row, it walks the weight matrix row and accumulates a dot product with the input vector:
This scalar implementation can leave Neon and SVE vector units underused if the compiler cannot efficiently autovectorize it. Because `matmul` is called hundreds of times per token, explicitly optimizing this kernel guarantees SIMD execution where most of the available compute is spent.
93
+
This scalar implementation can leave Neon and SVE vector units underused if the compiler can't efficiently autovectorize it. Because `matmul` is called hundreds of times per token, explicitly optimizing this kernel guarantees SIMD execution where most of the available compute is spent.
94
94
95
95
## Build and run the baseline
96
96
97
-
Configure and build the project with CMake. The project uses `-O2 -g`, which keeps optimization enabled while preserving debug symbols for profiling.
97
+
Configure and build the project with CMake:
98
98
99
99
```bash
100
100
cmake -S . -B build -DBUILD_USER_MATMUL=ON
101
101
cmake --build build --parallel
102
102
```
103
+
The project uses `-O2 -g`, which keeps optimization enabled while preserving debug symbols for profiling.
103
104
104
105
Run the scalar baseline binary:
105
106
106
107
```bash
107
108
./build/gpt2 --model gpt2-medium "Once upon a time" -n 20
108
109
```
109
110
110
-
The output is:
111
+
The output is similar to:
111
112
112
113

Copy file name to clipboardExpand all lines: content/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-3.md
+10-8Lines changed: 10 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -12,7 +12,7 @@ Before you optimize, identify where the application spends most of its time. Use
12
12
13
13
Open Arm Performix and select the **Code Hotspots** recipe. If this is your first run on the target, complete tool deployment as prompted.
14
14
15
-
Set the launch command to your baseline binary with the number of tokens (`-n`) set to 150. This value keeps startup overhead small compared to inference time, so the profile minimizes the time taken to load the model weights.
15
+
Set the launch command to your baseline binary with the number of tokens (`-n`) set to `150`. This value keeps startup overhead small compared to inference time, so the profile minimizes the time taken to load the model weights.
16
16
17
17
Copy and paste the following command as the Performix Workload. If your path is different, adjust it to point to `gpt2`:
18
18
@@ -22,9 +22,9 @@ Copy and paste the following command as the Performix Workload. If your path is
22
22
23
23

24
24
25
-
The results show that `kernels::matmul_ref()` is the hottest function. Double-clicking on the function shows which lines of source code the samples are mostly attributed to the accumulate step of `kernels::matmul_ref()`.
25
+
The results show that `kernels::matmul_ref()` is the hottest function. Double-click on the function to see which lines of source code are mostly attributed to the accumulate step of `kernels::matmul_ref()`.
26
26
27
-

27
+

28
28
29
29
This confirms that matrix multiplication is the highest-impact optimization target.
30
30
@@ -34,7 +34,9 @@ You can use online tools such as [Compiler Explorer](https://godbolt.org/) to co
34
34
35
35
{{< godbolt width="100%" height="400px" mode="assembly" opt="-O2 -g" src="void matmul_ref(float *out, const float *x, const float *W, const float *b, int n_in, int n_out)\n{\n for (int i = 0; i < n_out; i++) {\n float acc = b ? b[i] : 0.f;\n const float *row = W + (unsigned long long)i * (unsigned long long)n_in;\n for (int j = 0; j < n_in; j++) {\n acc += row[j] * x[j];\n }\n out[i] = acc;\n }\n}" >}}
36
36
37
-
This view helps you spot missed vectorization opportunities. In an optimized build, you'd expect the accumulation step to use SIMD instructions, for example `fmla v0.4s, v3.4s, v2.4s` with use of the vector register (`v0->v3`). However, assembly inspection has limitations. First, you need familiarity with SIMD mnemonics to recognize vectorized code. Second, this narrow snippet doesn't show whether changing compiler flags introduces regressions in other parts of the codebase. Third, and most importantly, this static view doesn't show which instructions in this function run most often on the CPU.
37
+
With this view, you can spot missed vectorization opportunities. In an optimized build, you'd expect the accumulation step to use SIMD instructions, for example `fmla v0.4s, v3.4s, v2.4s` with use of the vector register (`v0->v3`). However, assembly inspection has limitations.
38
+
39
+
First, you need familiarity with SIMD mnemonics to recognize vectorized code. Second, this narrow snippet doesn't show whether changing compiler flags introduces regressions in other parts of the codebase. Third, and most importantly, this static view doesn't show which instructions in this function run most often on the CPU.
38
40
39
41
The Instruction Mix recipe helps fill this gap.
40
42
@@ -56,22 +58,22 @@ Use the same model and prompt arguments as your baseline terminal run so the mea
56
58
57
59
### Analyze static disassembly
58
60
59
-
After the run completes, review static disassembly first. This view is ordered by percentage contribution and provides a high-level profile of the application’s generated instruction stream. It can help you identify broad characteristics, such as whether the code is branch-heavy, dominated by memory operations, or making effective use of SIMD instructions.
61
+
After the run completes, review static disassembly first. This view is ordered by percentage contribution and provides a high-level profile of the application’s generated instruction stream. By reviewing static disassembly, you can identify broad characteristics, such as whether the code is branch-heavy, dominated by memory operations, or making effective use of SIMD instructions.
60
62
61
63
Use this static view to understand overall code generation patterns rather than to attribute performance to specific functions or source lines. Dynamic analysis is typically more relevant for optimization because it reflects the instructions that are actually executed at runtime.
62
64
63
-

65
+

64
66
65
67
### Dynamic analysis
66
68
67
69
Then, inspect dynamic analysis bar chart to see where sampled runtime work is concentrated. Dynamic data is typically more useful for optimization because it reflects actual execution behavior for your input, runtime settings, and call frequencies.
68
70
69
-

71
+

70
72
71
73
Finally, in dynamic functions, you can break down operation types to individual functions. This is particularly useful when no single function dominates the profile, allowing you to inspect dynamic instruction patterns for specific functions.
72
74
73
75
## What you've accomplished and what's next
74
76
75
77
You've now used Instruction Mix to confirm that baseline runtime is dominated by scalar-heavy `matmul` execution.
76
78
77
-
Next, you can optionally learn to optimize matmul with vector intrinsics and use the Arm MCP Server with Performix. You can also skip to [Compare Neon and SVE with the Arm Performix Instruction Mix recipe](/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-5/) to compare updated Instruction Mix and throughput across scalar, Neon, SVE, and KleidiAI variants.
79
+
Next, you can optionally learn to optimize `matmul` with vector intrinsics and use the Arm MCP Server with Performix. You can also skip to [Compare Neon and SVE with the Arm Performix Instruction Mix recipe](/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-5/) to compare updated Instruction Mix and throughput across scalar, Neon, SVE, and KleidiAI variants.
Copy file name to clipboardExpand all lines: content/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-4.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@ layout: learningpathall
8
8
9
9
## Complete the challenge
10
10
11
-
In this project, `src/kernels/matmul_user.cpp` is your editable implementation file. The baseline behavior in this file is scalar, and the build uses `-O2 -g`, so compiler optimization is enabled but vector hardware is still underused in the hot loop.
11
+
`src/kernels/matmul_user.cpp` is your editable implementation file. The baseline behavior in this file is scalar, and the build uses `-O2 -g`, so compiler optimization is enabled but vector hardware is still underused in the hot loop.
12
12
13
13
Use the profiling evidence from Performix to implement your own Neon or SVE intrinsics in `src/kernels/matmul_user.cpp`, then rebuild and profile `gpt2_user`.
Then profile the `build/gpt2_user` binary with the same runtime arguments and compare the Instruction Mix and throughput against baseline.
28
+
Then, profile the `build/gpt2_user` binary with the same runtime arguments and compare the Instruction Mix and throughput against baseline.
29
29
30
30
Example solutions are available in:
31
31
@@ -64,7 +64,7 @@ args = [
64
64
65
65
Restart your coding assistant, then prompt it to run Performix Instruction Mix and Code Hotspots on your `gpt2_user` binary and suggest Arm intrinsics improvements.
66
66
67
-

67
+

Copy file name to clipboardExpand all lines: content/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-5.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -30,9 +30,9 @@ Rename each run with a descriptive name, such as baseline and Neon, so you can i
30
30
31
31
The baseline profile is mostly scalar instructions. After you add Neon intrinsics, the instruction mix shifts toward Advanced SIMD (Neon) instructions, showing that the code is using Arm Neon hardware more effectively.
32
32
33
-

33
+

34
34
35
-
You can also compare SVE variants in the same way. The increase in SVE operations shows that this path is now utilizing SVE hardware.
35
+
You can also compare SVE variants in the same way. The increase in SVE operations shows that this path is now using SVE hardware.
36
36
37
37

38
38
@@ -54,7 +54,7 @@ For variable-length vectorization, compare with an explicit SVE implementation t
0 commit comments