Skip to content

Commit 1293c9e

Browse files
another set of edits
1 parent af71432 commit 1293c9e

7 files changed

Lines changed: 38 additions & 36 deletions

File tree

content/learning-paths/servers-and-cloud-computing/performix-instruction-mix/_index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ further_reading:
5555
type: website
5656
- resource:
5757
title: KleidiAI GitHub Repository
58-
link: https://github.com/ARM-software/kleidiai
58+
link: https://gitlab.arm.com/kleidi/kleidiai
5959
type: website
6060
- resource:
6161
title: GPT-2 Example repository

content/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-1.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ weight: 2
66
layout: learningpathall
77
---
88

9-
## What the Arm Performix Instruction Mix recipe is
9+
## Why to use the Arm Performix Instruction Mix recipe
1010

1111
The Arm Performix Instruction Mix recipe shows the types and proportions of machine instructions your workload executes at runtime and in static analysis, so you can see how efficiently your code uses Arm CPU hardware resources.
1212

@@ -23,7 +23,7 @@ The Instruction Mix result gives you two complementary views:
2323
- static analysis, which inspects compiled machine code without running it
2424
- dynamic analysis, which measures instruction usage during real execution
2525

26-
Together, these views help you verify whether architecture-specific features are actually active in hot code paths.
26+
Together, these views help you verify whether architecture-specific features are active in hot code paths.
2727

2828
Instruction Mix is useful when you need to confirm that performance-critical code uses Arm CPU features effectively. This is especially helpful when you are, for example, validating the effectiveness of compiler autovectorization.
2929

@@ -39,6 +39,6 @@ You'll also try implementing your own `matmul` kernels that target Neon and SVE,
3939

4040
## What you've learned and what's next
4141

42-
You now know what instruction mix represents and why it matters for LLM inference optimization on Arm.
42+
You now know what the Arm Instruction Mix recipe represents and why it matters for LLM inference optimization on Arm.
4343

4444
Next, you'll set up the GPT-2 example, build the binaries, and run a baseline test.

content/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-2.md

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ layout: learningpathall
88

99
## Prepare the environment
1010

11-
Use an Arm Linux target, such as an Arm Neoverse-based cloud instance. The results in this Learning Path were collected on a Graviton 3-based instance based on Neoverse V1 running Ubuntu 24.04 LTS.
11+
Use an Arm Linux target, such as an Arm Neoverse-based cloud instance. The results in this Learning Path were collected on a Graviton 3-based Amazon EC2 instance running Ubuntu 24.04 LTS.
1212

1313
If you've not configured Arm Performix yet, complete setup and target connection using the [Arm Performix install guide](/install-guides/performix/).
1414

@@ -26,8 +26,6 @@ git checkout tags/v0.0.2
2626

2727
The C++ runtime expects exported model binaries. Create a Python virtual environment, install dependencies, and export GPT-2 Medium weights and vocabulary:
2828

29-
The workload uses [openai-community/gpt2-medium on Hugging Face](https://huggingface.co/openai-community/gpt2-medium), which corresponds to the GPT-2 Medium model from the original OpenAI GPT-2 release in 2019. The model has 355 million parameters, and in this workflow it runs with unquantized FP32 (32-bit floating-point) weights.
30-
3129
```bash
3230
python3 -m venv venv
3331
source venv/bin/activate
@@ -40,9 +38,11 @@ This creates:
4038
- `models/gpt2-medium/weights.bin`
4139
- `models/gpt2-medium/vocab.bin`
4240

41+
The workload uses [openai-community/gpt2-medium on Hugging Face](https://huggingface.co/openai-community/gpt2-medium), which corresponds to the GPT-2 Medium model from the original OpenAI GPT-2 release in 2019. The model has 355 million parameters and runs with unquantized FP32 (32-bit floating-point) weights.
42+
4343
## Review the source code
4444

45-
The `src/gpt2.cpp` file implements the end-to-end GPT-2 inference loop. Each generated token triggers a forward pass over all 24 transformer layers. Inside each layer, `matmul` is called multiple times: for the query/key/value projection, the attention output projection, and both feed-forward layers. It is called once more at the end for logits projection over the vocabulary:
45+
The `src/gpt2.cpp` file implements the end-to-end GPT-2 inference loop. Each generated token triggers a forward pass over all 24 transformer layers. Inside each layer, `matmul` is called multiple times for the query, key, and value projection, the attention output projection, and both feed-forward layers. It's called once more at the end for `logits` projection over the vocabulary:
4646

4747
```cpp
4848
// Attention QKV projection
@@ -76,7 +76,7 @@ static void matmul(float *out, const float *x, const float *W, const float *b,
7676
}
7777
```
7878

79-
The baseline kernel (`src/kernels/matmul_ref.cpp`) is a straightforward scalar nested for loop: for each output row, it walks the weight matrix row and accumulates a dot product with the input vector:
79+
The baseline kernel (`src/kernels/matmul_ref.cpp`) is a scalar nested for loop: for each output row, it walks the weight matrix row and accumulates a dot product with the input vector:
8080

8181
```cpp
8282
void matmul_ref(float *out, const float *x, const float *W, const float *b,
@@ -90,24 +90,25 @@ void matmul_ref(float *out, const float *x, const float *W, const float *b,
9090
}
9191
```
9292
93-
This scalar implementation can leave Neon and SVE vector units underused if the compiler cannot efficiently autovectorize it. Because `matmul` is called hundreds of times per token, explicitly optimizing this kernel guarantees SIMD execution where most of the available compute is spent.
93+
This scalar implementation can leave Neon and SVE vector units underused if the compiler can't efficiently autovectorize it. Because `matmul` is called hundreds of times per token, explicitly optimizing this kernel guarantees SIMD execution where most of the available compute is spent.
9494
9595
## Build and run the baseline
9696
97-
Configure and build the project with CMake. The project uses `-O2 -g`, which keeps optimization enabled while preserving debug symbols for profiling.
97+
Configure and build the project with CMake:
9898
9999
```bash
100100
cmake -S . -B build -DBUILD_USER_MATMUL=ON
101101
cmake --build build --parallel
102102
```
103+
The project uses `-O2 -g`, which keeps optimization enabled while preserving debug symbols for profiling.
103104

104105
Run the scalar baseline binary:
105106

106107
```bash
107108
./build/gpt2 --model gpt2-medium "Once upon a time" -n 20
108109
```
109110

110-
The output is:
111+
The output is similar to:
111112

112113
![Animated terminal output showing GPT-2 baseline inference running on Arm Linux, including generated text and the final tokens-per-second summary.#center](./gpt2-baseline.gif "GPT-2 baseline runtime output on Arm Linux")
113114

content/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-3.md

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ Before you optimize, identify where the application spends most of its time. Use
1212

1313
Open Arm Performix and select the **Code Hotspots** recipe. If this is your first run on the target, complete tool deployment as prompted.
1414

15-
Set the launch command to your baseline binary with the number of tokens (`-n`) set to 150. This value keeps startup overhead small compared to inference time, so the profile minimizes the time taken to load the model weights.
15+
Set the launch command to your baseline binary with the number of tokens (`-n`) set to `150`. This value keeps startup overhead small compared to inference time, so the profile minimizes the time taken to load the model weights.
1616

1717
Copy and paste the following command as the Performix Workload. If your path is different, adjust it to point to `gpt2`:
1818

@@ -22,9 +22,9 @@ Copy and paste the following command as the Performix Workload. If your path is
2222

2323
![Arm Performix Code Hotspots recipe configuration showing launch arguments for the GPT-2 baseline run with -n 150 to emphasize inference runtime.#center](./code_hotspot.webp "Code Hotspots recipe configuration for GPT-2 baseline")
2424

25-
The results show that `kernels::matmul_ref()` is the hottest function. Double-clicking on the function shows which lines of source code the samples are mostly attributed to the accumulate step of `kernels::matmul_ref()`.
25+
The results show that `kernels::matmul_ref()` is the hottest function. Double-click on the function to see which lines of source code are mostly attributed to the accumulate step of `kernels::matmul_ref()`.
2626

27-
![Arm Performix hotspot results table showing matmul_ref as the dominant runtime function during GPT-2 baseline inference.#center](./code_hotspot_results.webp "Hotspot results highlighting matmul_ref")
27+
![Arm Performix hotspot results table showing `matmul_ref` as the dominant runtime function during GPT-2 baseline inference.#center](./code_hotspot_results.webp "Hotspot results highlighting `matmul_ref`")
2828

2929
This confirms that matrix multiplication is the highest-impact optimization target.
3030

@@ -34,7 +34,9 @@ You can use online tools such as [Compiler Explorer](https://godbolt.org/) to co
3434

3535
{{< godbolt width="100%" height="400px" mode="assembly" opt="-O2 -g" src="void matmul_ref(float *out, const float *x, const float *W, const float *b, int n_in, int n_out)\n{\n for (int i = 0; i < n_out; i++) {\n float acc = b ? b[i] : 0.f;\n const float *row = W + (unsigned long long)i * (unsigned long long)n_in;\n for (int j = 0; j < n_in; j++) {\n acc += row[j] * x[j];\n }\n out[i] = acc;\n }\n}" >}}
3636

37-
This view helps you spot missed vectorization opportunities. In an optimized build, you'd expect the accumulation step to use SIMD instructions, for example `fmla v0.4s, v3.4s, v2.4s` with use of the vector register (`v0->v3`). However, assembly inspection has limitations. First, you need familiarity with SIMD mnemonics to recognize vectorized code. Second, this narrow snippet doesn't show whether changing compiler flags introduces regressions in other parts of the codebase. Third, and most importantly, this static view doesn't show which instructions in this function run most often on the CPU.
37+
With this view, you can spot missed vectorization opportunities. In an optimized build, you'd expect the accumulation step to use SIMD instructions, for example `fmla v0.4s, v3.4s, v2.4s` with use of the vector register (`v0->v3`). However, assembly inspection has limitations.
38+
39+
First, you need familiarity with SIMD mnemonics to recognize vectorized code. Second, this narrow snippet doesn't show whether changing compiler flags introduces regressions in other parts of the codebase. Third, and most importantly, this static view doesn't show which instructions in this function run most often on the CPU.
3840

3941
The Instruction Mix recipe helps fill this gap.
4042

@@ -56,22 +58,22 @@ Use the same model and prompt arguments as your baseline terminal run so the mea
5658

5759
### Analyze static disassembly
5860

59-
After the run completes, review static disassembly first. This view is ordered by percentage contribution and provides a high-level profile of the application’s generated instruction stream. It can help you identify broad characteristics, such as whether the code is branch-heavy, dominated by memory operations, or making effective use of SIMD instructions.
61+
After the run completes, review static disassembly first. This view is ordered by percentage contribution and provides a high-level profile of the application’s generated instruction stream. By reviewing static disassembly, you can identify broad characteristics, such as whether the code is branch-heavy, dominated by memory operations, or making effective use of SIMD instructions.
6062

6163
Use this static view to understand overall code generation patterns rather than to attribute performance to specific functions or source lines. Dynamic analysis is typically more relevant for optimization because it reflects the instructions that are actually executed at runtime.
6264

63-
![Arm Performix static disassembly view showing instruction category breakdown for GPT-2 hot paths, highlighting scalar-heavy sections in baseline matmul code.#center](./static_disassembly.webp "Static disassembly instruction classification")
65+
![Arm Performix static disassembly view showing instruction category breakdown for GPT-2 hot paths, highlighting scalar-heavy sections in baseline `matmul` code.#center](./static_disassembly.webp "Static disassembly instruction classification")
6466

6567
### Dynamic analysis
6668

6769
Then, inspect dynamic analysis bar chart to see where sampled runtime work is concentrated. Dynamic data is typically more useful for optimization because it reflects actual execution behavior for your input, runtime settings, and call frequencies.
6870

69-
![Arm Performix dynamic functions table showing most runtime samples in matmul-related functions for baseline GPT-2 inference.#center](./instruction_mix_dynamic_analysis.webp "Dynamic function sample distribution")
71+
![Arm Performix dynamic functions table showing most runtime samples in functions related to `matmul` for baseline GPT-2 inference.#center](./instruction_mix_dynamic_analysis.webp "Dynamic function sample distribution")
7072

7173
Finally, in dynamic functions, you can break down operation types to individual functions. This is particularly useful when no single function dominates the profile, allowing you to inspect dynamic instruction patterns for specific functions.
7274

7375
## What you've accomplished and what's next
7476

7577
You've now used Instruction Mix to confirm that baseline runtime is dominated by scalar-heavy `matmul` execution.
7678

77-
Next, you can optionally learn to optimize matmul with vector intrinsics and use the Arm MCP Server with Performix. You can also skip to [Compare Neon and SVE with the Arm Performix Instruction Mix recipe](/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-5/) to compare updated Instruction Mix and throughput across scalar, Neon, SVE, and KleidiAI variants.
79+
Next, you can optionally learn to optimize `matmul` with vector intrinsics and use the Arm MCP Server with Performix. You can also skip to [Compare Neon and SVE with the Arm Performix Instruction Mix recipe](/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-5/) to compare updated Instruction Mix and throughput across scalar, Neon, SVE, and KleidiAI variants.

content/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-4.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ layout: learningpathall
88

99
## Complete the challenge
1010

11-
In this project, `src/kernels/matmul_user.cpp` is your editable implementation file. The baseline behavior in this file is scalar, and the build uses `-O2 -g`, so compiler optimization is enabled but vector hardware is still underused in the hot loop.
11+
`src/kernels/matmul_user.cpp` is your editable implementation file. The baseline behavior in this file is scalar, and the build uses `-O2 -g`, so compiler optimization is enabled but vector hardware is still underused in the hot loop.
1212

1313
Use the profiling evidence from Performix to implement your own Neon or SVE intrinsics in `src/kernels/matmul_user.cpp`, then rebuild and profile `gpt2_user`.
1414

@@ -25,7 +25,7 @@ cmake -S . -B build -DBUILD_USER_MATMUL=ON
2525
cmake --build build --parallel
2626
```
2727

28-
Then profile the `build/gpt2_user` binary with the same runtime arguments and compare the Instruction Mix and throughput against baseline.
28+
Then, profile the `build/gpt2_user` binary with the same runtime arguments and compare the Instruction Mix and throughput against baseline.
2929

3030
Example solutions are available in:
3131

@@ -64,7 +64,7 @@ args = [
6464

6565
Restart your coding assistant, then prompt it to run Performix Instruction Mix and Code Hotspots on your `gpt2_user` binary and suggest Arm intrinsics improvements.
6666

67-
![Screenshot of a coding assistant prompt configured to use Arm MCP Server tools for running Performix recipes and analyzing matmul_user optimization opportunities in the GPT-2 workload.#center](./mcp-performix-prompt.webp "Coding assistant prompt for Performix analysis through Arm MCP Server")
67+
![Screenshot of a coding assistant prompt configured to use Arm MCP Server tools for running Performix recipes and analyzing `matmul_user` optimization opportunities in the GPT-2 workload.#center](./mcp-performix-prompt.webp "Coding assistant prompt for Performix analysis through Arm MCP Server")
6868

6969
## What you've accomplished and what's next
7070

content/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-5.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,9 +30,9 @@ Rename each run with a descriptive name, such as baseline and Neon, so you can i
3030

3131
The baseline profile is mostly scalar instructions. After you add Neon intrinsics, the instruction mix shifts toward Advanced SIMD (Neon) instructions, showing that the code is using Arm Neon hardware more effectively.
3232

33-
![Instruction Mix comparison view showing scalar-dominant baseline versus Neon variant with increased ASIMD instruction share in the hot matmul path.#center](./neon_scalar_instruction_mix.webp "Neon versus scalar instruction mix")
33+
![Instruction Mix comparison view showing scalar-dominant baseline versus Neon variant with increased ASIMD instruction share in the hot `matmul` path.#center](./neon_scalar_instruction_mix.webp "Neon versus scalar instruction mix")
3434

35-
You can also compare SVE variants in the same way. The increase in SVE operations shows that this path is now utilizing SVE hardware.
35+
You can also compare SVE variants in the same way. The increase in SVE operations shows that this path is now using SVE hardware.
3636

3737
![Instruction Mix comparison between baseline and SVE variant showing increased vector instruction usage and reduced scalar share in hot execution paths.#center](./sve_vs_baseline.webp "SVE versus baseline instruction mix")
3838

@@ -54,7 +54,7 @@ For variable-length vectorization, compare with an explicit SVE implementation t
5454
{{< godbolt width="100%" height="400px" mode="assembly" opt="-O2 -g -march=armv8.2-a+sve" src="#include <arm_sve.h>\n#include <stddef.h>\n\nvoid matmul_sve(float *out, const float *x, const float *W, const float *b,\n int n_in, int n_out) {\n for (int i = 0; i < n_out; i++) {\n float acc = b ? b[i] : 0.f;\n const float *row = W + (size_t)i * n_in;\n svfloat32_t vacc = svdup_f32(0.f);\n int j = 0;\n while (j < n_in) {\n svbool_t pg = svwhilelt_b32((uint64_t)j, (uint64_t)n_in);\n svfloat32_t vw = svld1(pg, row + j);\n svfloat32_t vx = svld1(pg, x + j);\n vacc = svmla_f32_m(pg, vacc, vw, vx);\n j += svcntw();\n }\n acc += svaddv_f32(svptrue_b32(), vacc);\n out[i] = acc;\n }\n}" >}}
5555

5656

57-
For a full-page view, open a [Godbolt session with all three matmul kernels](https://godbolt.org/z/E4a7Wxh8K).
57+
For a full-page view, open a [Godbolt session with all three `matmul` kernels](https://godbolt.org/z/E4a7Wxh8K).
5858

5959
## Measure speedup
6060

0 commit comments

Comments
 (0)