Skip to content

Commit d6fec68

Browse files
some skill-based updates
1 parent dd7e618 commit d6fec68

7 files changed

Lines changed: 31 additions & 27 deletions

File tree

content/learning-paths/servers-and-cloud-computing/performix-instruction-mix/_index.md

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,17 @@
11
---
2-
title: Profile GPT-2 instruction mix with Arm Performix
2+
title: Profile GPT-2 inference with the Arm Performix Instruction Mix recipe
33

4-
description: Learn how to profile GPT-2 inference on Arm Neoverse with the Arm Performix Instruction Mix recipe, identify scalar versus vector execution patterns, and improve throughput with NEON, SVE, and KleidiAI kernels.
4+
description: Profile GPT-2 inference on Arm Neoverse with the Arm Performix Instruction Mix recipe, identify scalar versus vector execution patterns, and improve throughput with Neon, SVE, and KleidiAI kernels.
55

66
minutes_to_complete: 45
77

88
who_is_this_for: This is an introductory topic for developers who want to get started using the Arm Performix Instruction Mix recipe through a practical example.
99

1010
learning_objectives:
11-
- Understand how the Instruction Mix recipe combines static disassembly with runtime sampling to show execution behavior
11+
- Use the Instruction Mix recipe to combine static disassembly with runtime sampling to show execution behavior
1212
- Build and run the GPT-2 inference example on an Arm Linux server
1313
- Identify why matrix multiplication dominates runtime and how vectorization changes the instruction mix
14-
- Compare throughput and instruction mix across scalar, NEON, SVE, and KleidiAI implementations
14+
- Compare throughput and instruction mix across scalar, Neon, SVE, and KleidiAI implementations
1515

1616
prerequisites:
1717
- Access to Arm Performix configured with a remote Arm Linux target. For setup, see the [Arm Performix install guide](/install-guides/performix/).
@@ -32,7 +32,7 @@ tools_software_languages:
3232
- Arm Performix
3333
- C
3434
- LLM
35-
- NEON
35+
- Neon
3636
- SVE
3737
operatingsystems:
3838
- Linux
@@ -53,6 +53,10 @@ further_reading:
5353
title: Arm MCP Server GitHub Repository
5454
link: https://github.com/arm/mcp
5555
type: website
56+
- resource:
57+
title: KleidiAI GitHub Repository
58+
link: https://github.com/ARM-software/kleidiai
59+
type: website
5660
- resource:
5761
title: GPT-2 Example repository
5862
link: https://github.com/arm-education/GPT-2-Example
@@ -65,4 +69,4 @@ further_reading:
6569
weight: 1 # _index.md always has weight of 1 to order correctly
6670
layout: "learningpathall" # All files under learning paths have this same wrapper
6771
learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content.
68-
---
72+
---

content/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-1.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ The Instruction Mix recipe classifies each instruction into a group. The availab
1616
- memory loads and stores (including exclusive operations)
1717
- control flow instructions, such as branches and loops
1818
- specialized instructions, such as cryptographic operations
19-
- SIMD (Single Instruction, Multiple Data) instructions, including NEON (fixed 128-bit) and SVE (scalable vector length)
19+
- SIMD (Single Instruction, Multiple Data) instructions, including Neon (fixed 128-bit) and SVE (scalable vector length)
2020

2121
The Instruction Mix result gives you two complementary views:
2222

@@ -27,18 +27,18 @@ Together, these views help you verify whether architecture-specific features are
2727

2828
Instruction Mix is useful when you need to confirm that performance-critical code uses Arm CPU features effectively. This is especially helpful when you are, for example, validating the effectiveness of compiler autovectorization.
2929

30-
For example, if a hot function is mostly scalar at runtime when you expected NEON or SVE activity, that often indicates missed vectorization opportunities. You can then focus optimization work on compiler flags, data layout, loop structure, and kernel implementation to improve throughput where it matters most.
30+
For example, if a hot function is mostly scalar at runtime when you expected Neon or SVE activity, that often indicates missed vectorization opportunities. You can then focus optimization work on compiler flags, data layout, loop structure, and kernel implementation to improve throughput where it matters most.
3131

3232
## GPT-2 as a test workload
3333

34-
You can run the [GPT-2 Medium](https://huggingface.co/openai-community/gpt2-medium) model on a minimal C++ inference engine to analyze instruction mix and throughput. This model is available under a [modified MIT License](https://github.com/openai/gpt-2/blob/master/LICENSE). You will confirm that matrix multiplication (`matmul`) is the hot path, then compare how scalar, NEON, and SVE implementations change instruction behavior and token generation speed.
34+
You can run the [GPT-2 Medium](https://huggingface.co/openai-community/gpt2-medium) model on a minimal C++ inference engine to analyze instruction mix and throughput. This model is available under a [modified MIT License](https://github.com/openai/gpt-2/blob/master/LICENSE). You will confirm that matrix multiplication (`matmul`) is the hot path, then compare how scalar, Neon, and SVE implementations change instruction behavior and token generation speed.
3535

3636
You'll implement only the forward inference path, with no back propagation or training. You don't need to understand the full transformer architecture to complete this Learning Path. Familiarity with matrix multiplication is enough. For background on GPT-2, see the original 2019 paper, [Language Models are Unsupervised Multitask Learners](https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf).
3737

38-
You'll also try implementing your own `matmul` kernels that target NEON and SVE, then use instruction mix data to verify that these vector paths are active and improving throughput.
38+
You'll also try implementing your own `matmul` kernels that target Neon and SVE, then use instruction mix data to verify that these vector paths are active and improving throughput.
3939

4040
## What you've learned and what's next
4141

4242
You now know what instruction mix represents and why it matters for LLM inference optimization on Arm.
4343

44-
Next, you'll set up the GPT-2 example, build the binaries, and run a baseline test.
44+
Next, you'll set up the GPT-2 example, build the binaries, and run a baseline test.

content/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-2.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ layout: learningpathall
88

99
## Prepare the environment
1010

11-
Use an Arm Linux target, such as an Arm Neoverse cloud instance. The results in this Learning Path were collected on a Graviton 3-based instance based on Neoverse V1 running Ubuntu 24.04 LTS.
11+
Use an Arm Linux target, such as an Arm Neoverse-based cloud instance. The results in this Learning Path were collected on a Graviton 3-based instance based on Neoverse V1 running Ubuntu 24.04 LTS.
1212

1313
If you've not configured Arm Performix yet, complete setup and target connection using the [Arm Performix install guide](/install-guides/performix/).
1414

@@ -90,7 +90,7 @@ void matmul_ref(float *out, const float *x, const float *W, const float *b,
9090
}
9191
```
9292
93-
This scalar implementation can leave NEON and SVE vector units underused if the compiler cannot efficiently autovectorize it. Because `matmul` is called hundreds of times per token, explicitly optimizing this kernel guarantees SIMD execution where most of the available compute is spent.
93+
This scalar implementation can leave Neon and SVE vector units underused if the compiler cannot efficiently autovectorize it. Because `matmul` is called hundreds of times per token, explicitly optimizing this kernel guarantees SIMD execution where most of the available compute is spent.
9494
9595
## Build and run the baseline
9696
@@ -115,4 +115,4 @@ The output is:
115115

116116
You now have a working baseline binary and model files.
117117

118-
Next, you'll use the Instruction Mix recipe in Arm Performix to inspect static disassembly and dynamic runtime behavior.
118+
Next, you'll use the Instruction Mix recipe in Arm Performix to inspect static disassembly and dynamic runtime behavior.

content/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-3.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Profile with the Arm Performix Instruction Mix recipe
2+
title: Find GPT-2 hotspots and profile with the Arm Performix Instruction Mix recipe
33
weight: 4
44

55
### FIXED, DO NOT MODIFY
@@ -74,4 +74,4 @@ Finally, in dynamic functions, you can break down operation types to individual
7474

7575
You've now used Instruction Mix to confirm that baseline runtime is dominated by scalar-heavy `matmul` execution.
7676

77-
Next, you'll compare updated Instruction Mix and throughput across scalar, NEON, SVE, and KleidiAI variants.
77+
Next, you can optionally learn to optimize matmul with vector intrinsics and use the Arm MCP Server with Performix. You can also skip to [Compare Neon and SVE with the Arm Performix Instruction Mix recipe](/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-5/) to compare updated Instruction Mix and throughput across scalar, Neon, SVE, and KleidiAI variants.

content/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-4.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,9 +10,9 @@ layout: learningpathall
1010

1111
In this project, `src/kernels/matmul_user.cpp` is your editable implementation file. The baseline behavior in this file is scalar, and the build uses `-O2 -g`, so compiler optimization is enabled but vector hardware is still underused in the hot loop.
1212

13-
Use the profiling evidence from Performix to implement your own NEON or SVE intrinsics in `src/kernels/matmul_user.cpp`, then rebuild and profile `gpt2_user`.
13+
Use the profiling evidence from Performix to implement your own Neon or SVE intrinsics in `src/kernels/matmul_user.cpp`, then rebuild and profile `gpt2_user`.
1414

15-
{{% notice Hint %}}
15+
{{% notice Note %}}
1616

1717
Focus on the accumulation loop in `matmul_user` (`acc += row[j] * x[j];`). Think about lane utilization, loop unrolling, and handling the tail when the input width is not an exact multiple of the vector width.
1818

@@ -70,4 +70,4 @@ Restart your coding assistant, then prompt it to run Performix Instruction Mix a
7070

7171
You've now optionally implemented and profiled a custom `matmul_user` kernel using the same workflow you used for baseline analysis.
7272

73-
Next, you'll compare instruction mix and throughput across scalar, NEON, SVE, and KleidiAI variants.
73+
Next, you'll compare Instruction Mix and throughput across scalar, Neon, SVE, and KleidiAI variants.

content/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-5.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ Rename each run with a descriptive name, such as baseline and Neon, so you can i
3030

3131
The baseline profile is mostly scalar instructions. After you add Neon intrinsics, the instruction mix shifts toward Advanced SIMD (Neon) instructions, showing that the code is using Arm Neon hardware more effectively.
3232

33-
![Instruction Mix comparison view showing scalar-dominant baseline versus NEON variant with increased ASIMD instruction share in the hot matmul path.#center](./neon_scalar_instruction_mix.webp "NEON versus scalar instruction mix")
33+
![Instruction Mix comparison view showing scalar-dominant baseline versus Neon variant with increased ASIMD instruction share in the hot matmul path.#center](./neon_scalar_instruction_mix.webp "Neon versus scalar instruction mix")
3434

3535
You can also compare SVE variants in the same way. The increase in SVE operations shows that this path is now utilizing SVE hardware.
3636

@@ -56,11 +56,11 @@ For variable-length vectorization, compare with an explicit SVE implementation t
5656

5757
For a full-page view, open a [Godbolt session with all three matmul kernels](https://godbolt.org/z/E4a7Wxh8K).
5858

59-
## Measure speed up
59+
## Measure speedup
6060

6161
Run the provided comparison script to measure tokens per second across all available binaries:
6262

63-
```bash bash { command_line="ubuntu@ip | 2-30"}
63+
```bash { command_line="ubuntu@ip | 2-30"}
6464
./compare_gpt2_variants.sh
6565
Model: gpt2-medium
6666
Prompt: Once upon a time
@@ -88,6 +88,6 @@ These results show that intrinsics increase throughput from about 3 tok/s in the
8888

8989
## What you've accomplished and what's next
9090

91-
You've now verified vectorization using the Instruction Mix recipe, compared throughput across Neon and SVE, and measure tokens per second across the available binaries.
91+
You've now verified vectorization using the Instruction Mix recipe, compared throughput across Neon and SVE, and measured tokens per second across the available binaries.
9292

93-
Next, you'll use optimized libraries to push performance further.
93+
Next, you'll use optimized libraries to push performance further.

content/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-6.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ Getting close to peak matmul performance is complex, especially in transformer i
1414

1515
- *Tiling* breaks large matrices into smaller blocks that fit better in cache, so data is reused more often and memory bandwidth pressure is lower.
1616

17-
Arm created [KleidiAI](https://github.com/ARM-software/kleidiai) to provide the fastest Arm CPU microkernels on packing and tiled matrix multiplication, so you can use these optimizations without writing and tuning every low-level kernel yourself.
17+
Arm created [KleidiAI](https://github.com/ARM-software/kleidiai) to provide accelerated Arm CPU microkernels on packing and tiled matrix multiplication, so you can use these optimizations without writing and tuning every low-level kernel yourself.
1818

1919
## KleidiAI integration in the GPT-2 example
2020

@@ -24,7 +24,7 @@ The file `src/kernels/matmul_kai_sve.cpp` is the runtime bridge between your mod
2424
- `kai_get_rhs_packed_offset_matmul_clamp_f32_f32_f32p4vlx1b_6x4vl_sve_mla`
2525
- `kai_run_matmul_clamp_f32_f32_f32p4vlx1b_6x4vl_sve_mla`
2626

27-
The `kai_run_matmul_clamp_f32_f32_f32p4vlx1b_6x4vl_sve_mla` is our entry. As per [the naming convention](https://github.com/ARM-software/kleidiai/blob/main/kai/ukernels/matmul/README.md), this kernel performs an FP32 matrix multiplication, computing output tiles of 6 × 4VL (six rows by four SVE vector lengths of columns) using SVE (multiply–accumulate) MLA instructions on prepacked RHS weights for efficient cache and SIMD utilization.
27+
`kai_run_matmul_clamp_f32_f32_f32p4vlx1b_6x4vl_sve_mla` is your entry. According to [the naming convention](https://github.com/ARM-software/kleidiai/blob/main/kai/ukernels/matmul/README.md), this kernel performs an FP32 matrix multiplication, computing output tiles of 6 × 4VL (six rows by four SVE vector lengths of columns) using SVE (multiply–accumulate) MLA instructions on prepacked RHS weights for efficient cache and SIMD utilization.
2828

2929
For more details on KleidiAI, see the [official GitLab repository](https://gitlab.arm.com/kleidi/kleidiai).
3030

@@ -105,7 +105,7 @@ Compared to the non-vectorized baseline (`gpt2`), this FP32 KleidiAI microkernel
105105
You can increase throughput by running the KleidiAI path with multiple matmul threads. For this 355M model, tune `--matmul-threads` heuristically on your target system to find the optimal value. For a Graviton 3-based instance, you'll observe a max token generation speed of 34.5 token/s with 4 threads.
106106

107107

108-
``` bash { command_line="ubuntu@ip | 3-50"}
108+
```bash { command_line="ubuntu@ip | 3-50"}
109109
cd build
110110
./gpt2_kai_sve --model gpt2-medium "Once upon a time" -n 150 --matmul-threads 4
111111
Weights path: /home/ubuntu/GPT-2-DEMO/GPT-2-Example/models/gpt2-medium/weights.bin

0 commit comments

Comments
 (0)