You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/learning-paths/cross-platform/simd-loops/1-about.md
+7-25Lines changed: 7 additions & 25 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,35 +8,17 @@ layout: learningpathall
8
8
9
9
Writing high-performance software on Arm often means using single-instruction, multiple-data (SIMD) technologies. Many developers start with NEON, a familiar fixed-width vector extension. As Arm architectures evolve, so do the SIMD capabilities available to you.
10
10
11
-
This Learning Path uses the **Scalable Vector Extension (SVE)** and the **Scalable Matrix Extension (SME)** to teach modern SIMD patterns. They are two powerful, scalable vector extensions designed for modern workloads. Unlike NEON, these architecture extensions are not just wider; they are fundamentally different. They introduce predication, vector-length-agnostic (VLA) programming, gather/scatter, streaming modes, and tile-based compute with ZA state. The result is more power and flexibility, with a learning curve to match.
11
+
This Learning Path uses the **Scalable Vector Extension (SVE)** and the **Scalable Matrix Extension (SME)** to demonstrate modern SIMD patterns. They are two powerful, scalable vector extensions designed for modern workloads. Unlike NEON, these architecture extensions are not just wider; they are fundamentally different. They introduce predication, vector-length-agnostic (VLA) programming, gather/scatter, streaming modes, and tile-based compute with ZA state. The result is more power and flexibility, but there can be a learning curve to match.
12
12
13
-
**SIMD Loops**offer a hands-on way to climb the learning curve. It is a public codebase of self-contained, real loop kernels written in C, Arm C Language Extensions (ACLE) intrinsics, and selected inline assembly. Kernels span tasks such as matrix multiply, sorting, and string processing. You can build them, run them, step through them, and adapt them for your own SIMD workloads.
13
+
The **SIMD Loops**project offers a hands-on way to climb the learning curve. It is a public codebase of self-contained, real loop kernels written in C, Arm C Language Extensions (ACLE) intrinsics, and selected inline assembly. Kernels span tasks such as matrix multiply, sorting, and string processing. You can build them, run them, step through them, and adapt them for your own SIMD workloads.
project, licensed under BSD 3-Clause, built to help you learn how to write SIMD code for modern Arm
21
-
architectures, specifically using SVE and SME.
22
-
It is designed for programmers who already know
23
-
their way around NEON intrinsics but are now facing the more powerful and
24
-
complex world of SVE and SME.
25
-
26
-
The goal of SIMD Loops is to provide working, readable examples that demonstrate
27
-
how to use the full range of features available in SVE, SVE2, and SME2. Each
28
-
example is a self-contained loop kernel, a small piece of code that performs
29
-
a specific task like matrix multiplication, vector reduction, histogram, or
30
-
memory copy. These examples show how that task can be implemented across different
31
-
vector instruction sets.
32
-
33
-
Unlike a cookbook that tries to provide a recipe for every problem, SIMD Loops
34
-
takes the opposite approach. It aims to showcase the architecture rather than
35
-
the problem. The loop kernels are chosen to be realistic and meaningful, but the
36
-
main goal is to demonstrate how specific features and instructions work in
37
-
practice. If you are trying to understand scalability, predication,
38
-
gather/scatter, streaming mode, ZA storage, compact instructions, or the
39
-
mechanics of matrix tiles, this is where you will see them in action.
17
+
SIMD Loops is an open-source project, licensed under BSD 3-Clause, built to help you learn how to write SIMD code for modern Arm architectures, specifically using SVE and SME. It is designed for programmers who already know their way around NEON intrinsics but are now facing the more powerful and complex world of SVE and SME.
18
+
19
+
The goal of SIMD Loops is to provide working, readable examples that demonstrate how to use the full range of features available in SVE, SVE2, and SME2. Each example is a self-contained loop kernel, a small piece of code that performs a specific task like matrix multiplication, vector reduction, histogram, or memory copy. These examples show how that task can be implemented across different vector instruction sets.
20
+
21
+
Unlike a cookbook that attempts to provide a recipe for every problem, SIMD Loops takes the opposite approach. It aims to showcase the architecture rather than the problem itself. The loop kernels are chosen to be realistic and meaningful, but the main goal is to demonstrate how specific features and instructions work in practice. If you are trying to understand scalability, predication, gather/scatter, streaming mode, ZA storage, compact instructions, or the mechanics of matrix tiles, this is where you can see them in action.
40
22
41
23
The project includes:
42
24
- Many numbered loop kernels, each focused on a specific feature or pattern
In the SIMD Loops project, the source code for the loops is organized under the `loops` directory. The complete
37
-
list of loops is documented in the `loops.inc` file, which includes a brief
38
-
description and the purpose of each loop. Every loop is associated with a
39
-
uniquely named source file following the pattern `loop_<NNN>.c`, where
40
-
`<NNN>` represents the loop number.
36
+
In the SIMD Loops project, the source code for the loops is organized under the `loops` directory. The complete list of loops is documented in the `loops.inc` file, which includes a brief description and the purpose of each loop. Every loop is associated with a uniquely named source file following the pattern `loop_<NNN>.c`, where `<NNN>` represents the loop number.
// Main of loop: buffer allocation, loop function call, result checking
85
81
```
86
82
87
-
Each loop is implemented in several SIMD extension variants. Conditional
88
-
compilation selects one of the implementations for the
89
-
`inner_loop_<NNN>` function.
83
+
Each loop is implemented in several SIMD extension variants. Conditional compilation selects one of the implementations for the `inner_loop_<NNN>` function.
90
84
91
-
The native C implementation is written first, and
92
-
it can be generated either when building natively with `-DHAVE_NATIVE` or through
93
-
compiler auto-vectorization with `-DHAVE_AUTOVEC`.
85
+
The native C implementation is written first, and it can be generated either when building natively with `-DHAVE_NATIVE` or through compiler auto-vectorization with `-DHAVE_AUTOVEC`.
94
86
95
-
When SIMD ACLE is supported (SME, SVE, or NEON),
96
-
the code is compiled using high-level intrinsics. If ACLE
97
-
support is not available, the build process falls back to handwritten inline
98
-
assembly targeting one of the available SIMD extensions, such as SME2.1, SME2,
99
-
SVE2.1, SVE2, and others.
87
+
When SIMD ACLE is supported (SME, SVE, or NEON), the code is compiled using high-level intrinsics. If ACLE support is not available, the build process falls back to handwritten inline assembly targeting one of the available SIMD extensions, such as SME2.1, SME2, SVE2.1, SVE2, and others.
100
88
101
-
The overall code structure also includes setup and
102
-
cleanup code in the main function, where memory buffers are allocated, the
103
-
selected loop kernel is executed, and results are verified for correctness.
89
+
The overall code structure also includes setup and cleanup code in the main function, where memory buffers are allocated, the selected loop kernel is executed, and results are verified for correctness.
104
90
105
-
At compile time, you can select which loop optimization to compile, whether it
106
-
is based on SME or SVE intrinsics, or one of the available inline assembly
107
-
variants.
91
+
At compile time, you can select which loop optimization to compile, whether it is based on SME or SVE intrinsics, or one of the available inline assembly variants.
Copy file name to clipboardExpand all lines: content/learning-paths/cross-platform/simd-loops/3-example.md
+19-21Lines changed: 19 additions & 21 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,13 +6,13 @@ weight: 4
6
6
layout: learningpathall
7
7
---
8
8
9
-
To illustrate the structure and design principles of SIMD Loops, consider loop 202 as an example
9
+
To illustrate the structure and design principles of SIMD Loops, consider loop 202 as an example.
10
10
11
-
Use a text editor to open `loops/loop_202.c`
11
+
Use a text editor to open `loops/loop_202.c`.
12
12
13
13
The function `inner_loop_202()` is defined around lines 60–70 in `loops/loop_202.c` and calls the `matmul_fp32` routine defined in `loops/matmul_fp32.c`
14
14
15
-
Open `loops/matmul_fp32.c` in your editor
15
+
Open `loops/matmul_fp32.c` in your editor.
16
16
17
17
This loop implements single-precision floating-point matrix multiplication of the form:
18
18
@@ -24,8 +24,7 @@ You can view matrix multiplication in two equivalent ways:
24
24
25
25
## Data structure
26
26
27
-
The loop begins by defining a data structure that captures the matrix
28
-
dimensions (`M`, `K`, `N`) along with input and output buffers:
27
+
The loop begins by defining a data structure that captures the matrix dimensions (`M`, `K`, `N`) along with input and output buffers:
29
28
30
29
```C
31
30
struct loop_202_data {
@@ -43,19 +42,19 @@ For this loop:
43
42
- Matrix `b` is stored in row-major order
44
43
- The memory regions referenced by `a`, `b`, and `c` do not alias, as indicated by the `restrict` keyword
45
44
46
-
This layout helps optimize memory access patterns across the targeted SIMD architectures
45
+
This layout helps optimize memory access patterns across the targeted SIMD architectures.
47
46
48
47
## Loop attributes
49
48
50
49
Loop attributes are specified per target architecture:
51
50
-**SME targets** — `inner_loop_202` is invoked with the `__arm_streaming` attribute and uses a shared `ZA` register context (`__arm_inout("za")`). These attributes are wrapped in the `LOOP_ATTR` macro
52
51
-**SVE or NEON targets** — no additional attributes are required
53
52
54
-
This design enables portability across SIMD extensions
53
+
This design enables portability across SIMD extensions.
55
54
56
55
## Function implementation
57
56
58
-
`loops/matmul_fp32.c` provides several optimizations of matrix multiplication, including ACLE intrinsics and hand-optimized assembly
57
+
`loops/matmul_fp32.c` provides several optimizations of matrix multiplication, including ACLE intrinsics and hand-optimized assembly.
59
58
60
59
### Scalar code
61
60
@@ -79,7 +78,7 @@ A scalar C implementation appears around lines 40–52. It follows the dot-produ
79
78
80
79
### SVE-optimized code
81
80
82
-
The SVE version uses indexed floating-point multiply–accumulate (`fmla`) to optimize the matrix multiplication operation. The outer product is decomposed into indexed multiply steps, and results accumulate directly in `Z` registers
81
+
The SVE version uses indexed floating-point multiply–accumulate (`fmla`) to optimize the matrix multiplication operation. The outer product is decomposed into indexed multiply steps, and results accumulate directly in `Z` registers.
83
82
84
83
In the intrinsics version (lines 167–210), the innermost loop is structured as follows:
85
84
@@ -130,23 +129,23 @@ In the intrinsics version (lines 167–210), the innermost loop is structured as
130
129
}
131
130
```
132
131
133
-
At the beginning of the loop, the accumulators (`Z` registers) are zeroed using `svdup` (or `dup` in assembly), encapsulated in the `ZERO_PAIR` macro
132
+
At the beginning of the loop, the accumulators (`Z` registers) are zeroed using `svdup` (or `dup` in assembly), encapsulated in the `ZERO_PAIR` macro.
134
133
135
134
Within each iteration over the `K` dimension:
136
-
- 128 bits (four consecutive floating-point values) are loaded from `A` using replicate loads `svld1rq` (or `ld1rqw`), via`LOADA_PAIR`
135
+
- 128 bits (four consecutive floating-point values) are loaded from `A` using replicate loads `svld1rq` (or `ld1rqw`), through`LOADA_PAIR`
137
136
- Two vectors are loaded from `B` using SVE vector loads, using `LOADB_PAIR`
138
137
- Indexed `fmla` operations compute element–vector products and accumulate into 16 `Z` register accumulators
139
138
- Partial sums build up the output tile
140
139
141
-
After all `K` iterations, results in the `Z` registers are stored to `C` using the `STORE_PAIR` macro
140
+
After all `K` iterations, results in the `Z` registers are stored to `C` using the `STORE_PAIR` macro.
142
141
143
-
The equivalent SVE hand-optimized assembly appears around lines 478–598
142
+
The equivalent SVE hand-optimized assembly appears around lines 478–598.
144
143
145
-
This loop shows how SVE registers and indexed `fmla` enable efficient decomposition of the outer-product formulation into parallel, vectorized accumulation
144
+
This loop shows how SVE registers and indexed `fmla` enable efficient decomposition of the outer-product formulation into parallel, vectorized accumulation.
146
145
147
-
For SVE/SVE2 semantics and optimization guidance, see the [Scalable Vector Extensions resources](https://developer.arm.com/Architectures/Scalable%20Vector%20Extensions)
146
+
For SVE/SVE2 semantics and optimization guidance, see the [Scalable Vector Extensions resources](https://developer.arm.com/Architectures/Scalable%20Vector%20Extensions).
148
147
149
-
###SME2-optimized code
148
+
## SME2-optimized code
150
149
151
150
The SME2 implementation leverages the outer-product formulation of the matrix
152
151
multiplication function, utilizing the `fmopa` SME instruction to perform the
@@ -226,17 +225,16 @@ For instruction semantics and SME/SME2 optimization guidance, see the [SME Progr
226
225
227
226
## Other optimizations
228
227
229
-
Beyond the SME2 and SVE implementations, this loop also includes additional
230
-
optimized versions that leverage architecture-specific features
228
+
Beyond the SME2 and SVE implementations, this loop also includes additional optimized versions that leverage architecture-specific features.
231
229
232
230
### NEON
233
231
234
-
The NEON version (lines 612–710) uses structure load/store combined with indexed `fmla` to vectorize the computation
232
+
The NEON version (lines 612–710) uses structure load/store combined with indexed `fmla` to vectorize the computation.
235
233
236
234
### SVE2.1
237
235
238
-
The SVE2.1 version (lines 355–462) extends the base SVE approach using multi-vector loads and stores
236
+
The SVE2.1 version (lines 355–462) extends the base SVE approach using multi-vector loads and stores.
239
237
240
238
### SME2.1
241
239
242
-
The SME2.1 version uses `movaz` / `svreadz_hor_za8_u8_vg4` to reinitialize `ZA` tile accumulators while moving data out to registers
240
+
The SME2.1 version uses `movaz`/`svreadz_hor_za8_u8_vg4` to reinitialize `ZA` tile accumulators while moving data out to registers.
Copy file name to clipboardExpand all lines: content/learning-paths/cross-platform/simd-loops/4-conclusion.md
+1-2Lines changed: 1 addition & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -15,5 +15,4 @@ Whether you are moving from NEON or starting directly with SVE and SME, the proj
15
15
16
16
Use the repository to explore, modify, and benchmark kernels so you can understand tradeoffs and apply the patterns to your own workloads.
17
17
18
-
For more information and to get started, visit the GitLab project and see the [README.md](https://gitlab.arm.com/architecture/simd-loops/-/blob/main/README.md)
19
-
for the latest instructions on building and running the code.
18
+
For more information and to get started, visit the GitLab project and see the [README.md](https://gitlab.arm.com/architecture/simd-loops/-/blob/main/README.md) for the latest instructions on building and running the code.
0 commit comments