Skip to content

Commit f64a1ae

Browse files
incorporate technical feedback into PR before merge
1 parent 27fba09 commit f64a1ae

4 files changed

Lines changed: 18 additions & 7 deletions

File tree

88.9 KB
Loading

content/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-2.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -105,7 +105,7 @@ Run the scalar baseline binary:
105105
./build/gpt2 --model gpt2-medium "Once upon a time" -n 20
106106
```
107107

108-
![Animated terminal output showing GPT-2 baseline inference running on Arm Linux, including generated text and the final tokens-per-second summary used for baseline comparison.#center](./gpt2-baseline.gif "GPT-2 baseline runtime output on Arm Linux")
108+
![Animated terminal output showing GPT-2 neon inference running on Arm Linux, including generated text and the final tokens-per-second summary.#center](./gpt2-baseline.gif "GPT-2 Neon runtime output on Arm Linux")
109109

110110
## What you've learned and what's next
111111

content/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-3.md

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,11 @@ Before you optimize, identify where the application spends most of its time. Use
1212

1313
Open Arm Performix and select the **Code Hotspots** recipe. If this is your first run on the target, complete tool deployment as prompted.
1414

15-
Set the launch command to your baseline binary with the number of tokens (`-n`) set to 150. This value keeps startup overhead small compared to inference time, so the profile minimizes the time taken to load the model weights:
15+
Set the launch command to your baseline binary with the number of tokens (`-n`) set to 150 as per the command below. This value keeps startup overhead small compared to inference time, so the profile minimizes the time taken to load the model weights:
16+
17+
```out
18+
<path to repository>/build/gpt2 --model gpt2-medium "Once upon a time" -n 150
19+
```
1620

1721
![Arm Performix Code Hotspots recipe configuration showing launch arguments for the GPT-2 baseline run with -n 150 to emphasize inference runtime.#center](./code_hotspot.webp "Code Hotspots recipe configuration for GPT-2 baseline")
1822

@@ -24,8 +28,7 @@ This confirms that matrix multiplication is the highest-impact optimization targ
2428

2529
## Assess compiler output
2630

27-
We can use online tools such as [Compiler Explorer](https://godbolt.org/) to conveniently see how this function is being compiled with the `-O2 -g` flags.
28-
31+
We can use online tools such as [Compiler Explorer](https://godbolt.org/) to conveniently see how this function is being compiled with the `-O2 -g` flags. The example below uses `GCC 12.1.0`. You can check your installed compiler version with the `g++ --version` command and select the corresponding version from the Compiler Explorer drop-down menu. The generated assembly may differ slightly across compiler versions.
2932

3033
{{< godbolt width="100%" height="400px" mode="assembly" opt="-O2 -g" src="void matmul_ref(float *out, const float *x, const float *W, const float *b, int n_in, int n_out)\n{\n for (int i = 0; i < n_out; i++) {\n float acc = b ? b[i] : 0.f;\n const float *row = W + (unsigned long long)i * (unsigned long long)n_in;\n for (int j = 0; j < n_in; j++) {\n acc += row[j] * x[j];\n }\n out[i] = acc;\n }\n}" >}}
3134

content/learning-paths/servers-and-cloud-computing/performix-instruction-mix/how-to-5.md

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,15 @@ layout: learningpathall
88

99
## Confirm instruction-mix changes after vectorization
1010

11-
In the earlier build step, you created `gpt2_neon` and `gpt2_sve`. These binaries use the reference solutions in `matmul_neon.cpp` and `matmul_sve.cpp`, respectively.
11+
In the earlier build step, you created `gpt2_neon` and `gpt2_sve`. These binaries use the reference solutions in `matmul_neon.cpp` and `matmul_sve.cpp`, respectively. Run the `gpt2_neon` binary with the following command to observe the speedup.
12+
13+
```bash
14+
./build/gpt2_neon --model gpt2-medium "Once upon a time" -n 20
15+
```
16+
17+
![Animated terminal output showing GPT-2 baseline inference running on Arm Linux, including generated text and the final tokens-per-second summary used for baseline comparison.#center](./gpt_neon.gif "GPT-2 Neon runtime output on Arm Linux")
18+
19+
The Neon implementation delivers a noticeable increase in token generation throughput. To evaluate the SVE implementation, run the same workload using the `gpt2_sve` binary instead of `gpt2_neon`. Performance gains will vary across systems, largely depending on the Scalable Vector Length (SVL) supported by the Arm Linux platform.
1220

1321
Rerun the Instruction Mix recipe with the `gpt2_neon` binary using the same recipe settings and workload arguments as baseline. After the run completes, select both runs and click **Compare** to open a comparison view.
1422

@@ -30,12 +38,12 @@ You can also compare SVE variants in the same way. The increase in SVE operation
3038

3139
#### Neon kernel
3240

33-
You can also inspect the Neon intrinsic implementation using Compiler Explorer, where the hot accumulation step (`vacc`) runs in aSIMD (Neon) registers such as `v0`:
41+
You can also inspect the Neon intrinsic implementation using Compiler Explorer, where the hot accumulation step (`vacc`) runs in ASIMD (Neon) registers such as `v0`:
3442

3543
{{< godbolt width="100%" height="400px" mode="assembly" opt="-O2 -g -march=armv8.2-a+simd" src="#include <arm_neon.h>\n\nvoid matmul_neon(float *out, const float *x, const float *W, const float *b,\n int n_in, int n_out) {\n for (int i = 0; i < n_out; i++) {\n float acc = b ? b[i] : 0.f;\n const float *row = W + (unsigned long long)i * (unsigned long long)n_in;\n int j = 0;\n float32x4_t vacc = vdupq_n_f32(0.f);\n for (; j + 4 <= n_in; j += 4) {\n const float32x4_t vw = vld1q_f32(row + j);\n const float32x4_t vx = vld1q_f32(x + j);\n vacc = vfmaq_f32(vacc, vw, vx);\n }\n acc += vaddvq_f32(vacc);\n for (; j < n_in; j++) acc += row[j] * x[j];\n out[i] = acc;\n }\n}" >}}
3644

3745

38-
### SVE kernel
46+
#### SVE kernel
3947

4048
For variable-length vectorization, compare with an explicit SVE implementation that assumes SVE support, where the hot accumulation step (`vacc`) runs in SVE z registers with predicate-controlled loads and multiply-accumulate:
4149

0 commit comments

Comments
 (0)