Skip to content

Commit 9e63ebd

Browse files
Update learning path content for ERNIE-4.5: refine section titles, enhance clarity, and adjust skill level to Advanced
1 parent 25bfecd commit 9e63ebd

4 files changed

Lines changed: 19 additions & 12 deletions

File tree

content/learning-paths/cross-platform/ernie_moe_v9/2_llamacpp_installation.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ weight: 3
44
layout: "learningpathall"
55
---
66

7-
## Set up llama.cpp on an Armv9 development board
7+
## Overview
88

99
In the previous section, you learned how Mixture of Experts (MoE) models reduce resource consumption by activating only a fraction of parameters.
1010
Now, you'll walk through how to prepare your environment to deploy ERNIE-4.5 models on an Armv9 platform using llama.cpp.
@@ -121,4 +121,6 @@ This confirms the GGUF model is successfully loaded, the llama.cpp build functio
121121

122122
The prompt "Please introduce Mixture of Experts in Chinese." was chosen because it tests two important capabilities. First, it verifies bilingual capability since the instruction is in English but the answer is expected in Chinese, confirming that ERNIE-4.5's multilingual support works correctly. Second, it engages MoE behavior by having the model explain a technical concept in Chinese, which likely activates different experts within the model during inference. Even though routing isn't explicitly logged, the richness and precision of the output suggest that MoE routing functions as designed. This type of prompt increases the likelihood of multiple experts being activated simultaneously, including language generation, machine learning knowledge, and Chinese translation experts.
123123

124-
You've now successfully set up llama.cpp on Armv9 hardware, deployed the ERNIE-4.5 model, and confirmed that multilingual inference works correctly. With this foundation in place, you're ready to explore expert routing behavior and benchmark performance optimizations in the next sections.
124+
## What you've accomplished and what's next
125+
126+
You've successfully set up llama.cpp on Armv9 hardware, deployed the ERNIE-4.5 model, and confirmed that multilingual inference works correctly. With this foundation in place, you're ready to explore expert routing behavior and compare model variants in the next section.

content/learning-paths/cross-platform/ernie_moe_v9/3_erine_moe.md

Lines changed: 12 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ Here is the PT variant:
4141
--jinja
4242
```
4343

44-
The answer looks like this:
44+
The output is similar to:
4545

4646
```output
4747
Assistant: 1. **Top Motivation**: Achieving visible results and maintaining progress through efficient recovery nutrition.
@@ -65,7 +65,7 @@ Here is the Thinking variant:
6565
--jinja
6666
```
6767

68-
You see a more comprehensive answer like this:
68+
The output is similar to:
6969

7070
```output
7171
assistant
@@ -161,18 +161,23 @@ Run inference with the same prompt and monitor the console for lines like this:
161161
---[DEBUG]--- entering build_moe_ffn at layer 27 with 64 experts (use 64)
162162
```
163163

164-
This reveals how many experts (for example, 6) and how many tokens (for example, 16) were routed at that layer.
164+
This output shows that each layer has 64 total experts available. The actual number of experts activated per token (typically 6 for ERNIE-4.5) is determined by the router during inference and isn't directly visible in this debug output.
165165

166166
{{% notice Note %}}
167-
You can also trace the function `llm_graph_context::build_moe_ffn()` in `src/llama-graph.cpp` to see how expert selection works.
167+
You can also trace the function `llm_graph_context::build_moe_ffn()` in `src/llama-graph.cpp` to see how expert selection works at a deeper level.
168168
{{% /notice %}}
169169

170-
Remove the print statement from `src/models/ernie4-5-moe.cpp` before moving to the next section.
170+
Remove the print statement from `src/models/ernie4-5-moe.cpp` and rebuild llama.cpp before moving to the next section.
171171

172172
As you review the debug output, observe whether the number of active experts changes between the PT and Thinking models. Look for patterns in routing, such as different token batches routing to differing expert sets. You can also correlate routing behavior with output differences, as deeper routing variety might align with more detailed responses.
173173

174-
## Summary
174+
## What you've accomplished and what's next
175175

176-
This task highlights the advantage of MoE fine-tuning. Even with the same architecture, thoughtful tuning can significantly change a model's reasoning behavior. The Thinking model is better suited for applications that need analytical depth, making it ideal for edge AI scenarios like customer profiling or real-time recommendations.
176+
In this section, you:
177+
- Compared ERNIE-4.5 PT and Thinking model outputs on the same task
178+
- Observed how fine-tuning affects reasoning depth and response structure
179+
- Learned how to add debug instrumentation to examine MoE expert routing
180+
181+
This comparison highlights the advantage of MoE fine-tuning. Even with the same architecture, thoughtful tuning can significantly change a model's reasoning behavior. The Thinking model is better suited for applications that need analytical depth, making it ideal for edge AI scenarios like customer profiling or real-time recommendations.
177182

178183
In the next section, you switch focus from model behavior to system-level performance by compiling with Armv9 instruction sets and measuring the impact on inference speed.

content/learning-paths/cross-platform/ernie_moe_v9/4_v9_optimization.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -114,8 +114,8 @@ After running benchmarks with and without Armv9-specific instructions, the resul
114114

115115
Vectorized kernels (i8mm, dotprod, SVE) drastically improve inference throughput. The pp512 test shows the most significant acceleration with a 2.57× improvement. Other patterns like tg128 and pp128+tg128 also achieve measurable gains. These results demonstrate the broad benefit of hardware-aware builds and show that Armv9 optimization enables practical real-time inference for 21 B models on edge-class hardware.
116116

117-
### Summary
117+
## What you've accomplished
118118

119-
Throughout this Learning Path, you deployed a 21 B parameter Chinese MoE model on edge-class Armv9 hardware. You started by understanding how MoE reduces memory usage by activating only a small subset of parameters per token. After setting up llama.cpp on a Radxa O6 board, you compared ERNIE-4.5 Thinking and PT model behaviors while examining expert routing logic with debug instrumentation. Finally, you applied Armv9 hardware optimizations and achieved over 2.5× speed improvements in token throughput.
119+
Throughout this Learning Path, you deployed a 21-billion-parameter Chinese MoE model on edge-class Armv9 hardware. You started by understanding how MoE reduces memory usage by activating only a small subset of parameters per token. After setting up llama.cpp on a Radxa O6 board, you compared ERNIE-4.5 Thinking and PT model behaviors while examining expert routing logic with debug instrumentation. Finally, you applied Armv9 hardware optimizations and achieved over 2.5× speed improvements in token throughput.
120120

121121
You can now deploy, profile, and tune Chinese LLMs for efficient inference on modern Arm CPUs.

content/learning-paths/cross-platform/ernie_moe_v9/_index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ prerequisites:
2020
author: Odin Shen
2121

2222
### Tags
23-
skilllevels: Introductory
23+
skilllevels: Advanced
2424
subjects: ML
2525
armips:
2626
- Cortex-A

0 commit comments

Comments
 (0)