Skip to content

Commit 2388a1c

Browse files
Update ERNIE-MoE v9 learning path
1 parent 25d3c8a commit 2388a1c

4 files changed

Lines changed: 16 additions & 17 deletions

File tree

content/learning-paths/cross-platform/ernie_moe_v9/1_mixture_of_experts.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -20,15 +20,15 @@ In a typical MoE setup, the model consists of many expert sub-networks (for exam
2020

2121
MoE architecture provides several advantages that make it particularly well-suited for edge deployment and large-scale model development:
2222

23-
**Scalable model size**: You can increase total parameter count without linearly increasing inference cost, allowing for larger, more capable models within the same resource constraints.
23+
Scalable model size: you can increase total parameter count without linearly increasing inference cost, allowing for larger, more capable models within the same resource constraints.
2424

25-
**Efficient inference**: The architecture requires lower memory and FLOPs per input compared to dense models of equivalent capacity, making real-time applications more feasible.
25+
Efficient inference: the architecture requires lower memory and FLOPs per input compared to dense models of equivalent capacity, making real-time applications more feasible.
2626

27-
**Modularity**: Each expert can learn domain-specific patterns such as finance, medicine, or language, enabling the model to handle diverse tasks without retraining the entire network.
27+
Modularity: each expert can learn domain-specific patterns such as finance, medicine, or language, enabling the model to handle diverse tasks without retraining the entire network.
2828

29-
**Specialization**: The architecture encourages the model to learn distinct processing behaviors across different experts, improving performance on specialized tasks while maintaining general capability.
29+
Specialization: the architecture encourages the model to learn distinct processing behaviors across different experts, improving performance on specialized tasks while maintaining general capability.
3030

31-
**Routing flexibility**: The dynamic expert selection makes it easier to adapt to specific tasks using fine-tuned routing strategies, allowing for task-specific optimizations without modifying the core model.
31+
Routing flexibility: the dynamic expert selection makes it easier to adapt to specific tasks using fine-tuned routing strategies, allowing for task-specific optimizations without modifying the core model.
3232

3333
## ERNIE-4.5: An MoE model for Chinese NLP
3434

content/learning-paths/cross-platform/ernie_moe_v9/2_llamacpp_installation.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -98,10 +98,10 @@ cd $HOME/llama.cpp/build
9898
```
9999

100100
Note the flags:
101-
- `-p`: Passes the input prompt directly as a string.
102-
- `-c 4096`: Sets the context length (in tokens). A longer context allows the model to "remember" more input text, which is crucial for long-form tasks. Here, you use the recommended 4096 tokens.
103-
- `-t 12`: Specifies the number of CPU threads used for inference. You should match this number to the physical cores (or logical threads) available on your system to maximize performance.
104-
- `--jinja`: Enables Jinja-style prompt templates. Many Chinese-oriented MoE models rely on this template format for structured inputs.
101+
- `-p`: passes the input prompt directly as a string.
102+
- `-c 4096`: sets the context length (in tokens). A longer context allows the model to "remember" more input text, which is crucial for long-form tasks. Here, you use the recommended 4096 tokens.
103+
- `-t 12`: specifies the number of CPU threads used for inference. You should match this number to the physical cores (or logical threads) available on your system to maximize performance.
104+
- `--jinja`: enables Jinja-style prompt templates. Many Chinese-oriented MoE models rely on this template format for structured inputs.
105105

106106
If everything is set up correctly, you see metadata output from llama.cpp indicating the model's architecture and size:
107107

@@ -113,7 +113,7 @@ print_info: general.name = Ernie-4.5-21B-A3B-Thinking
113113

114114
Once inference is complete, the expected output looks like this (in Chinese):
115115

116-
![img1 alt-text#center](meo_result.png "Introduce Mixture of Experts in Chinese")
116+
![Terminal output showing ERNIE-4.5 model response in Chinese with structured markdown formatting explaining Mixture of Experts concept alt-txt#center](meo_result.png "Introducing Mixture of Experts in Chinese")
117117

118118
This answer demonstrates the model's multilingual and structured reasoning ability. It begins with a narrative explanation introducing the concept of Mixture of Experts (MoE), followed by a well-organized markdown-style summary with section headers and bullet points. The response describes MoE as a model combining multiple specialized sub-models (experts) and a routing mechanism (gateway) to activate only a few experts per input. It explains how routing selects top experts based on input features and how the system reduces computation by activating only a small number of experts per token. The answer also gives examples such as word segmentation and translation, explaining how different experts might focus on distinct subtasks in Chinese NLP applications.
119119

content/learning-paths/cross-platform/ernie_moe_v9/3_ernie_moe.md

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,8 @@ Now that both ERNIE-4.5 models are installed and verified, you can compare their
1111
In this section, you compare the inference styles of PT and Thinking models, and learn how to inspect internal MoE expert routing behavior during generation.
1212

1313
Both ERNIE-4.5 models share the same MoE architecture and parameter count (around 21 B total, around 3 B activated at runtime), but they're tuned for different objectives:
14-
- PT: General-purpose model trained on multilingual corpora for versatile tasks
15-
- Thinking: Tuned for multi-step reasoning, long context, and structured response generation
14+
- PT: general-purpose model trained on multilingual corpora for versatile tasks
15+
- Thinking: tuned for multi-step reasoning, long context, and structured response generation
1616

1717
You can now observe how these different tuning objectives affect output behavior.
1818

@@ -49,11 +49,10 @@ Assistant: 1. **Top Motivation**: Achieving visible results and maintaining prog
4949
2. **New Product Line**: *Smart Recovery Meal Kits* – Customizable, nutrient-dense, and easy-to-prepare post-workout meals designed for quick consumption and recovery optimization.
5050
3. **Marketing Tagline**: "Fuel Progress, Recover Faster – Smart Nutrition for the Hard Worker."
5151
```
52-
5352
The answer shows:
54-
- Delivers conclusions directly: motivations and pain points are briefly mentioned with little reasoning.
55-
- Product ideas are sensible but templated: suggestions like "Smart Recovery Meal Kits" are plausible but lack contextual grounding in user behavior.
56-
- Tagline reads like a standard advertisement: for example, "Fuel Progress, Recover Faster" feels promotional rather than personalized.
53+
- Direct conclusions with minimal reasoning
54+
- Sensible but general product suggestions (such as "Smart Recovery Meal Kits")
55+
- Standard advertising tone in the tagline ("Fuel Progress, Recover Faster")
5756

5857
Here is the Thinking variant:
5958

content/learning-paths/cross-platform/ernie_moe_v9/_index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ title: Run ERNIE-4.5 Mixture of Experts model on Armv9 with llama.cpp
33

44
minutes_to_complete: 60
55

6-
who_is_this_for: This Learning Path is for developers and engineers who want to deploy Mixture of Experts (MoE) models, such as ERNIE 4.5, on edge devices. MoE architectures allow large LLMs with 21 billion or more parameters to run with only a fraction of their weights active per inference, making them ideal for resource constrained environments.
6+
who_is_this_for: This is an advanced topic for developers and engineers who want to deploy Mixture of Experts (MoE) models, such as ERNIE 4.5, on edge devices. MoE architectures allow large LLMs with 21 billion or more parameters to run with only a fraction of their weights active per inference, making them ideal for resource constrained environments.
77

88
learning_objectives:
99
- Deploy MoE models like ERNIE-4.5 on edge devices using llama.cpp

0 commit comments

Comments
 (0)