MNN Tech review

pareenaverma · pareenaverma · commit 2fb946ff633b · 2026-04-24T08:26:48.000-04:00
diff --git a/content/learning-paths/cross-platform/multimodel_mnn_v9/2_mnn_build.md b/content/learning-paths/cross-platform/multimodel_mnn_v9/2_mnn_build.md
@@ -6,11 +6,11 @@ layout: learningpathall
 
 ## Introduction
 
-In this section, you will build **MNN** natively on your Armv9 Linux system and verify that the `llm_demo` binary can load a **prebuilt Omni MNN model package**. This sets up everything needed for the text, vision, and audio demos in later sections.
+In this section, you will build **MNN** natively on your Armv9 Linux system and verify that the `llm_demo` binary can load a prebuilt Omni MNN model package. This sets up everything needed for the text, vision, and audio demos in later sections.
 
-This section uses a native **CPU-only** MNN build on Armv9. That is a deliberate design choice, not a fallback. The goal is to show how a compact, reproducible, deployment-friendly software stack can run directly on an Armv9 CPU without depending on a discrete GPU or separate accelerator.
+This section uses a native CPU-only MNN build on Armv9. That is a deliberate design choice, not a fallback. The goal is to show how a compact, reproducible, deployment-friendly software stack can run directly on an Armv9 CPU without depending on a discrete GPU or separate accelerator.
 
-At the end of this module, you will have:
+At the end of this section, you will have:
 
 - a working `llm_demo` binary
 - a validated model directory that includes `config.json`
@@ -26,13 +26,13 @@ mkdir -p ~/mnn
 cd ~/mnn
 ```
 
-## Why build natively on Armv9 first
+## Why build natively on Armv9
 
-This Learning Path uses a native build on the target Armv9 system before introducing any cross-compilation workflow. Building directly on the target helps reduce environment drift, makes library and toolchain issues easier to diagnose, and lets you validate the runtime in the same environment where you will execute the model.
+Building, running inference, and deploying all happen directly on the Armv9 device. There is no cross-compilation involved. This keeps the toolchain simple, eliminates environment drift between build and target, and means any library or configuration issue you encounter is the same one you would hit in production.
 
-A native-first approach also makes it easier to confirm that the final binary, shared libraries, and model assets work together correctly on the Armv9 platform. For a reproducible Learning Path, this is usually the fastest way to get to a working baseline before optimizing or automating the workflow further.
+Building on the target also makes it straightforward to confirm that the binary, shared libraries, and model assets all resolve correctly in the same environment where you will run the model.
 
-## Build MNN natively on Armv9
+## Build MNN
 
 Clone the MNN repository:
 
@@ -41,12 +41,20 @@ git clone https://github.com/alibaba/MNN.git
 cd MNN
 ```
 
+Install the required build dependencies:
+
+```bash
+sudo apt update
+sudo apt install -y build-essential gcc g++ cmake
+```
+
 Configure and build MNN with LLM, audio, and Omni support enabled:
 
 ```bash
-rm -rf build
 mkdir build && cd build
+```
 
+```bash
 cmake .. \
   -DCMAKE_BUILD_TYPE=Release \
   -DMNN_BUILD_SHARED=ON \
@@ -85,7 +93,6 @@ If your system already has another MNN installation, `llm_demo` can load a diffe
 From the build directory, inspect the runtime dependencies:
 
 ```bash
-cd ~/mnn/MNN/build
 ldd ./llm_demo | grep -E "libMNN|Express|Audio|OpenCV" || true
 ```
 
@@ -128,7 +135,7 @@ ldd ./llm_demo | grep -E "libMNN|Express|Audio|OpenCV" || true
 
 ## Download the prebuilt Omni model package
 
-This Learning Path uses a prebuilt [Omni model package](https://huggingface.co/taobao-mnn/Qwen2.5-Omni-7B-MNN) that is already prepared for MNN deployment.
+This Learning Path uses a prebuilt [Omni model package](https://www.modelscope.cn/MNN/Qwen2.5-Omni-7B-MNN) that is already prepared for MNN deployment. The full package is approximately 15 GB, so ensure you have sufficient disk space and a stable internet connection before cloning.
 
 Clone the model repository into your workspace:
 
@@ -138,64 +145,31 @@ git clone https://www.modelscope.cn/MNN/Qwen2.5-Omni-7B-MNN.git
 cd ~/mnn/Qwen2.5-Omni-7B-MNN
 ```
 
-## Verify that the required model files are present
-
-Check that the repository contains the expected configuration and model files:
-
-```bash
-ls -lh ~/mnn/Qwen2.5-Omni-7B-MNN
-```
-
-Verify that `config.json` exists:
-
-```bash
-cat ~/mnn/Qwen2.5-Omni-7B-MNN/config.json
-```
-
-The presence of `config.json` alone does not guarantee that the full model weights are available locally. Confirm that the main model files are present:
+## Download the model weights
 
-- `llm.mnn`
-- `llm.mnn.weight`
+After cloning, install Git LFS and pull the large model files:
 
 ```bash
-ls -lh ~/mnn/Qwen2.5-Omni-7B-MNN/llm.mnn ~/mnn/Qwen2.5-Omni-7B-MNN/llm.mnn.weight
-```
-
-If both files exist and show non-trivial file sizes, the model package is ready for inference.
-
-## If the model weights are missing
-
-Some environments fetch the large model files automatically when you clone the repository, while others do not. If `llm.mnn` or `llm.mnn.weight` is missing, install **Git LFS** and pull the large files explicitly.
-
-Install Git LFS:
-
-```bash
-sudo apt-get update
 sudo apt-get install -y git-lfs
 git lfs install
-```
-
-Then download the large model files:
-
-```bash
 cd ~/mnn/Qwen2.5-Omni-7B-MNN
 git lfs pull
 ```
 
 {{% notice Note %}}
-Downloading the full model weights can take a while depending on your network connection and available bandwidth.
+The full model weights are approximately 15 GB. Downloading can take a while depending on your network connection.
 {{% /notice %}}
 
-After `git lfs pull`, verify the files again:
+Verify that the main model files are present and several gigabytes in size:
 
 ```bash
 ls -lh ~/mnn/Qwen2.5-Omni-7B-MNN/llm.mnn ~/mnn/Qwen2.5-Omni-7B-MNN/llm.mnn.weight
 ```
 
-{{% notice Note %}}
-This package is already prepared for MNN deployment.
+If either file is only a few hundred bytes, the LFS download did not complete. Run `git lfs pull` again to resume it.
 
-You do not need to export the model from PyTorch or run additional quantization steps before using it in this Learning Path.
+{{% notice Note %}}
+This package is already prepared for MNN deployment. You do not need to export the model from PyTorch or run additional quantization steps before using it in this Learning Path.
 {{% /notice %}}
 
 ## Check your setup
diff --git a/content/learning-paths/cross-platform/multimodel_mnn_v9/3_mnn_text_baseline.md b/content/learning-paths/cross-platform/multimodel_mnn_v9/3_mnn_text_baseline.md
@@ -97,7 +97,9 @@ Prepare for tuning opt Begin
 Prepare for tuning opt End
 main, 282, cost time: 784.388000 ms
 prompt file is /home/radxa/mnn/text_baseline_prompt.txt
- The benefits of multimodal on-device inference are that it can save on cloud usage and also it can be more private and secure. It can save on cloud usage and also it can be more private and secure. So, what do you think about it? Do you have any other ideas on how it could be improved? Well, it could be improved by making it even faster, you know, like reducing the processing time even further. And also, maybe adding more features to it to make it more versatile. What do you think? Let's keep the conversation going. What do you think about the potential for multimodal on-device inference to be used in healthcare applications? It could be really useful, like for things like analyzing medical images or monitoring patients remotely. It would be great if it could be used in more areas like that. What do you think? Let's talk more about it. What do you think about the potential for multimodal on-device inference to be used in healthcare applications? It could be really useful, like for things like analyzing medical images or monitoring patients remotely. It would be great if it could be used in more areas like that. What do you think? Let's talk more about it. What do you think about the potential for multimodal on-device inference to be used in healthcare applications? What do you think about the potential for Arm is a processor architecture. It's used in mobile phones, embedded systems, etc. It has a RISC design, efficient performance, and low power consumption. Arm is popular in mobile devices, especially for its efficiency. It's used in many mobile phones, like iPhones, Androids, and embedded systems, etc. It has a RISC design, efficient performance, and low power consumption.
+ The benefits of multimodal on-device inference are that it can save on cloud usage and also it can be more private and secure.
+[...]
+Arm is a processor architecture. It's used in mobile phones, embedded systems, etc. It has a RISC design, efficient performance, and low power consumption.
 ```
 
 {{% notice Note %}}
@@ -111,7 +113,7 @@ cd ~/mnn/MNN/build
 ./llm_demo ~/mnn/Qwen2.5-Omni-7B-MNN/config.json
 ```
 
-Enter a short prompt and confirm that the model returns a reply. Type `exit` or press Ctrl+C to quit.
+Enter a short prompt and confirm that the model returns a reply. Press Ctrl+C to quit.
 
 ## Check your results
 
diff --git a/content/learning-paths/cross-platform/multimodel_mnn_v9/4_mnn_vision.md b/content/learning-paths/cross-platform/multimodel_mnn_v9/4_mnn_vision.md
@@ -40,7 +40,7 @@ You should see output similar to:
 JPEG image data, JFIF standard 1.02, resolution (DPI), density 72x72, segment length 16, Exif Standard: [TIFF image data, little-endian, direntries=12, description=                               , manufacturer=SONY, model=DSC-W50, orientation=upper-left, xresolution=203, yresolution=211, resolutionunit=2, software=Adobe Photoshop CS Macintosh, datetime=2007:04:10 17:45:47], progressive, precision 8, 2816x2112, components 3
 ```
 
-This confirms that the image asset is ready for the vision prompt.
+The key detail is that the output starts with `JPEG image data`, confirming the file is a valid JPEG. The remaining EXIF metadata can be ignored.
 
 ![Pet food aisle shelf with multiple levels stocked with pet food products, used as the vision prompt input for the shelf audit#center](pet_food_aisle.jpg "Pet food aisle shelf used for the vision audit prompt")
 
@@ -67,6 +67,10 @@ EOF
 
 Run `llm_demo` with the model `config.json` file and the vision prompt:
 
+{{% notice Note %}}
+Vision inference takes longer than text-only inference because the model encodes the image into tokens before generation begins. On an Armv9 CPU, expect the initial load and image encoding step to take several minutes. The terminal will appear idle during this time — this is expected.
+{{% /notice %}}
+
 ```bash
 cd ~/mnn/MNN/build
 ./llm_demo ~/mnn/Qwen2.5-Omni-7B-MNN/config.json ~/mnn/prompt_picture_coverage.txt
diff --git a/content/learning-paths/cross-platform/multimodel_mnn_v9/5_mnn_vision_audio.md b/content/learning-paths/cross-platform/multimodel_mnn_v9/5_mnn_vision_audio.md
@@ -6,37 +6,28 @@ layout: learningpathall
 
 ## Introduction
 
-In this module, you use an audio prompt and an MNN Omni model to convert a spoken restock note into a structured one-line ticket on an Armv9 system.
+In this section, you use an audio prompt and an MNN Omni model to convert a spoken restock note into a structured one-line ticket on an Armv9 system.
 
 This mirrors a simple retail workflow. A store associate records a short voice note during a shelf walk, and the system converts it into a task that can be reviewed by staff or passed to another tool.
 
-To keep the output predictable, this module uses a single-line format with semicolon-separated fields. This also helps when terminal output does not preserve line breaks consistently.
+To keep the output predictable, this section uses a single-line format with semicolon-separated fields. This also helps when terminal output does not preserve line breaks consistently.
 
 ## Prepare the audio asset
 
 To simplify the demonstration and improve clarity for this learning path, directly download the prepared audio file:
 
 ```bash
-mkdir ~/mnn/assets
-wget wget https://github.com/odincodeshen/multimodel_mnn_armv9/raw/main/assets/restock_note.wav -P ~/mnn/assets
+mkdir -p ~/mnn/assets
+wget https://github.com/odincodeshen/multimodel_mnn_armv9/raw/main/assets/restock_note.wav -P ~/mnn/assets
 ```
 
-This audio `restock_note.wav` contains brief spoken instructions describing restocking tasks, such as:
+The audio file contains a spoken restocking instruction. The content is similar to:
 
-- please restock the bottom-left large pet food bags, add ten bags
-- also restock the middle-left canned pet food, add twenty-four cans
-- finish before 3 PM today
-- if something is out of stock, use a similar substitute
-
-The model’s goal is to extract structured operational information from this audio, specifically:
-
-- items
-- quantities
-- shelf zones
-- deadline
-- substitution policy
+```text
+This is the pet food aisle, left shelf. Please restock the bottom left large pet food bags, add 10 bags. Also restock the middle left canned food, add 24 cans. Finish before 3 PM today. If something is out of stock, use a similar substitute.
+```
 
-The objective isn’t comprehensive speech analytics but efficiently converting spoken instructions into a compact operational ticket.
+From this spoken note, the model extracts item names, quantities, shelf zones, a deadline, and a substitution policy. The goal isn't comprehensive speech analytics — it's converting a short spoken instruction into a compact operational ticket.
 
 ## Verify the audio format
 
@@ -48,23 +39,17 @@ file ~/mnn/assets/restock_note.wav
 
 You should see output indicating that the file is a RIFF/WAV audio file.
 
-If you'd like to record the audio by yourself as MP3 format, please convert it to a speech-friendly WAV format:
+If you have your own MP3 recording, convert it to a speech-friendly WAV format:
 
 ```bash
 ffmpeg -y -i input.mp3 -ac 1 -ar 16000 -c:a pcm_s16le ~/mnn/assets/restock_note.wav
 ```
 
 This creates a mono, 16 kHz, 16-bit PCM WAV file, which is a practical format for speech input.
 
-The spoken content can be similar to:
-
-```text
-This is the pet food aisle, left shelf. Please restock the bottom left large pet food bags, add 10 bags. Also restock the middle left canned food, add 24 cans. Finish before 3 PM today. If something is out of stock, use a similar substitute.
-```
-
 In store inspection scenarios, unreliable connectivity, or environments where teams prefer not to upload audio in real time, local speech understanding on Armv9 has clear deployment value.
 
-In this module, the audio input is not just an audio prompt. It acts as on-device spoken task capture for store operations, turning a short spoken restocking note into structured operational input.
+In this section, the audio input is not just an audio prompt. It acts as on-device spoken task capture for store operations, turning a short spoken restocking note into structured operational input.
 
 ## Create the audio prompt
 
@@ -77,13 +62,11 @@ Create a prompt file that attaches the WAV file and asks the model to return a s
 Create the prompt file:
 
 ```bash
-cat > ~/mnn/prompt_audio_ticket.txt <<'EOF'
-<audio>/home/radxa/mnn/assets/restock_note.wav</audio> You are a retail store replenishment assistant. Convert the spoken note into a restocking ticket. Output EXACTLY ONE line using bullet-style segments separated by semicolons: Restock ticket; - Location: pet food aisle left shelf; - Task 1: <generic item> | qty <number or NOT_SURE> | zone <top|middle|bottom>-<left|center|right or NOT_SURE>; - Task 2: <generic item> | qty <number or NOT_SURE> | zone <top|middle|bottom>-<left|center|right or NOT_SURE>; - Deadline: <time or NOT_SURE>; - Substitution: <use similar substitute|no substitute|NOT_SURE>; - Notes: <short note>; - Confidence: <high|medium|low>. Rules: do not invent quantities if not in audio.
+cat > ~/mnn/prompt_audio_ticket.txt <<EOF
+<audio>$HOME/mnn/assets/restock_note.wav</audio> You are a retail store replenishment assistant. Convert the spoken note into a restocking ticket. Output EXACTLY ONE line using bullet-style segments separated by semicolons: Restock ticket; - Location: pet food aisle left shelf; - Task 1: <generic item> | qty <number or NOT_SURE> | zone <top|middle|bottom>-<left|center|right or NOT_SURE>; - Task 2: <generic item> | qty <number or NOT_SURE> | zone <top|middle|bottom>-<left|center|right or NOT_SURE>; - Deadline: <time or NOT_SURE>; - Substitution: <use similar substitute|no substitute|NOT_SURE>; - Notes: <short note>; - Confidence: <high|medium|low>. Rules: do not invent quantities if not in audio.
 EOF
 ```
 
-If your username or home directory is different, replace `/home/radxa` with the correct local path.
-
 This prompt asks the model to:
 
 - use generic item names instead of brands
@@ -125,7 +108,7 @@ Restock ticket;
 
 ```
 The exact wording can vary. What matters is that the response stays close to the requested structure and reflects the spoken content rather than invented details.
-![image2 Prompt Audio Ticket](prompt_audio_ticket.gif)
+![Animated terminal output showing llm_demo processing an audio prompt and returning a structured restock ticket on Armv9#center](prompt_audio_ticket.gif "llm_demo audio prompt output on Armv9")
 
 ## Verify the result
 
diff --git a/content/learning-paths/cross-platform/multimodel_mnn_v9/6_mnn_restock_ticket.md b/content/learning-paths/cross-platform/multimodel_mnn_v9/6_mnn_restock_ticket.md
