rewrite module 3~5 and upload images

Author: odinshen-git

content/learning-paths/cross-platform/multimodel_mnn_v9/3_mnn_text_baseline.md

@@ -61,6 +61,13 @@ Multimodal on-device inference has several benefits, including reducing the need
 ```
 
 
+{{% notice Note %}}
+When you run `llm_demo`, you may see a line like:
+`The device supports: i8sdot:1, fp16:1, i8mm: 1, sve2: 1, sme2: 0`
+This line reports hardware feature support on your CPU: **i8sdot**, **fp16**, **i8mm**, **sve2**, and **sme2**.  
+A value of `1` means the feature is supported in hardware, and `0` means it is not.
+{{% /notice %}}
+
 
 
 ## Step 3 - Add a second prompt to validate multi-line behavior

content/learning-paths/cross-platform/multimodel_mnn_v9/4_mnn_vision.md

@@ -6,37 +6,41 @@ layout: learningpathall
 
 ## Prepare a local image asset
 
-In this module you will implement a **smart retail shelf audit** using a single local image.
+In this module you will run a **retail shelf walk audit** using a single image and an Omni multimodal model on MNN.
 
-The model will:
+Instead of trying to do SKU-accurate counting (which is difficult and unreliable on real-world shelf photos), you will generate an **actionable store-operations signal**:
 
-- Inspect a **pet food aisle** image
-- Estimate **facing coverage** for three shelf levels: **top / middle / bottom**
-- Identify a **priority zone** that appears most sparse
-- Use `NOT_SURE` where the image is ambiguous
+- A coarse **facing coverage** estimate for **top / middle / bottom** shelf levels
+- A single **priority zone** that appears most sparse
+- A short **reason** grounded in what is visible
+- `NOT_SURE` when the image is ambiguous
 
+This is a common retail workflow: the goal is to decide **where to restock first**, not to perfectly count every unit.
 
-## Assets
 
-Create an `assets` directory:
+## Step 1 - Prepare the image asset
 
 ```bash
-mkdir -p ~/mnn_lp/assets
+mkdir -p ~/mnn/assets
 ```
 
-Download the tutorial image:
+Download the tutorial image and validate it:
 
 ```bash
-curl -L -o ~/mnn_lp/assets/pet_food_aisle.jpg \
-  "https://upload.wikimedia.org/wikipedia/commons/5/57/Pet_Food_Aisle.jpg"
-
-file ~/mnn_lp/assets/pet_food_aisle.jpg
+wget -P ~/mnn/assets https://upload.wikimedia.org/wikipedia/commons/e/e6/Pet_Food_Aisle.jpg
+file ~/mnn/assets/Pet_Food_Aisle.jpg
 ```
 
 The `file` command confirms that this is a valid JPEG image.
 
+```
+JPEG image data, JFIF standard 1.02, resolution (DPI), density 72x72, segment length 16, Exif Standard: [TIFF image data, little-endian, direntries=12, description=                               , manufacturer=SONY, model=DSC-W50, orientation=upper-left, xresolution=203, yresolution=211, resolutionunit=2, software=Adobe Photoshop CS Macintosh, datetime=2007:04:10 17:45:47], progressive, precision 8, 2816x2112, components 3
+```
+
+![image1 Pet Food Aisle](pet_food_aisle.jpg)
+
 
-## Prompt design
+## Step 2 - Create the vision prompt (single line)
 
 You will design a prompt that:
 
@@ -49,24 +53,43 @@ You will design a prompt that:
 Create the prompt file (adjust `/home/radxa` to your actual user home if needed):
 
 ```bash
-cat > ~/mnn_lp/prompt_picture_coverage.txt <<'EOF'
-<img>/home/radxa/mnn_lp/assets/pet_food_aisle.jpg</img> You are an on-device retail shelf auditing assistant. Audit ONLY the main left shelf (ignore the aisle on the right, hanging toys, and floor items). Do NOT count every item. Estimate facing coverage for top/middle/bottom as high|medium|low and identify the sparsest zone. Output ONE line only using bullet-style segments separated by semicolons: Shelf audit; - Coverage: top=<high|medium|low>, middle=<high|medium|low>, bottom=<high|medium|low>; - Priority zone: <top|middle|bottom>-<left|center|right>; - Reason: <one short sentence>; - Notes: <NOT_SURE if unclear>.
+cat > ~/mnn/prompt_picture_coverage.txt <<'EOF'
+<img>/home/radxa/mnn/assets/Pet_Food_Aisle.jpg</img> You are an on-device retail shelf auditing assistant. Audit ONLY the main left shelf (ignore the aisle on the right, hanging toys, and floor items). Do NOT count every item. Estimate facing coverage for top/middle/bottom as high|medium|low and identify the sparsest zone. Output ONE line only using bullet-style segments separated by semicolons: Shelf audit; - Coverage: top=<high|medium|low>, middle=<high|medium|low>, bottom=<high|medium|low>; - Priority zone: <top|middle|bottom>-<left|center|right>; - Reason: <one short sentence>; - Notes: <NOT_SURE if unclear>.
 EOF
 ```
 
-## Run the vision demo:
 
-```bash
-cd ~/mnn_lp/MNN/build
-./llm_demo ~/mnn_lp/Qwen2.5-Omni-7B-MNN/config.json ~/mnn_lp/prompt_picture_coverage.txt
-```
+## Step 3 - Run the vision demo
 
+Run llm_demo with your model config.json and the vision prompt file:
 
-## Checkpoint
+```bash
+cd ~/mnn/MNN/build
+./llm_demo ~/mnn/Qwen2.5-Omni-7B-MNN/config.json ~/mnn/prompt_picture_coverage.txt
+```
 
-You have completed this module when the output:
+The expected result will be: 
+```
+config path is /home/radxa/mnn/Qwen2.5-Omni-7B-MNN/config.json
+CPU Group: [ 1  2  3  4 ], 799999 - 1800968
+CPU Group: [ 7  8 ], 799897 - 2199795
+CPU Group: [ 5  6 ], 799897 - 2299896
+CPU Group: [ 9  10 ], 799897 - 2399998
+CPU Group: [ 0  11 ], 799897 - 2500100
+The device supports: i8sdot:1, fp16:1, i8mm: 1, sve2: 1, sme2: 0
+main, 274, cost time: 6046.771973 ms
+Prepare for tuning opt Begin
+Prepare for tuning opt End
+main, 282, cost time: 766.086975 ms
+prompt file is /home/radxa/mnn/prompt_picture_coverage.txt
+Shelf audit; - Coverage: top=high, middle=high, bottom=medium; - Priority zone: top-left; - Reason: top and middle have more variety than bottom, and left is where most items are; - Notes: NOT_SURE if unclear. How does this sound? Let me know if you need further clarification or have other questions! I'm here to help with more details if you want! 😊
+If you have any other questions about this shelf auditing, feel free to let me know! 😊
+```
 
+The exact words may differ, but the output should contain:
 - Includes coverage estimates for **top / middle / bottom**
 - Identifies a **priority zone** such as `middle-center`
 - Provides a **short reason** that clearly references visible sparsity on the shelf
-- Uses `NOT_SURE` only where the image is genuinely unclear
+- Uses `NOT_SURE` only where the image is genuinely unclear
+
+In the next module you will run an audio restock instruction demo using an <audio>...</audio> prompt.

content/learning-paths/cross-platform/multimodel_mnn_v9/5_mnn_vision_audio.md

@@ -4,25 +4,41 @@ weight: 6
 layout: learningpathall
 ---
 
-## Scenario
+In this module you will convert a **spoken restock note** into a **readable, single-line restock ticket** using MNN Omni.
 
-In this module you will convert a **spoken restock note** into a structured **restock ticket**.
+This mirrors a common retail workflow: an associate records a short voice note during a shelf walk, and the system converts it into an operational ticket that can be handed to staff or logged in a task system.
 
-For example, a store associate might say:
+Because `llm_demo` output may normalize newlines on some terminals, this module uses a **single-line** format with `;`-separated fields.
 
-> We need to restock large dog food bags on the bottom-left, four units, before tomorrow afternoon. If large bags are out of stock, use medium bags as a substitute.
+## Step 1 - Prepare the audio asset
 
-Your goal is to:
+Before you run the demo, make sure you know what your `restock_note.wav` contains. This learning path assumes the WAV is a short “store associate voice note” describing what to restock.
 
-- Extract **items, quantities, zones, deadlines, and substitution policy**
-- Encode them as a **single-line ticket** with clearly labeled fields
+### Suggested Audio Content Scenario
 
-## Assets
+Record (or provide) a short voice note similar to the following:
 
-Prepare a WAV file under `assets`:
+* Please restock the bottom-left large pet food bags, add ten bags.  
+* Also restock the middle-left canned pet food, add twenty-four cans.  
+* Finish before 3 PM today. If something is out of stock, use a similar substitute.
+
+Your goal in this module is to extract these fields from the audio:
+
+- **Items**
+- **Quantities**
+- **Zones**
+- **Deadline**
+- **Substitution policy**
+
+…and encode them as a **single-line ticket** with labeled fields.
+
+
+### Verify the WAV file format
+
+This demo expects a WAV (RIFF) file: For example `restock_note.wav` 
 
 ```bash
-file ~/mnn_lp/assets/restock_note.wav
+file ~/mnn/assets/restock_note.wav
 ```
 
 If you only have an MP3 file, convert it to a suitable WAV format:
@@ -33,33 +49,61 @@ ffmpeg -y -i input.mp3 -ac 1 -ar 16000 -c:a pcm_s16le ~/mnn_lp/assets/restock_no
 
 This produces a **mono, 16 kHz, 16-bit PCM** WAV file, which is commonly used for speech models.
 
+```
+This is the pet food aisle, left shelf. Please restock the bottom left large pet food bags, add 10 bags. Also restock the middle left canned food, add 24 cans. Finish before 3 PM today. If something is out of stock, use a similar substitute.
+```
 
-## Prompt design
+## Step 2 - Create the audio prompt
 
-You will ask the model to produce a **readable, single-line ticket** with:
+llm_demo commonly treats ***each line*** as a separate prompt. Keep this prompt file as ***one line***.
 
-- Item
-- Quantity (or `NOT_SURE` if not spoken)
-- Shelf zone
-- Deadline
-- Substitution policy
-- Notes and confidence
+This prompt instructs the model to:
+- Use ***generic*** item labels (avoid brands unless clearly spoken)
+- Output ***NOT_SURE*** for fields not stated in the audio
+- Produce a single-line ticket that is easy to read and parse
 
-Create the prompt file:
 
 ```bash
-cat > ~/mnn_lp/prompt_audio_ticket.txt <<'EOF'
-<audio>/home/radxa/mnn_lp/assets/restock_note.wav</audio> You are a retail store replenishment assistant. Convert the spoken note into a restocking ticket. Output EXACTLY ONE line using bullet-style segments separated by semicolons: Restock ticket; - Location: pet food aisle left shelf; - Task 1: <generic item> | qty <number or NOT_SURE> | zone <top|middle|bottom>-<left|center|right or NOT_SURE>; - Task 2: <generic item> | qty <number or NOT_SURE> | zone <top|middle|bottom>-<left|center|right or NOT_SURE>; - Deadline: <time or NOT_SURE>; - Substitution: <use similar substitute|no substitute|NOT_SURE>; - Notes: <short note>; - Confidence: <high|medium|low>. Rules: do not invent quantities if not in audio.
+cat > ~/mnn/prompt_audio_ticket.txt <<'EOF'
+<audio>/home/radxa/mnn/assets/restock_note.wav</audio> You are a retail store replenishment assistant. Convert the spoken note into a restocking ticket. Output EXACTLY ONE line using bullet-style segments separated by semicolons: Restock ticket; - Location: pet food aisle left shelf; - Task 1: <generic item> | qty <number or NOT_SURE> | zone <top|middle|bottom>-<left|center|right or NOT_SURE>; - Task 2: <generic item> | qty <number or NOT_SURE> | zone <top|middle|bottom>-<left|center|right or NOT_SURE>; - Deadline: <time or NOT_SURE>; - Substitution: <use similar substitute|no substitute|NOT_SURE>; - Notes: <short note>; - Confidence: <high|medium|low>. Rules: do not invent quantities if not in audio.
 EOF
 ```
 
+## Step 3 - Run the audio demo
+
 Run the audio demo:
 
 ```bash
-cd ~/mnn_lp/MNN/build
-./llm_demo ~/mnn_lp/Qwen2.5-Omni-7B-MNN/config.json ~/mnn_lp/prompt_audio_ticket.txt
+cd ~/mnn/MNN/build
+./llm_demo ~/mnn/Qwen2.5-Omni-7B-MNN/config.json ~/mnn/prompt_audio_ticket.txt
+```
+
+The expected result will be: 
+```
+config path is /home/radxa/mnn/Qwen2.5-Omni-7B-MNN/config.json
+CPU Group: [ 1  2  3  4 ], 799999 - 1800968
+CPU Group: [ 7  8 ], 799897 - 2199795
+CPU Group: [ 5  6 ], 799897 - 2299896
+CPU Group: [ 9  10 ], 799897 - 2399998
+CPU Group: [ 0  11 ], 799897 - 2500100
+The device supports: i8sdot:1, fp16:1, i8mm: 1, sve2: 1, sme2: 0
+main, 274, cost time: 5745.955078 ms
+Prepare for tuning opt Begin
+Prepare for tuning opt End
+main, 282, cost time: 767.578979 ms
+prompt file is /home/radxa/mnn/prompt_audio_ticket.txt
+Restock ticket;
+- Location: pet food aisle left shelf;
+- Task 1: large pet food bag | qty NOT_SURE | zone bottom-left;
+- Task 2: canned food | qty 24 | zone middle-left;
+- Deadline: before 3 p.m. today;
+- Substitution: use similar substitute;
+- Notes: restock before deadline;
+- Confidence: medium. Restock the large pet food bag and canned food as per the given quantities. If any item is out of stock, replace it with a similar substitute. Make sure it's done by 3 p.m. today. Let me know if you need further clarification or if there's anything else on your mind.
+
 ```
 
+![image2 Prompt Audio Ticket](prompt_audio_ticket.gif)
 
 ## Checkpoint
 

content/learning-paths/cross-platform/multimodel_mnn_v9/6_mnn_restock_ticket.md

@@ -6,25 +6,34 @@ layout: learningpathall
 
 ## Create the single-shot multimodal prompt (image + audio)
 
-In this final project you will combine:
+In this final project you will generate an **operational restock ticket** from **one single-shot multimodal prompt**.
 
-- A **shelf image** for coverage and priority zone
-- A **voice note** for tasks, quantities, deadlines, and substitution policy
+You will combine:
+- A **shelf image** to estimate shelf coverage and select a priority zone
+- A **voice note** to extract tasks, quantities, zones, deadlines, and substitution policy
 
-into **one multimodal prompt** that produces an **operational restock ticket** in a single inference.
+into **one multimodal inference** using MNN Omni.
+
+This mirrors a real retail workflow: a staff member captures a shelf photo and records a quick voice note, and the device produces a ticket that can be executed immediately.
 
 
 ## Inputs
 
-You will reuse the same assets:
+You will reuse the same local assets from previous modules:
+
+- Shelf image: `~/mnn/assets/Pet_Food_Aisle.jpg.jpg`
+- Voice note: `~/mnn/assets/restock_note.wav`
 
-- Shelf image: `~/mnn_lp/assets/pet_food_aisle.jpg`
-- Voice note: `~/mnn_lp/assets/restock_note.wav`
+{{% notice Note %}}
+Using local files avoids redirect and downloader issues.
+{{% /notice %}}
 
 
 ## Output format
 
-The output is a **single-line, bullet-style ticket** that must include:
+The output is a **single-line, bullet-style ticket** (segments separated by `;`), designed to remain readable even when terminals normalize newlines.
+
+The ticket must include:
 
 - **Coverage (image-derived):** `top=<high|medium|low>, middle=..., bottom=...`
 - **Priority zone (image-derived):** `<top|middle|bottom>-<left|center|right>`
@@ -34,27 +43,48 @@ The output is a **single-line, bullet-style ticket** that must include:
 - **Notes and confidence**
 
 
-## Prompt and run
+## Step 1 Create the multimodal prompt (single line)
 
-Create the multimodal prompt file:
+Create the multimodal prompt file. Keep the file as **one line**.
 
 ```bash
-cat > ~/mnn_lp/prompt_final_multimodal.txt <<'EOF'
-<img>/home/radxa/mnn_lp/assets/pet_food_aisle.jpg</img> <audio>/home/radxa/mnn_lp/assets/restock_note.wav</audio> You are an on-device retail replenishment assistant. Use the IMAGE to estimate facing coverage for the main LEFT shelf by level as high|medium|low and pick a priority zone. Use the AUDIO to extract item requests including quantities, zones, deadline, and substitution policy. Output EXACTLY ONE line using bullet-style segments separated by semicolons: Restock ticket; - Location: pet food aisle left shelf; - Coverage: top=<high|medium|low>, middle=<high|medium|low>, bottom=<high|medium|low>; - Priority zone: <top|middle|bottom>-<left|center|right>; - Task 1: <generic item> | qty <number or NOT_SURE> | zone <top|middle|bottom>-<left|center|right or NOT_SURE>; - Task 2: <generic item> | qty <number or NOT_SURE> | zone <top|middle|bottom>-<left|center|right or NOT_SURE>; - Deadline: <time or NOT_SURE>; - Substitution: <use similar substitute|no substitute|NOT_SURE>; - Notes: <short operational note>; - Confidence: <high|medium|low>. Rules: Qty/Deadline/Substitution must come from audio; if audio contains explicit numbers you must output them.
+cat > ~/mnn/prompt_final_multimodal.txt <<'EOF'
+<img>/home/radxa/mnn/assets/Pet_Food_Aisle.jpg.jpg</img> <audio>/home/radxa/mnn/assets/restock_note.wav</audio> You are an on-device retail replenishment assistant. Use the IMAGE to estimate facing coverage for the main LEFT shelf by level as high|medium|low and pick a priority zone. Use the AUDIO to extract item requests including quantities, zones, deadline, and substitution policy. Output EXACTLY ONE line using bullet-style segments separated by semicolons: Restock ticket; - Location: pet food aisle left shelf; - Coverage: top=<high|medium|low>, middle=<high|medium|low>, bottom=<high|medium|low>; - Priority zone: <top|middle|bottom>-<left|center|right>; - Task 1: <generic item> | qty <number or NOT_SURE> | zone <top|middle|bottom>-<left|center|right or NOT_SURE>; - Task 2: <generic item> | qty <number or NOT_SURE> | zone <top|middle|bottom>-<left|center|right or NOT_SURE>; - Deadline: <time or NOT_SURE>; - Substitution: <use similar substitute|no substitute|NOT_SURE>; - Notes: <short operational note>; - Confidence: <high|medium|low>. Rules: Qty/Deadline/Substitution must come from audio; if audio contains explicit numbers you must output them.
 EOF
 ```
 
-## Run the final multimodal demo:
+## Step 2 - Run the single-shot multimodal demo
+
+Run `llm_demo` with the multimodal prompt:
 
 ```bash
-cd ~/mnn_lp/MNN/build
+cd ~/mnn/MNN/build
+./llm_demo ~/mnn/Qwen2.5-Omni-7B-MNN/config.json ~/mnn/prompt_final_multimodal.txt
+```
 
-./llm_demo ~/mnn_lp/Qwen2.5-Omni-7B-MNN/config.json ~/mnn_lp/prompt_picture_coverage.txt
-./llm_demo ~/mnn_lp/Qwen2.5-Omni-7B-MNN/config.json ~/mnn_lp/prompt_audio_ticket.txt
-./llm_demo ~/mnn_lp/Qwen2.5-Omni-7B-MNN/config.json ~/mnn_lp/prompt_final_multimodal.txt
+Your output should be a single line similar to:
 ```
+CPU Group: [ 1  2  3  4 ], 799999 - 1800968
+CPU Group: [ 7  8 ], 799897 - 2199795
+CPU Group: [ 5  6 ], 799897 - 2299896
+CPU Group: [ 9  10 ], 799897 - 2399998
+CPU Group: [ 0  11 ], 799897 - 2500100
+The device supports: i8sdot:1, fp16:1, i8mm: 1, sve2: 1, sme2: 0
+main, 274, cost time: 5781.158203 ms
+Prepare for tuning opt Begin
+Prepare for tuning opt End
+main, 282, cost time: 787.130981 ms
+prompt file is /home/radxa/mnn/prompt_final_multimodal.txt
+Restock the bottom left large pet food bag, add 10 bags; also restock the middle left canned food, add 24 cans. Finish before 3 PM today. If something is out of stock, use a similar substitute.- Restock ticket: Location: pet food aisle left shelf, Coverage: top=low, middle=low, bottom=high, Priority zone: bottom-left, Task 1: pet food | qty 10 | zone bottom-left, Task 2: canned food | qty 24 | zone middle-left, Deadline: 3 PM today, Substitution: use similar substitute, Notes: Ensure proper storage conditions, Confidence: high.
+
+```
+
+Exact wording may differ, but the coverage/priority should be image-driven and the quantities/deadline/substitution should be audio-driven.
+
+
+## Step 3 - Verify image and audio are both used (optional)
 
-## Verify image and audio are both used (optional)
+To verify true multimodal behavior, perform a simple A/B test.
 
 - Swap the **audio** file: quantities/deadline/substitution should change.
 - Swap the **image** file: coverage/priority zone should change.