File tree Expand file tree Collapse file tree
content/learning-paths/cross-platform/multimodel_mnn_v9 Expand file tree Collapse file tree Original file line number Diff line number Diff line change 11---
2- title : Understand MNN and Omni for Armv9 on-device inference
2+ title : Multimodal On-Device Inference on Arm v9 with MNN for Audio and Vision
33weight : 2
44
55layout : learningpathall
Original file line number Diff line number Diff line change @@ -14,7 +14,7 @@ mkdir -p ~/mnn_lp
1414cd ~ /mnn_lp
1515```
1616
17- Clone MNN and build:
17+ Clone and build MNN :
1818
1919``` bash
2020git clone https://github.com/alibaba/MNN.git
Original file line number Diff line number Diff line change @@ -4,7 +4,7 @@ weight: 4
44layout : learningpathall
55---
66
7- ## Steps
7+ ## Run the text baseline
88
99Create a simple text prompt file:
1010
Original file line number Diff line number Diff line change @@ -4,7 +4,7 @@ weight: 5
44layout : learningpathall
55---
66
7- ## Scenario
7+ ## Prepare a local image asset
88
99In this module you will implement a ** smart retail shelf audit** using a single local image.
1010
@@ -54,7 +54,7 @@ cat > ~/mnn_lp/prompt_picture_coverage.txt <<'EOF'
5454EOF
5555```
5656
57- Run the vision demo:
57+ ## Run the vision demo:
5858
5959``` bash
6060cd ~ /mnn_lp/MNN/build
Original file line number Diff line number Diff line change @@ -4,7 +4,7 @@ weight: 7
44layout : learningpathall
55---
66
7- ## Goal
7+ ## Create the single-shot multimodal prompt (image + audio)
88
99In this final project you will combine:
1010
@@ -44,7 +44,7 @@ cat > ~/mnn_lp/prompt_final_multimodal.txt <<'EOF'
4444EOF
4545```
4646
47- Run the final multimodal demo:
47+ ## Run the final multimodal demo:
4848
4949``` bash
5050cd ~ /mnn_lp/MNN/build
@@ -54,14 +54,10 @@ cd ~/mnn_lp/MNN/build
5454./llm_demo ~ /mnn_lp/Qwen2.5-Omni-7B-MNN/config.json ~ /mnn_lp/prompt_final_multimodal.txt
5555```
5656
57- This sequence lets you compare:
57+ ## Verify image and audio are both used (optional)
5858
59- 1 . Vision-only output
60- 2 . Audio-only output
61- 3 . Combined multimodal output
62-
63-
64- ## Verification
59+ - Swap the ** audio** file: quantities/deadline/substitution should change.
60+ - Swap the ** image** file: coverage/priority zone should change.
6561
6662To verify that the model is truly multimodal, perform a simple A/B test:
6763
You can’t perform that action at this time.
0 commit comments