Skip to content

Commit 9a8b021

Browse files
authored
Whisper app updates (#192)
1 parent a99905e commit 9a8b021

11 files changed

Lines changed: 1137 additions & 145 deletions

File tree

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,26 @@
11
# Whisper Demo App
22

3-
This app runs the Whisper model with the Qualcomm backend in ExecuTorch. The model itself runs on the Qualcomm HTP (Hexagon Tensor Processor), an AI accelerator in Qualcomm SoCs. You need an Android smartphone with a [supported Qualcomm chipset](https://github.com/pytorch/executorch/tree/main/backends/qualcomm).
3+
This app runs the Whisper model in ExecuTorch.
44

5-
### Build the ExecuTorch Android library
5+
## Build the ExecuTorch Android library
66

77
Build the [ExecuTorch Android library with QNN backend](https://github.com/pytorch/executorch/blob/main/examples/demo-apps/android/LlamaDemo/docs/delegates/qualcomm_README.md).
88

9-
### Export the audio processing and model .pte files
9+
## Export the audio processing and model .pte files
1010

1111
There are two steps, audio processing and the Whisper model (encoder+decoder), which are both done via ExecuTorch.
1212

13-
1) Run the script `extension/audio/mel_spectrogram.py` to export `whisper_preprocess.pte`
14-
2) Run the script `examples/qualcomm/oss_scripts/whisper/whisper.py` to export `whisper_qnn_16a8w.pte`
13+
1. Run `extension/audio/mel_spectrogram.py` to export `whisper_preprocess.pte`
14+
2. Run `examples/qualcomm/oss_scripts/whisper/whisper.py` to export `whisper_qnn_16a8w.pte`
1515

16-
Move these two .pte files along with `tokenizer.json` to `/data/local/tmp/whisper` on device.
16+
Move these two `.pte` files along with `tokenizer.json` to `/data/local/tmp/whisper` on device.
1717

18-
### Run app
19-
20-
Open WhisperApp in Android studio. Copy the Android library `executorch.aar` which should have audio JNI bindings and the Qualcomm HTP libraries, into `app/libs`.
18+
## Run the app
2119

20+
1. Open WhisperApp in Android Studio
21+
2. Copy the Android library `executorch.aar` (with audio JNI bindings) into `app/libs`
22+
3. Build and run on device
2223

24+
## Demo
2325

2426
https://github.com/user-attachments/assets/ff8c71c5-b734-4ed4-8382-70a429830665

0 commit comments

Comments
 (0)