Skip to content

Commit 2681a80

Browse files
kirklandsignkirklandsign
authored
DL3 demo improvement (#157)
1 parent d0d845a commit 2681a80

10 files changed

Lines changed: 457 additions & 350 deletions

File tree

‎dl3/android/DeepLabV3Demo/README.md‎

Lines changed: 70 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -2,15 +2,26 @@
22

33
This guide explains how to setup ExecuTorch for Android using a demo app. The app employs a [DeepLab v3](https://pytorch.org/hub/pytorch_vision_deeplabv3_resnet101/) model for image segmentation tasks. Models are exported to ExecuTorch using [XNNPACK FP32 backend](https://pytorch.org/executorch/main/backends-xnnpack.html#xnnpack-backend).
44

5+
## Features
6+
- **Image Segmentation**: Detects and highlights all 21 PASCAL VOC classes (Person, Dog, Cat, Car, etc.)
7+
- **Overlay Visualization**: Segmentation mask blends with the original image at 50% opacity
8+
- **Inference Time Display**: Shows model inference latency in milliseconds
9+
- **In-App Model Download**: Download the model directly from the app
10+
- **Image Picker**: Select any image from your device's gallery
11+
- **Sample Images**: 3 built-in sample images for quick testing
12+
513
## Prerequisites
614
* Download and install [Android Studio and SDK 34](https://developer.android.com/studio).
715
* (For exporting the DL3 model) Python 3.10+ with `executorch` package installed.
816

9-
## Step 1: Export the model
10-
Run the script in `dl3/python/export.py` to export the model.
17+
## Step 1: Export the Model (Optional)
18+
The app can download the model automatically. If you want to export it yourself:
19+
```bash
20+
cd dl3/python
21+
python export.py
22+
```
1123

12-
## Step 2: Set up your device or emulator
13-
You can run the app on either a physical device or an emulator. To set up your device or emulator, follow these steps:
24+
## Step 2: Set Up Your Device or Emulator
1425

1526
### Using a Physical Device
1627
* Connect your device to your computer via USB.
@@ -20,52 +31,73 @@ You can run the app on either a physical device or an emulator. To set up your d
2031
* Open Android Studio and create a new virtual device.
2132
* Start the emulator by clicking the "Play" button next to the device name.
2233

23-
## Step 3: Build, install, and run the app on your phone
24-
### On your terminal
25-
(`cd dl3/android/DeepLabV3Demo` first)
26-
```
34+
## Step 3: Build and Run the App
35+
36+
### Using Terminal
37+
```bash
38+
cd dl3/android/DeepLabV3Demo
2739
./gradlew installDebug
28-
adb shell am start -W -S -n org.pytorch.executorchexamples.dl3/.MainActivity
40+
adb shell am start -n org.pytorch.executorchexamples.dl3/.MainActivity
2941
```
3042

31-
### On Android Studio
32-
Open Android Studio and open the project path `dl3/android/DeepLabV3Demo`. Wait for gradle sync to complete.
33-
Then simply press "Run app" button (Control + r) to run the app either on physical device / emulator.
43+
### Using Android Studio
44+
1. Open the project at `dl3/android/DeepLabV3Demo`
45+
2. Wait for Gradle sync to complete
46+
3. Click "Run app" (Control + R)
3447

35-
## Step 4: Push the model to the phone or emulator
36-
The app loads a hardcoded model path (`/data/local/tmp/dl3_xnnpack_fp32.pte`) on the phone.
37-
Run the following adb command to push the model.
38-
```
39-
adb push dl3_xnnpack_fp32.pte /data/local/tmp/dl3_xnnpack_fp32.pte
40-
```
48+
## Step 4: Get the Model
4149

42-
Note: If you want to use a QNN lowered model, you mush modify the maven executorch dependency to [executorch-qnn](https://mvnrepository.com/artifact/org.pytorch/executorch-android-qnn) and rebuild the app.
50+
### Option A: Download from the App (Recommended)
51+
When the app launches, tap the **"Download Model"** button. The model will be downloaded and extracted automatically.
4352

44-
## Step 5: Load and Test Custom Images (No APK Rebuild Needed)
45-
You can now test image segmentation on your own images (supported formats: .jpg, .jpeg, .png) without rebuilding the APK. The app supports loading .jpg, .jpeg, and .png images from the /sdcard/Pictures/ directory, with user-granted permissions.
53+
### Option B: Export and Push Manually
54+
If you exported the model yourself or want to use a custom model, you need to copy it to the app's private storage. Since the app is built in debug mode, we can use `run-as`:
4655

47-
### How to Use
48-
#### a. Push your image to the device:
49-
```
50-
adb push <path to your image> /sdcard/Pictures/
56+
```bash
57+
# 1. Push to device temporary storage
58+
adb push dl3_xnnpack_fp32.pte /data/local/tmp/
5159

60+
# 2. Copy to app's private storage using run-as
61+
adb shell "run-as org.pytorch.executorchexamples.dl3 cp /data/local/tmp/dl3_xnnpack_fp32.pte files/"
5262
```
53-
#### b. After the push, In the app:
54-
- Tap the "Load And Refresh" button.
55-
- If prompted, grant the app permission to access the /sdcard/Pictures/ folder.
56-
- The image should appear immediately in the app.
57-
- You can now run dynamic image segmentation and view the results.
5863

59-
Tip: You can use images containing Dog, Sheep, Person, or any combination of these classes.
64+
> **Note:** For QNN backend, change the maven dependency to [executorch-qnn](https://mvnrepository.com/artifact/org.pytorch/executorch-android-qnn) and rebuild the app.
6065
61-
This feature enhances the testing/validation process by allowing users to quickly test different images without the need to rebuild the application.
66+
## Step 5: Using the App
6267

63-
## Step 6: Run unit test
64-
### On your terminal
65-
```
68+
### Sample Images
69+
Tap **"Next sample image"** to cycle through 3 built-in sample images.
70+
71+
### Pick Your Own Image
72+
1. Tap **"Pick Image"** to open your device's gallery
73+
2. Select any image (it will be automatically resized to 224x224)
74+
3. Tap **"Run"** to perform segmentation
75+
76+
### Run Segmentation
77+
1. Tap **"Run"** to start inference
78+
2. The segmentation overlay appears blended with the original image
79+
3. Inference time is displayed below the image
80+
81+
### Reset
82+
Tap **"Reset"** to restore the original image without the segmentation overlay.
83+
84+
## Supported Classes
85+
The app detects all 21 PASCAL VOC classes with distinct color overlays:
86+
87+
| Class | Color | Class | Color |
88+
|-------|-------|-------|-------|
89+
| Person | Red | Dog | Green |
90+
| Cat | Magenta | Car | Cyan |
91+
| Bird | Yellow | Bicycle | Green |
92+
| Boat | Blue | Bottle | Orange |
93+
| And 13 more... | | | |
94+
95+
## Step 6: Run Unit Tests
96+
97+
### Using Terminal
98+
```bash
6699
./gradlew connectedAndroidTest
67100
```
68101

69-
### On Android Studio
70-
In Android Studio project, open file `app/src/androidTest/java/org/pytorch/executorchexamples/dl3/SanityCheck.java`,
71-
and click the "Play" button for `public class SanityCheck` (Control + Shift + r).
102+
### Using Android Studio
103+
Open `app/src/androidTest/java/org/pytorch/executorchexamples/dl3/SanityCheck.java` and click the Play button.

‎dl3/android/DeepLabV3Demo/app/build.gradle.kts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ android {
4343

4444
dependencies {
4545
implementation("androidx.appcompat:appcompat:1.7.0")
46+
implementation("com.google.android.material:material:1.12.0")
4647
implementation("androidx.constraintlayout:constraintlayout:2.2.0")
4748
implementation("org.pytorch:executorch-android:1.0.0")
4849
testImplementation("junit:junit:4.13.2")

‎dl3/android/DeepLabV3Demo/app/src/main/AndroidManifest.xml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99

1010
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
1111
<uses-permission android:name="android.permission.READ_MEDIA_IMAGES" />
12+
<uses-permission android:name="android.permission.INTERNET" />
1213

1314
<application
1415
android:allowBackup="false"

0 commit comments

Comments
 (0)