Skip to content

Commit 0baaf7a

Browse files
BmanClarkBmanClark
committed
plank tutor first pages
1 parent 8284635 commit 0baaf7a

3 files changed

Lines changed: 172 additions & 0 deletions

File tree

‎content/learning-paths/mobile-graphics-and-gaming/ai-plank-tutor/1-introduce-plank-tutor.md‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
---
2+
title: Introduction
3+
weight: 2
4+
5+
### FIXED, DO NOT MODIFY
6+
layout: learningpathall
7+
---
8+
9+
## What you will build
10+
11+
In this Learning Path, you will build a simple on-device AI fitness tutor for Android.
12+
13+
The app watches a learner hold a plank, compares their body position with a stored instructor reference, asks a local LLM for one short correction, and speaks the correction using Android text-to-speech.
14+
15+
This project is based on the [AI Yoga Tutor](https://developer.arm.com/community/arm-community-blogs/b/ai-blog/posts/ai-yoga-tutor) prototype. The Learning Path keeps the same core pipeline, but narrows the app to one static pose so you can focus on how a pipeline that includes Android camera, pose detector, local LLM, and speech output fits together.
16+
17+
<!-- TODO: Add final app screenshot here.
18+
Suggested image path: images/final-ui.png
19+
Suggested caption: Figure 1: AI Plank Tutor showing the instructor plank image, live camera view, score, and spoken correction caption.
20+
-->
21+
22+
The finished app has two main visual areas:
23+
24+
- An instructor plank image on the left.
25+
- A live front-camera preview on the right.
26+
27+
The app overlays a pose score and a short caption that matches the spoken coaching feedback.
28+
29+
## App pipeline
30+
31+
The app uses a small pipeline of on-device components:
32+
33+
```text
34+
reference image and landmarks
35+
-> CameraX live frames
36+
-> Pose Landmarker
37+
-> joint-angle scoring
38+
-> compact text prompt
39+
-> Arm AI Chat + LLM
40+
-> Text-To-Speech
41+
```
42+
43+
Each stage passes structured data to a subsequent stage. The LLM does not receive camera frames or images. It receives a short text prompt describing the largest joint-angle differences between the learner and the reference plank pose.
44+
45+
This keeps the LLM prompt small, reduces latency, and makes the behavior easier to tune.
46+
47+
## Get the starter project
48+
49+
Download the starter project from:
50+
51+
```text
52+
TODO: <starter app download URL>
53+
```
54+
55+
Unzip the starter project and open the Android project in Android Studio.
56+
57+
{{% notice Note %}}
58+
The starter project contains the app structure, layout, image asset, MediaPipe pose model, and several Kotlin shell files. You will fill in the missing code over the next pages.
59+
{{% /notice %}}
60+
61+
## Open the project in Android Studio
62+
63+
1. Start Android Studio.
64+
2. Select **Open**.
65+
3. Open the starter project's `android` folder.
66+
4. Wait for Gradle sync to finish.
67+
68+
If Android Studio prompts you to trust the project, accept the prompt.
69+
70+
The starter app is intentionally incomplete, but it should sync successfully before you add code.
71+
72+
## Inspect the provided files
73+
74+
Start by looking at the files that are already provided for you.
75+
76+
Open `app/build.gradle` and confirm that the Android, CameraX, lifecycle, and MediaPipe dependencies are already present.
77+
Arm's AI Chat dependency is not included yet. You will add it later, when you implement local LLM inference.
78+
79+
Open `app/src/main/AndroidManifest.xml` and confirm that the app requests camera access:
80+
81+
```xml
82+
<uses-permission android:name="android.permission.CAMERA" />
83+
```
84+
85+
Open `app/src/main/res/layout/activity_main.xml` and review the main UI. The layout already contains:
86+
87+
- An `ImageView` for the instructor plank image.
88+
- A `PreviewView` for the live camera.
89+
- A score label.
90+
- A caption label for spoken feedback.
91+
92+
Open `app/src/main/assets/videos/plank.jpg` and review the instructor reference image.
93+
94+
Open `PlankPoseData.kt` and note the hard-coded plank reference data. This file contains the reference landmarks and angle weights used by the scoring step. This was generated from the image in an offline step so it doesn't need any runtime compute.

‎content/learning-paths/mobile-graphics-and-gaming/ai-plank-tutor/_index.md‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
---
2+
title: Build an on-device AI plank tutor on Android
3+
4+
minutes_to_complete: 60
5+
6+
who_is_this_for: This Learning Path is for Android developers who want to explore creating an ML/GenAI pipeline, including camera input, local LLM inference, and speech.
7+
8+
learning_objectives:
9+
- Detect human pose landmarks from live Android camera frames with MediaPipe.
10+
- Structure data and build compact prompts that turn raw data into useful LLM input.
11+
- Run a mobile-sized LLM on-device with Arm's AI Chat library.
12+
- Speak generated output with Text-To-Speech.
13+
14+
prerequisites:
15+
- A development machine with Android Studio installed.
16+
- A recent Arm-powered Android phone in Developer Mode, with a USB data cable.
17+
- Basic familiarity with Kotlin and Android app development
18+
19+
author: Ben Clark
20+
21+
### Tags
22+
skilllevels: Introductory
23+
subjects: ML
24+
armips:
25+
- Cortex-A
26+
- Arm AI Chat library
27+
tools_software_languages:
28+
- Android Studio
29+
- Kotlin
30+
- CameraX
31+
- MediaPipe
32+
- LLM
33+
- Neon
34+
- SVE2
35+
- SME2
36+
operatingsystems:
37+
- Android
38+
39+
further_reading:
40+
- resource:
41+
title: AI Yoga Tutor
42+
link: https://developer.arm.com/community/arm-community-blogs/b/ai-blog/posts/ai-yoga-tutor
43+
type: blog
44+
- resource:
45+
title: AI Chat - Explore and evaluate LLMs on Android and ChromeOS
46+
link: https://developer.arm.com/community/arm-community-blogs/b/announcements/posts/ai-chat-explore-and-evaluate-llms-on-android-and-chromeos
47+
type: blog
48+
- resource:
49+
title: AI Chat library on GitHub
50+
link: https://github.com/arm/ai-chat
51+
type: website
52+
- resource:
53+
title: AI Chat library @ Maven Central
54+
link: https://central.sonatype.com/artifact/com.arm/ai-chat
55+
type: documentation
56+
- resource:
57+
title: MediaPipe Pose Landmarker
58+
link: https://ai.google.dev/edge/mediapipe/solutions/vision/pose_landmarker
59+
type: documentation
60+
- resource:
61+
title: Android TextToSpeech
62+
link: https://developer.android.com/reference/android/speech/tts/TextToSpeech
63+
type: documentation
64+
65+
### FIXED, DO NOT MODIFY
66+
# ================================================================================
67+
weight: 1 # _index.md always has weight of 1 to order correctly
68+
layout: "learningpathall" # All files under learning paths have this same wrapper
69+
learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content.
70+
---

‎content/learning-paths/mobile-graphics-and-gaming/ai-plank-tutor/_next-steps.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
---
2+
# ================================================================================
3+
# FIXED, DO NOT MODIFY THIS FILE
4+
# ================================================================================
5+
weight: 21 # The weight controls the order of the pages. _index.md always has weight 1.
6+
title: "Next Steps" # Always the same, html page title.
7+
layout: "learningpathall" # All files under learning paths have this same wrapper for Hugo processing.
8+
---

0 commit comments

Comments
 (0)