Skip to content

Commit a537504

Browse files
BmanClarkBmanClark
committed
Adding Arm AI Chat library simple Android chatbot learning path.
1 parent 436fc99 commit a537504

8 files changed

Lines changed: 602 additions & 0 deletions

File tree

‎content/learning-paths/mobile-graphics-and-gaming/android-ai-chat-lib/_index.md‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
---
2+
title: Add an LLM to your Android app with Arm's AI Chat library
3+
4+
minutes_to_complete: 15
5+
6+
who_is_this_for: Android App Developers who wish to add LLM capabilities to an app.
7+
8+
learning_objectives:
9+
- Easily add an on-device LLM to any Android app
10+
11+
prerequisites:
12+
- Android Studio
13+
- Android phone for testing (in Developer Mode, with USB cable to connect)
14+
15+
author: Ben Clark
16+
17+
### Tags
18+
skilllevels: Beginner
19+
subjects: ML
20+
armips:
21+
- Arm AI Chat library
22+
tools_software_languages:
23+
- Kotlin
24+
- Neon
25+
- SVE2
26+
- SME2
27+
- LLM
28+
operatingsystems:
29+
- Android
30+
31+
32+
33+
further_reading:
34+
- resource:
35+
title: AI Chat - Explore and evaluate LLMs on Android and ChromeOS
36+
link: https://developer.arm.com/community/arm-community-blogs/b/announcements/posts/ai-chat-explore-and-evaluate-llms-on-android-and-chromeos
37+
type: blog
38+
- resource:
39+
title: Arm AI Chat LLM test app
40+
link: https://play.google.com/store/apps/details?id=com.arm.aichat
41+
type: example app
42+
- resource:
43+
title: AI Chat library @ Maven Central
44+
link: https://central.sonatype.com/artifact/com.arm/ai-chat
45+
type: documentation
46+
- resource:
47+
title: AI Chat library on GitHub
48+
link: https://github.com/arm/ai-chat
49+
type: website
50+
- resource:
51+
title: Arm KleidiAI - Helping AI frameworks elevate their performance on Arm CPUs
52+
link: https://developer.arm.com/community/arm-community-blogs/b/ai-blog/posts/kleidiai
53+
type: blog
54+
- resource:
55+
title: SME2
56+
link: https://www.arm.com/technologies/sme2
57+
type: website
58+
59+
60+
### FIXED, DO NOT MODIFY
61+
# ================================================================================
62+
weight: 1 # _index.md always has weight of 1 to order correctly
63+
layout: "learningpathall" # All files under learning paths have this same wrapper
64+
learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content.
65+
---

‎content/learning-paths/mobile-graphics-and-gaming/android-ai-chat-lib/_next-steps.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
---
2+
# ================================================================================
3+
# FIXED, DO NOT MODIFY THIS FILE
4+
# ================================================================================
5+
weight: 21 # The weight controls the order of the pages. _index.md always has weight 1.
6+
title: "Next Steps" # Always the same, html page title.
7+
layout: "learningpathall" # All files under learning paths have this same wrapper for Hugo processing.
8+
---

‎content/learning-paths/mobile-graphics-and-gaming/android-ai-chat-lib/add-ai-chat-lib-2.md‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
---
2+
title: Add AI Chat Library
3+
weight: 3
4+
5+
### FIXED, DO NOT MODIFY
6+
layout: learningpathall
7+
---
8+
9+
Your freshly created Android Studio project should already have top-level repositories including `google()` and `mavenCentral()` in `settings.gradle.kts`, but check that it does, to ensure the app can find the library.
10+
11+
## Add the Maven Dependency
12+
We need to add the AI Chat library in the app module build file, `app/build.gradle.kts` (not the project `build.gradle.kts` in the root of the project).
13+
14+
In here we add into the `dependencies` section at the bottom:
15+
```kotlin
16+
implementation("com.arm:ai-chat:0.1.0")
17+
```
18+
19+
This adds the library to your project, and all the LLM functionality. Also in this file, check that:
20+
- `targetSdk` and `compileSdk` are 36
21+
- the Java version in `compileOptions` is `JavaVersion.VERSION_17`
22+
- the `jvmTarget` in `kotlinOptions` is 17
23+
24+
In `libs.version.toml` we need to change the `kotlin` version to `2.2.20`, as the library requires a more recent version than the default.
25+
26+
Finally, there will be a banner at the top of these settings files saying the "Gradle files have changed since last project sync." Choose the option to "Sync Now", and let the project update.
27+
28+
## Adjust the Manifest
29+
We need to adjust the `AndroidManifest.xml` file, which is in `app\src\main`.
30+
31+
Most of the file is an xml tag `<application ...>`. Add in among the other similar looking options in the tag:
32+
```xml
33+
<application
34+
android:extractNativeLibs="true"
35+
... >
36+
```
37+
This enables the app to load the needed llama.cpp native libraries.

‎content/learning-paths/mobile-graphics-and-gaming/android-ai-chat-lib/add-layouts-adapter-3.md‎

Lines changed: 174 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,174 @@
1+
---
2+
title: Add UI Code
3+
weight: 4
4+
5+
### FIXED, DO NOT MODIFY
6+
layout: learningpathall
7+
---
8+
9+
## Add Layouts
10+
Currently the `activity_main.xml` layout file in your `app\src\res\layout` directory is nearly empty, containing just a "Hello World!" piece of text that we wish to remove. Instead, replace it with the following, which will create a status area at the top, a place for messages in the middle, and a place for you to type at the bottom with a button to send:
11+
```xml
12+
<?xml version="1.0" encoding="utf-8"?>
13+
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
14+
xmlns:app="http://schemas.android.com/apk/res-auto"
15+
android:id="@+id/root"
16+
android:layout_width="match_parent"
17+
android:layout_height="match_parent">
18+
19+
<TextView
20+
android:id="@+id/status_text"
21+
android:layout_width="0dp"
22+
android:layout_height="wrap_content"
23+
android:layout_marginStart="16dp"
24+
android:layout_marginTop="16dp"
25+
android:layout_marginEnd="16dp"
26+
android:text="Loading model..."
27+
android:textAppearance="@style/TextAppearance.MaterialComponents.Body1"
28+
app:layout_constraintEnd_toEndOf="parent"
29+
app:layout_constraintStart_toStartOf="parent"
30+
app:layout_constraintTop_toTopOf="parent" />
31+
32+
<androidx.recyclerview.widget.RecyclerView
33+
android:id="@+id/messages"
34+
android:layout_width="0dp"
35+
android:layout_height="0dp"
36+
android:layout_marginStart="16dp"
37+
android:layout_marginTop="12dp"
38+
android:layout_marginEnd="16dp"
39+
android:clipToPadding="false"
40+
android:paddingBottom="8dp"
41+
app:layout_constraintBottom_toTopOf="@+id/input_row"
42+
app:layout_constraintEnd_toEndOf="parent"
43+
app:layout_constraintStart_toStartOf="parent"
44+
app:layout_constraintTop_toBottomOf="@+id/status_text" />
45+
46+
<LinearLayout
47+
android:id="@+id/input_row"
48+
android:layout_width="0dp"
49+
android:layout_height="wrap_content"
50+
android:layout_margin="16dp"
51+
android:orientation="horizontal"
52+
app:layout_constraintBottom_toBottomOf="parent"
53+
app:layout_constraintEnd_toEndOf="parent"
54+
app:layout_constraintStart_toStartOf="parent">
55+
<EditText
56+
android:id="@+id/user_input"
57+
android:layout_width="0dp"
58+
android:layout_height="wrap_content"
59+
android:layout_weight="1"
60+
android:enabled="false"
61+
android:hint="Model is loading..."
62+
android:inputType="textMultiLine"
63+
android:maxLines="4"
64+
android:minHeight="48dp"
65+
android:padding="12dp" />
66+
<com.google.android.material.button.MaterialButton
67+
android:id="@+id/send_button"
68+
android:layout_width="wrap_content"
69+
android:layout_height="wrap_content"
70+
android:layout_marginStart="12dp"
71+
android:enabled="false"
72+
android:text="Import model" />
73+
</LinearLayout>
74+
</androidx.constraintlayout.widget.ConstraintLayout>
75+
```
76+
77+
We also need two small "helper" layouts to format the messages from the user and the AI assistant. In the `layout` folder alongside the main layout, first create `item_message_user.xml` and insert the following:
78+
```xml
79+
<?xml version="1.0" encoding="utf-8"?>
80+
<FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"
81+
android:layout_width="match_parent"
82+
android:layout_height="wrap_content"
83+
android:paddingTop="6dp"
84+
android:paddingBottom="6dp">
85+
86+
<TextView
87+
android:id="@+id/msg_content"
88+
android:layout_width="wrap_content"
89+
android:layout_height="wrap_content"
90+
android:layout_gravity="end"
91+
android:background="#D7F0FF"
92+
android:padding="12dp"
93+
android:textAppearance="@style/TextAppearance.MaterialComponents.Body1" />
94+
</FrameLayout>
95+
```
96+
97+
Then create `item_message_assistant.xml` and put the following as its contents:
98+
```xml
99+
<?xml version="1.0" encoding="utf-8"?>
100+
<FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"
101+
android:layout_width="match_parent"
102+
android:layout_height="wrap_content"
103+
android:paddingTop="6dp"
104+
android:paddingBottom="6dp">
105+
106+
<TextView
107+
android:id="@+id/msg_content"
108+
android:layout_width="wrap_content"
109+
android:layout_height="wrap_content"
110+
android:layout_gravity="start"
111+
android:background="#EFEFEF"
112+
android:padding="12dp"
113+
android:textAppearance="@style/TextAppearance.MaterialComponents.Body1" />
114+
</FrameLayout>
115+
```
116+
117+
## Add the MessageAdaptor class
118+
As a final bit of UI code, we add a `MessageAdapter.kt` code file to put our messages into the correct bit of layout. The file should sit alongside the `MainActivity` class that is auto-created with the project.
119+
120+
The file contents of `MessageAdapter.kt` are:
121+
```kotlin
122+
package com.example.simpleaichat
123+
124+
import android.view.LayoutInflater
125+
import android.view.View
126+
import android.view.ViewGroup
127+
import android.widget.TextView
128+
import androidx.recyclerview.widget.RecyclerView
129+
130+
data class Message(
131+
val id: String,
132+
val content: String,
133+
val isUser: Boolean
134+
)
135+
136+
class MessageAdapter(
137+
private val messages: List<Message>
138+
) : RecyclerView.Adapter<RecyclerView.ViewHolder>() {
139+
140+
override fun getItemViewType(position: Int): Int {
141+
return if (messages[position].isUser) VIEW_TYPE_USER else VIEW_TYPE_ASSISTANT
142+
}
143+
144+
override fun onCreateViewHolder(parent: ViewGroup, viewType: Int): RecyclerView.ViewHolder {
145+
val inflater = LayoutInflater.from(parent.context)
146+
return if (viewType == VIEW_TYPE_USER) {
147+
UserMessageViewHolder(inflater.inflate(R.layout.item_message_user, parent, false))
148+
} else {
149+
AssistantMessageViewHolder(inflater.inflate(R.layout.item_message_assistant, parent, false))
150+
}
151+
}
152+
153+
override fun onBindViewHolder(holder: RecyclerView.ViewHolder, position: Int) {
154+
holder.itemView.findViewById<TextView>(R.id.msg_content).text = messages[position].content
155+
}
156+
157+
override fun getItemCount(): Int = messages.size
158+
159+
class UserMessageViewHolder(view: View) : RecyclerView.ViewHolder(view)
160+
class AssistantMessageViewHolder(view: View) : RecyclerView.ViewHolder(view)
161+
162+
companion object {
163+
private const val VIEW_TYPE_USER = 1
164+
private const val VIEW_TYPE_ASSISTANT = 2
165+
}
166+
}
167+
```
168+
169+
Quickly going through the important parts:
170+
- `Message` is the smallest useful chat piece: one id, the message text, and a flag saying whether it came from the user or the assistant.
171+
- `getItemViewType(...)` decides which row layout to use for each message.
172+
- `onCreateViewHolder(...)` inflates either the user bubble or the assistant bubble layout.
173+
- `onBindViewHolder(...)` writes the current message text into the row.
174+
- `getItemCount()` tells the `RecyclerView` how many chat rows it should render.

‎content/learning-paths/mobile-graphics-and-gaming/android-ai-chat-lib/add-model-run-5.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
title: Add Model And Run
3+
weight: 6
4+
5+
### FIXED, DO NOT MODIFY
6+
layout: learningpathall
7+
---
8+
9+
## Download a Mobile-compatible GGUF
10+
Before you run you'll need to download a GGUF model file to run. To be mobile compatible, it'll need to run within your test Android phone's memory. A common Android memory size is 8GB, although some modern premium phones have more. Other things will also need to fit in memory, so the model size will need to be noticeably less than 8GB.
11+
12+
A good example model is [google_gemma-3-4b-it-Q4_0.gguf](https://huggingface.co/bartowski/google_gemma-3-4b-it-GGUF/blob/main/google_gemma-3-4b-it-Q4_0.gguf). Gemma 3 is a powerful model, and this 4 billion parameter version has been int4 quantized with the Q4_0 schema that works particularly well with Arm's [KleidiAI library](https://developer.arm.com/ai/kleidi-libraries), enabling speed-ups on phones with [SME2](https://www.arm.com/technologies/sme2), [SVE2](https://developer.arm.com/documentation/102340/0100/Introducing-SVE2) and [Neon](https://www.arm.com/technologies/neon).
13+
14+
Download Gemma 3 or another suitable model onto your phone, ready to run the app.
15+
16+
## Run the App!
17+
In Android Studio, if you connect your test Android phone with a USB cable to your computer, you should now be able to run your LLM chatbot app. Make sure when you connect the phone it is in Developer Mode and you allow USB debugging.
18+
19+
In the bottom right there is a button "Import model". Clicking this will take you to downloads to be able to select the model you've downloaded, so the app can download it. Once it has finished copying and loading the model it will say "Model ready" at the top of the screen. Now if you click the text entry area at the bottom, you can type your questions and chat with the LLM.
20+
21+
![App screenshot#center](app_screenshot.jpg "Figure 1. AI Chat app screenshot interacting with Gemma 3 4B")

‎content/learning-paths/mobile-graphics-and-gaming/android-ai-chat-lib/app_screenshot.jpg‎

77 KB
Loading

‎content/learning-paths/mobile-graphics-and-gaming/android-ai-chat-lib/setup-1.md‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
---
2+
title: Project Setup
3+
weight: 2
4+
5+
### FIXED, DO NOT MODIFY
6+
layout: learningpathall
7+
---
8+
9+
## Objective
10+
In this learning path you will create a small chatbot Android app from scratch. The app will load a GGUF model of your choosing, and then run it in a chatbot format.
11+
12+
The app will use Arm's AI Chat library available from Maven Central, which provides an Android wrapper around llama.cpp, providing high-performance running of LLM models in the GGUF format.
13+
14+
For other examples of chatbots using this library you can use:
15+
- the fully featured [Arm AI Chat app on Google Play](https://play.google.com/store/apps/details?id=com.arm.aichat), which can be used to test the performance and capabilities of mobile models, or
16+
- the [AI Chat library GitHub example](https://github.com/arm/ai-chat/tree/use-maven-library/examples/llama.android), which is only slightly more complicated than this Learning Path.
17+
18+
## Project Setup
19+
Open Android Studio and create a new project of the type "Empty Views Activity". Name it however you like, and leave other options on default - for instance Minimum SDK will be 33.
20+

0 commit comments

Comments
 (0)