|
| 1 | +# Android Emulator Agent Guide |
| 2 | + |
| 3 | +Instructions for Claude agents working autonomously with the Android emulator. Follow these sections in order when starting from scratch, or jump to the relevant section for ongoing work. |
| 4 | + |
| 5 | +## 1. Emulator Setup and Launch |
| 6 | + |
| 7 | +### Check if the emulator is already running |
| 8 | +```bash |
| 9 | +adb devices |
| 10 | +``` |
| 11 | +If you see `emulator-5554 device`, the emulator is ready — skip to section 2. |
| 12 | +If the list is empty or shows `offline`, you need to start the emulator. |
| 13 | + |
| 14 | +### First-time setup (only needed once) |
| 15 | +If no AVD has been created yet: |
| 16 | +```bash |
| 17 | +make setup |
| 18 | +``` |
| 19 | +This downloads the Android SDK, system image, and creates the `kolibri-test` AVD. |
| 20 | + |
| 21 | +### Start the emulator |
| 22 | +```bash |
| 23 | +make emulator |
| 24 | +``` |
| 25 | +This launches the emulator in the background. Wait for it to finish booting: |
| 26 | +```bash |
| 27 | +adb wait-for-device |
| 28 | +adb shell getprop sys.boot_completed |
| 29 | +``` |
| 30 | +Poll `sys.boot_completed` until it returns `1`. The boot can take 30-60 seconds. |
| 31 | + |
| 32 | +**If the emulator segfaults or crashes**, it's likely a GPU issue. Start with software rendering instead: |
| 33 | +```bash |
| 34 | +android_root/emulator/emulator -avd kolibri-test -gpu guest -no-snapshot & |
| 35 | +``` |
| 36 | + |
| 37 | +## 2. Build and Install the App |
| 38 | + |
| 39 | +This project uses a Makefile as the primary build interface. Run `make help` to see all available targets. |
| 40 | + |
| 41 | +### Build and install in one step |
| 42 | +```bash |
| 43 | +make install |
| 44 | +``` |
| 45 | +This builds the debug APK via Gradle and installs it on the connected emulator. The first build takes several minutes; subsequent builds are faster. |
| 46 | + |
| 47 | +Watch for: |
| 48 | +- **BUILD SUCCESSFUL**: Proceed to install |
| 49 | +- **Compilation errors**: Fix before continuing |
| 50 | +- **Python errors**: Check Chaquopy output for syntax issues |
| 51 | + |
| 52 | +Or just build without installing: |
| 53 | +```bash |
| 54 | +make kolibri.apk.unsigned |
| 55 | +``` |
| 56 | + |
| 57 | +### Verify the app is installed |
| 58 | +```bash |
| 59 | +adb shell pm list packages | grep kolibri |
| 60 | +``` |
| 61 | +Should output: `package:org.learningequality.Kolibri` |
| 62 | + |
| 63 | +### Launch the app |
| 64 | +```bash |
| 65 | +adb shell am start -n org.learningequality.Kolibri/org.learningequality.Kolibri.WebViewActivity |
| 66 | +``` |
| 67 | + |
| 68 | +### Force stop the app |
| 69 | +```bash |
| 70 | +adb shell am force-stop org.learningequality.Kolibri |
| 71 | +``` |
| 72 | + |
| 73 | +### Clear app data (resets to fresh state) |
| 74 | +```bash |
| 75 | +adb shell pm clear org.learningequality.Kolibri |
| 76 | +``` |
| 77 | +This is needed after Python code changes since Chaquopy caches bytecode. |
| 78 | + |
| 79 | +### Uninstall and reinstall (for signing key mismatches) |
| 80 | +```bash |
| 81 | +make uninstall && make install |
| 82 | +``` |
| 83 | + |
| 84 | +### Makefile reference |
| 85 | + |
| 86 | +| Target | Description | |
| 87 | +|--------|-------------| |
| 88 | +| `make setup` | Complete SDK + emulator setup (first time) | |
| 89 | +| `make emulator` | Start the emulator | |
| 90 | +| `make kolibri.apk.unsigned` | Build debug APK to dist/ | |
| 91 | +| `make install` | Build and install debug APK | |
| 92 | +| `make uninstall` | Uninstall app from device | |
| 93 | +| `make logcat` | View filtered Kolibri logs | |
| 94 | +| `make clean` | Clean build artifacts | |
| 95 | +| `make test` | Run unit tests | |
| 96 | +| `make lint` | Run Android linter | |
| 97 | + |
| 98 | +### Quick commands |
| 99 | + |
| 100 | +Build + Install + Launch: |
| 101 | +```bash |
| 102 | +make install && adb shell am start -n org.learningequality.Kolibri/org.learningequality.Kolibri.WebViewActivity |
| 103 | +``` |
| 104 | + |
| 105 | +Clear logs and monitor: |
| 106 | +```bash |
| 107 | +adb logcat -c && make logcat |
| 108 | +``` |
| 109 | + |
| 110 | +## 3. Visual Inspect-Act Loop |
| 111 | + |
| 112 | +This is the core workflow for autonomous UI interaction. Use `/project:screenshot` to run the full loop with instructions, or follow these steps: |
| 113 | + |
| 114 | +### Capture the screen |
| 115 | +```bash |
| 116 | +mkdir -p /tmp/claude |
| 117 | +adb exec-out screencap -p > /tmp/claude/screenshot.png |
| 118 | +``` |
| 119 | +Read the screenshot image at `/tmp/claude/screenshot.png` to see the screen visually. |
| 120 | + |
| 121 | +### Inspect: CDP vs uiautomator |
| 122 | + |
| 123 | +Kolibri is a WebView app. **WebView content and native Android UI require different tools:** |
| 124 | + |
| 125 | +| What you see | Tool | Why | |
| 126 | +|---|---|---| |
| 127 | +| Kolibri UI (buttons, forms, nav, text) | `python3 scripts/cdp_helper.py dump` | WebView DOM is invisible to uiautomator | |
| 128 | +| Native Android dialogs (permissions, system prompts) | `adb shell uiautomator dump /sdcard/window_dump.xml && adb shell cat /sdcard/window_dump.xml` | System dialogs are invisible to CDP | |
| 129 | + |
| 130 | +**Rule of thumb:** If a system dialog with rounded corners is overlaying the app, use uiautomator. For everything else, use CDP. |
| 131 | + |
| 132 | +### Interact: CDP vs adb input |
| 133 | + |
| 134 | +**WebView elements** — click by text via CDP (no coordinate math needed): |
| 135 | +```bash |
| 136 | +python3 scripts/cdp_helper.py click "CONTINUE" |
| 137 | +python3 scripts/cdp_helper.py click "EXPLORE" |
| 138 | +``` |
| 139 | + |
| 140 | +**Native elements** — tap by coordinates from uiautomator bounds: |
| 141 | +```bash |
| 142 | +# bounds="[137,1177][943,1331]" → center at (540, 1254) |
| 143 | +adb shell input tap 540 1254 |
| 144 | +``` |
| 145 | + |
| 146 | +**Other interactions:** |
| 147 | +```bash |
| 148 | +adb shell input text "<text>" # Type text (encode spaces as %s) |
| 149 | +adb shell input swipe 540 1500 540 500 300 # Scroll down |
| 150 | +adb shell input swipe 540 500 540 1500 300 # Scroll up |
| 151 | +adb shell input keyevent 4 # Press BACK |
| 152 | +adb shell input keyevent 66 # Press ENTER |
| 153 | +adb shell input keyevent 3 # Press HOME |
| 154 | +``` |
| 155 | + |
| 156 | +### Verify |
| 157 | +Take another screenshot after every interaction. Confirm the UI changed as expected before proceeding. |
| 158 | + |
| 159 | +### CDP helper reference |
| 160 | + |
| 161 | +The CDP helper (`scripts/cdp_helper.py`) uses Chrome DevTools Protocol over ADB to access the WebView DOM. Requires `websockets` (`uv pip install websockets`). |
| 162 | + |
| 163 | +```bash |
| 164 | +python3 scripts/cdp_helper.py dump # List visible DOM elements as JSON |
| 165 | +python3 scripts/cdp_helper.py click "Button" # Click element by exact text match |
| 166 | +python3 scripts/cdp_helper.py js "expr" # Evaluate arbitrary JavaScript |
| 167 | +``` |
| 168 | + |
| 169 | +### Key event codes |
| 170 | +| Code | Key | Code | Key | |
| 171 | +|------|-----|------|-----| |
| 172 | +| 3 | HOME | 4 | BACK | |
| 173 | +| 19 | DPAD_UP | 20 | DPAD_DOWN | |
| 174 | +| 21 | DPAD_LEFT | 22 | DPAD_RIGHT | |
| 175 | +| 61 | TAB | 66 | ENTER | |
| 176 | +| 67 | DEL | 111 | ESCAPE | |
| 177 | + |
| 178 | +## 4. Maestro Flow Development |
| 179 | + |
| 180 | +Maestro flows live in `.maestro/`. Use them for repeatable UI test sequences. |
| 181 | + |
| 182 | +### Develop a new flow |
| 183 | +1. **Discover UI elements** using the CDP helper: `python3 scripts/cdp_helper.py dump`. Note the `text` content — Maestro matches WebView elements by text when `androidWebViewHierarchy: devtools` is set. |
| 184 | +2. **Write the flow** as a YAML file in `.maestro/`: |
| 185 | + ```yaml |
| 186 | + appId: org.learningequality.Kolibri |
| 187 | + androidWebViewHierarchy: devtools |
| 188 | + --- |
| 189 | + - launchApp |
| 190 | + - tapOn: "CONTINUE" |
| 191 | + ``` |
| 192 | +3. **Run the flow**: |
| 193 | + ```bash |
| 194 | + ~/.maestro/bin/maestro test .maestro/your-flow.yaml |
| 195 | + ``` |
| 196 | +4. **Iterate**: If the flow fails, screenshot to see the actual state, adjust selectors or add waits, re-run. |
| 197 | + |
| 198 | +### Install Maestro (if not present) |
| 199 | +```bash |
| 200 | +make maestro-install |
| 201 | +``` |
| 202 | + |
| 203 | +### Common Maestro commands |
| 204 | +- `launchApp` / `clearState` / `clearKeychain` |
| 205 | +- `tapOn: "text"` / `tapOn: { id: "resource-id" }` |
| 206 | +- `inputText: "value"` |
| 207 | +- `assertVisible: "text"` / `assertNotVisible: "text"` |
| 208 | +- `extendedWaitUntil: { visible: "text", timeout: 30000 }` |
| 209 | +- `scroll` / `swipe` |
| 210 | +- `back` / `hideKeyboard` |
| 211 | + |
| 212 | +## 5. Log Inspection |
| 213 | + |
| 214 | +### Kolibri-filtered logs (streaming) |
| 215 | +```bash |
| 216 | +make logcat |
| 217 | +``` |
| 218 | + |
| 219 | +### Python stdout/stderr |
| 220 | +```bash |
| 221 | +adb logcat -s python.stdout:V python.stderr:V |
| 222 | +``` |
| 223 | + |
| 224 | +### Specific component tags |
| 225 | +```bash |
| 226 | +adb logcat -s KolibriWebView:V KolibriServer:V TaskWorkerImpl:V BaseTaskWorker:V |
| 227 | +``` |
| 228 | + |
| 229 | +### Crash logs |
| 230 | +```bash |
| 231 | +adb logcat -s AndroidRuntime:E |
| 232 | +``` |
| 233 | + |
| 234 | +### Recent log snapshot (non-streaming) |
| 235 | +```bash |
| 236 | +adb logcat -d -t 50 |
| 237 | +``` |
| 238 | + |
| 239 | +### Clear log buffer |
| 240 | +```bash |
| 241 | +adb logcat -c |
| 242 | +``` |
| 243 | + |
| 244 | +## 6. Troubleshooting |
| 245 | + |
| 246 | +### App crashes on startup |
| 247 | +1. Check crash logs: `adb logcat -s AndroidRuntime:E` |
| 248 | +2. Look for Python import errors: `adb logcat -s python.stderr:V` |
| 249 | + |
| 250 | +### Python changes not appearing |
| 251 | +Chaquopy caches Python bytecode. Clear app data: |
| 252 | +```bash |
| 253 | +adb shell pm clear org.learningequality.Kolibri |
| 254 | +``` |
| 255 | + |
| 256 | +Or uninstall and reinstall: |
| 257 | +```bash |
| 258 | +make uninstall && make install |
| 259 | +``` |
| 260 | + |
| 261 | +### INSTALL_FAILED_UPDATE_INCOMPATIBLE |
| 262 | +Signing key mismatch. Uninstall first: |
| 263 | +```bash |
| 264 | +make uninstall && make install |
| 265 | +``` |
| 266 | + |
| 267 | +### Service Worker issues |
| 268 | +1. Open Chrome DevTools: `chrome://inspect` |
| 269 | +2. Find the Kolibri WebView and inspect |
| 270 | +3. Check Application > Service Workers |
| 271 | + |
| 272 | +### WorkManager tasks not running |
| 273 | +Check task logs: |
| 274 | +```bash |
| 275 | +adb logcat -s TaskWorkerImpl:V BaseTaskWorker:V WM-WorkerWrapper:V |
| 276 | +``` |
| 277 | + |
| 278 | +### Emulator not found |
| 279 | +```bash |
| 280 | +make setup # Creates SDK + AVD |
| 281 | +make emulator |
| 282 | +``` |
| 283 | + |
| 284 | +## 7. Iterating |
| 285 | + |
| 286 | +1. Make code changes |
| 287 | +2. `make install` |
| 288 | +3. Launch app and test |
| 289 | +4. `make logcat` in another terminal |
| 290 | +5. Repeat |
| 291 | + |
| 292 | +For Python-only changes, builds are fast since Java doesn't need recompilation. |
| 293 | + |
| 294 | +## Key Facts |
| 295 | + |
| 296 | +| Fact | Value | |
| 297 | +|------|-------| |
| 298 | +| Package name | `org.learningequality.Kolibri` | |
| 299 | +| Main activity | `org.learningequality.Kolibri.WebViewActivity` | |
| 300 | +| AVD name | `kolibri-test` | |
| 301 | +| JAVA_HOME | Set by Makefile; see `JAVA_HOME` in `Makefile` | |
| 302 | +| GPU workaround | Use `-gpu guest` if default GPU segfaults | |
| 303 | +| Python caching | Clear app data after Python changes (Chaquopy caches bytecode) | |
| 304 | +| Build system | Gradle via Makefile wrappers — use `make` targets | |
| 305 | +| CDP helper | `python3 scripts/cdp_helper.py` — requires `websockets` package | |
0 commit comments