Skip to content

Commit d77a921

Browse files
thymikeethymikee
authored
feat: add Android snapshot helper (#454)
* feat: add android snapshot helper * fix: harden android snapshot helper packaging * fix: harden android xml attribute parsing * fix: capture android helper window roots * fix: validate android helper install args * fix: harden android helper manifest and traversal * refactor: drop unused helper timeout parameter * feat: bundle android snapshot helper in npm package * feat: enable bundled android snapshot helper by default * refactor: simplify android snapshot helper resolution * fix: harden android snapshot helper artifacts * refactor: split android snapshot helper modules * fix: use type-only snapshot helper imports
1 parent 83042cc commit d77a921

36 files changed

Lines changed: 3031 additions & 199 deletions

‎.fallowrc.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
"src/remote-config.ts",
1010
"src/install-source.ts",
1111
"src/android-apps.ts",
12+
"src/android-snapshot-helper.ts",
1213
"src/contracts.ts",
1314
"src/selectors.ts",
1415
"src/finders.ts",

‎.gitattributes‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
android-snapshot-helper/debug.keystore binary

‎.github/workflows/ci.yml‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,39 @@ jobs:
9090
- name: Run typecheck
9191
run: pnpm typecheck
9292

93+
android-snapshot-helper:
94+
name: Android Snapshot Helper Package
95+
runs-on: ubuntu-latest
96+
timeout-minutes: 15
97+
steps:
98+
- name: Checkout
99+
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
100+
101+
- name: Setup toolchain
102+
uses: ./.github/actions/setup-node-pnpm
103+
104+
- name: Install Android SDK packages
105+
run: |
106+
SDK_ROOT="${ANDROID_HOME:-${ANDROID_SDK_ROOT:-/usr/local/lib/android/sdk}}"
107+
SDKMANAGER="$SDK_ROOT/cmdline-tools/latest/bin/sdkmanager"
108+
if [ ! -x "$SDKMANAGER" ]; then
109+
SDKMANAGER="$SDK_ROOT/cmdline-tools/bin/sdkmanager"
110+
fi
111+
if [ ! -x "$SDKMANAGER" ]; then
112+
echo "sdkmanager not found under $SDK_ROOT" >&2
113+
exit 1
114+
fi
115+
yes | "$SDKMANAGER" --licenses >/dev/null
116+
"$SDKMANAGER" "platforms;android-36" "build-tools;36.0.0"
117+
118+
- name: Check Java toolchain
119+
run: |
120+
javac --version
121+
java --version
122+
123+
- name: Package npm-bundled Android snapshot helper
124+
run: pnpm package:android-snapshot-helper:npm
125+
93126
integration-smoke:
94127
name: Integration Smoke
95128
runs-on: macos-26

‎.github/workflows/release-android-snapshot-helper.yml‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
name: Release Android Snapshot Helper
2+
3+
on:
4+
release:
5+
types:
6+
- published
7+
workflow_dispatch:
8+
inputs:
9+
release_tag:
10+
description: GitHub Release tag to upload assets to, for example v0.13.3.
11+
required: true
12+
type: string
13+
checkout_ref:
14+
description: Optional branch, tag, or commit SHA to build. Defaults to the selected workflow ref.
15+
required: false
16+
type: string
17+
18+
permissions:
19+
contents: write
20+
21+
jobs:
22+
publish-android-snapshot-helper:
23+
name: Publish Android Snapshot Helper
24+
runs-on: ubuntu-latest
25+
timeout-minutes: 30
26+
steps:
27+
- name: Checkout
28+
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
29+
with:
30+
ref: ${{ github.event.inputs.checkout_ref || github.ref }}
31+
32+
- name: Setup Node.js
33+
uses: actions/setup-node@6044e13b5dc448c55e2357c09f80417699197238 # v6.2.0
34+
with:
35+
node-version: "22"
36+
37+
- name: Install Android SDK packages
38+
run: |
39+
SDK_ROOT="${ANDROID_HOME:-${ANDROID_SDK_ROOT:-/usr/local/lib/android/sdk}}"
40+
SDKMANAGER="$SDK_ROOT/cmdline-tools/latest/bin/sdkmanager"
41+
if [ ! -x "$SDKMANAGER" ]; then
42+
SDKMANAGER="$SDK_ROOT/cmdline-tools/bin/sdkmanager"
43+
fi
44+
if [ ! -x "$SDKMANAGER" ]; then
45+
echo "sdkmanager not found under $SDK_ROOT" >&2
46+
exit 1
47+
fi
48+
yes | "$SDKMANAGER" --licenses >/dev/null
49+
"$SDKMANAGER" "platforms;android-36" "build-tools;36.0.0"
50+
51+
- name: Check Java toolchain
52+
run: |
53+
javac --version
54+
java --version
55+
56+
- name: Resolve release metadata
57+
id: meta
58+
run: |
59+
set -euo pipefail
60+
PACKAGE_VERSION="$(node -p "JSON.parse(require('node:fs').readFileSync('package.json', 'utf8')).version")"
61+
TAG_NAME="${{ github.event.release.tag_name || github.event.inputs.release_tag }}"
62+
VERSION="${TAG_NAME#v}"
63+
if [ "$VERSION" != "$PACKAGE_VERSION" ]; then
64+
echo "Release tag $TAG_NAME does not match package.json version $PACKAGE_VERSION" >&2
65+
exit 1
66+
fi
67+
echo "tag=$TAG_NAME" >> "$GITHUB_OUTPUT"
68+
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
69+
shell: bash
70+
71+
- name: Package Android snapshot helper
72+
id: package
73+
env:
74+
RELEASE_ASSET_DIR: ${{ github.workspace }}/.tmp/release-assets
75+
run: |
76+
set -euo pipefail
77+
mkdir -p "${RELEASE_ASSET_DIR}"
78+
sh ./scripts/package-android-snapshot-helper.sh \
79+
"${{ steps.meta.outputs.version }}" \
80+
"${{ steps.meta.outputs.tag }}" \
81+
"${RELEASE_ASSET_DIR}"
82+
shell: bash
83+
84+
- name: Upload helper assets to GitHub Release
85+
env:
86+
GH_TOKEN: ${{ github.token }}
87+
run: |
88+
set -euo pipefail
89+
gh release upload "${{ steps.meta.outputs.tag }}" \
90+
"${{ steps.package.outputs.apk_path }}" \
91+
"${{ steps.package.outputs.checksum_path }}" \
92+
"${{ steps.package.outputs.manifest_path }}" \
93+
--clobber
94+
shell: bash

‎.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ node_modules/
22
.pnpm-store/
33
.fallow/
44
dist/
5+
.tmp/
56
.DS_Store
67
__pycache__/
78
*.pyc
@@ -23,9 +24,12 @@ xcuserdata/
2324
*.xcsettings
2425
*.xcresult
2526
*.ipa
27+
*.apk
2628
*.dSYM
2729
*.dSYM.zip
2830
*.app
2931
*.xctestrun
3032
*.xcarchive
3133
.skillgym-results/
34+
android-snapshot-helper/build/
35+
android-snapshot-helper/dist/

‎android-snapshot-helper/AndroidManifest.xml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
<manifest
2+
xmlns:android="http://schemas.android.com/apk/res/android"
3+
package="com.callstack.agentdevice.snapshothelper">
4+
<uses-sdk android:minSdkVersion="23" android:targetSdkVersion="36" />
5+
6+
<application
7+
android:debuggable="true"
8+
android:label="Agent Device Snapshot Helper"
9+
android:theme="@android:style/Theme.NoDisplay" />
10+
11+
<instrumentation
12+
android:name=".SnapshotInstrumentation"
13+
android:targetPackage="com.callstack.agentdevice.snapshothelper"
14+
android:label="Agent Device Snapshot Helper"
15+
android:functionalTest="true" />
16+
</manifest>

‎android-snapshot-helper/README.md‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
# Android Snapshot Helper
2+
3+
Small instrumentation APK used to capture Android accessibility snapshots without relying on
4+
`uiautomator dump`'s fixed idle wait behavior. The helper enables Android's interactive-window
5+
retrieval flag and serializes every accessible window root returned by `UiAutomation.getWindows()`
6+
so keyboards and system overlays can appear in the same snapshot. If interactive window roots are
7+
unavailable, it falls back to the active-window root.
8+
9+
The helper is intentionally provider-neutral. Local `adb`, cloud ADB tunnels, and remote device
10+
providers can all install and run the same APK as long as they can execute ADB-style operations.
11+
Released helper APKs use the committed `debug.keystore`; do not rotate it casually, because Android
12+
requires a stable signing certificate for `adb install -r` upgrades.
13+
14+
## Build
15+
16+
```sh
17+
sh ./scripts/build-android-snapshot-helper.sh 0.13.3 .tmp/android-snapshot-helper
18+
```
19+
20+
The build uses Android SDK command-line tools directly. It expects `ANDROID_HOME` or
21+
`ANDROID_SDK_ROOT` to point at an SDK with `platforms/android-36` and matching build tools.
22+
`pnpm prepack` builds the npm-bundled helper into `android-snapshot-helper/dist`; npm users get
23+
that APK in the package and the first helper-backed `snapshot` installs it automatically when
24+
missing or outdated.
25+
26+
## Run
27+
28+
```sh
29+
adb install -r -t .tmp/android-snapshot-helper/agent-device-android-snapshot-helper-0.13.3.apk
30+
adb shell am instrument -w \
31+
-e waitForIdleTimeoutMs 500 \
32+
-e timeoutMs 8000 \
33+
-e maxDepth 128 \
34+
-e maxNodes 5000 \
35+
com.callstack.agentdevice.snapshothelper/.SnapshotInstrumentation
36+
```
37+
38+
`maxDepth` also caps recursive traversal depth inside the helper.
39+
The `-t` install flag is required because the helper is a debuggable instrumentation/test APK.
40+
Devices or providers that block test-package installs must allow this package before helper capture
41+
can run.
42+
43+
## Output Contract
44+
45+
The APK emits instrumentation status records using
46+
`agentDeviceProtocol=android-snapshot-helper-v1`.
47+
48+
Each XML chunk is sent with:
49+
50+
- `outputFormat=uiautomator-xml`
51+
- `chunkIndex`
52+
- `chunkCount`
53+
- `payloadBase64`
54+
55+
The final instrumentation result includes:
56+
57+
- `ok=true`
58+
- `helperApiVersion=1`
59+
- `waitForIdleTimeoutMs`
60+
- `timeoutMs`
61+
- `maxDepth`
62+
- `maxNodes`
63+
- `rootPresent`
64+
- `captureMode` (`interactive-windows` or `active-window`)
65+
- `windowCount`
66+
- `nodeCount`
67+
- `truncated`
68+
- `elapsedMs`
69+
70+
Failures return `ok=false`, `errorType`, and `message` in the final result.
71+
72+
The release manifest is a stable provider contract for the current helper protocol. Providers should
73+
resolve the APK from `apkUrl`, verify `sha256`, install using `installArgs`, and run
74+
`instrumentationRunner`. `installArgs` must start with `install`; extra arguments are limited to the
75+
allowlisted adb install flags `-r`, `-t`, `-d`, and `-g`, and the consumer appends the APK path.

‎android-snapshot-helper/debug.keystore‎

2.65 KB
Binary file not shown.

0 commit comments

Comments
 (0)