Add Android touchscreen target support and controller app#1450
Add Android touchscreen target support and controller app#1450Batestinha wants to merge 77 commits into
Conversation
|
steadying seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| byte(y >> 8), | ||
| contactCount, | ||
| }) | ||
| } |
There was a problem hiding this comment.
Touchscreen input doesn't reset user input timer
High Severity
TouchscreenReport never calls u.resetUserInputTime() after a successful HID write, unlike every other HID report function (AbsMouseReport, AbsMouseWheelReport, RelMouseReport, and keyboard functions). The jiggler uses gadget.GetLastUserInputTime() to detect inactivity, so active touchscreen use won't prevent the jiggler from firing, causing unwanted mouse movement during Android touch interaction.
Reviewed by Cursor Bugbot for commit 5fc6830. Configure here.
| lastAbsPos.current = { x, y }; | ||
| }, | ||
| [mouseMode, send, setMousePosition], | ||
| ); |
There was a problem hiding this comment.
Digitizer coordinate math conflicts with object-fill styling
High Severity
getDigitizerMoveHandler computes letterbox/pillarbox offsets assuming the video uses object-contain, but when isAndroidTarget is true, the video element is styled with object-fill. With object-fill, the video stretches to fill the entire element with no black bars, making offsetX/offsetY already map 1:1 to the video content. The handler's offset calculations produce incorrect coordinates since it compensates for non-existent bars, causing touch input to be misaligned on Android targets.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit cda3c8b. Configure here.
| container width constraint. | ||
| - After the README-only artifact-policy commit, GitHub CI had Go/UI lint | ||
| passing while build and Bugbot were still in progress when work stopped. | ||
|
|
There was a problem hiding this comment.
WIP.md contains personal development notes for production
Medium Severity
WIP.md contains personal development journal entries with internal commit SHAs, local machine paths (/userdata/jetkvm/bin/jetkvm_app), binary SHA-256 hashes, specific device references (Pixel 8), and Bugbot finding notes. This is working-notes content that belongs in a personal development log, not in a PR to an upstream repository.
Reviewed by Cursor Bugbot for commit cda3c8b. Configure here.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
There are 13 total unresolved issues (including 12 from previous reviews).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit dab06da. Configure here.
Batestinha
force-pushed
the
android-support-issues-pr-20260507
branch
from
7eea105 to
adffdc9
Compare
Tested on JetKVM hardware after deploy. Digitizer emulation is perfect, coordinates are well aligned and smooth, and the aspect ratio is correct. Removes the stray Android Back/Home/Recents overlay from the known-good 657c72b baseline.
Tested on JetKVM hardware after deploy. Desktop usability is unchanged, phone usability remains good, and the Android controller view hides the desktop chrome and button strip automatically without manual URL arguments.
Add the Android controller APK source and backend handoff so Android WebView clients use the native login activity instead of the vanilla web login page. The APK now asks only for JetKVM IP/host plus password, preserves Autofill support, submits to /auth/login-local, stores returned cookies, and uses the JetKVM primary blue for the login button. The backend serves a no-store native-login bridge for Android controller clients on /login-local and for unauthenticated Android HTML navigation through the SPA catch-all, while normal desktop browsers still receive the vanilla SPA. Tested: go test ./... with local Go caches; release APK build; installed controller APK 1.10 on the Pixel 7a; deployed and verified the JetKVM backend hash; confirmed Android root requests receive the native handoff, normal browser root requests receive the SPA, Bitwarden Autofill appears, and manual logout returns to the native login activity. Known limitation: after selecting a password from the Autofill service, the on-screen keyboard may remain open. This is app-side polish and will be addressed in the next commit by explicitly collapsing the IME after Autofill/login input.
Fixes the known limitation documented in ec4ded3: after selecting a password from the Autofill service, the on-screen keyboard could remain open over the native login screen. Use a small AutofillAwareEditText wrapper for the password field so the APK responds to Android's actual autofill callback rather than guessing from text timing. When Autofill populates the password, the app waits briefly, clears password focus, and hides the IME. Tested: release APK build, installed on the Pixel 7a, and verified the Autofill password selection now collapses the OSK while preserving the prior native login flow.
Restore and update the JetKVM companion APK source on this branch. Remove the arm companion and trusted keyguard dismiss test buttons from the settings UI, simplify permission button labels, and show direct granted/not granted permission status text. Align the companion settings background and button blue with the native Android login activity, and bump the companion package to versionCode 6 / versionName 1.5. Built the release APK and installed it on the Pixel 8 for visual verification; backend Go tests also passed.
Prefer the relative mouse HID path for wheel reports when it is available, keeping absolute mouse wheel as a fallback. Track the current relative mouse button state and include it in wheel reports so scrolling does not synthesize a button release. Add focused tests for relative mouse and wheel report byte layout. Validated on the Pixel 8 target: direct relative HID wheel reports scroll Android Settings, the deployed experiment backend scrolls through the viewer, and digitizer/aspect behavior remained unchanged.
Remove the inert fullscreen entry from the Android compact hamburger menu. Expose a controller-side IME bridge that opens Android's OSK and forwards committed text through the existing JetKVM keyboard macro path, preserving desktop virtual keyboard fallback behavior.
Replace the fork README with a fresh upstream README copy plus a clearly separated Android support overview. Describe the validated touchscreen/aspect baseline, controller APK, compact overlay, native login, Android OSK bridge, display toggle, wheel scrolling, and target companion APK so the fork purpose and implementation are clear before the vanilla upstream documentation.
The upstream wake HID function and the Android touchscreen digitizer both need hid.usb3 on this hardware. The device kernel does not expose hid.usb4, so keep wake_hid enabled only when touchscreen is disabled and route wake behavior through a harmless no-touch touchscreen report when touchscreen owns the slot. Also preserve enabled config symlinks when a disabled gadget item shares the same config path, which prevents disabled wake_hid from removing the active touchscreen hid.usb3 link during config writes. Validated with go test ./internal/usbgadget, make build_dev, and a live JetKVM sanity check showing the debug binary hash matched, USB was configured, and hid.usb3 used the touchscreen report length.
Batestinha
force-pushed
the
android-support-issues-pr-20260507
branch
from
919216b to
7fd2aa9
Compare
2 participants
Summary
This PR adds an Android support path for using JetKVM against Android targets and from Android phones:
Issues Addressed
Directly addresses:
Also related, but not claimed as fixed here:
Validation
./scripts/dev_deploy.sh -r 192.168.137.254 -i --disable-docker./userdata/jetkvm/bin/jetkvm_appand rebooted the device.androidCompactController,JetKVMWebView/1) and the JetKVM web endpoints respond over LAN/Tailscale.go test ./internal/usbgadget.Notes
The Android companion does not bypass credential security. When Android requires credentials, it can bring up the bouncer after wake; PIN/password entry still goes through JetKVM keyboard input or the Android controller IME/OSK path.
Note
High Risk
High risk because it adds new USB HID device types, new JSON-RPC surface area, and Android companion pairing/target-metadata flows that affect input handling and device identity/EDID defaults.
Overview
Adds an Android-target control path by introducing a USB HID touchscreen/digitizer gadget (
/dev/hidg3) and a newtouchscreenReportJSON-RPC method, alongside tweaks to relative wheel reporting to preserve button state.Extends configuration and device identity handling with
target_type, companion pairing storage, and per-device USB/EDID defaults (serial/product strings and monitor name/serial derived from device ID), plus JSON-RPCgetTargetType/setTargetTypebacked by companion-provided target leases.Introduces two new Android projects: an Android controller APK (native login + Autofill + IME text bridge into the web UI) and an Android target companion APK (pairing, foreground service, keyguard-dismiss assist, external-display “pulse” presentation, and periodic
/companion/targetlease reporting). Also tightens local deployment scripts with a baseline-branch/install guard to prevent accidentally installing non-Android-support binaries.Reviewed by Cursor Bugbot for commit 8bc8eb8. Bugbot is set up for automated code reviews on this repo. Configure here.