Skip to content

Add support for accessibility navigation gestures (TalkBack)#2144

Open
keymapper-ai wants to merge 1 commit into
developfrom
claude/issue-262-talkback-gestures
Open

Add support for accessibility navigation gestures (TalkBack)#2144
keymapper-ai wants to merge 1 commit into
developfrom
claude/issue-262-talkback-gestures

Conversation

@keymapper-ai
Copy link
Copy Markdown
Collaborator

Closes #262

Summary

  • Adds a new "TalkBack gesture" action (ActionId.TALKBACK_GESTURE) that lets users trigger TalkBack navigation gestures via a key mapping.
  • Supports all 25 gestures listed in the issue: 1-finger swipes (4), 1-finger angular/chained swipes (5), 2-finger gestures (4), 3-finger gestures (5), and 4-finger gestures (7).
  • The action is editable: tapping it opens a picker dialog showing each gesture's TalkBack action label and its default gesture name (e.g. "Move reading control up or backwards (Swipe up)").
  • Gestures are dispatched via the accessibility service's dispatchGesture() API using computed screen-center coordinates.
  • Error checking shows a warning if the TalkBack app is not installed or disabled.

Files changed

File Change
TalkBackGestureType.kt New enum of 25 gesture types
TalkBackGestureStrings.kt String-resource lookup helpers for action labels and gesture names
ActionId.kt New TALKBACK_GESTURE entry
ActionData.kt New ActionData.TalkBackGesture(gesture) data class
ActionEntity.kt New EXTRA_TALKBACK_GESTURE_TYPE extra constant
ActionDataEntityMapper.kt Bidirectional entity ↔ data mapping
ActionUtils.kt Category (INTERFACE), title, icon (Icons.Outlined.Accessibility), editability
ActionUiHelper.kt Human-readable instance title (e.g. "TalkBack: Move reading control up or backwards")
ActionErrorSnapshot.kt Error if TalkBack package not installed/enabled
CreateActionDelegate.kt Picker dialog to choose gesture at action-creation time
PerformActionsUseCase.kt Dispatches gesture at execution time
IAccessibilityService.kt New performTalkBackGesture() method in interface
BaseAccessibilityService.kt Full gesture dispatch implementation using dispatchGesture()
strings.xml Action title, formatted title, picker title, and 50 label strings (25 action labels + 25 gesture names)
CHANGELOG.md Unreleased entry

Known limitations

  • Samsung TalkBack: Samsung ships its own TalkBack variant under a different package name. The error check only covers the standard Google TalkBack package (com.google.android.marvin.talkback).
  • Gesture timing: Tap durations and swipe speeds are approximations computed from screen density. They work for standard TalkBack configurations but may need tuning for non-default TalkBack gesture sensitivity settings.
  • Multi-finger tap stroke count: Android limits GestureDescription to 10 strokes. Four-finger triple-tap would require 12 strokes and is not included in the gesture set for this reason. The implemented set stays within this limit.
  • Coordinate-based dispatch: Gestures are dispatched relative to the screen center and are not aware of focused elements, so they simulate physical gestures rather than semantic TalkBack commands.

Generated by Claude Code

Adds a new "TalkBack gesture" action that dispatches TalkBack navigation
gestures (swipes, multi-finger taps, and multi-directional swipes) via
the accessibility service's dispatchGesture API.

https://claude.ai/code/session_01K15jUcBgeWMmHcmWJ3DWgK
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for accessibility navigation gestures (talkback)

2 participants