feat(analytics): add autocapture frustration signals (click, rage click, dead click)

[rahul-mixpanel]

Summary

• Implements autocapture Phase 1: $mp_click, $mp_rage_click, and $mp_dead_click event detection for both XML Views and Jetpack Compose UIs
• Adds AutocaptureOptions configuration (disabled by default, opt-in via MixpanelOptions builder) with per-feature options (ClickOptions, RageClickOptions, DeadClickOptions)
• Intercepts touch events via Window.Callback wrapping, with WindowSpy for dialog/popup/menu window tracking
• Full Compose support via native SemanticsNode tree traversal (graceful degradation when Compose is not present)

Privacy

Autocapture does not capture visible text content ($el_text) from tapped elements. Tracking text can be invasive and raise privacy concerns. The complexity of nested view hierarchies can also cause text extraction to capture content from unintended views — for example, tapping a LinearLayout container might extract text from a deeply nested TextView that isn't semantically related to the tap.

The captured properties ($el_id, $el_tag_name, $attr-aria-label, $attr-role, $elements, $x, $y) are purely structural UI metadata with no PII risk.

Captured Properties

Property	Description
$x, $y	Touch coordinates
$el_id	Element identifier (contentDescription → resource ID → fallback hash)
$el_tag_name	Element class name (e.g., Button, TextView)
$attr-aria-label	Accessibility label (contentDescription)
$attr-role	Semantic role (e.g., Button, Switch, Slider)
$elements	View hierarchy string (max 5 levels)

Architecture

Component	Responsibility
TouchInterceptor	Per-window touch interception via Window.Callback
SemanticExtractor	Element info extraction (XML View + Compose routing)
ComposeSemanticHelper	Compose-specific semantics via RootForTest / SemanticsNode
RageClickTracker	4+ clicks within 1000ms / 44dp radius
DeadClickDetector	No UI change within 500ms (XML: view hash, Compose: semantic snapshot)
AutocaptureManager	Lifecycle coordination, event dispatch
WindowSpy	Root view tracking for dialogs, popups, menus

Tests

• 7 XML autocapture instrumented tests (AutocaptureInstrumentedTest)
• 7 Compose autocapture instrumented tests (ComposeAutocaptureInstrumentedTest)
• All 14 autocapture tests pass on emulator (API 35)

Demo app

• XML and Compose test screens added to mixpaneldemo for manual validation

Hardening (code review fixes)

• Depth-limited recursion (max 20) on all tree traversals to prevent StackOverflowError
• Fixed WeakHashMap iteration in stop() to avoid ConcurrentModificationException
• Fixed WindowSpy.getRootViews() returning stale data
• Fixed ClickEvent.isComposeClick() unreliability from WeakReference GC
• Fixed AccessibilityNodeInfo leak on exception

Test plan

• Run XML tests: ./gradlew :analytics:connectedAndroidTest -Pandroid.testInstrumentationRunnerArguments.class=com.mixpanel.android.mpmetrics.AutocaptureInstrumentedTest
• Run Compose tests: ./gradlew :analytics:connectedAndroidTest -Pandroid.testInstrumentationRunnerArguments.class=com.mixpanel.android.mpmetrics.ComposeAutocaptureInstrumentedTest
• Manual validation using demo app XML and Compose test screens
• Verify existing SDK tests still pass: ./gradlew :analytics:connectedAndroidTest

[tylerjroach]

I haven't finished reviewing the code, but had these questions on autocapture doc

[greptile-apps]

Confidence Score: 3/5

Several correctness issues from previous review rounds remain unresolved, including manager teardown not wired into any MixpanelAPI cleanup path and touch listeners for dialog/popup windows not removed on dismissal.

The core detection pipeline works correctly in the happy path, multi-touch handling was fixed, and null guards were added to AutocaptureOptions.Builder. However, AutocaptureManager.stop() is still never called from MixpanelAPI, so the manager and its lifecycle callbacks leak on any cleanup path. Dialog/popup window touch listeners registered via onRootViewChanged are not cleaned up on dismissal. The accessibility fallback still emits visible text as $el_id and omits $attr-aria-label when a resource ID is present, contradicting the documented privacy and property contracts.

AutocaptureManager.java (stop() not wired to any MixpanelAPI cleanup), SemanticExtractor.java (accessibility fallback text capture + ariaLabel omission), MixpanelAPI.java (no teardown path for mAutocaptureManager).

Important Files Changed

Filename	Overview
analytics/src/main/java/com/mixpanel/android/autocapture/AutocaptureManager.java	Core lifecycle coordinator; stop() is never called from MixpanelAPI (open issue from previous review), and dialog/popup window touch listeners added via onRootViewChanged are not removed on dismissal.
analytics/src/main/java/com/mixpanel/android/autocapture/SemanticExtractor.java	Handles View + Compose semantics extraction; two open issues from previous review: node.getText() used as $el_id (contradicts privacy claim), and $attr-aria-label dropped when viewIdResourceName is present.
analytics/src/main/java/com/mixpanel/android/autocapture/CurtainsHelper.kt	Touch interception; multi-touch false-tap fix applied (ACTION_POINTER_DOWN resets downTime), single-tap filtering looks correct.
analytics/src/main/java/com/mixpanel/android/autocapture/DeadClickDetector.java	Baseline/hash based change detection; depth-limited tree walk for XML; Compose monitors structural hash. AccessibilityNodeInfo lifecycle appears correct.
analytics/src/main/java/com/mixpanel/android/autocapture/ComposeSemanticHelper.java	Compose-specific semantic extraction and snapshot hashing; coordinate conversion to window-relative space is correct; no text captured as element ID (unlike the accessibility fallback path).
analytics/src/main/java/com/mixpanel/android/autocapture/RageClickTracker.java	Time-window + spatial-radius rage click detection; iterator-safe pruning; injectable TimeProvider for testability. No issues found.
analytics/src/main/java/com/mixpanel/android/mpmetrics/MixpanelAPI.java	trackAutocaptureEvent correctly uses 2-arg track() with isAutomaticEvent=false, so events are not gated by trackAutomaticEvents. mAutocaptureManager.stop() still not wired to any cleanup path.
analytics/src/main/java/com/mixpanel/android/mpmetrics/AutocaptureOptions.java	Null guards added to all three Builder setters; defaults all sub-options to enabled. Clean.
analytics/src/main/java/com/mixpanel/android/autocapture/WindowSpy.java	Curtains singleton; static listener list with CopyOnWriteArrayList is thread-safe. uninstall() exists but is never called from AutocaptureManager.stop(), leaving the Curtains hook as a permanent singleton (benign but intentional).
analytics/src/main/java/com/mixpanel/android/autocapture/ClickEvent.java	isCompose flag is now a primitive boolean set at construction (not from WeakReference), fixing the GC reliability issue called out in a previous review thread.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant User
    participant CurtainsHelper
    participant AutocaptureManager
    participant SemanticExtractor
    participant RageClickTracker
    participant DeadClickDetector
    participant MixpanelAPI

    User->>CurtainsHelper: finger down / up (MotionEvent)
    CurtainsHelper->>CurtainsHelper: "filter: single-pointer, slop, <500ms"
    CurtainsHelper->>AutocaptureManager: onTap(x, y, decorView)
    AutocaptureManager->>SemanticExtractor: extract(decorView, x, y)
    SemanticExtractor-->>AutocaptureManager: ClickEvent.Builder
    AutocaptureManager->>MixpanelAPI: trackClick(clickEvent)
    AutocaptureManager->>RageClickTracker: recordClick(clickEvent)
    RageClickTracker-->>AutocaptureManager: rageClick? (4+ in window)
    AutocaptureManager->>MixpanelAPI: trackRageClick(rageClick)
    AutocaptureManager->>DeadClickDetector: startDetection(clickEvent, rootView)
    DeadClickDetector->>DeadClickDetector: captureBaseline (view hash / semantic snapshot)
    Note over DeadClickDetector: wait 500ms
    DeadClickDetector->>DeadClickDetector: hasChanged()?
    DeadClickDetector-->>AutocaptureManager: onDeadClickDetected(clickEvent)
    AutocaptureManager->>MixpanelAPI: trackDeadClick(clickEvent)

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant User
    participant CurtainsHelper
    participant AutocaptureManager
    participant SemanticExtractor
    participant RageClickTracker
    participant DeadClickDetector
    participant MixpanelAPI

    User->>CurtainsHelper: finger down / up (MotionEvent)
    CurtainsHelper->>CurtainsHelper: "filter: single-pointer, slop, <500ms"
    CurtainsHelper->>AutocaptureManager: onTap(x, y, decorView)
    AutocaptureManager->>SemanticExtractor: extract(decorView, x, y)
    SemanticExtractor-->>AutocaptureManager: ClickEvent.Builder
    AutocaptureManager->>MixpanelAPI: trackClick(clickEvent)
    AutocaptureManager->>RageClickTracker: recordClick(clickEvent)
    RageClickTracker-->>AutocaptureManager: rageClick? (4+ in window)
    AutocaptureManager->>MixpanelAPI: trackRageClick(rageClick)
    AutocaptureManager->>DeadClickDetector: startDetection(clickEvent, rootView)
    DeadClickDetector->>DeadClickDetector: captureBaseline (view hash / semantic snapshot)
    Note over DeadClickDetector: wait 500ms
    DeadClickDetector->>DeadClickDetector: hasChanged()?
    DeadClickDetector-->>AutocaptureManager: onDeadClickDetected(clickEvent)
    AutocaptureManager->>MixpanelAPI: trackDeadClick(clickEvent)

Loading

_{Reviews (29): Last reviewed commit: "Align rage click distance with iOS/JS SD..." | Re-trigger Greptile}

[tylerjroach]

Not to overwhelm here but some notable findings to look into from Fable.

1. Autocapture events silently dropped when trackAutomaticEvents=false

MixpanelAPI.java:2204 — The emit callback calls track(eventName, properties, /*isAutomaticEvent=*/true), and track() (line 2805) early-returns on (isAutomaticEvent && !mTrackAutomaticEvents). An app created with getInstance(context, token, false, optionsWithAutocaptureEnabled) starts the full autocapture machinery (interceptors, WindowSpy, timers) yet emits zero events — "Autocapture initialized" is logged and nothing hints at the drop. Autocapture opt-in should gate itself rather than piggyback on the automatic-events flag.

2. WindowSpy mViews swap is an unsynchronized race that can crash the host app

WindowSpy.java:80-84 — install() copies WindowManagerGlobal.mViews into a new list and writes the field back while holding only WindowSpy's own lock (not the framework's mLock), running synchronously on whatever thread calls getInstance(). A root view added by the main thread between the copy and the field write vanishes from mViews; its later removeView throws IllegalArgumentException("View not attached to window manager") in the host app — violating the never-crash rule. This swap technique is only safe when performed on the main thread.

3. Every single-pointer ACTION_UP is treated as a click — scrolls and swipes emit $mp_click

TouchInterceptor.java:107-125 — ACTION_DOWN is never recorded, so there is no touch-slop or duration check, and ACTION_CANCEL/ACTION_MOVE are ignored (no mitigation exists anywhere downstream). Every scroll stroke on a RecyclerView/LazyColumn produces a spurious click on whatever was under the finger at lift-off, and a few quick strokes in one region synthesize a false $mp_rage_click. A slop/duration check at this layer is standard.

4. Responsive buttons falsely reported as dead clicks (XML path)

DeadClickDetector.java:201-224 — The baseline snapshot is taken 150ms after the click, and onGlobalLayout/onScrollChanged explicitly ignore anything before mBaselineCaptured ("expected settling"). A click handler that updates the UI within a frame or two — the common case — has its response absorbed into the baseline; at the 500ms check the tree matches and a false $mp_dead_click fires. The Compose path does this correctly by snapshotting synchronously at click time.

5. No retroactive attach — deferred SDK init captures nothing on the current screen

AutocaptureManager.java:101-122 — start() only registers forward-looking hooks: registerActivityLifecycleCallbacks doesn't replay onActivityResumed for an already-resumed activity, and DelegatingViewList's copy-constructor bulk copy bypasses the overridden add(), so existing root views never fire onRootViewChanged. With deferred init (e.g., after a consent dialog), the foreground activity gets no interceptor until navigation or a config change.

6. Dialogs, popups, and menus are never captured — WindowSpy attaches to nothing

AutocaptureManager.java:339-357 — getWindowFromView() only handles context instanceof Activity directly (no ContextWrapper unwrapping; the "reflection fallback" comment at line 341 has no code behind it). Dialog decor views carry a DecorContext/ContextThemeWrapper, so the method returns null and no interceptor attaches. When the context is the Activity (popups), it returns the activity's already-registered window, so the containsKey guard skips attachment. Net effect: the entire WindowSpy mechanism currently attaches interceptors to nothing.

7. Compose hit-testing mixes coordinate spaces

ComposeSemanticHelper.java:226 — TouchInterceptor captures getRawX()/getRawY() (screen space) and passes them unconverted to findNodeAtPositionRecursive, which tests against SemanticsNode.getBoundsInWindow() (window space). The XML and accessibility paths correctly use screen space (getLocationOnScreen, getBoundsInScreen). For any Compose window not at the screen origin — dialogs, popups, multi-window — the tap misses or attributes to the wrong node.

8. Two Window.Callback methods are not delegated

TouchInterceptor.java:128-281 — onProvideKeyboardShortcuts (API 24) and onPointerCaptureChanged (API 26) are not overridden. Both are default no-ops in the interface, so this compiles silently, but once wrapped those framework notifications never reach the host Activity — breaking the keyboard-shortcuts sheet (Meta+/) and pointer-capture notifications. The existing Api23Helper + animalsniffer-ignore pattern covers this cleanly.

9 $tap_count documented but never emitted

RageClickTracker.java:92-95 — autocapture.md:150 documents $tap_count ("Number of taps in the rage click sequence") on $mp_rage_click, but recordClick clears the click history and returns the triggering event unchanged, and ClickEvent.toProperties() has no tap-count field. The documented property never appears in any emitted event, and once cleared the actual count is unrecoverable.

10. stop() / uninstall() latent defects

TouchInterceptor.java:67-76 — uninstall() only restores the callback when getCallback() == this; if another SDK wrapped the callback after Mixpanel, it silently no-ops and the stale interceptor keeps emitting clicks forever — there is no detached/stopped flag anywhere on the event path. install() is also not idempotent (no check for an
already-installed TouchInterceptor), so a stop/start cycle would double-wrap and double-emit every click. Currently unreachable — nothing in the SDK calls the public stop() — but worth fixing before it becomes reachable.

Autocapture PR Review — Additional Findings (design, performance, cleanup)

Design / architecture

D1. "Disabled by default" is a hand-built template, not a real off switch

MixpanelOptions.java:147-152 — The default is implemented by building an AutocaptureOptions that individually disables each of the three current sub-options, and AutocaptureOptions.isEnabled() is an OR over sub-options. When Phase 2 adds a new sub-option
whose builder defaults to enabled=true (the existing convention in ClickOptions/RageClickOptions/DeadClickOptions), every app that never called autocaptureOptions() silently gets that capture enabled. Fix: a master enabled flag or an
AutocaptureOptions.disabled() factory so "off" is defined once, in the class that owns the sub-options. (flagged independently by 2 agents)

D2. Dead-click detection is two divergent mechanisms instead of one abstraction

DeadClickDetector.java:188 — DetectionSession branches on mIsComposeClick throughout start/captureBaseline/checkResult/cleanup, and the paths already disagree: baselineDelayMs silently doesn't apply to Compose, layout/scroll cancellation signals only exist
for XML, and onWindowFocusChanged() (line 124) is never called by anyone — dead code. Every new change signal or Phase 2 surface must be added to both branches. Fix: a UI-change-monitor interface (captureBaseline / hasChanged / attach / detach) chosen once per
click.

D3. Compose change-detection sensitivity is a hardcoded heuristic

ComposeSemanticHelper.java:136 — The Compose path classifies a click as dead unless the node count changes by more than ±5, bypassing both AutocaptureDefaults and DeadClickOptions, while the XML path uses exact count/hash equality. A Compose click that
adds/removes 1-4 nodes (toggling an icon, revealing a badge) is a false dead click, and there's no option to tune it. Sensitivity should be defined once and honored by both snapshot implementations.

D4. MAX_ACCESSIBILITY_NODES doesn't enforce its documented contract

SemanticExtractor.java:215 — The constant (500, documented as "maximum number of accessibility nodes to probe") is actually used as a recursion depth bound in findNodeAtPosition. A broad-but-shallow tree probes unbounded nodes per tap, while 500 as a depth
limit is far too deep to prevent the StackOverflowError it presumably targets (other traversals cap depth at 20). Fix: a shared visited-node counter for total work, with depth reusing MAX_RECURSION_DEPTH.

D5. WindowSpy install is irreversible and failure is silent-permanent

WindowSpy.java:118 — There is no uninstall path (stop() removes listeners but the swapped list stays forever), sInstalled is set true even when the reflection fails, so degradation is permanent and undetectable by the caller, and DelegatingViewList only
intercepts add/remove — not addAll/clear/set — so coverage depends on OS internals. If another library uses the same mViews-swap technique, whichever installs second wraps the other and neither can be removed. Fix: keep the original list for restore-on-stop,
and surface install failure.

D6. Duplicate ActivityLifecycleCallbacks registration

AutocaptureManager.java:109 — The SDK already registers MixpanelActivityLifecycleCallbacks on the same Application (MixpanelAPI.java:2178); AutocaptureManager registers a second, and with multiple Mixpanel instances each adds another. Dispatching
autocapture's resume/pause/destroy hooks from the existing callbacks would keep one lifecycle path to maintain.

D7. XML hit-testing picks the deepest view, not the clickable target

SemanticExtractor.java:506 — findViewAtPosition returns the deepest visible view with no preference for clickable views or walk-up to a clickable ancestor (unlike the accessibility path). A tap on a clickable container built as ViewGroup + TextView is
attributed to the inner TextView: wrong $el_id, role text instead of button, and isInteractive=false — which also silently skips dead-click detection for that element.

Performance (tap hot path — all on the main thread)

P1. Extraction runs before the app receives the touch event

TouchInterceptor.java:90 — On every ACTION_UP, the full hit test, hierarchy string build, and (for Compose) semantics traversal + snapshot run synchronously before mOriginalCallback.dispatchTouchEvent(event), adding latency between finger-up and the app's

Performance (tap hot path — all on the main thread)

P1. Extraction runs before the app receives the touch event

TouchInterceptor.java:90 — On every ACTION_UP, the full hit test, hierarchy string build, and (for Compose) semantics traversal + snapshot run synchronously before mOriginalCallback.dispatchTouchEvent(event), adding latency between finger-up and the app's click handling. Nothing in the extraction depends on running first — forward the event,
then extract (or post it).

P2. Per-node getLocationOnScreen during hit-test DFS

SemanticExtractor.java:483 — Each visited view allocates an int[2] and calls getLocationOnScreen (which walks the parent chain), making the hit test O(visited × depth) per tap. Converting the tap to root coordinates once and translating incrementally while descending (as ViewGroup.dispatchTouchEvent does) eliminates both costs.

P3. Dead-click snapshots walk the tree twice, twice

DeadClickDetector.java:250 — captureBaseline() and checkResult() each traverse the full hierarchy twice (countViews + computeContentHash) — 4 full traversals per interactive click where 1-2 suffice. Compute count and hash in a single pass (as ComposeSemanticHelper.computeTreeHash already does), or drop countViews since the hash
already reflects count changes.

P4. Per-node allocation in Compose snapshots

ComposeSemanticHelper.java:207 — computeTreeHash allocates an int[2] per semantics node, and the snapshot runs at click time and again at timeout — hundreds of short-lived arrays per interactive click on large screens. Pack count+hash into a single long return, or use one holder per snapshot.

P5. Hot-path debug logs build strings regardless of log level

SemanticExtractor.java:60 (also ComposeSemanticHelper.extract, captureSnapshot, AutocaptureManager.emitEvent) — MPLog.d checks the level inside the call, so concatenation and getSimpleName() run on every tap even at the default WARN level. Guard with a level check or remove per-tap logs.

Dead code / simplification

• AutocaptureManager.java:66 — mCurrentActivityRef is write-only: assigned every onActivityResumed, never read. Delete. (flagged independently by 3 agents)
• WindowSpy.java:128 — getRootViews() and the sOriginalViews field exist only to serve it; zero callers. Delete both. (flagged independently by 2 agents)
• ClickEvent.java:49 — timestamp is set, threaded through the 10-arg constructor, and never serialized or read (RageClickTracker keeps its own timestamps). Drop it.
• DeadClickDetector.java:167 — DetectionSession caches mIsComposeClick/mComposeRootRef duplicating state already available from the mClickEvent it holds (double-wrapping an already-weak reference). Use the ClickEvent accessors.
• ComposeSemanticHelper.java:46 — ExtractResult wrapper + two-value ExtractionResult enum encode exactly "nullable builder". Return @Nullable ClickEvent.Builder and delete both types.
• SemanticExtractor.java:158,183,293 — Dead SDK-version guards for API 16/18; minSdk is 21, so the early-return is unreachable and the gates are always true.
• SemanticExtractor.java:516 / ComposeSemanticHelper.java:303 — The $el_id priority chain (contentDescription > id/testTag > fallback hash) and role mapping are copy-pasted across the XML, accessibility, and Compose paths — three places to edit, already drifting. Extract one shared resolver.

Conventions (per CLAUDE.md)

• AutocaptureOptions.java:91 — Builder setters (clickOptions/rageClickOptions/deadClickOptions) store @NonNull params with no null guard; a Java caller passing null NPEs later in AutocaptureOptions.isEnabled(), which runs in the MixpanelAPI constructor outside any try/catch — reachable host-app crash during getInstance(). The
  sibling MixpanelOptions.Builder.autocaptureOptions added in this PR does guard; mirror it.
• AutocaptureManager.java:57,59 — mRageClickTracker and mDeadClickDetector are assigned only in the constructor but declared non-final ("Final fields for immutability").
• ClickEvent.java:23 — Instance fields (x, y, elementId, …) omit the m prefix used by every other new class in this PR ("Member variables prefixed with 'm'").

[tylerjroach]

Many apps are mixed between compose and xml. Will the change monitor detect a click that goes from a compose view to an xml view since these monitors look to be specific for the view type that came from the click?

Or a better example would be if a compose view is added to an xml view. If a button came from xml but changed the contents inside compose (or vise versa) would we detect this?

[linear-code]

SDK-27