Skip to content

AppKit: Add tablet tool support#4593

Open
bytesandwich wants to merge 2 commits into
rust-windowing:masterfrom
bytesandwich:macos-tablet-tool
Open

AppKit: Add tablet tool support#4593
bytesandwich wants to merge 2 commits into
rust-windowing:masterfrom
bytesandwich:macos-tablet-tool

Conversation

@bytesandwich

Copy link
Copy Markdown
Contributor

Add support to winit-appkit for tablet tools (pen/eraser) with rich inputs like pressure and tilt.

Apple documents that tool type is only available in proximity events (pointingDeviceType is "valid for mouse events with subtype NSTabletProximityEventSubtype and for NSTabletProximity events") so we keep that information cached in the ViewState.

Tilt gets converted from -1,1 to -90,90. y is negated, since AppKit is y-up and winit documents positive y as toward the user.

  • Tested on all platforms changed
    Tested with a toy app that tracks a Wacom tablet pressure (sphere size) and tilt (the vector)
Screenshot 2026-06-09 at 8 41 47 PM
  • Added an entry to the changelog module if knowledge of this change could be valuable to users
  • Updated documentation to reflect any user-facing changes, including notes of platform-specific behavior
  • Created or updated an example program if it would help users understand this functionality

@bytesandwich bytesandwich requested a review from madsmtm as a code owner June 10, 2026 01:27

@kchibisov kchibisov left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just by looking into docs for tablet, it seems that tablet events has a property of

NSTabletPointEventSubtype or NSTabletPoint

Also, how handling works if you hover with tablet over window and also use mouse? is mouse treated as tablet in such case?

Comment thread winit-appkit/src/view.rs Outdated
Comment on lines +1222 to +1231
if device == NSPointingDeviceType::Eraser {
TabletToolKind::Eraser
} else if device == NSPointingDeviceType::Cursor {
// AppKit's `Cursor` is the tablet puck; winit's closest tool kind is `Mouse`.
TabletToolKind::Mouse
} else {
// `Pen` and `Unknown` both map to a pen; it is the most common tool and a
// reasonable default for hardware that does not report a specific type.
TabletToolKind::Pen
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like match

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment thread winit-appkit/src/view.rs Outdated
Comment on lines +1105 to +1117
Some((kind, data)) => ButtonSource::TabletTool {
kind,
button: match button {
// Mirror winit-core's `TabletToolButton` to `MouseButton` conversion table.
MouseButton::Left => TabletToolButton::Contact,
MouseButton::Right => TabletToolButton::Barrel,
MouseButton::Middle => TabletToolButton::Other(1),
MouseButton::Back => TabletToolButton::Other(3),
MouseButton::Forward => TabletToolButton::Other(4),
other => TabletToolButton::Other(other as u16),
},
data,
},

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe extract all of that to tablet button?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@bytesandwich

bytesandwich commented Jun 15, 2026

Copy link
Copy Markdown
Contributor Author

Thank you for the review!

how handling works if you hover with tablet over window and also use mouse? is mouse treated as tablet in such case?

Good catch. mouse_enter/mouse_exited can't reliably know the device (I just checked that .subtype() throws there) so for example if the pen just hovers (and sets the cached ViewState's tablet_tool) and then there are mouse enter/exit events they will report the tablet tool cache value and that's not right.

I changed it to kind=unknown for now, because that seems to be the only 100% correct option and winit is a foundation library.
It could go back to always "Mouse" if you want, but always Mouse isn't right for tablets and actually might cause problems for people who have tablets and mice working around each other.

Besides mouse_enter/mouse_exited though, the mouse isn't treated like the tablet with this PR change. If they are both active at the same time then they both try to move the cursor. They both send the same .type values (like MouseMoved or LeftMouseDown), but the .subtype values are distinct (MouseEvent or TabletPoint) so you can tell which device an event came from. I'll share a log example below.

Just by looking into docs for tablet, it seems that tablet events has a property of NSTabletPointEventSubtype or NSTabletPoint

Right, and I think we need the subtype one here. The tablet NSEvents look the same as mouse events if you only look at .type (NSEventType) but the .subtype field (NSEventSubtype) has the tablet details we want. I think the logs make it a little clearer.

Inline comments addressed

I also addressed the inline comments. The device-type chain is now a match, and the button mapping is extracted into a tablet_button helper.

Here are summarized logs of (NSEvent) event.type and (NSEvent) event.subtype for just my wacom tablet pen being used:

* .type = MouseEntered
* .type = MouseMoved // pen crosses into proximity (near the surface)
	.subtype = TabletProximity (entering=true device=Pen)
* .type = MouseMoved // pen hovering above the surface (force = 0.0)
	.subtype = TabletPoint
* .type = LeftMouseDown // pen makes contact (force > 0)
	.subtype = TabletPoint
* .type = LeftMouseDragged // pen moves while in contact
	.subtype = TabletPoint
* .type = LeftMouseUp // pen lifts off the surface
	.subtype = TabletPoint
* .type = MouseMoved // pen hovering again (force = 0.0)
	.subtype = TabletPoint
* .type = MouseMoved // pen leaves proximity
	.subtype = TabletProximity (entering=false device=Pen)

Here are summarized logs of (NSEvent) event.type and (NSEvent) event.subtype for just a mouse being used:

* .type = MouseEntered (no subtype)
* .type = MouseMoved // moving the mouse (hover, no button)
	.subtype = MouseEvent
* .type = LeftMouseDown // button press
	.subtype = MouseEvent
* .type = LeftMouseDragged // move while button held
	.subtype = MouseEvent
* .type = LeftMouseUp // button release
	.subtype = MouseEvent
* .type = MouseMoved // moving the mouse afterward
	.subtype = MouseEvent

Here are summarized logs of (NSEvent) event.type and (NSEvent) event.subtype for using the mouse and pen at the same time

(hover the pen near the surface (no contact) then use the mouse)

* .type = MouseMoved 
	.subtype=TabletProximity entering=true  device=Pen
* .type = MouseMoved 
	.subtype=TabletPoint (hover sample, force 0)
* .type = MouseMoved 
	.subtype=TabletProximity

// now use the mouse while pen is hovering. Note that mouse appears normal.
* .type = MouseMoved       
	.subtype=MouseEvent
* .type = LeftMouseDown    
	.subtype=MouseEvent
* .type = LeftMouseDragged 
	.subtype=MouseEvent
* .type = LeftMouseUp      
	.subtype=MouseEvent
* .type = MouseMoved       
	.subtype=MouseEvent

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants