Scripts that leverage the HoloLens input features namely Gaze, Gesture and Voice.
Prefabs related to the input features.
Torus shaped basic cursor that follows the user's gaze around.
Torus shaped CursorOnHolograms when user is gazing at holograms and point light CursorOffHolograms when user is gazing away from holograms.
Torus shaped cursor that follows the user's gaze and HandDetectedFeedback asset to give feedback to user when their hand is detected in the ready state.
Keyword manager pre-wired to send messages to object being currently focused via FocusedObjectMessageSender component. You can simply drop this into your scene and be able to send arbitrary messages to currently focused object.
Keyword manager pre-wired to send messages to object being currently selected via SelectedObjectMessageSender comoponent. You can simply drop this into your scene and be able to send arbitrary messages to currently selected object.
Scripts related to the input features.
- Decides when to show the cursor.
- Positions the cursor at the gazed hit location.
- Rotates the cursor to match hologram normals.
CursorFeedback class takes GameObjects to give cursor feedback to users based on different states.
CursorManager class takes Cursor GameObjects. One that is a Cursor on Holograms and another Cursor off Holograms.
- Shows the appropriate Cursor when a hologram is hit.
- Places the appropriate Cursor at the hit position.
- Matches the Cursor normal to the hit surface.
You must provide GameObjects for the CursorOnHologram and CursorOffHologram public fields.
CursorOnHologram Cursor object to display when you are gazing at a hologram.
CursorOffHologram Cursor object to display when you are not gazing at a hologram.
DistanceFromCollision Distance, in meters, to offset the cursor from a collision with a hologram in the scene. This is to prevent the cursor from being occluded.
Perform Physics.Raycast in the user's gaze direction to get the position and normals of any collisions.
MaxGazeDistance The maximum distance to Raycast. Any holograms beyond this value will not be raycasted to.
RaycastLayerMask The Unity layers to raycast against. If you have holograms that should not be raycasted against, like a cursor, do not include their layers in this mask.
Stabilize the user's gaze to account for head jitter.
StoredStabilitySamples Number of samples that you want to iterate on. A larger number will be more stable.
PositionDropOffRadius Position based distance away from gravity well.
DirectionDropOffRadius Direction based distance away from gravity well.
PositionStrength Position lerp interpolation factor.
DirectionStrength Direction lerp interpolation factor.
StabilityAverageDistanceWeight Stability average weight multiplier factor.
StabilityVarianceWeight Stability variance weight multiplier factor.
GestureManager creates a gesture recognizer and signs up for a tap gesture. When a tap gesture is detected, GestureManager uses GazeManager to find the game object. GestureManager then sends a message to that game object.
It also has an OverrideFocusedObject which lets you send gesture input to a specific object by overriding the gaze.
Show a GameObject when a gesturing hand is about to lose tracking.
You must provide GameObjects for the Cursor and HandGuidanceIndicator public fields.
Cursor The object in your scene that is being used as the cursor. The hand guidance indicator will be rendered around this cursor.
HandGuidanceIndicator GameObject to display when your hand is about to lose tracking.
HandGuidanceThreshold When to start showing the HandGuidanceIndicator. 1 is out of view, 0 is centered in view.
Keeps track of when the user's hand has been detected in the ready position.
Allows you to specify keywords and methods in the Unity Inspector, instead of registering them explicitly in code.
IMPORTANT: Please make sure to add the microphone capability in your app, in Unity under
Edit -> Project Settings -> Player -> Settings for Windows Store -> Publishing Settings -> Capabilities
or in your Visual Studio Package.appxmanifest capabilities.
KeywordsAndResponses Set the size as the number of keywords you'd like to listen for, then specify the keywords and method responses to complete the array.
RecognizerStart Set this to determine whether the keyword recognizer will start immediately or if it should wait for your code to tell it to start.
Sends Unity message to currently focused object. FocusedObjectMessageSender.SendMessageToFocusedObject needs to be registered as a response in KeywordManager to enable arbitrary messages to be sent to currently focused object.
Sends Unity message to currently selected object. SelectedObjectMessageSender.SendMessageToSelectedObject needs to be registered as a response in KeywordManager to enable arbitrary messages to be sent to currently selected object.
Example on how to handle messages send by FocusedObjectMessageSender. In this particular implementation, focused object color it toggled on gaze enter/exit events.
Example on how to handle messages send by SelectedObjectMessageSender. In this particular implementation, selected object color it toggled on selecting object and clearing selected object.
A grid of dynamic objects to illustrate sending messages to prefab instances created at runtime as opposed to only static objects that already exist in the scene.
Tests related to the input features. To use the scene:
- Navigate to the Tests folder.
- Double click on the test scene you wish to explore.
- Either click "Play" in the unity editor or File -> Build Settings.
- Add Open Scenes, Platform -> Windows Store, SDK -> Universal 10, Build Type -> D3D, Check 'Unity C# Projects'.
- Click 'Build' and create an App folder. When compile is done, open the solution and deploy to device.
Shows the basic cursor following the user's gaze and hugging the test sphere in the scene.
Shows the cursor on holograms hugging the test sphere in the scene and cursor off holograms when not gazing at the sphere.
Shows the cursor hugging the test sphere in the scene and displays hand detected asset when hand is detected in ready state.
Example on how to send keyword messages to currently focused dynamically instantiated object. Gazing on an object and saying "Make Smaller" and "Make Bigger" will adjust object size.
Example on how to send keyword messages to currently selected dynamically instantiated object. Gazing on an object and saying "Select Object" will persistently select that object for interaction with voice commands, after which the user can also adjust object size with "Make Smaller" and "Make Bigger" voice commands and finally clear currently selected object by saying "Clear Selection".