Manus plugin: HMD hand tracking, installer, docs, and visualizer

docs/source/device/index.rst

@@ -20,3 +20,4 @@ See the `Plugins directory <https://github.com/NVIDIA/IsaacTeleop/tree/main/src/
    trackers
    add_device
    body_tracking
+   manus

docs/source/device/manus.rst

@@ -0,0 +1,183 @@
+..
+   SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+   SPDX-License-Identifier: Apache-2.0
+
+Manus Gloves
+============
+
+A Linux-only plugin for integrating `Manus <https://www.manus-meta.com/>`_ gloves
+into the Isaac Teleop framework. It provides full hand-joint tracking via the
+Manus SDK and injects the resulting poses into the OpenXR hand-tracking layer so
+any downstream retargeter can consume them transparently.
+
+Components
+----------
+
+- **Core library** (``manus_plugin_core``) — wraps the Manus SDK
+  (``libIsaacTeleopPluginsManus.so``) and exposes per-joint tracking data.
+- **Plugin executable** (``manus_hand_plugin``) — the main plugin binary that
+  integrates with the Teleop system via CloudXR / OpenXR.
+- **CLI tool** (``manus_hand_tracker_printer``) — a standalone diagnostic tool
+  that prints tracked joint data to the terminal and opens a real-time
+  **MANUS Data Visualizer** window showing the hand skeleton from two orthographic
+  views per hand.
+
+Prerequisites
+-------------
+
+- **Linux** — x86_64 (tested on Ubuntu 22.04 / 24.04).
+- **Manus SDK** for Linux — downloaded automatically by the install script.
+- **System dependencies** — the install script installs required packages automatically.
+
+Installation
+------------
+
+Automated (recommended)
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+The install script handles SDK download, dependency installation, and building:
+
+.. code-block:: bash
+
+   cd src/plugins/manus
+   ./install_manus.sh
+
+The script will:
+
+1. Install the required system packages for MANUS Core Integrated.
+2. Download MANUS SDK v3.1.1.
+3. Extract and place the SDK in the correct location.
+4. Build the plugin and the diagnostic tool
+
+Manual
+~~~~~~
+
+If you prefer to install manually:
+
+1. Download the MANUS Core SDK from
+   `MANUS Downloads <https://docs.manus-meta.com/3.1.1/Resources/>`_.
+2. Extract and place the ``ManusSDK`` folder inside ``src/plugins/manus/``, or
+   point CMake at a different path by setting ``MANUS_SDK_ROOT``.
+3. Follow the
+   `MANUS Getting Started guide for Linux <https://docs.manus-meta.com/3.1.1/Plugins/SDK/Linux/>`_
+   to install the dependencies and configure device permissions.
+
+Expected directory layout after placing the SDK:
+
+.. code-block:: text
+
+   src/plugins/manus/
+     app/
+       main.cpp
+     core/
+       manus_hand_tracking_plugin.cpp
+     inc/
+       core/
+         manus_hand_tracking_plugin.hpp
+     tools/
+       manus_hand_tracker_printer.cpp
+     ManusSDK/        <-- placed here
+       include/
+       lib/
+
+Then build from the root:
+
+.. code-block:: bash
+
+   cd ../../..  # navigate to root
+   cmake -S . -B build
+   cmake --build build --target manus_hand_plugin manus_hand_tracker_printer -j
+   cmake --install build --component manus
+
+Running the Plugin
+------------------
+
+1. Set up the CloudXR environment
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Source the CloudXR environment and start the runtime before running the plugin:
+
+.. code-block:: bash
+
+   export NV_CXR_RUNTIME_DIR=~/.cloudxr/run
+   export XR_RUNTIME_JSON=~/.cloudxr/openxr_cloudxr.json
+
+2. Verify with the CLI tool
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Verify that the gloves are working using the CLI tool:
+
+.. code-block:: bash
+
+   ./build/bin/manus_hand_tracker_printer
+
+The tool prints joint positions to the terminal and opens a **MANUS Data
+Visualizer** window showing a top-down and side view of each hand.
+
+3. Run the plugin
+~~~~~~~~~~~~~~~~~~
+
+The plugin is installed to the `install` directory, please ensure the CLI tool is not running when running the plugin.
+
+.. code-block:: bash
+
+   ./install/plugins/manus/manus_hand_plugin
+
+Wrist Positioning — Controllers vs Optical Hand Tracking
+---------------------------------------------------------
+
+Two sources are available for positioning the Manus gloves in 3D space:
+
+- **Controller adapters** — attach Quest 3 controllers to the Manus Universal
+  Mount on the back of the glove. The controller pose drives wrist placement.
+- **Optical hand tracking** — use the HMD's built-in optical hand tracking to
+  position the hands. No physical controller adapter required.
+
+The plugin selects the source automatically at runtime: optical hand tracking is
+preferred when ``XR_MNDX_xdev_space`` is supported and the runtime reports an
+actively tracked wrist pose; otherwise it falls back to the controller pose.
+
+.. note::
+
+   When using controller adapters it is advisable to disable the HMD's automatic
+   hand-tracking–to–controller switching to avoid unexpected source changes
+   mid-session.
+
+Troubleshooting
+---------------
+
+.. list-table::
+   :widths: 40 60
+   :header-rows: 1
+
+   * - Symptom
+     - Resolution
+   * - SDK download fails
+     - Check your internet connection and re-run the install script.
+   * - Manus SDK not found at build time
+     - With manual installation, ensure ``ManusSDK`` is inside
+       ``src/plugins/manus/`` or set ``MANUS_SDK_ROOT`` to your installation path.
+   * - Manus SDK not found at runtime
+     - The build configures RPATH automatically. If you moved the SDK after
+       building, set ``LD_LIBRARY_PATH`` to its ``lib/`` directory.
+   * - No data received
+     - Ensure Manus Core is running and the gloves are connected and calibrated.
+   * - CloudXR runtime errors
+     - Make sure ``scripts/setup_cloudxr_env.sh`` has been sourced before running.
+   * - Permission denied for USB devices
+     - The install script configures udev rules. If the rules were not reloaded,
+       run:
+
+       .. code-block:: bash
+
+          sudo udevadm control --reload-rules
+          sudo udevadm trigger
+
+       Then reconnect your Manus devices.
+
+License
+-------
+
+Source files are covered by their stated licenses (Apache-2.0). The Manus SDK is
+proprietary to Manus and is subject to its own license; it is **not** redistributed
+by this project.

examples/native_openxr/xdev_list/main.cpp

@@ -1,4 +1,4 @@
-// SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 // SPDX-License-Identifier: Apache-2.0
 
 #include "common/main_helpers.hpp"

examples/oxr/cpp/oxr_session_sharing.cpp

@@ -1,4 +1,4 @@
-// SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 // SPDX-License-Identifier: Apache-2.0
 
 #include <deviceio_session/deviceio_session.hpp>

src/core/oxr_utils/cpp/inc/oxr_utils/oxr_funcs.hpp

@@ -1,4 +1,4 @@
-// SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 // SPDX-License-Identifier: Apache-2.0
 
 #pragma once
@@ -24,6 +24,10 @@ extern "C"
     XRAPI_ATTR XrResult XRAPI_CALL xrGetInstanceProcAddr(XrInstance instance,
                                                          const char* name,
                                                          PFN_xrVoidFunction* function);
+    XRAPI_ATTR XrResult XRAPI_CALL xrEnumerateInstanceExtensionProperties(const char* layerName,
+                                                                          uint32_t propertyCapacityInput,
+                                                                          uint32_t* propertyCountOutput,
+                                                                          XrExtensionProperties* properties);
 }
 
 namespace core

src/core/schema/python/pedals_bindings.h

@@ -1,4 +1,4 @@
-// SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 // SPDX-License-Identifier: Apache-2.0
 
 // Python bindings for the Pedals FlatBuffer schema.

src/plugins/manus/70-manus-hid.rules

@@ -0,0 +1,5 @@
+# HIDAPI/libusb
+SUBSYSTEMS=="usb", ATTRS{idVendor}=="3325", MODE:="0666"
+SUBSYSTEMS=="usb", ATTRS{idVendor}=="1915", ATTRS{idProduct}=="83fd", MODE:="0666"
+# HIDAPI/hidraw
+KERNEL=="hidraw*", ATTRS{idVendor}=="3325", MODE:="0666"

src/plugins/manus/CMakeLists.txt

@@ -107,4 +107,5 @@ add_subdirectory(tools)
 # Install Manus SDK shared library
 install(FILES "${_MANUS_LIB}"
     DESTINATION ${CMAKE_INSTALL_LIBDIR}
+    COMPONENT manus
 )

src/plugins/manus/README.md

@@ -1,5 +1,5 @@
 <!--
-SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 SPDX-License-Identifier: Apache-2.0
 -->
 
@@ -11,20 +11,38 @@ This folder provides a Linux-only example of using the Manus SDK for hand tracki
 
 - **Core Library** (`manus_plugin_core`): Interfaces with the Manus SDK (`libIsaacTeleopPluginsManus.so`).
 - **Plugin Executable** (`manus_hand_plugin`): The main plugin executable that integrates with the Teleop system.
-- **CLI Tool** (`manus_hand_tracker_printer`): A standalone tool to print tracked joint data for verification.
+- **CLI Tool** (`manus_hand_tracker_printer`): A standalone tool that prints tracked joint data to the terminal **and** opens a real-time Vulkan visualizer window ("MANUS Data Visualizer") showing the hand skeleton from two orthographic views.
 
 ## Prerequisites
 
-- **Linux** (x86_64, tested on Ubuntu 22.04)
-- **Manus SDK** for Linux x86_64
+- **Linux** (x86_64 tested on Ubuntu 22.04/24.04)
+- **Manus SDK** for Linux (automatically downloaded by install script)
+- **System dependencies**: The install script installs required packages automatically
 
-## Getting the Manus SDK
+## Installation
 
-The Manus SDK must be downloaded separately due to licensing.
+### Automated Installation (Recommended)
 
-1. Obtain a Manus account and credentials.
-2. Download the MANUS Core SDK from [Manus Downloads](https://my.manus-meta.com/resources/downloads).
-3. Extract and place the `ManusSDK` folder inside `src/plugins/manus/`, or set the `MANUS_SDK_ROOT` environment variable to your installation path.
+Use the provided installation script which handles SDK download, dependency installation, and building:
+
+```bash
+cd src/plugins/manus
+./install_manus.sh
+```
+
+The script will:
+1. Install required system packages for MANUS Core Integrated
+2. Automatically download the MANUS SDK v3.1.1
+3. Extract and configure the SDK in the correct location
+4. Build the plugin
+
+### Manual Installation
+
+If you prefer to install manually:
+
+1. Download the MANUS Core SDK from [MANUS Downloads](https://docs.manus-meta.com/3.1.1/Resources/)
+2. Extract and place the `ManusSDK` folder inside `src/plugins/manus/`, or set the `MANUS_SDK_ROOT` environment variable to your installation path
+3. Follow the [MANUS Getting Started guide for Linux](https://docs.manus-meta.com/3.1.1/Plugins/SDK/Linux/) to install the dependencies and setup device permissions.
 
 Expected layout:
 ```text
@@ -43,41 +61,64 @@ src/plugins/manus/
     lib/
 ```
 
-## Build
-
-This plugin is built as part of the Isaac Teleop project. It will be automatically included if the Manus SDK is found.
-
-Run the project build from the repository root:
+4. Build from the TeleopCore root:
 
 ```bash
+cd ../../..  # Navigate to TeleopCore root
 cmake -S . -B build
-cmake --build build -j
+cmake --build build --target manus_hand_plugin manus_hand_tracker_printer -j
+cmake --install build --component manus
 ```
 
-If the SDK is not found, the build will skip the Manus plugin with a warning.
+## Running the Plugin
 
-## Running the Example
+### 1. Setup CloudXR Environment
+Before running the plugin, ensure CloudXR environment is configured:
 
-### 1. Verify with CLI Tool
-Ensure Manus Core is running and gloves are connected.
-Run the printer tool to verify data reception:
+The following environment variables must be set before running either the CLI tool or the plugin (adjust paths if your CloudXR installation differs from the defaults):
+
+```bash
+export NV_CXR_RUNTIME_DIR=~/.cloudxr/run
+export XR_RUNTIME_JSON=~/.cloudxr/openxr_cloudxr.json
+```
+
+### 2. Verify with CLI Tool
+Verify that the gloves are working using the CLI tool:
 
 ```bash
 ./build/bin/manus_hand_tracker_printer
 ```
 
-### 2. Run the Plugin
-The plugin is installed to the `install` directory.
+The tool prints joint positions to the terminal and opens a **MANUS Data Visualizer** window with a top-down and side view of each hand.
+
+### 3. Run the Plugin
+The plugin is installed to the `install` directory, please ensure the CLI tool is not running when running the plugin.
 
 ```bash
 ./install/plugins/manus/manus_hand_plugin
 ```
 
+## Controller positioning vs Optical hand tracking positioning
+To position the MANUS gloves in 3D space two avenues are available:
+
+- Use the MANUS Quest 3 controller adapters to attach the Quest 3 controllers to the MANUS Universal Mount on the back of the glove.
+- Use the HMD's optical hand tracking to position the hands.
+
+The system will switch dynamically based on the available tracking source. When using controllers it's advised to turn off hand tracking entirely or turn off automatic switching.
+
 ## Troubleshooting
 
-- **Manus SDK not found at build time**: Ensure `ManusSDK` is in `src/plugins/manus/` or `MANUS_SDK_ROOT` is set correctly.
-- **Manus SDK not found at runtime**: The build configures RPATH to find the SDK libraries. If you moved the SDK or built without RPATH support, you may need to set `LD_LIBRARY_PATH`.
-- **No data available**: Ensure Manus Core is running and gloves are properly connected and calibrated.
+- **SDK download fails**: Check your internet connection and try running the install script again
+- **Manus SDK not found at build time**: If using manual installation, ensure `ManusSDK` is in `src/plugins/manus/` or `MANUS_SDK_ROOT` is set correctly
+- **Manus SDK not found at runtime**: The CMake build configures RPATH to find the SDK libraries. If you moved the SDK, you may need to set `LD_LIBRARY_PATH`
+- **No data available**: Ensure Manus Core is running and gloves are properly connected and calibrated
+- **CloudXR runtime errors**: Make sure you've sourced `scripts/setup_cloudxr_env.sh` before running the plugin
+- **Permission denied for USB devices**: The install script configures udev rules. You may need to run:
+  ```bash
+  sudo udevadm control --reload-rules
+  sudo udevadm trigger
+  ```
+  Then reconnect your Manus devices.
 
 ## License
 

src/plugins/manus/app/CMakeLists.txt

@@ -26,8 +26,10 @@ set_target_properties(manus_hand_plugin PROPERTIES
 
 install(TARGETS manus_hand_plugin
     RUNTIME DESTINATION plugins/manus
+    COMPONENT manus
 )
 
 install(FILES "${CMAKE_CURRENT_SOURCE_DIR}/../plugin.yaml" "${CMAKE_CURRENT_SOURCE_DIR}/../README.md"
     DESTINATION plugins/manus
+    COMPONENT manus
 )

src/plugins/manus/core/CMakeLists.txt

@@ -38,4 +38,5 @@ set_target_properties(manus_plugin_core PROPERTIES
 install(TARGETS manus_plugin_core
     LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
     RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}
+    COMPONENT manus
 )