Skip to content

[OvPhysX] Add JointWrenchSensor backend#5706

Open
AntoineRichard wants to merge 14 commits into
isaac-sim:developfrom
AntoineRichard:antoiner/feat/ovphysx_joint_wrench
Open

[OvPhysX] Add JointWrenchSensor backend#5706
AntoineRichard wants to merge 14 commits into
isaac-sim:developfrom
AntoineRichard:antoiner/feat/ovphysx_joint_wrench

Conversation

@AntoineRichard

Copy link
Copy Markdown
Collaborator

Description

Adds the OVPhysX backend for isaaclab.sensors.JointWrenchSensor, so that scenes running under OvPhysxManager get a working joint reaction wrench sensor that exposes force [N] and torque [N·m] in the child-side joint frame (incoming-joint-frame convention) — the same contract the PhysX and Newton backends already provide. With this PR the joint-wrench sensor stack is unified across all three backends.

Highlights:

  • New isaaclab_ovphysx.sensors.JointWrenchSensor + JointWrenchSensorData. The sensor owns an OVPhysX LINK_INCOMING_JOINT_FORCE TensorBinding, runs a Warp split kernel that mirrors the PhysX one (OVPhysX wraps PhysX, so the source wrench is in the same convention), and discovers per-body child-side joint frames via the same USD walk as the PhysX backend.
  • Removes isaaclab_ovphysx.assets.ArticulationData.body_incoming_joint_wrench_b to align with the PhysX and Newton backends, which already migrated their joint-wrench accessor onto the dedicated sensor. The changelog fragment includes the migration text users will need.
  • Adds a full real-backend test file that mirrors the PhysX joint-wrench tests one-for-one (only the fixtures, sim_context helper, and raw-tensor read helper change — physical assertions are byte-identical so both backends report the same convention).

Fixes # (issue)

Type of change

  • New feature (non-breaking change which adds functionality)
  • Breaking change (removes the OVPhysX-only ArticulationData.body_incoming_joint_wrench_b accessor — matches the PhysX/Newton migration; users move to JointWrenchSensorCfg)
  • Documentation update (auto-generated docs picked up the new sensor symbols via ./isaaclab.sh -d)

Screenshots

N/A — backend addition with no UI surface.

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package (do not edit CHANGELOG.rst or bump extension.toml — CI handles that)
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

Test plan

Ran the new source/isaaclab_ovphysx/test/sensors/test_joint_wrench_sensor.py one test at a time on both devices (per the OVPhysX wheel's process-global device-mode lock — see the _ovphysx_skip_other_device autouse fixture):

  • 11 tests on CPU: all pass
  • 10 device-parametrized tests on CUDA (cuda:0): all pass

Tests exercise:

  • Pre-init force/torque contract returning None.
  • Buffer shapes and body_names ordering for single-joint, cartpole, and Ant (nested articulation root) scenes.
  • Physical correctness in static rest, with external force/torque, on interior joints, and through a non-identity child-side joint frame.
  • Reset semantics (full + partial via env_ids).
  • Sensor __str__.

Also verified ./isaaclab.sh -f clean and ./isaaclab.sh -d builds the docs with the new symbols picked up.

The sensor was computing num_envs as binding.count // body_count.
For a LINK_INCOMING_JOINT_FORCE tensor binding, .count is already
the number of articulation instances (one per env) -- the division
made num_envs collapse to 1 even with 2+ envs, so the float32 alias
view (sized from binding.shape) overran the spatial_vectorf scratch
buffer (sized from the truncated num_envs). The wheel then read
garbage into freed heap, corrupting state that physx.reset() walked
on shutdown and aborting.

Take binding.count directly, mirroring how OVPhysX Articulation
reads sample.count as num_instances. Restore the symmetric teardown
that drops the float32 alias view before the spatial_vectorf buffer
it aliases.
The previous helper read into sensor._wrench_buf and returned a torch
view of it.  When the caller then accessed sensor.data, the sensor's
own _update_buffers_impl re-read the binding into _wrench_buf,
silently mutating the snapshot the caller had just obtained.

For PhysX this was harmless because ArticulationView.get_link_incoming_joint_force
returns a freshly-allocated wp.array per call.  Under OVPhysX the
binding writes directly into the shared scratch.  In CUDA Ant runs
with large gear-ratio forces, the second read produced last-bit
different values from the first, so the kernel transform and the
torch transform of 'raw_wrench' compared two slightly different
inputs and torch.testing.assert_close tripped its relative tolerance.

Allocate an independent scratch buffer in the helper to mirror the
PhysX semantics.
@github-actions github-actions Bot added the isaac-lab Related to Isaac Lab team label May 20, 2026

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Isaac Lab Review Bot

Summary

This PR adds the OVPhysX backend for JointWrenchSensor, unifying the joint-wrench sensor stack across PhysX, Newton, and OVPhysX backends. The implementation follows established patterns from the existing PhysX joint-wrench sensor closely.


📋 Architecture & Design

Strengths:

  • Clean separation of concerns with JointWrenchSensor, JointWrenchSensorData, and kernels.py
  • Factory pattern integration via __backend_name__ = "ovphysx" is consistent with existing backends
  • Proper use of TensorBinding for reading incoming joint wrenches from OVPhysX
  • USD joint-frame discovery via UsdPhysics.Joint mirrors the PhysX implementation

Observations:

  • The child-side joint frame transform logic in joint_wrench_split_kernel correctly shifts torque to the joint anchor and rotates into the joint frame
  • Breaking change (removing ArticulationData.body_incoming_joint_wrench_b) is properly documented in the changelog fragment with migration guidance

🔍 Code Quality

joint_wrench_sensor.py:

  • Line 135-139: The binding count validation and num_envs reconciliation logic handles the OVPhysX clone_usd=False quirk appropriately
  • The _resolve_articulation_root_prim_path method has robust error handling for missing prims and multiple articulation roots
  • The _invalidate_initialize_callback properly cleans up resources in the correct order (alias view before the buffer it aliases)

kernels.py:

  • Warp kernels are correctly decorated and use appropriate dtype annotations
  • The joint_wrench_split_kernel correctly applies quat_rotate_inv for the body-to-joint frame transform

joint_wrench_sensor_data.py:

  • Lazy ProxyArray initialization pattern is consistent with other sensor data classes
  • Clear docstrings documenting shape, dtype, and None-before-init behavior

🧪 Test Coverage

Excellent coverage including:

  • Pre-init None contract validation
  • Buffer shapes and body name ordering across single-joint, cartpole, and nested articulation root (Ant) scenes
  • Physical correctness: static rest, external force/torque, interior joints, non-identity joint frames
  • Reset semantics (full and partial via env_ids)
  • __str__ representation

Note on device-lock fixture:

  • The _ovphysx_skip_other_device autouse fixture is a pragmatic workaround for OVPhysX's process-global device binding limitation

💡 Suggestions (Non-blocking)

  1. Type annotations: Consider adding explicit return type hints on _create_joint_frame_buffers (currently returns None)

  2. Error message clarity: In _update_buffers_impl (lines 247-253), the error messages could include the sensor's current initialization state to aid debugging

  3. Test docstring consistency: Some test functions have detailed docstrings while others have brief ones — consider standardizing


📊 CI Status

CI checks are in progress. Please ensure all checks pass before merging.


✅ Verdict

This is a well-structured implementation that follows established patterns and includes comprehensive test coverage. The breaking change is documented with clear migration guidance. No blocking issues identified.


🔍 Reviewed commit: bdeaa737eaca93adb0a87dc65613460939ce002d

@greptile-apps

greptile-apps Bot commented May 20, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds the OVPhysX backend for JointWrenchSensor, completing sensor parity across all three physics backends (PhysX, Newton, OVPhysX). It also removes ArticulationData.body_incoming_joint_wrench_b from the OVPhysX articulation, aligning it with the PhysX and Newton backends that already migrated to the dedicated sensor.

  • New isaaclab_ovphysx.sensors.JointWrenchSensor reads a LINK_INCOMING_JOINT_FORCE OVPhysX TensorBinding, runs a Warp split kernel to shift the torque from body origin to the child-side joint anchor and rotate both components into the child-side joint frame — same convention as PhysX.
  • ArticulationData.body_incoming_joint_wrench_b is removed with a migration note pointing users to JointWrenchSensorCfg; the factory class in isaaclab.sensors is updated to include the OVPhysX types in its return-type unions.
  • A full real-backend test suite mirrors the PhysX tests (11 CPU, 10 CUDA tests) covering shapes, body ordering, physical correctness, reset semantics, non-identity joint frames, and the OVPhysX process-global device-lock constraint.

Confidence Score: 4/5

The change is safe to merge; the new sensor correctly mirrors the PhysX backend convention and all three critical paths (initialization, update, reset) have appropriate guards and are covered by the test suite.

The kernel math, quaternion convention, and buffer lifetime management are correct. The one notable gap is that _create_joint_frame_buffers silently uses an identity transform when a USD joint's body1 leaf name doesn't match an OVPhysX body name — no log or warning is emitted, so a naming-convention mismatch would produce wrong per-body joint frames without any observable signal at startup.

source/isaaclab_ovphysx/isaaclab_ovphysx/sensors/joint_wrench/joint_wrench_sensor.py — specifically the _create_joint_frame_buffers silent-fallback path around line 226.

Important Files Changed

Filename Overview
source/isaaclab_ovphysx/isaaclab_ovphysx/sensors/joint_wrench/joint_wrench_sensor.py New OVPhysX JointWrenchSensor — initialization, USD traversal, binding setup, and buffer lifecycle all look correct; minor silent-failure risk on name mismatch in _create_joint_frame_buffers.
source/isaaclab_ovphysx/isaaclab_ovphysx/sensors/joint_wrench/kernels.py Warp split and reset kernels; torque-shift formula and quaternion rotation are mathematically correct and consistent with the test helper.
source/isaaclab_ovphysx/isaaclab_ovphysx/sensors/joint_wrench/joint_wrench_sensor_data.py Data container with lazy ProxyArray caching; force/torque return None before create_buffers is called, satisfying the pre-init contract.
source/isaaclab_ovphysx/test/sensors/test_joint_wrench_sensor.py Comprehensive real-backend test suite; covers shapes, body names, reset semantics, external forces, non-identity joint frames, and OVPhysX device-lock. Physical assertions validate consistency against the raw OVPhysX tensor rather than independently derived ground-truth values.
source/isaaclab/isaaclab/sensors/joint_wrench/joint_wrench_sensor.py Factory class updated to include OvPhysxJointWrenchSensor in the return-type union and the data class union — mechanical, no logic change.

Sequence Diagram

sequenceDiagram
    participant User
    participant JointWrenchSensor
    participant OvPhysxManager
    participant TensorBinding
    participant WarpKernel

    User->>JointWrenchSensor: sim.reset() → _initialize_impl()
    JointWrenchSensor->>OvPhysxManager: get_physx_instance()
    OvPhysxManager-->>JointWrenchSensor: physx_instance
    JointWrenchSensor->>TensorBinding: create_tensor_binding(pattern, LINK_INCOMING_JOINT_FORCE)
    TensorBinding-->>JointWrenchSensor: binding (body_count, body_names, shape)
    JointWrenchSensor->>JointWrenchSensor: _create_joint_frame_buffers() [USD walk → joint_pos_b / joint_quat_b]
    JointWrenchSensor->>JointWrenchSensor: allocate _wrench_buf + float32 alias view

    User->>JointWrenchSensor: scene.update() → _update_buffers_impl(env_mask)
    JointWrenchSensor->>TensorBinding: binding.read(_wrench_read_view)
    TensorBinding-->>JointWrenchSensor: raw spatial wrenches (body frame, body origin)
    JointWrenchSensor->>WarpKernel: joint_wrench_split_kernel(wrench_buf, joint_pos_b, joint_quat_b)
    WarpKernel-->>JointWrenchSensor: force/torque in child-side joint frame

    User->>JointWrenchSensor: sensor.data.force / .torque
    JointWrenchSensor-->>User: ProxyArray (wp.vec3f, shape num_envs×num_bodies)
Loading

Reviews (1): Last reviewed commit: "Read raw wrench into an independent scra..." | Re-trigger Greptile

Comment on lines +217 to +228
link_name_to_index = {name: index for index, name in enumerate(self._data._body_names)}

for prim in Usd.PrimRange(first_env_matching_prim):
joint = UsdPhysics.Joint(prim)
if not joint or joint.GetJointEnabledAttr().Get() is False:
continue
body1_targets = joint.GetBody1Rel().GetTargets()
if len(body1_targets) == 0:
continue
body_index = link_name_to_index.get(body1_targets[0].name)
if body_index is None:
continue

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Silent identity fallback on unmatched body name

When link_name_to_index.get(body1_targets[0].name) returns None, the joint frame for that body silently remains as identity and the joint USD entry is quietly skipped. If OVPhysX body names and USD leaf path names diverge (e.g., naming convention differences across assets), every affected body would report wrenches in the body frame rather than the child-side joint frame, with no visible indication. Adding a logger.debug (or at least logger.warning) when body_index is None would surface such mismatches at initialization time.

@marcodiiga marcodiiga left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking finding from reviewing this against current develop:

source/isaaclab_ovphysx/isaaclab_ovphysx/sensors/joint_wrench/kernels.py:19-31 splits whatever the OVPhysX binding returns whenever env_mask[env] is true, and _update_buffers_impl() reads the binding and launches that kernel at source/isaaclab_ovphysx/isaaclab_ovphysx/sensors/joint_wrench/joint_wrench_sensor.py:261-272 without passing the sensor timestamp. The PhysX backend has an explicit timestamp[env] == 0.0 guard at source/isaaclab_physx/isaaclab_physx/sensors/joint_wrench/kernels.py:24-27, with a public reset regression at source/isaaclab_physx/test/sensors/test_joint_wrench_sensor.py:430-456.

That guard matters after scene.reset(env_ids): the sensor reset zeros the Isaac Lab buffers and timestamps, but the native wrench buffer can still contain the previous step's values until a new physics step runs. A public sensor.data read in that window will lazy-refetch the stale OVPhysX binding and repopulate the reset env with non-zero force/torque.

Please mirror the timestamp guard in the OVPhysX kernel/launch and add the same public reset regression for the OVPhysX test (scene.reset(env_ids) followed by sensor.data before another sim step).

@marcodiiga

Copy link
Copy Markdown

This PR is currently conflicting with latest develop in source/isaaclab_ovphysx/test/assets/test_articulation.py (git merge-tree --write-tree origin/develop origin/pr/5706 reports a content conflict). Can you rebase before the next review pass?

@marcodiiga marcodiiga left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving after re-review. Existing review comments still stand where applicable.

Resolve the OVPhysX sensor export conflict by keeping both IMU and joint-wrench exports. Keep the PR's removal of the articulation-level incoming joint wrench test, since OVPhysX now exposes that behavior through the joint-wrench sensor.

Port the stale-reset timestamp guard from the PhysX/Newton joint-wrench kernels to OVPhysX and add matching regression coverage.
Keep the OVPhysX sensor exports for IMU, joint wrench, and PVA together after the latest develop merge.

Relax the OVPhysX joint wrench raw-tensor comparison tolerance to account for CUDA float32 operation-order differences between the Warp sensor kernel and the PyTorch reference path.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants