Adds heterogeneous dexsuite to Newton backend

[ooctipus]

Description

This PR enables heterogeneous dexsuite with newton backend

Fixes # (issue)

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (existing functionality will not work without user modification)
• Documentation update

Screenshots

Please attach before and after screenshots of the change if applicable.

Checklist

• I have read and understood the contribution guidelines
• I have run the pre-commit checks with ./isaaclab.sh --format
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• I have updated the changelog and the corresponding version in the extension's config/extension.toml file
• I have added my name to the CONTRIBUTORS.md or my name already exists there

[greptile-apps]

Greptile Summary

This PR enables heterogeneous dexsuite (multi-asset object spawning) with the Newton physics backend. It switches shape primitives from ShapeCfg variants (CuboidCfg, SphereCfg, CapsuleCfg, ConeCfg) to MeshCfg variants (MeshCuboidCfg, MeshSphereCfg, MeshCapsuleCfg, MeshConeCfg) for Newton compatibility, tunes solver parameters, adds body armature to the Newton ModelBuilder, and removes the validation guard that previously blocked Newton + multi-asset configurations.

• Newton cloner: Adds default_body_armature = 0.001 to both the global and per-prototype ModelBuilder and reorders imports
• Dexsuite env config: Replaces Shape primitives with Mesh equivalents, adds collision_props with contact_offset=0.002, lowers ground plane by 1cm, reduces env_spacing from 3 to 2.0, and removes the Newton multi-asset validation error
• Kuka Allegro config: Increases nconmax (70→200), switches use_mujoco_contacts to False, reduces env_spacing
• Issue found: The object_physics_material EventTerm field was renamed to OBJECT_PHYSICS_material, which breaks the project's snake_case naming convention and could break downstream curriculum configurations that reference the field by name

Confidence Score: 3/5

• PR is mostly safe but contains a field rename that breaks naming conventions and may break curriculum references
• The core physics changes (Mesh primitives, armature, solver tuning) appear correct and well-motivated. However, the rename of object_physics_material to OBJECT_PHYSICS_material is a naming convention violation that could silently break curriculum configurations referencing the old field name. The removal of the Newton multi-asset validation guard is intentional per the PR's purpose.
• Pay close attention to source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/dexsuite/dexsuite_env_cfg.py — the OBJECT_PHYSICS_material rename needs to be reverted to object_physics_material

Important Files Changed

Filename	Overview
source/isaaclab_newton/isaaclab_newton/cloner/newton_replicate.py	Reorders imports and adds default_body_armature = 0.001 to both the global and per-prototype ModelBuilder instances. Changes are straightforward and consistent.
source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/dexsuite/config/kuka_allegro/dexsuite_kuka_allegro_env_cfg.py	Tunes Newton solver parameters (nconmax 70→200, use_mujoco_contacts True→False) and reduces env_spacing from 3 to 2.0 for Newton backend compatibility.
source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/dexsuite/dexsuite_env_cfg.py	Switches shape primitives from ShapeCfg (CuboidCfg, SphereCfg, etc.) to MeshCfg variants (MeshCuboidCfg, MeshSphereCfg, etc.) for Newton compatibility, adds collision_props/contact_offset, removes Newton multi-asset validation guard, and renames object_physics_material to OBJECT_PHYSICS_material which breaks naming conventions and may break curriculum references.
source/isaaclab_tasks/isaaclab_tasks/utils/presets.py	Adds missing trailing newline required by pre-commit end-of-file-fixer hook. No functional change.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[DexsuiteReorientEnvCfg] -->|scene config| B[SceneCfg]
    B -->|object spawn| C[ObjectCfg / MultiAssetSpawnerCfg]
    C -->|"was: CuboidCfg, SphereCfg..."| D[ShapeCfg variants - Rigid only]
    C -->|"now: MeshCuboidCfg, MeshSphereCfg..."| E[MeshCfg variants - Newton compatible]
    E -->|collision_props + contact_offset| F[OBJECT_PHYSICS dict]
    
    A -->|physics config| G{Physics Backend?}
    G -->|PhysX| H[PhysxCfg]
    G -->|Newton| I[NewtonCfg]
    
    I -->|cloner| J[newton_replicate.py]
    J -->|ModelBuilder| K[default_body_armature = 0.001]
    J -->|per-prototype builder| K
    
    I -->|solver tuning| L[MJWarpSolverCfg]
    L -->|nconmax: 200| M[Increased contact capacity]
    L -->|use_mujoco_contacts: False| N[Disabled MuJoCo contacts]
    
    A -->|validate_config| O{Newton + MultiAsset?}
    O -->|"was: raise ValueError"| P[Blocked]
    O -->|"now: allowed"| Q[Heterogeneous spawn enabled]

Loading

_{Last reviewed commit: b99c260}

[greptile-apps]

Field name breaks naming convention

The rename from object_physics_material to OBJECT_PHYSICS_material deviates from the project's consistent snake_case convention for EventTerm field names. Every other similar field across the codebase (robot_physics_material, object_physics_material in inhand_env_cfg.py, shadow_hand_env_cfg.py, etc.) uses lowercase snake_case. The MIXED_CASE name also appears to be an unintentional collision with the new OBJECT_PHYSICS dict constant defined on line 40.

Additionally, the curriculum system references this field by name (e.g., "events.object_physics_material.func.material_buckets" in curriculums.py), so this rename could break downstream curriculum configurations that depend on the original name.

Suggested change

	OBJECT_PHYSICS_material = EventTerm(
	object_physics_material = EventTerm(

[AntoineRichard]

LGTM. @ooctipus did you end-up trying Tobi's PR so that we don't have to offset the base?

[AntoineRichard]

@ooctipus could you create an issue to Newton about these defaults?

[ooctipus]

newton-physics/newton#2121

[AntoineRichard]

The famous trick! Did you end-up trying Tobi's PR?

[isaaclab-review-bot]

🤖 Isaac Lab Review Bot

Summary

This PR enables heterogeneous object shapes (multi-asset spawning) for the DexSuite manipulation task when using the Newton physics backend. The changes include: (1) switching from primitive shape configs (CuboidCfg, SphereCfg, etc.) to mesh-based configs (MeshCuboidCfg, MeshSphereCfg, etc.), (2) removing a validation check that previously blocked multi-asset spawning with Newton, (3) tuning Newton solver parameters, and (4) reducing environment spacing from 3.0 to 2.0.

Architecture Impact

• Cross-module: The shape config changes in dexsuite_env_cfg.py affect all DexSuite environments (Kuka Allegro and potentially other robot configs). The switch from primitive to mesh-based shapes could have different collision behavior characteristics.
• PhysX compatibility: The change from CuboidCfg/SphereCfg to MeshCuboidCfg/MeshSphereCfg may affect existing PhysX-based training runs if they relied on the primitive collision shapes.
• Newton solver tuning: The nconmax increase (70→200) and use_mujoco_contacts=False are significant physics behavior changes.

Implementation Verdict

Minor fixes needed — The core changes appear correct for enabling Newton multi-asset support, but there's a concerning regression in the collision physics configuration and an incomplete validation removal.

Test Coverage

No tests added or modified. Given this PR changes physics behavior and removes a safety validation, regression tests demonstrating that Newton multi-asset spawning now works correctly would be valuable. At minimum, a smoke test showing the environments can be instantiated and run with Newton+shapes preset would catch future regressions.

CI Status

No CI checks available yet — unable to verify these changes don't break existing PhysX or Newton workflows.

Findings

🟡 Warning: dexsuite_env_cfg.py:37-40 — contact_offset added to OBJECT_PHYSICS may change collision behavior for all backends
The new OBJECT_PHYSICS dict adds collision_props=sim_utils.CollisionPropertiesCfg(contact_offset=0.002) which wasn't present in the original per-shape definitions. This contact_offset value could subtly change contact dynamics. Was this intentional for Newton compatibility, or an unintended side effect? If intentional, document why; if not, consider whether it should only apply to Newton.

🟡 Warning: dexsuite_env_cfg.py:434-439 — Validation function body is now effectively empty
The validate_config method now only validates camera renderer types. The empty first statement after the docstring (line 435) suggests incomplete cleanup. If no validation is needed, consider documenting why, or add a pass statement to make intent clear. More importantly, was the Newton+multi-asset validation truly unnecessary, or did the mesh-based shapes actually enable support?

def validate_config(self):
    """Check for invalid preset combinations after resolution."""

    warp_supported = {"rgb", "depth", "distance_to_image_plane"}

🔵 Improvement: dexsuite_env_cfg.py:76-82 — ObjectCfg.newton preset is now stale

newton = cube  # newton does not support multi-asset spawning yet

This comment claims Newton doesn't support multi-asset spawning, but this PR's purpose is to enable exactly that. The newton preset should either be removed (if it's no longer needed as a fallback) or the comment should be updated to explain why a single-cube fallback is still provided.

🟡 Warning: dexsuite_kuka_allegro_env_cfg.py:55 — use_mujoco_contacts=False is a significant physics change
Switching from use_mujoco_contacts=True to False changes the contact handling strategy in the Newton solver. This could affect training stability and learned policies. The PR description doesn't explain why this change was necessary for heterogeneous objects. Was this required for correctness, or a performance optimization?

🔵 Improvement: dexsuite_kuka_allegro_env_cfg.py:83 — Environment spacing reduced from 3.0 to 2.0
This change reduces environment spacing from 3.0 to 2.0 meters. While 2.0 matches SceneCfg in the base class, reducing spacing could cause inter-environment collisions if objects are thrown or robots move aggressively. Verify this spacing is sufficient for the workspace bounds defined in TerminationsCfg.object_out_of_bound (x: -1.5 to 0.5, y: -2.0 to 2.0).

🔵 Improvement: dexsuite_env_cfg.py:115 — GroundPlaneCfg color parameter removed
The explicit color=(1.0, 1.0, 1.0) was removed from the ground plane config. This is a minor visual change but worth noting for reproducibility of any visual-based training that may depend on ground plane appearance.

🟡 Warning: Missing documentation — PR makes physics-significant changes without explanation
The changes to nconmax (70→200), use_mujoco_contacts, collision shape types (primitive→mesh), and contact_offset all affect simulation fidelity. Without documentation explaining why these changes enable heterogeneous dexsuite on Newton, future maintainers won't understand the constraints or whether they can be relaxed.

[isaaclab-review-bot]

🤖 Isaac Lab Review Bot

Follow-up Review

The new commit (598615a) addresses several previous concerns:

1. Previous concern about stale ObjectCfg.newton comment has been addressed — The comment # newton does not support multi-asset spawning yet has been removed, and newton = cube now stands alone as a fallback option without the misleading comment.

2. Previous concern about empty validation function has been addressed — The empty line after the docstring has been removed, making the function body cleaner.

3. Previous concerns about contact_offset, use_mujoco_contacts=False, nconmax tuning, environment spacing, and ground plane color remain unaddressed — These are physics-significant changes that still lack documentation explaining why they're necessary for Newton heterogeneous support. However, these were flagged as warnings/improvements rather than blockers.

No new issues introduced by this commit. The changes are incremental cleanups to the previous submission.

Implementation Verdict: Ship it — The code changes are reasonable for enabling Newton multi-asset support. The remaining warnings about documentation are valid but not blocking.

[isaaclab-review-bot]

🤖 Isaac Lab Review Bot

Follow-up Review

The new commit (acf23a9) makes no substantive changes to the code logic — only the ObjectCfg.newton line was modified from newton = cube to include a comment # newton does not support multi-asset spawning yet, which re-introduces the stale/misleading comment that was flagged in a previous review and subsequently removed.

🟡 Warning: dexsuite_env_cfg.py:79 — Stale comment re-introduced
The comment # newton does not support multi-asset spawning yet was previously removed but is now back. If this PR is specifically about enabling heterogeneous (multi-asset) dexsuite for Newton, this comment is contradictory and confusing. Either the comment should be removed, or the PR description is misleading.

Implementation Verdict: Minor fixes needed — Please clarify whether Newton now supports multi-asset spawning (remove the comment) or if the fallback is still required (update PR description).

[isaaclab-review-bot]

🤖 Isaac Lab Review Bot

Follow-up Review

The new commit (62cd5e3) addresses the previous concern by removing the stale newton = cube line entirely. The ObjectCfg now only has shapes, cube, and default = shapes presets, which is cleaner.

However, there's a remaining concern:

🟡 Warning: dexsuite_env_cfg.py:73-79 — Newton heterogeneous support unclear
The ObjectCfg no longer has a newton preset, but KukaAllegroPhysicsCfg still defines a newton physics preset. If the preset system auto-resolves presets=newton for objects, this will now fall through to default = shapes (multi-asset). The PR claims to "enable heterogeneous dexsuite with newton backend" — if multi-asset spawning now works with Newton, this is correct. If not, users selecting presets=newton may hit runtime errors when the multi-asset spawner is used.

Please confirm: Does Newton now support MultiAssetSpawnerCfg? If yes, this is ship-ready. If no, the newton preset for ObjectCfg should be restored to use cube.

[kellyguo11]

I think the failing test_rendering_dexsuite_kuka test might actually depend on golden images. perhaps we need to update the images with the latest changes in the environment? @huidongc for viz

[huidongc]

I think the failing test_rendering_dexsuite_kuka test might actually depend on golden images. perhaps we need to update the images with the latest changes in the environment? @huidongc for viz

Right. From the job log we can find the test artifact:

Artifact comparison-images-isaac-lab-tasks-2-test-25194568968-7 successfully finalized. Artifact ID 6759375450
Artifact comparison-images-isaac-lab-tasks-2-test-25194568968-7 has been successfully uploaded! Final size is 499163 bytes. Artifact ID is 6759375450
Artifact download URL: https://github.com/isaac-sim/IsaacLab/actions/runs/25194568968/artifacts/6759375450

You can then download the artifact and confirm whether the new images are expected or not. If so, you can overwrite the golden images (source/isaaclab_tasks/test/golden_images) with the new images.

[isaaclab-review-bot]

🤖 Isaac Lab Review Bot

Follow-up Review

The new commit (ca409e5) addresses the previous concern by removing the validation check that blocked Newton + multi-asset combinations, indicating Newton now supports heterogeneous objects. The ObjectCfg.newton = cube preset was removed earlier, and now the validator no longer raises errors for this combination.

The changes look consistent: Newton's PhysicsCfg preset is moved to the base config with nconmax=200 (increased from 70 in the removed Kuka-specific config), use_mujoco_contacts=False, and additional collision_cfg/default_shape_cfg parameters — all suggesting proper Newton heterogeneous support.

Ship it — the PR coherently enables heterogeneous dexsuite with Newton backend by centralizing physics config and removing artificial restrictions.

[isaaclab-review-bot]

🤖 Isaac Lab Review Bot

Follow-up Review

The new commit (471ed5d) adds ObjectCfg.newton_mjwarp = cube back to provide a single-geometry fallback for Newton, which is reasonable since multi-asset spawning support may still be experimental. This is a sensible addition.

However, I notice a potential issue:

🟡 Warning: dexsuite_env_cfg.py:80 — ObjectCfg.newton_mjwarp = cube but PhysicsCfg preset uses newton key

The physics preset key is newton (line 426) while the object preset key is newton_mjwarp. If preset resolution uses the same key for both, these won't align — users selecting presets=newton would get the Newton physics config but still get the default shapes object (multi-asset), potentially causing issues if Newton's multi-asset support is incomplete. Verify that preset key naming is intentional and that newton_mjwarp is the correct convention being used elsewhere.

Otherwise, ship it — the centralization of physics config and enabling heterogeneous dexsuite for Newton looks correct.

[isaaclab-review-bot]

🤖 Isaac Lab Review Bot

Follow-up Review

The new commits (ac2e1b0, 2ce44b7) address previous review concerns:

1. Ground color reverted — The explicit ground plane color has been reverted to defaults, addressing the reproducibility concern.
2. Environment spacing restored to 3.0 — Spacing is back to 3.0m (from the briefly reduced 2.0m), ensuring adequate clearance for workspace bounds.
3. Test flakiness markers removed — The @pytest.mark.flaky decorators have been removed from DexSuite rendering tests, suggesting improved test stability with these changes.
4. Golden images updated — New golden images for DexSuite Kuka rendering tests reflect the mesh-based shapes and Newton physics changes.

Remaining Observations

The PR now looks coherent. The centralized PhysicsCfg preset in the base dexsuite_env_cfg.py properly configures both PhysX and Newton backends, and the mesh-based shapes (MeshCuboidCfg, MeshSphereCfg, etc.) enable heterogeneous object spawning with Newton.

Key physics parameters in the Newton config:

• nconmax=200 (increased from 70 for multi-object contacts)
• use_mujoco_contacts=False (using Newton's collision pipeline)
• ccd_iterations=35 (reduced from 5000 — more reasonable)
• Added collision_cfg=NewtonCollisionPipelineCfg() and default_shape_cfg=NewtonShapeCfg()

Minor note: The ObjectCfg.newton_mjwarp = cube fallback was removed in an earlier commit. If users specify presets=newton_mjwarp for objects, they will now get the default multi-asset spawner, which should work with the updated Newton config.

Implementation Verdict: Ship it — The changes coherently enable heterogeneous DexSuite with Newton backend.

[kellyguo11]

hmm isaaclab_newton tests look like are timing out