[Bug Report] headless training behaves differently with collisionApproximateCylinders=true

[Zionshang]

Description

I found a strange issue in Isaac Lab. When enabling:

--kit_args="--/physics/collisionApproximateCylinders=true"

the training result becomes very different between headless and non-headless modes.

Observed behavior

With the same task, robot asset, and training config:

Setting	Result
no --headless, with collisionApproximateCylinders=true	good training
with --headless, with collisionApproximateCylinders=true	very poor training
with --headless, without collisionApproximateCylinders=true	good training

This seems unexpected because rendering should not significantly affect the physics simulation or RL training result.

Robot details

The robot is a quadruped, and several lower-leg links use cylindrical collision geometries.

This is the usd file

The task also uses contact-based rewards, including undesired contact penalties on thigh/calf links.

    undesired_contacts = RewTerm(
        func=mdp.undesired_contacts,
        weight=-0.1,
        params={"sensor_cfg": SceneEntityCfg("contact_forces", body_names=[".*_hip", ".*_thigh"]), "threshold": 0.1},
    )

The robot config is

ROBOT_CFG = ArticulationCfg(
    prim_path="{ENV_REGEX_NS}/Robot",
    spawn=sim_utils.UsdFileCfg(
        usd_path=f"{RL_LAB_ENV_ASSETS_DIR}/robots/go2/usd/unitree_go2.usd",
        activate_contact_sensors=True,
        rigid_props=sim_utils.RigidBodyPropertiesCfg(
            disable_gravity=False,
            retain_accelerations=False,
            linear_damping=0.0,
            angular_damping=0.0,
            max_linear_velocity=1000.0,
            max_angular_velocity=1000.0,
            max_depenetration_velocity=1.0,
        ),
        articulation_props=sim_utils.ArticulationRootPropertiesCfg(
            enabled_self_collisions=False, solver_position_iteration_count=4, solver_velocity_iteration_count=0
        ),
    ),

Environment

• Isaac Lab version: 2.3.2 (pip install)
• Isaac Sim version: 5.1
• OS: Ubuntu 22.04
• GPU: 5090D
• NVIDIA driver: 580.142

[Zionshang]

It almost feels like the simulation is in a quantum state: once I observe it with rendering, the training behaves well; once I do not observe it in headless mode, it collapses. 😅