You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A few days ago, while configuring iassc-lab（main version）, I found that according to the official documentation, I was able to run iassc-sim correctly and run empty-routines and demos after configuring iassc-lab. However, I was unable to run the rsl-rl framework (such as the demo in the official documentation. the affected frameworks include rsl-rl, rl-games, etc.).
I have resolved the issue and shared it with you. If you also encounter a similar situation, you can refer to it:
The cause is that the Tensordict library make a major version iteration in April 2026 from 0.11.0 ->0.12.0 (the latest default installation version is 0.12.3). According to the official description, this update made significant adjustments to the underlying structure (directly causing Python 3.9 to not rely on this library, which actually affects the compatibility of subsequent versions such as Python 3.10 and Python 3.11, i think so). If you also have a similar memory removal issue, please roll back the latest version of the Tensordict library to version 0.11.0. There are no restrictions on the Tensordict library version in the RL framework setup. py file of IASSC-Lab. Please be aware of this. Therefore, I suggest rolling back the Tensordict library version first after completing the configuration of the IASSC-lab and RL frameworks. I will put my running log at the end:
(env_isaaclab) PS F:\IsaacLab> isaaclab.bat -p scripts/reinforcement_learning/rsl_rl/train.py --task=Isaac-Ant-v0 --headless
[INFO] Using python from: F:\conda\envs\env_isaaclab\python.exe
[INFO][AppLauncher]: Using device: cuda:0
[INFO][AppLauncher]: Loading experience file: F:\IsaacLab\apps\isaaclab.python.headless.kit
Loading user config located at: 'f:/conda/envs/env_isaaclab/lib/site-packages/isaacsim/kit/data/Kit/Isaac-Sim/5.1/user.config.json'
[Info] [carb] Logging to file: f:/conda/envs/env_isaaclab/lib/site-packages/isaacsim/kit/logs/Kit/Isaac-Sim/5.1/kit_20260516_012826.log
2026-05-15T17:28:26Z [43ms] [Warning] [carb.crashreporter-breakpad.plugin] [previous crash] preventing upload of minidump due to user opt-out: 'f:/conda/envs/env_isaaclab/lib/site-packages/isaacsim/kit/data/Kit/Isaac-Sim/5.1\2ae79b34-1944-421d-8746-0256900a2b54.dmp'
2026-05-15T17:28:27Z [213ms] [Warning] [omni.usd_config.extension] Enable omni.materialx.libs extension to use MaterialX
2026-05-15T17:28:27Z [864ms] [Warning] [carb] Acquiring non optional plugin interface which is not listed as dependency: [omni::physx::IPhysxBenchmarks v1.0] (plugin: ), by client: omni.physics.physx.plugin. Add it to CARB_PLUGIN_IMPL_DEPS() macro of a client.
2026-05-15T17:28:27Z [878ms] [Warning] [omni.isaac.dynamic_control] omni.isaac.dynamic_control is deprecated as of Isaac Sim 4.5. No action is needed from end-users.
F:\conda\envs\env_isaaclab\Lib\site-packages\networkx\utils\backends.py:102: RuntimeWarning: networkx backend name is not a valid identifier: 'nx-loopback'. Ignoring.
backends = _get_backends("networkx.backends")
Loading user config located at: 'f:/conda/envs/env_isaaclab/lib/site-packages/isaacsim/kit/data/Kit/Isaac-Sim/5.1/user.config.json'
[Info] [carb] Logging to file: f:/conda/envs/env_isaaclab/lib/site-packages/isaacsim/kit/logs/Kit/Isaac-Sim/5.1/kit_20260516_012849.log
2026-05-15T17:28:49Z [33ms] [Warning] [carb.crashreporter-breakpad.plugin] [previous crash] preventing upload of minidump due to user opt-out: 'f:/conda/envs/env_isaaclab/lib/site-packages/isaacsim/kit/data/Kit/Isaac-Sim/5.1\f960d45f-2882-475b-b8eb-c5bcc68abdf2.dmp'
2026-05-15T17:28:49Z [37ms] [Warning] [carb.crashreporter-breakpad.plugin] [previous crash] preventing upload of minidump due to user opt-out: 'f:/conda/envs/env_isaaclab/lib/site-packages/isaacsim/kit/data/Kit/Isaac-Sim/5.1\2ae79b34-1944-421d-8746-0256900a2b54.dmp'
2026-05-15T17:28:49Z [203ms] [Warning] [omni.usd_config.extension] Enable omni.materialx.libs extension to use MaterialX
2026-05-15T17:28:50Z [849ms] [Warning] [carb] Acquiring non optional plugin interface which is not listed as dependency: [omni::physx::IPhysxBenchmarks v1.0] (plugin: ), by client: omni.physics.physx.plugin. Add it to CARB_PLUGIN_IMPL_DEPS() macro of a client.
2026-05-15T17:28:50Z [863ms] [Warning] [omni.isaac.dynamic_control] omni.isaac.dynamic_control is deprecated as of Isaac Sim 4.5. No action is needed from end-users.
F:\conda\envs\env_isaaclab\Lib\site-packages\networkx\utils\backends.py:102: RuntimeWarning: networkx backend name is not a valid identifier: 'nx-loopback'. Ignoring.
If you are not sure if your problem is similar, observe two characteristics. One is whether there is a memory leak reported in the log and check the memory and system hardware configuration multiple times in the log. The other is whether the fault tracing is as high as 207. Using AI (such as GPT or Deepseek) to assist you in analyzing the log is also a good way for you to refer to
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
A few days ago, while configuring iassc-lab（main version）, I found that according to the official documentation, I was able to run iassc-sim correctly and run empty-routines and demos after configuring iassc-lab. However, I was unable to run the rsl-rl framework (such as the demo in the official documentation. the affected frameworks include rsl-rl, rl-games, etc.).
I have resolved the issue and shared it with you. If you also encounter a similar situation, you can refer to it:
The cause is that the Tensordict library make a major version iteration in April 2026 from 0.11.0 ->0.12.0 (the latest default installation version is 0.12.3). According to the official description, this update made significant adjustments to the underlying structure (directly causing Python 3.9 to not rely on this library, which actually affects the compatibility of subsequent versions such as Python 3.10 and Python 3.11, i think so). If you also have a similar memory removal issue, please roll back the latest version of the Tensordict library to version 0.11.0. There are no restrictions on the Tensordict library version in the RL framework setup. py file of IASSC-Lab. Please be aware of this. Therefore, I suggest rolling back the Tensordict library version first after completing the configuration of the IASSC-lab and RL frameworks. I will put my running log at the end:
(env_isaaclab) PS F:\IsaacLab> isaaclab.bat -p scripts/reinforcement_learning/rsl_rl/train.py --task=Isaac-Ant-v0 --headless
[INFO] Using python from: F:\conda\envs\env_isaaclab\python.exe
[INFO][AppLauncher]: Using device: cuda:0
[INFO][AppLauncher]: Loading experience file: F:\IsaacLab\apps\isaaclab.python.headless.kit
Loading user config located at: 'f:/conda/envs/env_isaaclab/lib/site-packages/isaacsim/kit/data/Kit/Isaac-Sim/5.1/user.config.json'
[Info] [carb] Logging to file: f:/conda/envs/env_isaaclab/lib/site-packages/isaacsim/kit/logs/Kit/Isaac-Sim/5.1/kit_20260516_012826.log
2026-05-15T17:28:26Z [43ms] [Warning] [carb.crashreporter-breakpad.plugin] [previous crash] preventing upload of minidump due to user opt-out: 'f:/conda/envs/env_isaaclab/lib/site-packages/isaacsim/kit/data/Kit/Isaac-Sim/5.1\2ae79b34-1944-421d-8746-0256900a2b54.dmp'
2026-05-15T17:28:27Z [213ms] [Warning] [omni.usd_config.extension] Enable omni.materialx.libs extension to use MaterialX
2026-05-15T17:28:27Z [864ms] [Warning] [carb] Acquiring non optional plugin interface which is not listed as dependency: [omni::physx::IPhysxBenchmarks v1.0] (plugin: ), by client: omni.physics.physx.plugin. Add it to CARB_PLUGIN_IMPL_DEPS() macro of a client.
2026-05-15T17:28:27Z [878ms] [Warning] [omni.isaac.dynamic_control] omni.isaac.dynamic_control is deprecated as of Isaac Sim 4.5. No action is needed from end-users.
F:\conda\envs\env_isaaclab\Lib\site-packages\networkx\utils\backends.py:102: RuntimeWarning: networkx backend name is not a valid identifier: 'nx-loopback'. Ignoring.
backends = _get_backends("networkx.backends")
|---------------------------------------------------------------------------------------------|
| Driver Version: 581.42 | Graphics API: D3D12
|=============================================================================================|
| GPU | Name | Active | LDA | GPU Memory | Vendor-ID | LUID |
| | | | | | Device-ID | UUID |
| | | | | | Bus-ID | |
|---------------------------------------------------------------------------------------------|
| 0 | NVIDIA GeForce RTX 3070 Ti | Yes: 0 | | 8018 MB | 10de | a37f0803.. |
| | | | | | 2482 | 0 |
| | | | | | 7 | |
|=============================================================================================|
| OS: Windows 11 Pro, Version: 10.0 (25H2), Build: 26200, Kernel: 10.0.26100.6584
| Processor: AMD Ryzen 9 5900X 12-Core Processor
| Cores: 12 | Logical Cores: 24
|---------------------------------------------------------------------------------------------|
| Total Memory (MB): 16270 | Free Memory: 8946
| Total Page/Swap (MB): 62454 | Free Page/Swap: 48459
|---------------------------------------------------------------------------------------------|
Windows fatal exception: access violation
Current thread 0x00006444 (most recent call first):
File "", line 241 in _call_with_frames_removed
File "", line 1233 in create_module
File "", line 573 in module_from_spec
File "", line 676 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in _find_and_load
File "F:\conda\envs\env_isaaclab\Lib\site-packages\tensordict\utils.py", line 44 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in _find_and_load
File "F:\conda\envs\env_isaaclab\Lib\site-packages\tensordict_datasets.py", line 12 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in _find_and_load
File "F:\conda\envs\env_isaaclab\Lib\site-packages\tensordict\base.py", line 56 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in _find_and_load
File "F:\conda\envs\env_isaaclab\Lib\site-packages\tensordict_td.py", line 37 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in _find_and_load
File "F:\conda\envs\env_isaaclab\Lib\site-packages\tensordict_lazy.py", line 38 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in _find_and_load
File "F:\conda\envs\env_isaaclab\Lib\site-packages\tensordict_reductions.py", line 11 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in find_and_load
File "F:\conda\envs\env_isaaclab\Lib\site-packages\tensordict_init.py", line 6 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in _find_and_load
File "F:\conda\envs\env_isaaclab\Lib\site-packages\rsl_rl\algorithms\distillation.py", line 11 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in find_and_load
File "F:\conda\envs\env_isaaclab\Lib\site-packages\rsl_rl\algorithms_init.py", line 8 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in _find_and_load
File "F:\conda\envs\env_isaaclab\Lib\site-packages\rsl_rl\runners\on_policy_runner.py", line 13 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in find_and_load
File "F:\conda\envs\env_isaaclab\Lib\site-packages\rsl_rl\runners_init.py", line 8 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in _find_and_load
File "F:\IsaacLab\scripts\reinforcement_learning\rsl_rl\train.py", line 85 in
2026-05-15T17:28:33Z [0ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] A crash has occurred. If a debugger should be attached, please set the '/crashreporter/debuggerAttachTimeoutMs' setting to a timeout in milliseconds. This can be used to allow the crash reporter to wait for up to that long for a debugger to attach before processing or sending the crash report.
2026-05-15T17:28:33Z [5ms] [Warning] [carb.crashreporter-breakpad.plugin] Launched process 'py-spy.exe' as PID 13464
2026-05-15T17:28:33Z [103ms] [Error] [carb.crashreporter-breakpad.plugin] [crash] Wrote dump file 'f:/conda/envs/env_isaaclab/lib/site-packages/isaacsim/kit/data/Kit/Isaac-Sim/5.1\f960d45f-2882-475b-b8eb-c5bcc68abdf2.dmp'
2026-05-15T17:28:33Z [104ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] preventing upload of minidump due to user opt-out: 'f:/conda/envs/env_isaaclab/lib/site-packages/isaacsim/kit/data/Kit/Isaac-Sim/5.1\f960d45f-2882-475b-b8eb-c5bcc68abdf2.dmp'
2026-05-15T17:28:33Z [104ms] [Error] [carb.crashreporter-breakpad.plugin] [crash] dump file size is 5614447 bytes, file is readable.
2026-05-15T17:28:34Z [746ms] [Fatal] [carb.crashreporter-breakpad.plugin] [crash] Crash detected in pid 4980 thread 17224
2026-05-15T17:28:34Z [747ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] Crash metadata:
2026-05-15T17:28:34Z [747ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] BuildGitlabJobID = '209507878'
2026-05-15T17:28:34Z [748ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] CarbSdkVersion = '206.6+release.9587.07f17b1b.gl'
2026-05-15T17:28:34Z [749ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] CrashTime = 'Fri May 15 17:28:33 2026 GMT'
2026-05-15T17:28:34Z [749ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] DriverShaderCacheWrapper = 'disabled'
2026-05-15T17:28:34Z [750ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] DumpId = 'f960d45f-2882-475b-b8eb-c5bcc68abdf2'
2026-05-15T17:28:34Z [751ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] LastUploadStatus = '0'
2026-05-15T17:28:34Z [751ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] ProductName = 'OmniverseKit'
2026-05-15T17:28:34Z [752ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] PythonTracebackStatus = ''py-spy.exe' successfully wrote info to f:/conda/envs/env_isaaclab/lib/site-packages/isaacsim/kit/data/Kit/Isaac-Sim/5.1/f960d45f-2882-475b-b8eb-c5bcc68abdf2.py.txt'
2026-05-15T17:28:34Z [753ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] RetryCount = '0'
2026-05-15T17:28:34Z [753ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] StartupTime = '1778866106'
2026-05-15T17:28:34Z [754ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] UploadSuccessful = '0'
2026-05-15T17:28:34Z [755ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] UptimeSeconds = '7'
2026-05-15T17:28:34Z [755ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] UserStory = ''
2026-05-15T17:28:34Z [756ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] UserStoryStatus = 'Running in headless mode; not gathering user story'
2026-05-15T17:28:34Z [757ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] Version = '107.3.3+production.229672.69cbf6ad.gl'
2026-05-15T17:28:34Z [757ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] aftermath_enabled = '1'
2026-05-15T17:28:34Z [758ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] aftermath_flags = '3'
2026-05-15T17:28:34Z [759ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] aftermath_status = 'auto-enabled'
2026-05-15T17:28:34Z [759ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] aftermath_version = '2024.1'
2026-05-15T17:28:34Z [760ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] appName = 'Isaac-Sim'
2026-05-15T17:28:34Z [760ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] appState = 'started'
2026-05-15T17:28:34Z [761ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] appVersion = '5.1.0'
2026-05-15T17:28:34Z [762ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] autoloadExts = ''
2026-05-15T17:28:34Z [763ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildBranch = 'production'
2026-05-15T17:28:34Z [763ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildCi = 'gl'
2026-05-15T17:28:34Z [764ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildConfig = 'release'
2026-05-15T17:28:34Z [764ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildGitlabJobName = 'kit-build-release-windows-x86_64'
2026-05-15T17:28:34Z [765ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildGitlabJobStage = 'kit-build'
2026-05-15T17:28:34Z [766ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildGitlabPipelineID = '34875094'
2026-05-15T17:28:34Z [766ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildHash = '69cbf6ad'
2026-05-15T17:28:34Z [767ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildMajor = '107'
2026-05-15T17:28:34Z [767ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildMinor = '3'
2026-05-15T17:28:34Z [768ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildNumber = '229672'
2026-05-15T17:28:34Z [769ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildPatch = '3'
2026-05-15T17:28:34Z [769ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildVersion = '107.3.3'
2026-05-15T17:28:34Z [770ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] carboniteFrameworkVersion = '206.6+release.9587.07f17b1b.gl'
2026-05-15T17:28:34Z [770ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] carboniteSdkVersion = '206.6+release.9587.07f17b1b.gl'
2026-05-15T17:28:34Z [771ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] commandLine = 'F:\conda\envs\env_isaaclab\python.exe scripts/reinforcement_learning/rsl_rl/train.py --task=Isaac-Ant-v0 --headless'
2026-05-15T17:28:34Z [772ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] cpuCoreLimited = '-1'
2026-05-15T17:28:34Z [772ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] cpuId = 'AMD64 Family 25 Model 33 Stepping 0'
2026-05-15T17:28:34Z [773ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] cpuLogicalCoresBareMetal = '24'
2026-05-15T17:28:34Z [773ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] cpuName = 'AMD Ryzen 9 5900X 12-Core Processor '
2026-05-15T17:28:34Z [774ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] cpuPhysicalCoresBareMetal = '12'
2026-05-15T17:28:34Z [775ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] cpuUsageQuota = '-1.000000'
2026-05-15T17:28:34Z [775ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] cpuVendor = 'AuthenticAMD'
2026-05-15T17:28:34Z [776ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] desktopOrigin = '(0, 0)'
2026-05-15T17:28:34Z [778ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] desktopSize = '2560x1440'
2026-05-15T17:28:34Z [779ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] displayCount = '1'
2026-05-15T17:28:34Z [780ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] displayRes_0 = '3840x2160x32bit@60Hz'
2026-05-15T17:28:34Z [781ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] environmentName = 'Individual'
2026-05-15T17:28:34Z [782ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] experience = 'Isaac Sim'
2026-05-15T17:28:34Z [783ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] externalBuild = '1'
2026-05-15T17:28:34Z [784ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] gpuDriver_0 = '581.42'
2026-05-15T17:28:34Z [785ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] gpuVRAM_0 = '8407482368'
2026-05-15T17:28:34Z [786ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] gpu_0 = 'NVIDIA GeForce RTX 3070 Ti'
2026-05-15T17:28:34Z [786ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] lib_isaacSim_buildBranch = 'release'
2026-05-15T17:28:34Z [787ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] lib_isaacSim_buildDate = 'Fri Oct 17 03:55:10 2025 +0000'
2026-05-15T17:28:34Z [788ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] lib_isaacSim_buildHash = '9c81211'
2026-05-15T17:28:34Z [789ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] lib_isaacSim_buildRepo = 'https://gitlab-master.nvidia.com/omniverse/isaac/omni_isaac_sim.git'
2026-05-15T17:28:34Z [789ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] lib_isaacSim_buildVersion = '5.1.0-rc.19'
2026-05-15T17:28:34Z [790ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] lib_physx_buildBranch = 'HEAD'
2026-05-15T17:28:34Z [791ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] lib_physx_buildDate = 'Oct-03-2025'
2026-05-15T17:28:34Z [791ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] lib_physx_buildHash = '3a61992'
2026-05-15T17:28:34Z [792ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] lib_physx_buildRepo = 'gitlab-master.nvidia.com/omniverse/physics'
2026-05-15T17:28:34Z [793ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] lib_physx_buildVersion = '107.3.26'
2026-05-15T17:28:34Z [793ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] memoryStats = '(avail/total) RAM: 8.41907/15.8893GB, Swap: 46.8976/60.9908GB, VM: 131025/131072GB'
2026-05-15T17:28:34Z [794ms] [Warning] [carb.crashreporter-breakpad.plugin]
2026-05-15T17:28:34Z [795ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] osDistro = '25H2'
2026-05-15T17:28:34Z [795ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] osName = 'Windows 11 Pro'
2026-05-15T17:28:34Z [796ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] osVersion = '10.00.26200'
2026-05-15T17:28:34Z [797ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] portableMode = '1'
2026-05-15T17:28:34Z [797ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] primaryDisplayRes = '3840x2160x32bit@60Hz'
2026-05-15T17:28:34Z [798ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] runEnvironment = 'Individual'
2026-05-15T17:28:34Z [799ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] runningInContainer = '0'
|---------------------------------------------------------------------------------------------|
| Driver Version: 581.42 | Graphics API: D3D12
|=============================================================================================|
| GPU | Name | Active | LDA | GPU Memory | Vendor-ID | LUID |
| | | | | | Device-ID | UUID |
| | | | | | Bus-ID | |
|---------------------------------------------------------------------------------------------|
| 0 | NVIDIA GeForce RTX 3070 Ti | Yes: 0 | | 8018 MB | 10de | a37f0803.. |
| | | | | | 2482 | 0 |
| | | | | | 7 | |
|=============================================================================================|
| OS: Windows 11 Pro, Version: 10.0 (25H2), Build: 26200, Kernel: 10.0.26100.6584
| Processor: AMD Ryzen 9 5900X 12-Core Processor
| Cores: 12 | Logical Cores: 24
|---------------------------------------------------------------------------------------------|
| Total Memory (MB): 16270 | Free Memory: 8933
| Total Page/Swap (MB): 62454 | Free Page/Swap: 48487
|---------------------------------------------------------------------------------------------|
Windows fatal exception: access violation
Current thread 0x000034bc (most recent call first):
File "", line 241 in _call_with_frames_removed
File "", line 1233 in create_module
File "", line 573 in module_from_spec
File "", line 676 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in _find_and_load
File "F:\conda\envs\env_isaaclab\Lib\site-packages\tensordict\utils.py", line 44 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in _find_and_load
File "F:\conda\envs\env_isaaclab\Lib\site-packages\tensordict_datasets.py", line 12 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in _find_and_load
File "F:\conda\envs\env_isaaclab\Lib\site-packages\tensordict\base.py", line 56 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in _find_and_load
File "F:\conda\envs\env_isaaclab\Lib\site-packages\tensordict_td.py", line 37 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in _find_and_load
File "F:\conda\envs\env_isaaclab\Lib\site-packages\tensordict_lazy.py", line 38 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in _find_and_load
File "F:\conda\envs\env_isaaclab\Lib\site-packages\tensordict_reductions.py", line 11 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in find_and_load
File "F:\conda\envs\env_isaaclab\Lib\site-packages\tensordict_init.py", line 6 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in _find_and_load
File "F:\conda\envs\env_isaaclab\Lib\site-packages\rsl_rl\algorithms\distillation.py", line 11 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in find_and_load
File "F:\conda\envs\env_isaaclab\Lib\site-packages\rsl_rl\algorithms_init.py", line 8 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in _find_and_load
File "F:\conda\envs\env_isaaclab\Lib\site-packages\rsl_rl\runners\on_policy_runner.py", line 13 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in find_and_load
File "F:\conda\envs\env_isaaclab\Lib\site-packages\rsl_rl\runners_init.py", line 8 in
File "", line 241 in _call_with_frames_removed
File "", line 940 in exec_module
File "", line 690 in _load_unlocked
File "", line 1147 in _find_and_load_unlocked
File "", line 1176 in _find_and_load
File "F:\IsaacLab\scripts\reinforcement_learning\rsl_rl\train.py", line 85 in
2026-05-15T17:28:55Z [0ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] A crash has occurred. If a debugger should be attached, please set the '/crashreporter/debuggerAttachTimeoutMs' setting to a timeout in milliseconds. This can be used to allow the crash reporter to wait for up to that long for a debugger to attach before processing or sending the crash report.
2026-05-15T17:28:55Z [5ms] [Warning] [carb.crashreporter-breakpad.plugin] Launched process 'py-spy.exe' as PID 23844
2026-05-15T17:28:55Z [102ms] [Error] [carb.crashreporter-breakpad.plugin] [crash] Wrote dump file 'f:/conda/envs/env_isaaclab/lib/site-packages/isaacsim/kit/data/Kit/Isaac-Sim/5.1\68690a3e-31db-4e7a-b6e7-137d8fb9eda9.dmp'
2026-05-15T17:28:55Z [103ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] preventing upload of minidump due to user opt-out: 'f:/conda/envs/env_isaaclab/lib/site-packages/isaacsim/kit/data/Kit/Isaac-Sim/5.1\68690a3e-31db-4e7a-b6e7-137d8fb9eda9.dmp'
2026-05-15T17:28:55Z [104ms] [Error] [carb.crashreporter-breakpad.plugin] [crash] dump file size is 5410819 bytes, file is readable.
2026-05-15T17:28:56Z [745ms] [Fatal] [carb.crashreporter-breakpad.plugin] [crash] Crash detected in pid 14568 thread 28476
2026-05-15T17:28:56Z [745ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] Crash metadata:
2026-05-15T17:28:56Z [746ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] BuildGitlabJobID = '209507878'
2026-05-15T17:28:56Z [747ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] CarbSdkVersion = '206.6+release.9587.07f17b1b.gl'
2026-05-15T17:28:56Z [747ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] CrashTime = 'Fri May 15 17:28:55 2026 GMT'
2026-05-15T17:28:56Z [748ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] DriverShaderCacheWrapper = 'disabled'
2026-05-15T17:28:56Z [749ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] DumpId = '68690a3e-31db-4e7a-b6e7-137d8fb9eda9'
2026-05-15T17:28:56Z [749ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] LastUploadStatus = '0'
2026-05-15T17:28:56Z [750ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] ProductName = 'OmniverseKit'
2026-05-15T17:28:56Z [751ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] PythonTracebackStatus = ''py-spy.exe' successfully wrote info to f:/conda/envs/env_isaaclab/lib/site-packages/isaacsim/kit/data/Kit/Isaac-Sim/5.1/68690a3e-31db-4e7a-b6e7-137d8fb9eda9.py.txt'
2026-05-15T17:28:56Z [751ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] RetryCount = '0'
2026-05-15T17:28:56Z [752ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] StartupTime = '1778866129'
2026-05-15T17:28:56Z [753ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] UploadSuccessful = '0'
2026-05-15T17:28:56Z [753ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] UptimeSeconds = '6'
2026-05-15T17:28:56Z [754ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] UserStory = ''
2026-05-15T17:28:56Z [755ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] UserStoryStatus = 'Running in headless mode; not gathering user story'
2026-05-15T17:28:56Z [755ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] Version = '107.3.3+production.229672.69cbf6ad.gl'
2026-05-15T17:28:56Z [756ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] aftermath_enabled = '1'
2026-05-15T17:28:56Z [756ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] aftermath_flags = '3'
2026-05-15T17:28:56Z [757ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] aftermath_status = 'auto-enabled'
2026-05-15T17:28:56Z [758ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] aftermath_version = '2024.1'
2026-05-15T17:28:56Z [758ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] appName = 'Isaac-Sim'
2026-05-15T17:28:56Z [759ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] appState = 'started'
2026-05-15T17:28:56Z [760ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] appVersion = '5.1.0'
2026-05-15T17:28:56Z [760ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] autoloadExts = ''
2026-05-15T17:28:56Z [761ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildBranch = 'production'
2026-05-15T17:28:56Z [762ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildCi = 'gl'
2026-05-15T17:28:56Z [762ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildConfig = 'release'
2026-05-15T17:28:56Z [763ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildGitlabJobName = 'kit-build-release-windows-x86_64'
2026-05-15T17:28:56Z [764ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildGitlabJobStage = 'kit-build'
2026-05-15T17:28:56Z [764ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildGitlabPipelineID = '34875094'
2026-05-15T17:28:56Z [765ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildHash = '69cbf6ad'
2026-05-15T17:28:56Z [766ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildMajor = '107'
2026-05-15T17:28:56Z [766ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildMinor = '3'
2026-05-15T17:28:56Z [767ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildNumber = '229672'
2026-05-15T17:28:56Z [768ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildPatch = '3'
2026-05-15T17:28:56Z [768ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] buildVersion = '107.3.3'
2026-05-15T17:28:56Z [769ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] carboniteFrameworkVersion = '206.6+release.9587.07f17b1b.gl'
2026-05-15T17:28:56Z [770ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] carboniteSdkVersion = '206.6+release.9587.07f17b1b.gl'
2026-05-15T17:28:56Z [770ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] commandLine = 'F:\conda\envs\env_isaaclab\python.exe scripts/reinforcement_learning/rsl_rl/train.py --task=Isaac-Ant-v0 --headless'
2026-05-15T17:28:56Z [771ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] cpuCoreLimited = '-1'
2026-05-15T17:28:56Z [772ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] cpuId = 'AMD64 Family 25 Model 33 Stepping 0'
2026-05-15T17:28:56Z [772ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] cpuLogicalCoresBareMetal = '24'
2026-05-15T17:28:56Z [773ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] cpuName = 'AMD Ryzen 9 5900X 12-Core Processor '
2026-05-15T17:28:56Z [773ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] cpuPhysicalCoresBareMetal = '12'
2026-05-15T17:28:56Z [774ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] cpuUsageQuota = '-1.000000'
2026-05-15T17:28:56Z [775ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] cpuVendor = 'AuthenticAMD'
2026-05-15T17:28:56Z [775ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] desktopOrigin = '(0, 0)'
2026-05-15T17:28:56Z [776ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] desktopSize = '2560x1440'
2026-05-15T17:28:56Z [777ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] displayCount = '1'
2026-05-15T17:28:56Z [777ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] displayRes_0 = '3840x2160x32bit@60Hz'
2026-05-15T17:28:56Z [778ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] environmentName = 'Individual'
2026-05-15T17:28:56Z [779ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] experience = 'Isaac Sim'
2026-05-15T17:28:56Z [779ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] externalBuild = '1'
2026-05-15T17:28:56Z [780ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] gpuDriver_0 = '581.42'
2026-05-15T17:28:56Z [781ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] gpuVRAM_0 = '8407482368'
2026-05-15T17:28:56Z [781ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] gpu_0 = 'NVIDIA GeForce RTX 3070 Ti'
2026-05-15T17:28:56Z [782ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] lib_isaacSim_buildBranch = 'release'
2026-05-15T17:28:56Z [783ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] lib_isaacSim_buildDate = 'Fri Oct 17 03:55:10 2025 +0000'
2026-05-15T17:28:56Z [783ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] lib_isaacSim_buildHash = '9c81211'
2026-05-15T17:28:56Z [784ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] lib_isaacSim_buildRepo = 'https://gitlab-master.nvidia.com/omniverse/isaac/omni_isaac_sim.git'
2026-05-15T17:28:56Z [785ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] lib_isaacSim_buildVersion = '5.1.0-rc.19'
2026-05-15T17:28:56Z [785ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] lib_physx_buildBranch = 'HEAD'
2026-05-15T17:28:56Z [786ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] lib_physx_buildDate = 'Oct-03-2025'
2026-05-15T17:28:56Z [787ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] lib_physx_buildHash = '3a61992'
2026-05-15T17:28:56Z [787ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] lib_physx_buildRepo = 'gitlab-master.nvidia.com/omniverse/physics'
2026-05-15T17:28:56Z [788ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] lib_physx_buildVersion = '107.3.26'
2026-05-15T17:28:56Z [789ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] memoryStats = '(avail/total) RAM: 8.48206/15.8893GB, Swap: 46.9955/60.9908GB, VM: 131025/131072GB'
2026-05-15T17:28:56Z [789ms] [Warning] [carb.crashreporter-breakpad.plugin]
2026-05-15T17:28:56Z [790ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] osDistro = '25H2'
2026-05-15T17:28:56Z [791ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] osName = 'Windows 11 Pro'
2026-05-15T17:28:56Z [791ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] osVersion = '10.00.26200'
2026-05-15T17:28:56Z [792ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] portableMode = '1'
2026-05-15T17:28:56Z [793ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] primaryDisplayRes = '3840x2160x32bit@60Hz'
2026-05-15T17:28:56Z [793ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] runEnvironment = 'Individual'
2026-05-15T17:28:56Z [794ms] [Warning] [carb.crashreporter-breakpad.plugin] [crash] runningInContainer = '0'
If you are not sure if your problem is similar, observe two characteristics. One is whether there is a memory leak reported in the log and check the memory and system hardware configuration multiple times in the log. The other is whether the fault tracing is as high as 207. Using AI (such as GPT or Deepseek) to assist you in analyzing the log is also a good way for you to refer to
Beta Was this translation helpful? Give feedback.
All reactions