Skip to content

NullReferenceException in Unity SentisModelInfo when loading ONNX model exported by ML-Agents (PyTorch 2.x onnxscript dependency conflict) #6293

@NeonCityLight

Description

@NeonCityLight

Bug Report: NullReferenceException in Unity SentisModelInfo when loading ONNX model exported by ML-Agents (PyTorch 2.x onnxscript dependency conflict)

Describe the bug

  1. When training the ML-Agents 3DBall example environment with PyTorch 2.x, the training runs normally, but the PyTorch ONNX export system mandatorily requires the onnxscript package, which causes a critical dependency conflict with ML-Agents' required onnx==1.15.0 and protobuf<3.21.
  2. After forcibly exporting an ONNX model (via workarounds like mocking onnxscript), Unity throws a NullReferenceException in the SentisModelInfo.CheckContinuousOutputs() method when assigning the model to the Agent's Behavior Parameters.
  3. The root cause: The exported ONNX model has an invalid format/unrecognized output nodes for Unity Sentis, making the model unreadable by the ML-Agents inference engine.

To Reproduce

Steps to reproduce the behavior:

  1. Set up the ML-Agents 1.2.0.dev0 (develop branch from source) environment on macOS with PyTorch 2.x installed.
  2. Train the 3DBall example environment using the default PPO config (config/ppo/3DBall.yaml).
  3. Export the ONNX model using the ML-Agents native model exporter (requires mocking onnxscript to bypass PyTorch 2.x dependency checks).
  4. Import the exported ONNX model into the Unity 3DBall project.
  5. Assign the ONNX model to the Model field of the Agent's Behavior Parameters component in Unity.
  6. View the Inspector panel and see the NullReferenceException error.

Console logs / stack traces

NullReferenceException: Object reference not set to an instance of an object
Unity.MLAgents.Inference.SentisModelInfo.CheckContinuousOutputs () (at /Users/gwen/Team/AgentEngineer/Unity/tool-pkg/ml-agents/com.unity.ml-agents/Runtime/Inference/SentisModelInfo.cs:240)
Unity.MLAgents.Inference.SentisModelInfo.GetOutputNames () (at /Users/gwen/Team/AgentEngineer/Unity/tool-pkg/ml-agents/com.unity.ml-agents/Runtime/Inference/SentisModelInfo.cs:211)
Unity.MLAgents.Inference.SentisModelInfo.CacheModelInfo () (at /Users/gwen/Team/AgentEngineer/Unity/tool-pkg/ml-agents/com.unity.ml-agents/Runtime/Inference/SentisModelInfo.cs:112)
Unity.MLAgents.Inference.SentisModelInfo..ctor (Unity.InferenceEngine.Model model, System.Boolean deterministicInference) (at /Users/gwen/Team/AgentEngineer/Unity/tool-pkg/ml-agents/com.unity.ml-agents/Runtime/Inference/SentisModelInfo.cs:50)
Unity.MLAgents.Inference.SentisModelParamLoader.CheckModel (Unity.InferenceEngine.Model model, Unity.MLAgents.Policies.BrainParameters brainParameters, Unity.MLAgents.Sensors.ISensor[] sensors, Unity.MLAgents.Actuators.ActuatorComponent[] actuatorComponents, System.Int32 observableAttributeTotalSize, Unity.MLAgents.Policies.BehaviorType behaviorType, System.Boolean deterministicInference) (at /Users/gwen/Team/AgentEngineer/Unity/tool-pkg/ml-agents/com.unity.ml-agents/Runtime/Inference/SentisModelParamLoader.cs:151)
Unity.MLAgents.Editor.BehaviorParametersEditor.DisplayFailedModelChecks () (at /Users/gwen/Team/AgentEngineer/Unity/tool-pkg/ml-agents/com.unity.ml-agents/Editor/BehaviorParametersEditor.cs:159)
Unity.MLAgents.Editor.BehaviorParametersEditor.OnInspectorGUI () (at /Users/gwen/Team/AgentEngineer/Unity/tool-pkg/ml-agents/com.unity.ml-agents/Editor/BehaviorParametersEditor.cs:93)
UnityEditor.UIElements.InspectorElement+<>c__DisplayClass82_0.<CreateInspectorElementUsingIMGUI>b__2 () (at /Users/bokken/build/output/unity/unity/Editor/Mono/UIElements/Inspector/InspectorElement.cs:797)
UnityEngine.GUIUtility:ProcessEvent(EntityId, IntPtr, Boolean&) (at /Users/bokken/build/output/unity/unity/Modules/IMGUI/GUIUtility.cs:224)

Screenshots

Not applicable (error is a full stack trace in the Unity Console/Inspector).

Environment

  • Unity Version: 2022.3 LTS
  • OS + version: macOS Sonoma 14.0+
  • ML-Agents version: 1.2.0.dev0 (develop branch, built from source)
  • Torch version: 2.0.0+
  • Environment: 3DBall (official ML-Agents example environment)

Additional context

  1. PyTorch 2.x ONNX export has a hard dependency on onnxscript, which requires onnx>=1.16.0 — this conflicts with ML-Agents' required onnx==1.15.0 (the only version compatible with Unity Sentis).
  2. Mocking/faking the onnxscript module bypasses the training error but produces an invalid ONNX model that Unity cannot parse.
  3. Protobuf version conflict: onnxscript requires protobuf>=4.25.1, while ML-Agents requires protobuf<3.21.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugIssue describes a potential bug in ml-agents.

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions