Cannot create ONNXRuntimeAdapter for ONNX FP16 model

[itallix]

Describe the bug

Creating ONNXRuntimeAdapter for FP16 ONNX model fails with TypeError.

The model is exported in the Geti Tune app with the following snippet:

exported_onnx_model_path = getitune_engine.export(
            checkpoint=model_checkpoint_path,
            export_format=ExportFormat.ONNX,
            export_precision=Precision.FP16,
        )

The ONNX model in FP32 format works.

Steps to Reproduce

1. Train any detection model in Geti Tune
2. Download ONNX artifact (available in FP16 format after training succeeds)
3. Try to create ONNXRuntimeAdapter with downloaded model:

from model_api.adapters import ONNXRuntimeAdapter

inference_adapter = ONNXRuntimeAdapter("./mobilenetv2/model.onnx", ort_options={"providers" : ['CPUExecutionProvider']})

Stacktrace:

TypeError                                 Traceback (most recent call last)
[/tmp/ipykernel_25233/3774717722.py](https://localhost:8080/#) in <cell line: 0>()
----> 1 inference_adapter = ONNXRuntimeAdapter("./mobilenetv2/model.onnx", ort_options={"providers" : ['CPUExecutionProvider']})
      2 ssd_model = SSD(inference_adapter=inference_adapter, preload=True)
      3 results = ssd_model(input_data)

4 frames
[/usr/local/lib/python3.12/dist-packages/onnxruntime/tools/symbolic_shape_infer.py](https://localhost:8080/#) in _get_shape_rank(self, node, idx)
    416 
    417     def _get_shape_rank(self, node, idx):
--> 418         return len(self._get_shape(node, idx))
    419 
    420     def _get_sympy_shape(self, node, idx):

TypeError: object of type 'NoneType' has no len()

ONNX_model.zip

Environment

• OS: Ubuntu 22.04.5
• OpenVINO version: 2026.1.0-21367-63e31528c62-releases/2026/1
• Inference device model and memory: CPU AMD EPYC 7B12

[itallix]

Issue seems to be on ONNX side when converting to FP16.
getitune library runs the following code on export:

from onnxconverter_common import float16

onnx_model = float16.convert_float_to_float16(onnx_model)

I am not able to run the following snippet for the exported model:

import onnxruntime as ort

session = ort.InferenceSession("model-fp16.onnx")

Error:

Fail                                      Traceback (most recent call last)
[/tmp/ipykernel_18879/684761369.py](https://localhost:8080/#) in <cell line: 0>()
----> 1 session = ort.InferenceSession("rtdetrr50-fp16.onnx")

1 frames
[/usr/local/lib/python3.12/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py](https://localhost:8080/#) in _create_inference_session(self, providers, provider_options, disabled_optimizers)
    622 
    623         if self._model_path:
--> 624             sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
    625         else:
    626             sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)

Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from rtdetrr50-fp16.onnx failed:Type Error: Type (tensor(float16)) of output arg (_to_copy) of node (node__to_copy) does not match expected type (tensor(float)).

There are multiple tickets reported in onnxconverter-common related to FP16 conversion.

PR

[leoll2]

Possible fix in open-edge-platform/training_extensions#6292

[kprokofi]

The fix has been merged. Can you check if it resolves your issue?
It fixes similar error with MaskRCNN

[itallix]

@kprokofi I have trained DINOv3-DETR-S and ATSS-MobileNet-V2 on Chess dataset.

Both ONNX FP16 variants still fail when trying to either compile the model with openvino or creating inference session using onnxruntime:

DINOv3-DETR-S:

OpenVINO:

RuntimeError                              Traceback (most recent call last)
[/tmp/ipykernel_3922/3258758416.py](https://localhost:8080/#) in <cell line: 0>()
      1 import openvino as ov
      2 core = ov.Core()
----> 3 model = core.read_model(MODEL_PATH)
      4 compiled_model = core.compile_model(model, "CPU")

[/usr/local/lib/python3.12/dist-packages/openvino/_ov_api.py](https://localhost:8080/#) in read_model(self, model, weights, config)
    563             return Model(super().read_model(model, bytes() if weights is None else weights))
    564         elif weights is None:
--> 565             return Model(super().read_model(model, config=config))
    566         else:
    567             return Model(super().read_model(model, weights, config))

RuntimeError: Exception from src/inference/src/cpp/core.cpp:95:
Check 'false' failed at src/frontends/onnx/frontend/src/frontend.cpp:184:
FrontEnd API failed with GeneralFailure:
Errors during ONNX translation:
While validating ONNX node '<Node(Mul): node_mul_51>': Check 'element::Type::merge(result_et, node->get_input_element_type(0), node->get_input_element_type(1))' failed at src/core/src/op/util/elementwise_args.cpp:18:
While validating node 'opset1::Multiply Multiply_11824 (opset1::Convert node__to_copy[0]:f32[1,8400,1], opset1::Concat node_concat_1[0]:f16[1,8400,192]) -> (dynamic[...])' with friendly_name 'Multiply_11824':
Arguments do not have the same element type (arg0 element type: f32, arg1 element type: f16).

onnxruntime:

Fail                                      Traceback (most recent call last)
[/tmp/ipykernel_3922/3474116706.py](https://localhost:8080/#) in <cell line: 0>()
----> 1 session = ort.InferenceSession(MODEL_PATH)

1 frames
[/usr/local/lib/python3.12/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py](https://localhost:8080/#) in _create_inference_session(self, providers, provider_options, disabled_optimizers)
    622 
    623         if self._model_path:
--> 624             sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
    625         else:
    626             sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)

Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from model.onnx failed:Type Error: Type (tensor(float16)) of output arg (_to_copy) of node (node__to_copy) does not match expected type (tensor(float)).

ATSS-MobileNet-V2:

OpenVINO:

RuntimeError                              Traceback (most recent call last)
[/tmp/ipykernel_3922/3258758416.py](https://localhost:8080/#) in <cell line: 0>()
      1 import openvino as ov
      2 core = ov.Core()
----> 3 model = core.read_model(MODEL_PATH)
      4 compiled_model = core.compile_model(model, "CPU")

[/usr/local/lib/python3.12/dist-packages/openvino/_ov_api.py](https://localhost:8080/#) in read_model(self, model, weights, config)
    563             return Model(super().read_model(model, bytes() if weights is None else weights))
    564         elif weights is None:
--> 565             return Model(super().read_model(model, config=config))
    566         else:
    567             return Model(super().read_model(model, weights, config))

RuntimeError: Exception from src/inference/src/cpp/core.cpp:95:
Check 'false' failed at src/frontends/onnx/frontend/src/frontend.cpp:184:
FrontEnd API failed with GeneralFailure:
Errors during ONNX translation:
While validating ONNX node '<Node(Where): node_where_6>': Check 'element::Type::merge(result_et, get_input_element_type(1), get_input_element_type(2))' failed at src/core/src/op/select.cpp:68:
While validating node 'opset1::Select Select_8065 (opset1::Unsqueeze node_unsqueeze_30[0]:boolean[1,..2800,1], opset1::Tile node_repeat_10[0]:f16[1,..2800,5], opset3::Broadcast node_ConstantOfShape_3004[0]:f32[1]) -> (dynamic[...])' with friendly_name 'Select_8065':
Argument 1 and 2 element types must match.

onnxruntime:

Fail                                      Traceback (most recent call last)
[/tmp/ipykernel_3922/3474116706.py](https://localhost:8080/#) in <cell line: 0>()
----> 1 session = ort.InferenceSession(MODEL_PATH)

1 frames
[/usr/local/lib/python3.12/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py](https://localhost:8080/#) in _create_inference_session(self, providers, provider_options, disabled_optimizers)
    622 
    623         if self._model_path:
--> 624             sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
    625         else:
    626             sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)

Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from model.onnx 6 failed:Type Error: Type (tensor(float16)) of output arg (val_3004) of node (node_ConstantOfShape_3004) does not match expected type (tensor(float)).