OpenVino Async runs OK?

[alexanderwendt]

As api=async execution does not measure layer times, but is faster than the api=sync, we want to use async execution and nireq=1 as the standard option.

It shall be checked if the programm runs without errors for async mode for:
https://github.com/embedded-machine-learning/scripts-and-guides/blob/main/scripts/hardwaremodules/openvino/run_pb_bench_sizes.py

and

https://github.com/embedded-machine-learning/scripts-and-guides/blob/main/scripts/hardwaremodules/openvino/test_write_results.py

[mwessley]

async at nireq=1 would be equal to sync mode.
In general higher number of nireq increases latency but at the same time achieves higher throughput. So I`d say we need to cover both cases.

[alexanderwendt]

What is the optimal setting for min. latency for sequencial images on OpenVino? Is then sync mode still our best option if batchsize=1?