[AUTO] Model compilation device selection per device utilization

[yangwang201911]

Details:

• Get CPU/GPU/NPU utilization from IPF service
• Add new device utilization threshold to update device selection logic
• Documentation changes were split out into a separate PR for independent review: [AUTO][DOCS] Split AUTO telemetry documentation changes from PR #36745 #36816.

Tickets:

• CVS-190401

AI Assistance:

• Yes
• AI was used to configure the IPF service as the third-party for AUTO plugin. Reviewed by human.

[Copilot]

Pull request overview

Adds IPF-backed device utilization monitoring (CPU/GPU/NPU) and a new AUTO property to optionally filter device selection based on utilization thresholds, with corresponding unit-test and documentation updates.

Changes:

• Introduces device monitor abstractions and platform-specific implementations (Windows via IPF; stubbed behavior on Linux).
• Extends AUTO plugin selection to consume ov::intel_auto::devices_utilization_threshold and consult utilization data during device choice.
• Updates C++/Python APIs, unit tests, and user documentation/snippets for the new property.

Reviewed changes

Copilot reviewed 41 out of 41 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
thirdparty/ipf/CMakeLists.txt	Adds a CMake wrapper to expose an imported IPF::ClientApi target from an external IPF SDK.
thirdparty/dependencies.cmake	Wires optional IPF SDK integration on Windows via IPF_SDK_ROOT.
src/tests/test_utils/common_test_utils/tests/utils_tests.cpp	Adds a basic test for ov::util::get_device_utilization.
src/plugins/auto/tests/unit/stateful_model_test.cpp	Adds cleanup for registered priorities in a stateful AUTO unit test.
src/plugins/auto/tests/unit/startup_fallback_property_test.cpp	Updates mocked select_device signature to include utilization thresholds.
src/plugins/auto/tests/unit/set_log_level_test.cpp	Updates mocked select_device signature to include utilization thresholds.
src/plugins/auto/tests/unit/select_device_test.cpp	Updates select_device mocking and adds parameterized tests for utilization-threshold-based selection.
src/plugins/auto/tests/unit/select_device_failed_test.cpp	Updates mocked select_device signature to include utilization thresholds.
src/plugins/auto/tests/unit/release_helper_test.cpp	Updates mocked select_device signature to include utilization thresholds.
src/plugins/auto/tests/unit/key_network_priority_test.cpp	Updates calls/mocks for select_device signature change.
src/plugins/auto/tests/unit/include/gmock_plugin.hpp	Extends mock plugin with utilization and property getters; updates select_device signature.
src/plugins/auto/tests/unit/get_device_list.cpp	Adds internal properties include and test data for net precisions.
src/plugins/auto/tests/unit/compile_model_metric_test.cpp	Updates mocked select_device signature to include utilization thresholds.
src/plugins/auto/tests/unit/auto_unit_test.cpp	Updates default mock behaviors for new get_property/get_device_utilization and select_device signature.
src/plugins/auto/src/plugin.hpp	Extends select_device signature and adds get_device_utilization method to AUTO plugin.
src/plugins/auto/src/plugin.cpp	Loads utilization thresholds and applies utilization-based exclusion in device selection; calls shared util monitor API.
src/plugins/auto/src/plugin_config.cpp	Registers the new devices_utilization_threshold property with a default empty map.
src/plugins/auto/src/common.hpp	Adds utilization-threshold storage to the scheduling context.
src/plugins/auto/src/auto_schedule.cpp	Passes utilization thresholds into device selection.
src/inference/include/openvino/runtime/auto/properties.hpp	Adds new public AUTO property devices_utilization_threshold.
src/common/util/src/monitors/win/xpu_device_monitor.cpp	Implements Windows XPU monitor wrapper that delegates to an impl.
src/common/util/src/monitors/win/xpu_device_monitor_impl.hpp	Implements Windows XPU utilization retrieval via IPF + JSON.
src/common/util/src/monitors/win/cpu_device_monitor.cpp	Implements Windows CPU monitor wrapper that delegates to an impl.
src/common/util/src/monitors/win/cpu_device_monitor_impl.hpp	Implements Windows CPU utilization retrieval via IPF + JSON.
src/common/util/src/monitors/lin/xpu_device_monitor.cpp	Implements Linux XPU monitor wrapper (currently stubbed impl).
src/common/util/src/monitors/lin/xpu_device_monitor_impl.hpp	Stub Linux XPU utilization impl returning empty data.
src/common/util/src/monitors/lin/cpu_device_monitor.cpp	Implements Linux CPU monitor wrapper (stubbed impl).
src/common/util/src/monitors/lin/cpu_device_monitor_impl.hpp	Stub Linux CPU utilization impl returning sentinel Total=-1.
src/common/util/src/monitors/device_monitor.cpp	Adds ov::util::get_device_utilization facade for CPU vs XPU monitors.
src/common/util/include/openvino/util/xpu_device_monitor.hpp	Declares XPU monitor interface.
src/common/util/include/openvino/util/idevice_monitor.hpp	Declares monitor interfaces used by CPU/XPU monitors.
src/common/util/include/openvino/util/device_monitor.hpp	Declares the public get_device_utilization API.
src/common/util/include/openvino/util/cpu_device_monitor.hpp	Declares CPU monitor interface.
src/common/util/CMakeLists.txt	Adds monitor sources to openvino::util and links IPF target when present.
src/bindings/python/tests/test_runtime/test_properties.py	Adds a Python test asserting registration/typing behavior for the new AUTO property.
src/bindings/python/src/pyopenvino/utils/utils.hpp	Declares new helper conversions used in Python bindings.
src/bindings/python/src/pyopenvino/utils/utils.cpp	Extends Any→Python conversion for map<string,unsigned> and adds config conversion logic for the new property.
src/bindings/python/src/pyopenvino/core/properties/properties.cpp	Exposes devices_utilization_threshold in the intel_auto Python submodule.
docs/articles_en/openvino-workflow/running-inference/inference-devices-and-modes/auto-device-selection.rst	Documents the new utilization-threshold property and adds a “Load Balancing” section.
docs/articles_en/assets/snippets/ov_auto.py	Adds a Python snippet demonstrating utilization-threshold configuration.
docs/articles_en/assets/snippets/AUTO8.cpp	Adds a C++ snippet demonstrating utilization-threshold configuration.

[Copilot]

Pull request overview

Copilot reviewed 41 out of 41 changed files in this pull request and generated 7 comments.

Comments suppressed due to low confidence (1)

src/tests/test_utils/common_test_utils/tests/utils_tests.cpp:1

• On Windows, CPUDeviceMonitorImpl returns {\"Total\": -1.0f} when IPF is unavailable or parsing fails. This test currently requires Total >= 0.0f, which will fail in common CI/dev setups without a working IPF runtime. Consider accepting -1.0f as the documented 'unavailable' value (similar to the non-Windows branch), or conditionally skipping/relaxing the assertion when IPF is not present.

[yangwang201911]

@yangwang201911 Please move the doc/ folder changes to another draft PR.

#36816

[Copilot]

Pull request overview

Copilot reviewed 28 out of 28 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 28 out of 28 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 28 out of 28 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 28 out of 28 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 28 out of 28 changed files in this pull request and generated 1 comment.