Skip to content

build-run-openvino.md

Latest commit

 

History

History
135 lines (91 loc) · 5.35 KB

build-run-openvino.md

File metadata and controls

135 lines (91 loc) · 5.35 KB

Building and Running ExecuTorch with OpenVINO Backend

In this tutorial we will walk you through the process of setting up the prerequisites, building OpenVINO backend library, exporting .pte models with OpenVINO optimizations, and executing the exported models on Intel hardware.

::::{grid} 2 :::{grid-item-card} What you will learn in this tutorial: :class-card: card-prerequisites

Introduction to OpenVINO

OpenVINO is an open-source toolkit designed to enhance AI inference on Intel hardware by reducing latency and increasing throughput while preserving accuracy. It optimizes hardware utilization and simplifies AI development and deep learning integration across domains such as computer vision, large language models (LLMs), and generative AI.

OpenVINO is integrated as an Executorch delegate to accelerate AI applications deployed with Executorch APIs.

Supported Hardware

OpenVINO backend supports the following hardware:

  • Intel CPUs
  • Intel integrated GPUs
  • Intel discrete GPUs
  • Intel NPUs

For more information on the supported hardware, please refer to OpenVINO System Requirements page.

Instructions for Building OpenVINO Backend

Quick Start (pip wheel, Linux)

The easiest way to use the OpenVINO backend is via the prebuilt pip wheel:

pip install executorch[openvino]

The backend automatically discovers the OpenVINO C library from the pip-installed package — no environment variable setup is needed.

See the OpenVINO backend README for verification steps and troubleshooting.

Building from Source

If you need a custom build (different platform, custom configuration, etc.), follow the instructions below.

Prerequisites

Before you begin, ensure you have the OpenVINO runtime installed and configured on your system. For most users, we recommend installing it via pip:

pip install "openvino>=2025.1.0,<2026.0.0"

Alternatively, you can build OpenVINO from source or follow the official OpenVINO installation guide for platform-specific options:

git clone https://github.com/openvinotoolkit/openvino.git
cd openvino && git checkout releases/2025/1
git submodule update --init --recursive
sudo ./install_build_dependencies.sh
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -DENABLE_PYTHON=ON
make -j<N>

cd ..
cmake --install build --prefix <your_preferred_install_location>
cd <your_preferred_install_location>
source setupvars.sh

For more information about building OpenVINO from source, refer to the OpenVINO Build Instructions.

Setup

Follow the steps below to setup your build environment:

  1. Setup ExecuTorch Environment: Refer to the Environment Setup guide for detailed instructions on setting up the ExecuTorch environment.

  2. Setup OpenVINO Backend Environment

  • Install the dependent libs. Ensure that you are inside executorch/backends/openvino/ directory 
    pip install -r requirements.txt

  1. Navigate to scripts/ directory.

  2. Build OpenVINO Backend: Once the prerequisites are in place, run the openvino_build.sh script to start the build process, OpenVINO backend will be built under cmake-out/backends/openvino/ as libopenvino_backend.a

    ./openvino_build.sh

Build Instructions for Examples

AOT step:

Refer to the README.md in the executorch/examples/openvino folder for detailed instructions on exporting deep learning models from various model suites (TIMM, Torchvision, Hugging Face) to openvino backend using Executorch. Users can dynamically specify the model, input shape, and target device.

Below is an example to export a ResNet50 model from Torchvision model suite for CPU device with an input shape of [1, 3, 256, 256]

cd executorch/examples/openvino
python aot_optimize_and_infer.py --export --suite torchvision --model resnet50 --input_shape "(1, 3, 256, 256)" --device CPU

The exported model will be saved as 'resnet50.pte' in the current directory.

Build C++ OpenVINO Examples

After building the OpenVINO backend following the instructions above, the executable will be saved in <executorch_root>/cmake-out/.

The executable requires a model file (.pte file generated in the aot step) and the number of inference executions.

Example Usage

Run inference with a given model for 10 executions:

./executor_runner \
    --model_path=model.pte \
    --num_executions=10

Support

If you encounter any issues while reproducing the tutorial, please file a github issue on ExecuTorch repo and tag use #openvino tag