Skip to content

mps-overview.md

Latest commit

 

History

History
128 lines (102 loc) · 5.63 KB

mps-overview.md

File metadata and controls

128 lines (102 loc) · 5.63 KB

MPS Backend

:::{warning} Deprecated: The MPS backend is deprecated as of ExecuTorch 1.2 and will be removed in ExecuTorch 1.4. Please migrate to the CoreML backend for iOS/macOS GPU acceleration, or the Metal backend for macOS desktop GPU workloads. :::

MPS delegate is the ExecuTorch solution to take advantage of Apple's GPU for on-device ML using the MPS Graph framework and tuned kernels provided by MPS.

Target Requirements

Below are the minimum OS requirements on various hardware for running a MPS-delegated ExecuTorch model:

Development Requirements

To develop you need:

Before starting, make sure you install the Xcode Command Line Tools:

xcode-select --install

Using the MPS Backend

In this step, you will generate a simple ExecuTorch program that lowers MobileNetV3 model to the MPS delegate. You'll then pass this Program (the .pte file) during the runtime to run it using the MPS backend.

cd executorch
# Note: `mps_example` script uses by default the MPSPartitioner for ops that are not yet supported by the MPS delegate. To turn it off, pass `--no-use_partitioner`.
python3 -m examples.apple.mps.scripts.mps_example --model_name="mv3" --bundled --use_fp16

# To see all options, run following command:
python3 -m examples.apple.mps.scripts.mps_example --help

Runtime

Building the MPS executor runner:

# In this step, you'll be building the `mps_executor_runner` that is able to run MPS lowered modules:
cd executorch
./examples/apple/mps/scripts/build_mps_executor_runner.sh

Run the mv3 generated model using the mps_executor_runner

./cmake-out/examples/apple/mps/mps_executor_runner --model_path mv3_mps_float16_bundled.pte --bundled_program
  • You should see the following results. Note that no output file will be generated in this example:
I 00:00:00.003290 executorch:mps_executor_runner.mm:286] Model file mv3_mps_float16_bundled.pte is loaded.
I 00:00:00.003306 executorch:mps_executor_runner.mm:292] Program methods: 1
I 00:00:00.003308 executorch:mps_executor_runner.mm:294] Running method forward
I 00:00:00.003311 executorch:mps_executor_runner.mm:349] Setting up non-const buffer 1, size 606112.
I 00:00:00.003374 executorch:mps_executor_runner.mm:376] Setting up memory manager
I 00:00:00.003376 executorch:mps_executor_runner.mm:392] Loading method name from plan
I 00:00:00.018942 executorch:mps_executor_runner.mm:399] Method loaded.
I 00:00:00.018944 executorch:mps_executor_runner.mm:404] Loading bundled program...
I 00:00:00.018980 executorch:mps_executor_runner.mm:421] Inputs prepared.
I 00:00:00.118731 executorch:mps_executor_runner.mm:438] Model executed successfully.
I 00:00:00.122615 executorch:mps_executor_runner.mm:501] Model verified successfully.

[Optional] Run the generated model directly using pybind

  1. Make sure pybind MPS support was installed:
CMAKE_ARGS="-DEXECUTORCH_BUILD_MPS=ON" ./install_executorch.sh
  1. Run the mps_example script to trace the model and run it directly from python:
cd executorch
# Check correctness between PyTorch eager forward pass and ExecuTorch MPS delegate forward pass
python3 -m examples.apple.mps.scripts.mps_example --model_name="mv3" --no-use_fp16 --check_correctness
# You should see following output: `Results between ExecuTorch forward pass with MPS backend and PyTorch forward pass for mv3_mps are matching!`

# Check performance between PyTorch MPS forward pass and ExecuTorch MPS forward pass
python3 -m examples.apple.mps.scripts.mps_example --model_name="mv3" --no-use_fp16 --bench_pytorch

Profiling:

  1. [Optional] Generate an ETRecord while you're exporting your model.
cd executorch
python3 -m examples.apple.mps.scripts.mps_example --model_name="mv3" --generate_etrecord -b
  1. Run your Program on the ExecuTorch runtime and generate an ETDump.
./cmake-out/examples/apple/mps/mps_executor_runner --model_path mv3_mps_float16_bundled.pte --bundled_program --dump-outputs
  1. Create an instance of the Inspector API by passing in the ETDump you have sourced from the runtime along with the optionally generated ETRecord from step 1.
python3 -m devtools.inspector.inspector_cli --etdump_path etdump.etdp --etrecord_path etrecord.bin

Runtime integration

Step 1. Create the ExecuTorch core and MPS delegate frameworks to link on iOS

cd executorch
./scripts/build_apple_frameworks.sh --mps

mps_delegate.xcframework will be in cmake-out folder, along with executorch.xcframework and portable_delegate.xcframework:

cd cmake-out && ls

Step 2. Link the frameworks into your XCode project: Go to project Target’s Build Phases - Link Binaries With Libraries, click the + sign and add the frameworks: files located in Release folder.

  • executorch.xcframework
  • portable_delegate.xcframework
  • mps_delegate.xcframework

From the same page, include the needed libraries for the MPS delegate:

  • MetalPerformanceShaders.framework
  • MetalPerformanceShadersGraph.framework
  • Metal.framework

In this tutorial, you have learned how to lower a model to the MPS delegate, build the mps_executor_runner and run a lowered model through the MPS delegate, or directly on device using the MPS delegate static library.