diff --git a/docs/source/android-examples.md b/docs/source/android-examples.md index b56ed660f63..24af5750d56 100644 --- a/docs/source/android-examples.md +++ b/docs/source/android-examples.md @@ -2,9 +2,9 @@ - [Working with LLMs - Android Examples](https://github.com/meta-pytorch/executorch-examples/blob/main/llm/android/LlamaDemo/README.md) - ExecuTorch Llama Android Demo App - [Demo Apps](https://github.com/meta-pytorch/executorch-examples/tree/main/dl3/android/DeepLabV3Demo#executorch-android-demo-app) - DeepLab v3 model for image segmentation -- {doc}`backends/arm-vgf/tutorials/arm-vgf-tutorials` — Export a simple PyTorch model for the ExecuTorch VGF backend +- {doc}`backends/arm-vgf/tutorials/vgf-getting-started` — Export a simple PyTorch model for the ExecuTorch VGF backend ```{toctree} :hidden: -backends/arm-vgf/tutorials/arm-vgf-tutorials +backends/arm-vgf/tutorials/vgf-getting-started ``` diff --git a/docs/source/android-samsung-exynos.md b/docs/source/android-samsung-exynos.md index 4c5a470edca..d2c020ba0fd 100644 --- a/docs/source/android-samsung-exynos.md +++ b/docs/source/android-samsung-exynos.md @@ -1 +1,2 @@ -```{include} backends-samsung-exynos.md +```{include} backends/samsung/samsung-overview.md +``` diff --git a/docs/source/api-life-cycle.md b/docs/source/api-life-cycle.md index 3ccaa4eddb1..fcf82094d73 100644 --- a/docs/source/api-life-cycle.md +++ b/docs/source/api-life-cycle.md @@ -80,21 +80,72 @@ communicate state to developers. Language + API state + Code Documentation + + All + + Stable + + + +No annotation is required. APIs are considered stable if they are not marked as +experimental or deprecated. + + + + +No warning is required. + + + + + All + + Deleted + + + +Remove the API from code after the deprecation period has passed. + + + + +Remove references to the deleted API from documentation. + + + Python + Deprecated + Use the executorch.exir._warnings.deprecated decorator. -

+ + + +Use .. warning:: in the docstring. Clearly point to the +replacement API when one exists. + + + + + Python + + Experimental + + + Use the executorch.exir._warnings.experimental decorator. @@ -102,25 +153,22 @@ decorator. -Use .. warning:: in the docstrings of deprecated and experimental -APIs. See +Use .. warning:: in the docstring. State that the API is +experimental and may change or be removed without notice. See example usage. - C++ + Deprecated + Use the ET_DEPRECATED annotation macro. See example usage. -

-

-Use the ET_EXPERIMENTAL annotation macro. - @@ -128,23 +176,33 @@ Start Doxygen comments with DEPRECATED: See example usage. -

-

+ + + + C++ + + Experimental + + + +Use the ET_EXPERIMENTAL annotation macro. + + + + Start Doxygen comments with EXPERIMENTAL:. + Java + Deprecated + Use java.lang.Deprecated. -

-

- -Use androidx.annotation.RequiresOptIn. -

@@ -152,6 +210,19 @@ Use androidx.annotation.RequiresOptIn. + + +

/**
 * Warning: This API is experimental.
@@ -161,12 +232,11 @@ Use 
-

-

-__attribute__((deprecated("This API is experimental and may change without notice.")));

@@ -175,6 +245,18 @@ Use + +


 /**
@@ -186,17 +268,28 @@ Use 
+   
+   
 

+/// - Warning: Deprecated. Use `newMethod()` instead. + + + + Swift + + Experimental + +

@available(*, message: "This API is experimental")

-/// - Warning: Deprecated. Use `newMethod()` instead. -

/// - Warning: This API is experimental. diff --git a/docs/source/backends-overview.md b/docs/source/backends-overview.md index 3d54a150eac..ad3d3b507c1 100644 --- a/docs/source/backends-overview.md +++ b/docs/source/backends-overview.md @@ -65,4 +65,5 @@ backends/arm-vgf/arm-vgf-overview build-run-openvino backends/nxp/nxp-overview backends-cadence -backends/samsung/samsung-overview +Samsung Exynos Backend +``` diff --git a/docs/source/backends/arm-ethos-u/arm-ethos-u-overview.md b/docs/source/backends/arm-ethos-u/arm-ethos-u-overview.md index faffedece35..0b82cc9e56f 100644 --- a/docs/source/backends/arm-ethos-u/arm-ethos-u-overview.md +++ b/docs/source/backends/arm-ethos-u/arm-ethos-u-overview.md @@ -209,7 +209,7 @@ ExecuTorch for the Ethos-U backend, you automatically install the compiler conta **→{doc}`/backends/arm-ethos-u/arm-ethos-u-troubleshooting` — Troubleshooting and common issues.** -**→{doc}`/backends/arm-ethos-u/tutorials/arm-ethos-u-tutorials` — Tutorials.** +**→{doc}`/backends/arm-ethos-u/tutorials/ethos-u-getting-started` — Getting started tutorial.** **→{doc}`/backends/arm-ethos-u/U55_op_support` — Ethos-U55 supported operators.** @@ -224,7 +224,7 @@ ExecuTorch for the Ethos-U backend, you automatically install the compiler conta arm-ethos-u-partitioner arm-ethos-u-quantization arm-ethos-u-troubleshooting -tutorials/arm-ethos-u-tutorials +tutorials/ethos-u-getting-started U55_op_support U85_op_support ``` diff --git a/docs/source/backends/arm-ethos-u/tutorials/arm-ethos-u-tutorials.md b/docs/source/backends/arm-ethos-u/tutorials/arm-ethos-u-tutorials.md deleted file mode 100644 index 4b540f2179d..00000000000 --- a/docs/source/backends/arm-ethos-u/tutorials/arm-ethos-u-tutorials.md +++ /dev/null @@ -1,10 +0,0 @@ -# Arm Ethos-U Backend Tutorials - -**→{doc}`ethos-u-getting-started`** - -```{toctree} -:maxdepth: 2 -:hidden: -:caption: Tutorials - -ethos-u-getting-started diff --git a/docs/source/backends/arm-vgf/arm-vgf-overview.md b/docs/source/backends/arm-vgf/arm-vgf-overview.md index dd3b00eb356..2f4523a1eb9 100644 --- a/docs/source/backends/arm-vgf/arm-vgf-overview.md +++ b/docs/source/backends/arm-vgf/arm-vgf-overview.md @@ -129,7 +129,7 @@ described in the rest of this guide but with a concrete end-to-end sample. **→{doc}`/backends/arm-vgf/arm-vgf-troubleshooting` — Debug common issues.** -**→{doc}`/backends/arm-vgf/tutorials/arm-vgf-tutorials` — Tutorials.** +**→{doc}`/backends/arm-vgf/tutorials/vgf-getting-started` — Getting started tutorial.** **→{doc}`/backends/arm-vgf/VGF_op_support` — VGF supported operators.** @@ -142,6 +142,6 @@ described in the rest of this guide but with a concrete end-to-end sample. arm-vgf-partitioner arm-vgf-quantization arm-vgf-troubleshooting -tutorials/arm-vgf-tutorials +tutorials/vgf-getting-started VGF_op_support ``` diff --git a/docs/source/backends/arm-vgf/tutorials/arm-vgf-tutorials.md b/docs/source/backends/arm-vgf/tutorials/arm-vgf-tutorials.md deleted file mode 100644 index ceb4304a814..00000000000 --- a/docs/source/backends/arm-vgf/tutorials/arm-vgf-tutorials.md +++ /dev/null @@ -1,10 +0,0 @@ -# Arm VGF Backend Tutorials - -**→{doc}`vgf-getting-started`** - -```{toctree} -:maxdepth: 2 -:hidden: -:caption: Tutorials - -vgf-getting-started diff --git a/docs/source/backends/arm-vgf/tutorials/vgf-getting-started.md b/docs/source/backends/arm-vgf/tutorials/vgf-getting-started.md index 9600f3b9d54..44e1ca59d93 100644 --- a/docs/source/backends/arm-vgf/tutorials/vgf-getting-started.md +++ b/docs/source/backends/arm-vgf/tutorials/vgf-getting-started.md @@ -168,9 +168,9 @@ To produce a pte file equivalent to the one above, run `python -m backends.arm.scripts.aot_arm_compiler --model_name=add --delegate --quantize --output=simple_example.pte --target=vgf` ``` -### Runtime: +## Runtime -## Build executor runtime +### Build executor runtime After the AOT compilation flow is done, we can build the executor runner target. For this tutorial, the default runner can be used. Build it with the following configuration: @@ -200,7 +200,7 @@ The block diagram below demonstrates, at the high level, how the various build a ![](arm-delegate-runtime-build.svg) -## Deploying and running on device +### Deploying and running on device Since we are using the Vulkan emulation layer, we can run the executor runner with the VGF delegate on the host machine: diff --git a/docs/source/backends/xnnpack/xnnpack-quantization.md b/docs/source/backends/xnnpack/xnnpack-quantization.md index e0180393f9e..74d8eafba72 100644 --- a/docs/source/backends/xnnpack/xnnpack-quantization.md +++ b/docs/source/backends/xnnpack/xnnpack-quantization.md @@ -61,7 +61,7 @@ See [PyTorch 2 Export Post Training Quantization](https://docs.pytorch.org/ao/ma The XNNPACK backend also supports quantizing models with the [torchao](https://github.com/pytorch/ao) quantize_ API. This is most commonly used for LLMs, requiring more advanced quantization. Since quantize_ is not backend aware, it is important to use a config that is compatible with CPU/XNNPACK: -* Quantize embeedings with `IntxWeightOnlyConfig` (with weight_dtype torch.int2, torch.int4, or torch.int8, using PerGroup or PerAxis granularity) +* Quantize embeddings with `IntxWeightOnlyConfig` (with weight_dtype torch.int2, torch.int4, or torch.int8, using PerGroup or PerAxis granularity) * Quantize linear layers with 4 bit weight and 8bit dynamic activation, use `Int8DynamicActivationIntxWeightConfig` (with weight_dtype=torch.int4, using PerGroup or PerAxis granularity) Below is a simple example, but a more detailed tutorial including accuracy evaluation on popular LLM benchmarks can be found in the [torchao documentation](https://docs.pytorch.org/ao/main/serving.html#mobile-deployment-with-executorch). @@ -79,13 +79,13 @@ embedding_config = IntxWeightOnlyConfig( weight_dtype=torch.int8, granularity=PerAxis(0), ) -qunatize_( +quantize_( eager_model, lambda m, fqn: isinstance(m, torch.nn.Embedding), ) -# Quatize linear layers with 8-bit dynamic activations and 4-bit weights +# Quantize linear layers with 8-bit dynamic activations and 4-bit weights linear_config = Int8DynamicActivationIntxWeightConfig( weight_dtype=torch.int4, weight_granularity=PerGroup(32), diff --git a/docs/source/conf.py b/docs/source/conf.py index 75757b7da27..6fffab9f3d7 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -299,6 +299,8 @@ "llm/llama-demo-android": "../using-executorch-android.html", "tutorial-arm-ethos-u": "backends/arm-ethos-u/tutorials/ethos-u-getting-started.html", "tutorial-arm-vgf": "backends/arm-vgf/tutorials/vgf-getting-started.html", + "backends/arm-ethos-u/tutorials/arm-ethos-u-tutorials": "ethos-u-getting-started.html", + "backends/arm-vgf/tutorials/arm-vgf-tutorials": "vgf-getting-started.html", "visualization": "visualize.html", } diff --git a/docs/source/desktop-coreml.md b/docs/source/desktop-coreml.md index 48271326d87..129bc02ce64 100644 --- a/docs/source/desktop-coreml.md +++ b/docs/source/desktop-coreml.md @@ -1 +1,2 @@ -```{include} backends-coreml.md +```{include} backends/coreml/coreml-overview.md +``` diff --git a/docs/source/desktop-mps.md b/docs/source/desktop-mps.md index d6f305d33aa..6a0014dea0e 100644 --- a/docs/source/desktop-mps.md +++ b/docs/source/desktop-mps.md @@ -1 +1,2 @@ -```{include} backends-mps.md +```{include} backends/mps/mps-overview.md +``` diff --git a/docs/source/embedded-arm-ethos-u.md b/docs/source/embedded-arm-ethos-u.md index cdc544a6553..b0e161f1ca5 100644 --- a/docs/source/embedded-arm-ethos-u.md +++ b/docs/source/embedded-arm-ethos-u.md @@ -1 +1,2 @@ -```{include} backends-arm-ethos-u.md +```{include} backends/arm-ethos-u/arm-ethos-u-overview.md +``` diff --git a/docs/source/embedded-backends.md b/docs/source/embedded-backends.md index 147f6cfc151..31b04f1839d 100644 --- a/docs/source/embedded-backends.md +++ b/docs/source/embedded-backends.md @@ -13,7 +13,7 @@ Available hardware acceleration backends for embedded systems. ## NPU Acceleration -- {doc}`embedded-arm-ethos-u` — ARM Ethos-U NPU acceleration +- {doc}`embedded-arm-ethos-u` — Arm Ethos-U NPU acceleration - {doc}`embedded-nxp` — NXP eIQ Neutron Backend @@ -23,3 +23,4 @@ embedded-arm-cortex-m embedded-cadence embedded-arm-ethos-u embedded-nxp +``` diff --git a/docs/source/ir-exir.md b/docs/source/ir-exir.md index d3357d34a03..0c819cf558a 100644 --- a/docs/source/ir-exir.md +++ b/docs/source/ir-exir.md @@ -175,4 +175,19 @@ for all core ATen ops. ## Backend Dialect -See this [doc](compiler-backend-dialect.md) +Backend dialect is the optional, target-aware stage after Edge dialect. It is +used when a backend needs to rewrite the graph with backend-specific operators, +metadata, or delegated lowered modules before the program is converted to an +ExecuTorch program. For example, a backend pass can replace an Edge dialect +subgraph such as `addmm` followed by `relu` with a single backend operator that +the target can execute more efficiently. + +Unlike custom operators that may appear in eager mode, ATen dialect, or Edge +dialect, backend-specific operators are introduced only by passes that run after +Edge dialect. They are useful for target-specific fusions, lowering patterns, +or delegate integration where the resulting graph contains nodes meaningful only +to the selected backend. + +For details on when to use backend-specific operators, how they differ from +custom operators and delegates, and how to register replacement patterns with +`bind_pattern_to_op`, see [Backend Dialect](compiler-backend-dialect.md). diff --git a/docs/source/tools-section.md b/docs/source/tools-section.md index 6d8061dd33a..d2bb44086f8 100644 --- a/docs/source/tools-section.md +++ b/docs/source/tools-section.md @@ -13,7 +13,7 @@ In this section, explore ExecuTorch's comprehensive developer tools for profilin - {doc}`model-inspector` — Model Inspector - {doc}`memory-planning-inspection` — Memory Planning Inspection - {doc}`devtools-tutorial` — Development Utilities -- {doc}`visualize` — Model Visualization +- [Model Visualization](visualize.md) — Visualize exported models as computational graphs ```{toctree} :hidden: @@ -29,4 +29,5 @@ model-debugging model-inspector memory-planning-inspection devtools-tutorial -visualize +Model Visualization +```