Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/source/android-examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@

- [Working with LLMs - Android Examples](https://github.com/meta-pytorch/executorch-examples/blob/main/llm/android/LlamaDemo/README.md) - ExecuTorch Llama Android Demo App
- [Demo Apps](https://github.com/meta-pytorch/executorch-examples/tree/main/dl3/android/DeepLabV3Demo#executorch-android-demo-app) - DeepLab v3 model for image segmentation
- {doc}`backends/arm-vgf/tutorials/arm-vgf-tutorials` — Export a simple PyTorch model for the ExecuTorch VGF backend
- {doc}`backends/arm-vgf/tutorials/vgf-getting-started` — Export a simple PyTorch model for the ExecuTorch VGF backend

```{toctree}
:hidden:
backends/arm-vgf/tutorials/arm-vgf-tutorials
backends/arm-vgf/tutorials/vgf-getting-started
```
3 changes: 2 additions & 1 deletion docs/source/android-samsung-exynos.md
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
```{include} backends-samsung-exynos.md
```{include} backends/samsung/samsung-overview.md
```
133 changes: 113 additions & 20 deletions docs/source/api-life-cycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,78 +80,149 @@ communicate state to developers.
<tr>
<td><strong>Language</strong>
</td>
<td><strong>API state</strong>
</td>
<td><strong>Code</strong>
</td>
<td><strong>Documentation</strong>
</td>
</tr>
<tr>
<td>All
</td>
<td>Stable
</td>
<td>

No annotation is required. APIs are considered stable if they are not marked as
experimental or deprecated.

</td>
<td>

No warning is required.

</td>
</tr>
<tr>
<td>All
</td>
<td>Deleted
</td>
<td>

Remove the API from code after the deprecation period has passed.

</td>
<td>

Remove references to the deleted API from documentation.

</td>
</tr>
<tr>
<td>Python
</td>
<td>Deprecated
</td>
<td>

Use the
<a href="https://github.com/pytorch/executorch/blob/main/exir/_warnings.py">executorch.exir._warnings.deprecated</a>
decorator.

<p>
</td>
<td>

Use <code>.. warning::</code> in the docstring. Clearly point to the
replacement API when one exists.

</td>
</tr>
<tr>
<td>Python
</td>
<td>Experimental
</td>
<td>

Use the
<a href="https://github.com/pytorch/executorch/blob/main/exir/_warnings.py">executorch.exir._warnings.experimental</a>
decorator.

</td>
<td>

Use <code>.. warning::</code> in the docstrings of deprecated and experimental
APIs. See
Use <code>.. warning::</code> in the docstring. State that the API is
experimental and may change or be removed without notice. See
<a href="https://github.com/pytorch/pytorch/blob/main/torch/nn/utils/stateless.py#L176">example
usage</a>.

</ul>
</td>
</tr>
<tr>
<td>C++
</td>
<td>Deprecated
</td>
<td>

Use the <code>ET_DEPRECATED</code> annotation macro. See <a href="https://github.com/pytorch/executorch/blob/main/runtime/executor/program.h#L92">example usage</a>.

<p>
<p>
Use the <code>ET_EXPERIMENTAL</code> annotation macro.
</ul>
</td>
<td>

Start Doxygen comments with <code>DEPRECATED:</code> See
<a href="https://github.com/pytorch/executorch/blob/main/runtime/executor/program.h#L164">example
usage</a>.

<p>
<p>
</td>
</tr>
<tr>
<td>C++
</td>
<td>Experimental
</td>
<td>

Use the <code>ET_EXPERIMENTAL</code> annotation macro.

</td>
<td>

Start Doxygen comments with <code>EXPERIMENTAL:</code>.

</td>
</tr>
<tr>
<td>Java
</td>
<td>Deprecated
</td>
<td>

Use <a href="https://docs.oracle.com/javase/9/docs/api/java/lang/Deprecated.html">java.lang.Deprecated</a>.

<p>
<p>

Use <a href="https://cs.android.com/androidx/platform/frameworks/support/+/androidx-main:docs/api_guidelines/annotations.md">androidx.annotation.RequiresOptIn</a>.

</td>
<td>
<p>
<pre><code>/**
* @deprecated Use {@link #newMethod()} instead.
*/
</code></pre>
</td>
</tr>
<tr>
<td>Java
</td>
<td>Experimental
</td>
<td>

Use <a href="https://cs.android.com/androidx/platform/frameworks/support/+/androidx-main:docs/api_guidelines/annotations.md">androidx.annotation.RequiresOptIn</a>.

</td>
<td>
<p>
<pre><code>/**
* Warning: This API is experimental.
Expand All @@ -161,12 +232,11 @@ Use <a href="https://cs.android.com/androidx/platform/frameworks/support/+/andro
<tr>
<td>Objective-C
</td>
<td>Deprecated
</td>
<td>
<p>
<code>__attribute__((deprecated("Use newMethod instead")));</code>
<p>
<p>
<code>__attribute__((deprecated("This API is experimental and may change without notice.")));</code>
</td>
<td>
<p>
Expand All @@ -175,6 +245,18 @@ Use <a href="https://cs.android.com/androidx/platform/frameworks/support/+/andro
* @deprecated Use `newMethod` instead.
*/
</code></pre>
</td>
</tr>
<tr>
<td>Objective-C
</td>
<td>Experimental
</td>
<td>
<p>
<code>__attribute__((deprecated("This API is experimental and may change without notice.")));</code>
</td>
<td>
<p>
<pre><code>
/**
Expand All @@ -186,17 +268,28 @@ Use <a href="https://cs.android.com/androidx/platform/frameworks/support/+/andro
<tr>
<td>Swift
</td>
<td>Deprecated
</td>
<td>
<p>
<code>@available(*, deprecated, message: "Use newMethod instead")</code>
</td>
<td>
<p>
<code>/// - Warning: Deprecated. Use `newMethod()` instead.</code>
</td>
</tr>
<tr>
<td>Swift
</td>
<td>Experimental
</td>
<td>
<p>
<code>@available(*, message: "This API is experimental")</code>
</td>
<td>
<p>
<code>/// - Warning: Deprecated. Use `newMethod()` instead.</code>
<p>
<code>/// - Warning: This API is experimental.</code>
</td>
</tr>
Expand Down
3 changes: 2 additions & 1 deletion docs/source/backends-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,4 +65,5 @@ backends/arm-vgf/arm-vgf-overview
build-run-openvino
backends/nxp/nxp-overview
backends-cadence
backends/samsung/samsung-overview
Samsung Exynos Backend <backends/samsung/samsung-overview>
```
4 changes: 2 additions & 2 deletions docs/source/backends/arm-ethos-u/arm-ethos-u-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,7 @@ ExecuTorch for the Ethos-U backend, you automatically install the compiler conta

**→{doc}`/backends/arm-ethos-u/arm-ethos-u-troubleshooting` — Troubleshooting and common issues.**

**→{doc}`/backends/arm-ethos-u/tutorials/arm-ethos-u-tutorials` — Tutorials.**
**→{doc}`/backends/arm-ethos-u/tutorials/ethos-u-getting-started` — Getting started tutorial.**

**→{doc}`/backends/arm-ethos-u/U55_op_support` — Ethos-U55 supported operators.**

Expand All @@ -224,7 +224,7 @@ ExecuTorch for the Ethos-U backend, you automatically install the compiler conta
arm-ethos-u-partitioner
arm-ethos-u-quantization
arm-ethos-u-troubleshooting
tutorials/arm-ethos-u-tutorials
tutorials/ethos-u-getting-started
U55_op_support
U85_op_support
```

This file was deleted.

4 changes: 2 additions & 2 deletions docs/source/backends/arm-vgf/arm-vgf-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ described in the rest of this guide but with a concrete end-to-end sample.

**→{doc}`/backends/arm-vgf/arm-vgf-troubleshooting` — Debug common issues.**

**→{doc}`/backends/arm-vgf/tutorials/arm-vgf-tutorials` — Tutorials.**
**→{doc}`/backends/arm-vgf/tutorials/vgf-getting-started` — Getting started tutorial.**

**→{doc}`/backends/arm-vgf/VGF_op_support` — VGF supported operators.**

Expand All @@ -142,6 +142,6 @@ described in the rest of this guide but with a concrete end-to-end sample.
arm-vgf-partitioner
arm-vgf-quantization
arm-vgf-troubleshooting
tutorials/arm-vgf-tutorials
tutorials/vgf-getting-started
VGF_op_support
```
10 changes: 0 additions & 10 deletions docs/source/backends/arm-vgf/tutorials/arm-vgf-tutorials.md

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -168,9 +168,9 @@ To produce a pte file equivalent to the one above, run
`python -m backends.arm.scripts.aot_arm_compiler --model_name=add --delegate --quantize --output=simple_example.pte --target=vgf`
```

### Runtime:
## Runtime

## Build executor runtime
### Build executor runtime

After the AOT compilation flow is done, we can build the executor runner target. For this tutorial, the default runner can be used. Build it with the following configuration:

Expand Down Expand Up @@ -200,7 +200,7 @@ The block diagram below demonstrates, at the high level, how the various build a
![](arm-delegate-runtime-build.svg)


## Deploying and running on device
### Deploying and running on device

Since we are using the Vulkan emulation layer, we can run the executor runner with the VGF delegate on the host machine:

Expand Down
6 changes: 3 additions & 3 deletions docs/source/backends/xnnpack/xnnpack-quantization.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ See [PyTorch 2 Export Post Training Quantization](https://docs.pytorch.org/ao/ma

The XNNPACK backend also supports quantizing models with the [torchao](https://github.com/pytorch/ao) quantize_ API. This is most commonly used for LLMs, requiring more advanced quantization. Since quantize_ is not backend aware, it is important to use a config that is compatible with CPU/XNNPACK:

* Quantize embeedings with `IntxWeightOnlyConfig` (with weight_dtype torch.int2, torch.int4, or torch.int8, using PerGroup or PerAxis granularity)
* Quantize embeddings with `IntxWeightOnlyConfig` (with weight_dtype torch.int2, torch.int4, or torch.int8, using PerGroup or PerAxis granularity)
* Quantize linear layers with 4 bit weight and 8bit dynamic activation, use `Int8DynamicActivationIntxWeightConfig` (with weight_dtype=torch.int4, using PerGroup or PerAxis granularity)

Below is a simple example, but a more detailed tutorial including accuracy evaluation on popular LLM benchmarks can be found in the [torchao documentation](https://docs.pytorch.org/ao/main/serving.html#mobile-deployment-with-executorch).
Expand All @@ -79,13 +79,13 @@ embedding_config = IntxWeightOnlyConfig(
weight_dtype=torch.int8,
granularity=PerAxis(0),
)
qunatize_(
quantize_(
eager_model,
lambda m, fqn: isinstance(m, torch.nn.Embedding),
)


# Quatize linear layers with 8-bit dynamic activations and 4-bit weights
# Quantize linear layers with 8-bit dynamic activations and 4-bit weights
linear_config = Int8DynamicActivationIntxWeightConfig(
weight_dtype=torch.int4,
weight_granularity=PerGroup(32),
Expand Down
2 changes: 2 additions & 0 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -299,6 +299,8 @@
"llm/llama-demo-android": "../using-executorch-android.html",
"tutorial-arm-ethos-u": "backends/arm-ethos-u/tutorials/ethos-u-getting-started.html",
"tutorial-arm-vgf": "backends/arm-vgf/tutorials/vgf-getting-started.html",
"backends/arm-ethos-u/tutorials/arm-ethos-u-tutorials": "ethos-u-getting-started.html",
"backends/arm-vgf/tutorials/arm-vgf-tutorials": "vgf-getting-started.html",
"visualization": "visualize.html",
}

Expand Down
3 changes: 2 additions & 1 deletion docs/source/desktop-coreml.md
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
```{include} backends-coreml.md
```{include} backends/coreml/coreml-overview.md
```
3 changes: 2 additions & 1 deletion docs/source/desktop-mps.md
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
```{include} backends-mps.md
```{include} backends/mps/mps-overview.md
```
3 changes: 2 additions & 1 deletion docs/source/embedded-arm-ethos-u.md
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
```{include} backends-arm-ethos-u.md
```{include} backends/arm-ethos-u/arm-ethos-u-overview.md
```
3 changes: 2 additions & 1 deletion docs/source/embedded-backends.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Available hardware acceleration backends for embedded systems.

## NPU Acceleration

- {doc}`embedded-arm-ethos-u`ARM Ethos-U NPU acceleration
- {doc}`embedded-arm-ethos-u`Arm Ethos-U NPU acceleration
- {doc}`embedded-nxp` — NXP eIQ Neutron Backend


Expand All @@ -23,3 +23,4 @@ embedded-arm-cortex-m
embedded-cadence
embedded-arm-ethos-u
embedded-nxp
```
Loading
Loading