Skip to content

Commit 89a4f60

Browse files
authored
chore: created docs 0.8.x (#1009)
## Description <!-- Provide a concise and descriptive summary of the changes implemented in this PR. --> ### Introduces a breaking change? - [ ] Yes - [ ] No ### Type of change - [ ] Bug fix (change which fixes an issue) - [ ] New feature (change which adds functionality) - [ ] Documentation update (improves or adds clarity to existing documentation) - [ ] Other (chores, tests, code style improvements etc.) ### Tested on - [ ] iOS - [ ] Android ### Testing instructions <!-- Provide step-by-step instructions on how to test your changes. Include setup details if necessary. --> ### Screenshots <!-- Add screenshots here, if applicable --> ### Related issues <!-- Link related issues here using #issue-number --> ### Checklist - [ ] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have updated the documentation accordingly - [ ] My changes generate no new warnings ### Additional notes <!-- Include any additional information, assumptions, or context that reviewers might need to understand this PR. -->
1 parent d0d3e5b commit 89a4f60

File tree

402 files changed

+33642
-1
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

402 files changed

+33642
-1
lines changed
Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
---
2+
title: Getting Started
3+
slug: /fundamentals/getting-started
4+
keywords:
5+
[
6+
react native,
7+
react native ai,
8+
react native llm,
9+
react native qwen,
10+
react native llama,
11+
react native executorch,
12+
executorch,
13+
on-device ai,
14+
pytorch,
15+
mobile ai,
16+
]
17+
description: 'Get started with React Native ExecuTorch - a framework for running AI models on-device in your React Native applications.'
18+
---
19+
20+
import Tabs from '@theme/Tabs';
21+
import TabItem from '@theme/TabItem';
22+
23+
## What is ExecuTorch?
24+
25+
[ExecuTorch](https://executorch.ai) is a novel AI framework developed by Meta, designed to streamline deploying PyTorch models on a variety of devices, including mobile phones and microcontrollers. This framework enables exporting models into standalone binaries, allowing them to run locally without requiring API calls. ExecuTorch achieves state-of-the-art performance through optimizations and delegates such as Core ML and XNNPACK. It provides a seamless export process with robust debugging options, making it easier to resolve issues if they arise.
26+
27+
## React Native ExecuTorch
28+
29+
React Native ExecuTorch is our way of bringing ExecuTorch into the React Native world. Our API is built to be simple, declarative, and efficient. Plus, we’ll provide a set of pre-exported models for common use cases, so you won’t have to worry about handling exports yourself. With just a few lines of JavaScript, you’ll be able to run AI models (even LLMs 👀) right on your device—keeping user data private and saving on cloud costs.
30+
31+
## Compatibility
32+
33+
React Native Executorch supports only the [New React Native architecture](https://reactnative.dev/architecture/landing-page).
34+
35+
If your app still runs on the old architecture, please consider upgrading to the New Architecture.
36+
37+
For supported React Native and Expo versions, see the [Compatibility table](../07-other/01-compatibility.mdx).
38+
39+
## Installation
40+
41+
Installation is pretty straightforward, use your package manager of choice to install the package and some peer dependencies required to streamline model downloads. If you want to implement your custom model fetching logic, see [this document](../08-resource-fetcher/02-custom-adapter.md).
42+
43+
<Tabs>
44+
<TabItem value="npm" label="NPM">
45+
46+
```
47+
npm install react-native-executorch
48+
# For Expo projects
49+
npm install react-native-executorch-expo-resource-fetcher
50+
# For bare React Native projects
51+
npm install react-native-executorch-bare-resource-fetcher
52+
```
53+
54+
</TabItem>
55+
<TabItem value="pnpm" label="PNPM">
56+
57+
```
58+
pnpm install react-native-executorch
59+
# For Expo projects
60+
pnpm install react-native-executorch-expo-resource-fetcher
61+
# For bare React Native projects
62+
pnpm install react-native-executorch-bare-resource-fetcher
63+
64+
```
65+
66+
</TabItem>
67+
<TabItem value="yarn" label="YARN">
68+
69+
```
70+
yarn add react-native-executorch
71+
# For Expo projects
72+
yarn install react-native-executorch-expo-resource-fetcher
73+
# For bare React Native projects
74+
yarn install react-native-executorch-bare-resource-fetcher
75+
```
76+
77+
</TabItem>
78+
</Tabs>
79+
80+
:::warning
81+
Before using any other API, you must call `initExecutorch` with a resource fetcher adapter at the entry point of your app:
82+
83+
```js
84+
import { initExecutorch } from 'react-native-executorch';
85+
import { ExpoResourceFetcher } from 'react-native-executorch-expo-resource-fetcher';
86+
// or BareResourceFetcher for Expo projects
87+
88+
initExecutorch({ resourceFetcher: ExpoResourceFetcher });
89+
```
90+
91+
Calling any library API without initializing first will throw a `ResourceFetcherAdapterNotInitialized` error.
92+
:::
93+
94+
Our library offers support for both bare React Native and Expo projects. Please follow the instructions from [Loading models section](./02-loading-models.md) to make sure you setup your project correctly. We encourage you to use Expo project if possible. If you are planning to migrate from bare React Native to Expo project, the link (https://docs.expo.dev/bare/installing-expo-modules/) offers a guidance on setting up Expo Modules in a bare React Native environment.
95+
96+
If you plan on using your models via require() instead of fetching them from a url, you also need to add following lines to your `metro.config.js`:
97+
98+
```json
99+
// metro.config.js
100+
...
101+
defaultConfig.resolver.assetExts.push('pte')
102+
defaultConfig.resolver.assetExts.push('bin')
103+
...
104+
```
105+
106+
This allows us to use binaries, such as exported models or tokenizers for LLMs.
107+
108+
:::warning
109+
When using Expo, please note that you need to use a custom development build of your app, not the standard Expo Go app. This is because we rely on native modules, which Expo Go doesn’t support.
110+
:::
111+
112+
:::info
113+
Because we are using ExecuTorch under the hood, you won't be able to build iOS app for release with simulator selected as the target device. Make sure to test release builds on real devices.
114+
:::
115+
116+
Running the app with the library:
117+
118+
```bash
119+
yarn <ios | android> -d
120+
```
121+
122+
## Supporting new models in React Native ExecuTorch
123+
124+
Adding new functionality to the library follows a consistent three-step integration pipeline:
125+
126+
1. **Model Serialization:** We export PyTorch models for specific tasks (e.g., object detection) into the \*.pte format, which is optimized for the ExecuTorch runtime.
127+
128+
2. **Native Implementation:** We develop a C++ execution layer that interfaces with the ExecuTorch runtime to handle inference. This layer also manages model-dependent logic, such as data pre-processing and post-processing.
129+
130+
3. **TS Bindings:** Finally, we implement a TypeScript API that bridges the JavaScript environment to the native C++ logic, providing a clean, typed interface for the end user."
131+
132+
## Good reads
133+
134+
If you want to dive deeper into ExecuTorch or our previous work with the framework, we highly encourage you to check out the following resources:
135+
136+
- [ExecuTorch docs](https://pytorch.org/executorch/stable/index.html)
137+
- [React Native RAG](https://blog.swmansion.com/introducing-react-native-rag-fbb62efa4991)
138+
- [Offline Text Recognition on Mobile: How We Brought EasyOCR to React Native ExecuTorch](https://blog.swmansion.com/bringing-easyocr-to-react-native-executorch-2401c09c2d0c)
Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
---
2+
title: Loading Models
3+
---
4+
5+
There are three different methods available for loading model files, depending on their size and location.
6+
7+
## Prerequisites
8+
9+
In our library, you can use two different resource fetching mechanisms. One is implemented using Expo FileSystem, the other one uses external library. We encourage you to use implementation utilizing Expo if possible.
10+
11+
To use the Expo adapter, please add these libraries:
12+
13+
```bash
14+
yarn add react-native-executorch-expo-resource-fetcher
15+
yarn add expo-file-system expo-asset
16+
```
17+
18+
and then add the following code in your React Native app:
19+
20+
```typescript
21+
import { initExecutorch } from 'react-native-executorch';
22+
import { ExpoResourceFetcher } from 'react-native-executorch-expo-resource-fetcher';
23+
24+
initExecutorch({
25+
resourceFetcher: ExpoResourceFetcher,
26+
});
27+
```
28+
29+
If you cannot use Expo in your project, proceed with the following steps:
30+
31+
```bash
32+
yarn add react-native-executorch-bare-resource-fetcher
33+
yarn add @dr.pogodin/react-native-fs @kesha-antonov/react-native-background-downloader
34+
```
35+
36+
and
37+
38+
```typescript
39+
import { initExecutorch } from 'react-native-executorch';
40+
import { BareResourceFetcher } from '@react-native-executorch/bare-adapter';
41+
42+
initExecutorch({
43+
resourceFetcher: BareResourceFetcher,
44+
});
45+
```
46+
47+
**1. Load from React Native assets folder (For Files < 512MB)**
48+
49+
```typescript
50+
useExecutorchModule({
51+
modelSource: require('../assets/llama3_2.pte'),
52+
});
53+
```
54+
55+
**2. Load from remote URL:**
56+
57+
For files larger than 512MB or when you want to keep size of the app smaller, you can load the model from a remote URL (e.g. HuggingFace).
58+
59+
```typescript
60+
useExecutorchModule({
61+
modelSource: 'https://.../llama3_2.pte',
62+
});
63+
```
64+
65+
**3. Load from local file system:**
66+
67+
If you prefer to delegate the process of obtaining and loading model and tokenizer files to the user, you can use the following method:
68+
69+
```typescript
70+
useExecutorchModule({
71+
modelSource: 'file:///var/mobile/.../llama3_2.pte',
72+
});
73+
```
74+
75+
:::info
76+
The downloaded files are stored in documents directory of your application.
77+
:::
78+
79+
## Predefined Models
80+
81+
Our library offers out-of-the-box support for multiple models. To make things easier, we created aliases for our model exported to `pte` format. For full list of aliases, check out: [API Reference](../06-api-reference/index.md#models---classification)
82+
83+
## Example
84+
85+
The following code snippet demonstrates how to load model and tokenizer files using `useLLM` hook:
86+
87+
```typescript
88+
import { useLLM } from 'react-native-executorch';
89+
90+
const llama = useLLM({
91+
modelSource: 'https://.../llama3_2.pte',
92+
tokenizerSource: require('../assets/tokenizer.bin'),
93+
});
94+
```
Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
---
2+
title: Frequently Asked Questions
3+
---
4+
5+
This section is meant to answer some common community inquiries, especially regarding the ExecuTorch runtime or adding your own models. If you can't see an answer to your question, feel free to open up a [discussion](https://github.com/software-mansion/react-native-executorch/discussions/new/choose).
6+
7+
### What models are supported?
8+
9+
Each hook documentation subpage (useClassification, useLLM, etc.) contains a supported models section, which lists the models that are runnable within the library with close to no setup. For running your custom models, refer to `ExecuTorchModule` or `useExecuTorchModule`.
10+
11+
### How can I run my own AI model?
12+
13+
To run your own model, you need to directly access the underlying [ExecuTorch Module API](https://pytorch.org/executorch/stable/extension-module.html). We provide [React hook](../03-hooks/03-executorch-bindings/useExecutorchModule.md) along with a [TypeScript alternative](../04-typescript-api/03-executorch-bindings/ExecutorchModule.md), which serve as a way to use the aforementioned API without the need of diving into native code. In order to get a model in a format runnable by the runtime, you'll need to get your hands dirty with some ExecuTorch knowledge. For more guides on exporting models, please refer to the [ExecuTorch tutorials](https://pytorch.org/executorch/stable/tutorials/export-to-executorch-tutorial.html). Once you obtain your model in a `.pte` format, you can run it with `useExecuTorchModule` and `ExecuTorchModule`.
14+
15+
### How React Native ExecuTorch works under the hood?
16+
17+
The general workflow for each functionality in our library goes like this:
18+
19+
- You call a functionality from TypeScript
20+
- TypeScript calls C++ function like model inference or data processing via JSI
21+
- C++ returns result to TypeScript back via JSI
22+
- You get results in TypeScript
23+
24+
Using JSI enables us using **zero-copy data transfer** and **fast, low-level C++**.
25+
26+
### Can you do function calling with useLLM?
27+
28+
If your model supports tool calling (i.e. its chat template can process tools) you can use the method explained on the [useLLM page](../03-hooks/01-natural-language-processing/useLLM.md).
29+
30+
If your model doesn't support it, you can still work around it using context. For details, refer to [this comment](https://github.com/software-mansion/react-native-executorch/issues/173#issuecomment-2775082278).
31+
32+
### Can I use React Native ExecuTorch in bare React Native apps?
33+
34+
Yes, staring from version `0.8.x` you can use React Native ExecuTorch in bare React Native apps. You just need to use bare React Native resource fetcher instead of Expo one, see: [Loading models section](./02-loading-models.md) for more details.
35+
36+
### Do you support the old architecture?
37+
38+
The old architecture is not supported and we're currently not planning to add support.
39+
40+
### Can I run GGUF models using the library?
41+
42+
No, as of now ExecuTorch runtime doesn't provide a reliable way to use GGUF models, hence it is not possible.
43+
44+
### Are the models leveraging GPU acceleration?
45+
46+
While it is possible to run some models using Core ML on iOS, which is a backend that utilizes CPU, GPU and ANE, we currently don't have many models exported to Core ML. For Android, the current state of GPU acceleration is pretty limited. As of now, there are attempts of running the models using a Vulkan backend. However the operator support is very limited meaning that the resulting performance is often inferior to XNNPACK. Hence, most of the models use XNNPACK, which is a highly optimized and mature CPU backend that runs on both Android and iOS.
47+
48+
### Does this library support XNNPACK and Core ML?
49+
50+
Yes, all of the backends are linked, therefore the only thing that needs to be done on your end is to export the model with the backend that you're interested in using.
Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
# Glossary of Terms
2+
3+
This glossary defines key concepts used throughout the React Native ExecuTorch ecosystem, covering high-level machine learning terms and library-specific components.
4+
5+
## Backend
6+
7+
The execution engine responsible for running the actual computations of a model on specific hardware.
8+
9+
- **XNNPACK:** A highly optimized library for floating-point neural network inference on ARM, x86, and WebAssembly. It is the default CPU backend for ExecuTorch.
10+
11+
- **Core ML:** Apple's framework for optimizing and running machine learning models on iOS, macOS, and iPadOS devices. Using the Core ML backend allows ExecuTorch to delegate operations to the Apple Neural Engine (ANE) for significantly faster and more energy-efficient inference.
12+
13+
## Forward Function
14+
15+
The primary method of a PyTorch module (usually `forward()`) that defines the computation performed at every call. In the context of ExecuTorch, this is the logic that gets exported and compiled. When you run inference in React Native, you are essentially invoking this compiled forward function with new inputs.
16+
17+
## Inference
18+
19+
The process of using a trained machine learning model to make predictions or generate outputs for given input data.
20+
21+
## Out-of-the-Box Support
22+
23+
Refers to features, models, or architectures that work immediately with React Native ExecuTorch without requiring custom compilation, manual kernel registration, or complex configuration. For example, standard Llama architectures have out-of-the-box support, meaning you can download the `.pte` file and run it instantly.
24+
25+
## Prefill
26+
27+
The initial phase of generating text with an LLM (Large Language Model) where the model processes the entire input prompt (context) at once.
28+
29+
- **Why it matters:** This step is computationally intensive because the model must "understand" all provided tokens simultaneously.
30+
31+
- **Performance Metric:** "Time to First Token" (TTFT) usually measures the speed of the prefill phase.
32+
33+
## Quantization
34+
35+
A technique to reduce the size of a model and speed up inference by representing weights and activations with lower-precision data types (e.g., converting 32-bit floating-point numbers to 8-bit integers).
36+
37+
- **Benefits:** Drastically lowers memory usage (RAM) and saves battery life on mobile devices.
38+
39+
- **Trade-off:** Slight reduction in model accuracy, though often negligible for deployment.
40+
41+
## Tensor
42+
43+
The fundamental data structure in PyTorch and ExecuTorch. A tensor is a multi-dimensional array (like a matrix) that holds the inputs, weights, and outputs of a model.
44+
45+
- **Example:** An image might be represented as a tensor of shape `[3, 224, 224]` (3 color channels, 224 pixels high, 224 pixels wide).
46+
47+
## Token
48+
49+
The basic unit of text that an LLM reads and generates. A token can be a word, part of a word, or even a single character.
50+
51+
- **Rule of thumb:** 1,000 tokens is roughly equivalent to 750 words in English.
52+
53+
- **Context:** Models have a "Context Window" limit (e.g., 2048 tokens), which is the maximum number of tokens they can remember from the conversation history.
54+
55+
## Tokenization
56+
57+
The process of converting raw text (strings) into a sequence of numerical IDs (tokens) that the model can understand.
58+
59+
- **TokenizerModule (Component):** In React Native ExecuTorch, the `Tokenizer` is a utility class that handles encoding text into tensors and decoding output tensors back into readable text strings.
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"label": "Fundamentals",
3+
"link": {
4+
"type": "generated-index"
5+
}
6+
}
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"label": "Benchmarks",
3+
"link": {
4+
"type": "generated-index"
5+
}
6+
}

0 commit comments

Comments
 (0)