Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
493fc00
refactor!: make ClassificationModule a factory
chmjkb Mar 5, 2026
b56da84
refactor!: make ImageEmbeddingsModule a factory
chmjkb Mar 5, 2026
9a042c8
refactor!: make StyleTransferModule a factory
chmjkb Mar 5, 2026
ac676dd
refactor!: make VADModule a factory
chmjkb Mar 5, 2026
253eb99
feat: add modelName to all model constants for union discrimination a…
chmjkb Mar 5, 2026
22693ca
docs: update JSDoc to document modelName field
chmjkb Mar 5, 2026
ad11843
type fixes
chmjkb Mar 6, 2026
d139821
feat(ObjectDetection, SemanticSegmentation): rename fromCustomConfig …
chmjkb Mar 9, 2026
e491f29
feat: add fromCustomModel to Classification, StyleTransfer, ImageEmbe…
chmjkb Mar 9, 2026
d1f39d6
feat(OCR): migrate to factory pattern, add fromCustomModel, update us…
chmjkb Mar 9, 2026
9d17c42
chore: migrate VerticalOCR to factory pattern, rewrite OCR hooks with…
chmjkb Mar 9, 2026
b45a089
chore: migrate LLMModule to factory pattern, add LLMModelName type
chmjkb Mar 9, 2026
0189927
chore: migrate SpeechToTextModule to factory pattern, add SpeechToTex…
chmjkb Mar 9, 2026
86b5021
chore: migrate TextToImageModule to factory pattern, add TextToImageM…
chmjkb Mar 9, 2026
8845272
lint & fix model names
chmjkb Mar 12, 2026
864085f
docs: update docs, jsdocs
chmjkb Mar 12, 2026
d5f00a7
refactor: refactor TTS
chmjkb Mar 12, 2026
a22dd0f
final chores & docs
chmjkb Mar 12, 2026
a6b1216
bring back parallel lefthook
chmjkb Mar 12, 2026
f480e4e
fix useModuleFactory
chmjkb Mar 12, 2026
2d363fc
Update docs/docs/04-typescript-api/01-natural-language-processing/LLM…
chmjkb Mar 13, 2026
27403e4
fix: await loading the native module
chmjkb Mar 13, 2026
5e87cba
docs: fix docs
chmjkb Mar 13, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion .cspell-wordlist.txt
Original file line number Diff line number Diff line change
Expand Up @@ -128,4 +128,15 @@ detr
metaprogramming
ktlint
lefthook
espeak
espeak
NCHW
həlˈO
wˈɜɹld
mˈæn
dˈʌzᵊnt
tɹˈʌst
hɪmsˈɛlf
nˈɛvəɹ
ɹˈiᵊli
ˈɛniwˌʌn
ˈɛls
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,13 @@ TypeScript API implementation of the [useLLM](../../03-hooks/01-natural-language
```typescript
import { LLMModule, LLAMA3_2_1B_QLORA } from 'react-native-executorch';

// Creating an instance
const llm = new LLMModule({
tokenCallback: (token) => console.log(token),
messageHistoryCallback: (messages) => console.log(messages),
});

// Loading the model
await llm.load(LLAMA3_2_1B_QLORA, (progress) => console.log(progress));
// Creating an instance and loading the model
const llm = await LLMModule.fromModelName(
LLAMA3_2_1B_QLORA,
(progress) => console.log(progress),
(token) => console.log(token),
(messages) => console.log(messages),
);
Comment thread
chmjkb marked this conversation as resolved.

// Running the model - returns the generated response
const response = await llm.sendMessage('Hello, World!');
Expand All @@ -41,30 +40,26 @@ All methods of `LLMModule` are explained in details here: [LLMModule API Referen

## Loading the model

To create a new instance of `LLMModule`, use the [constructor](../../06-api-reference/classes/LLMModule.md#constructor) with optional callbacks:

- [`tokenCallback`](../../06-api-reference/classes/LLMModule.md#tokencallback) - Function called on every generated token.

- [`messageHistoryCallback`](../../06-api-reference/classes/LLMModule.md#messagehistorycallback) - Function called on every finished message.

Then, to load the model, use the [`load`](../../06-api-reference/classes/LLMModule.md#load) method. It accepts an object with the following fields:
Use the static [`fromModelName`](../../06-api-reference/classes/LLMModule.md#frommodelname) factory method:

- [`model`](../../06-api-reference/classes/LLMModule.md#model) - Object containing:
- [`modelSource`](../../06-api-reference/classes/LLMModule.md#modelsource) - The location of the used model.

- [`tokenizerSource`](../../06-api-reference/classes/LLMModule.md#tokenizersource) - The location of the used tokenizer.

- [`tokenizerConfigSource`](../../06-api-reference/classes/LLMModule.md#tokenizerconfigsource) - The location of the used tokenizer config.
```typescript
const llm = await LLMModule.fromModelName(
LLAMA3_2_3B, // model config constant
onDownloadProgress, // optional, progress 0–1
tokenCallback, // optional, called on every token
messageHistoryCallback // optional, called when generation finishes
);
```

- [`onDownloadProgressCallback`](../../06-api-reference/classes/LLMModule.md#ondownloadprogresscallback) - Callback to track download progress.
The model config object contains `modelSource`, `tokenizerSource`, `tokenizerConfigSource`, and optional `capabilities`. Pass one of the built-in constants (e.g. `LLAMA3_2_3B`) or construct it manually.

This method returns a promise, which can resolve to an error or void.
This method returns a promise resolving to an `LLMModule` instance.

For more information on loading resources, take a look at [loading models](../../01-fundamentals/02-loading-models.md) page.

## Listening for download progress

To subscribe to the download progress event, you can pass the [`onDownloadProgressCallback`](../../06-api-reference/classes/LLMModule.md#ondownloadprogresscallback) function to the [`load`](../../06-api-reference/classes/LLMModule.md#load) method. This function is called whenever the download progress changes.
To subscribe to the download progress event, you can pass the `onDownloadProgress` callback as the second argument to [`fromModelName`](../../06-api-reference/classes/LLMModule.md#frommodelname). This function is called whenever the download progress changes.

## Running the model

Expand Down Expand Up @@ -116,25 +111,26 @@ To configure model (i.e. change system prompt, load initial conversation history

## Vision-Language Models (VLM)

Some models support multimodal input — text and images together. To use them, pass `capabilities` in the model object when calling [`load`](../../06-api-reference/classes/LLMModule.md#load):
Some models support multimodal input — text and images together. To use them, pass `capabilities` in the model object when calling [`fromModelName`](../../06-api-reference/classes/LLMModule.md#frommodelname):

```typescript
import { LLMModule, LFM2_VL_1_6B_QUANTIZED } from 'react-native-executorch';

const llm = new LLMModule({
tokenCallback: (token) => console.log(token),
});

await llm.load(LFM2_VL_1_6B_QUANTIZED);
const llm = await LLMModule.fromModelName(
LFM2_VL_1_6B_QUANTIZED,
undefined,
(token) => console.log(token)
);
```

The `capabilities` field is already set on the model constant. You can also construct the model object explicitly:

```typescript
await llm.load({
modelSource: '...',
tokenizerSource: '...',
tokenizerConfigSource: '...',
const llm = await LLMModule.fromModelName({
modelName: 'lfm2.5-vl-1.6b-quantized',
modelSource: require('./path/to/model.pte'),
tokenizerSource: require('./path/to/tokenizer.json'),
tokenizerConfigSource: require('./path/to/tokenizer_config.json'),
capabilities: ['vision'],
});
```
Expand All @@ -161,6 +157,27 @@ const chat: Message[] = [
const response = await llm.generate(chat);
```

## Using a custom model

Use [`fromCustomModel`](../../06-api-reference/classes/LLMModule.md#fromcustommodel) to load your own exported LLM instead of a built-in preset:

```typescript
import { LLMModule } from 'react-native-executorch';

const llm = await LLMModule.fromCustomModel(
'https://example.com/model.pte',
'https://example.com/tokenizer.json',
'https://example.com/tokenizer_config.json',
(progress) => console.log(progress),
(token) => console.log(token),
(messages) => console.log(messages)
);
```

### Required model contract

The `.pte` model binary must be exported following the [ExecuTorch LLM export process](https://docs.pytorch.org/executorch/1.1/llm/export-llm.html). The native runner expects the standard ExecuTorch text-generation interface — KV-cache management, prefill/decode phases, and logit sampling are all handled by the runtime.

## Deleting the model from memory

To delete the model from memory, you can use the [`delete`](../../06-api-reference/classes/LLMModule.md#delete) method.
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,12 @@ TypeScript API implementation of the [useSpeechToText](../../03-hooks/01-natural
```typescript
import { SpeechToTextModule, WHISPER_TINY_EN } from 'react-native-executorch';

const model = new SpeechToTextModule();
await model.load(WHISPER_TINY_EN, (progress) => {
console.log(progress);
});
const model = await SpeechToTextModule.fromModelName(
WHISPER_TINY_EN,
(progress) => {
console.log(progress);
}
);

// Standard transcription (returns string)
const text = await model.transcribe(waveform);
Expand All @@ -40,18 +42,17 @@ All methods of `SpeechToTextModule` are explained in details here: [`SpeechToTex

## Loading the model

Create an instance of [`SpeechToTextModule`](../../06-api-reference/classes/SpeechToTextModule.md) and use the [`load`](../../06-api-reference/classes/SpeechToTextModule.md#load) method. It accepts an object with the following fields:

- [`model`](../../06-api-reference/classes/SpeechToTextModule.md#model) - Object containing:
- [`isMultilingual`](../../06-api-reference/interfaces/SpeechToTextModelConfig.md#ismultilingual) - Flag indicating if model is multilingual.
Use the static [`fromModelName`](../../06-api-reference/classes/SpeechToTextModule.md#frommodelname) factory method. It accepts an object with the following fields:

- [`modelSource`](../../06-api-reference/interfaces/SpeechToTextModelConfig.md#modelsource) - The location of the used model (bundled encoder + decoder functionality).
- [`isMultilingual`](../../06-api-reference/interfaces/SpeechToTextModelConfig.md#ismultilingual) - Flag indicating if model is multilingual.
- [`modelSource`](../../06-api-reference/interfaces/SpeechToTextModelConfig.md#modelsource) - The location of the used model (bundled encoder + decoder functionality).
- [`tokenizerSource`](../../06-api-reference/interfaces/SpeechToTextModelConfig.md#tokenizersource) - The location of the used tokenizer.

- [`tokenizerSource`](../../06-api-reference/interfaces/SpeechToTextModelConfig.md#tokenizersource) - The location of the used tokenizer.
And an optional second argument:

- [`onDownloadProgressCallback`](../../06-api-reference/classes/SpeechToTextModule.md#ondownloadprogresscallback) - Callback to track download progress.
- `onDownloadProgress` - Callback to track download progress.

This method returns a promise, which can resolve to an error or void.
This method returns a promise resolving to a `SpeechToTextModule` instance.

For more information on loading resources, take a look at [loading models](../../01-fundamentals/02-loading-models.md) page.

Expand All @@ -66,10 +67,12 @@ If you aim to obtain a transcription in other languages than English, use the mu
```typescript
import { SpeechToTextModule, WHISPER_TINY } from 'react-native-executorch';

const model = new SpeechToTextModule();
await model.load(WHISPER_TINY, (progress) => {
console.log(progress);
});
const model = await SpeechToTextModule.fromModelName(
WHISPER_TINY,
(progress) => {
console.log(progress);
}
);

const transcription = await model.transcribe(spanishAudio, { language: 'es' });
```
Expand Down Expand Up @@ -121,10 +124,12 @@ import * as FileSystem from 'expo-file-system';

const transcribeAudio = async () => {
// Initialize with the model config
const model = new SpeechToTextModule();
await model.load(WHISPER_TINY_EN, (progress) => {
console.log(progress);
});
const model = await SpeechToTextModule.fromModelName(
WHISPER_TINY_EN,
(progress) => {
console.log(progress);
}
);

// Download the audio file
const { uri } = await FileSystem.downloadAsync(
Expand Down Expand Up @@ -163,10 +168,12 @@ import { SpeechToTextModule, WHISPER_TINY_EN } from 'react-native-executorch';
import { AudioManager, AudioRecorder } from 'react-native-audio-api';

// Load the model
const model = new SpeechToTextModule();
await model.load(WHISPER_TINY_EN, (progress) => {
console.log(progress);
});
const model = await SpeechToTextModule.fromModelName(
WHISPER_TINY_EN,
(progress) => {
console.log(progress);
}
);

// Configure audio session
AudioManager.setAudioSessionOptions({
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,9 @@ import {
ALL_MINILM_L6_V2,
} from 'react-native-executorch';

// Creating an instance
const textEmbeddingsModule = new TextEmbeddingsModule();

// Loading the model
await textEmbeddingsModule.load(ALL_MINILM_L6_V2);
// Creating an instance and loading the model
const textEmbeddingsModule =
await TextEmbeddingsModule.fromModelName(ALL_MINILM_L6_V2);

// Running the model
const embedding = await textEmbeddingsModule.forward('Hello World!');
Expand All @@ -33,15 +31,12 @@ All methods of `TextEmbeddingsModule` are explained in details here: [`TextEmbed

## Loading the model

To load the model, use the [`load`](../../06-api-reference/classes/TextEmbeddingsModule.md#load) method. It accepts an object:

- [`model`](../../06-api-reference/classes/TextEmbeddingsModule.md#model) - Object containing:
- [`modelSource`](../../06-api-reference/classes/TextEmbeddingsModule.md#modelsource) - Location of the used model.
- [`tokenizerSource`](../../06-api-reference/classes/TextEmbeddingsModule.md#tokenizersource) - Location of the used tokenizer.
Use the static [`fromModelName`](../../06-api-reference/classes/TextEmbeddingsModule.md#frommodelname) factory method. It accepts a model config object (e.g. `ALL_MINILM_L6_V2`) containing:

- [`onDownloadProgressCallback`](../../06-api-reference/classes/TextEmbeddingsModule.md#ondownloadprogresscallback) - Callback to track download progress.
- [`modelSource`](../../06-api-reference/classes/TextEmbeddingsModule.md#modelsource) - Location of the used model.
- [`tokenizerSource`](../../06-api-reference/classes/TextEmbeddingsModule.md#tokenizersource) - Location of the used tokenizer.

This method returns a promise, which can resolve to an error or void.
And an optional `onDownloadProgress` callback. It returns a promise resolving to a `TextEmbeddingsModule` instance.

For more information on loading resources, take a look at [loading models](../../01-fundamentals/02-loading-models.md) page.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,15 +19,9 @@ import {
KOKORO_VOICE_AF_HEART,
} from 'react-native-executorch';

const model = new TextToSpeechModule();
await model.load(
{
model: KOKORO_MEDIUM,
voice: KOKORO_VOICE_AF_HEART,
},
(progress) => {
console.log(progress);
}
const model = await TextToSpeechModule.fromModelName(
{ model: KOKORO_MEDIUM, voice: KOKORO_VOICE_AF_HEART },
(progress) => console.log(progress)
);

await model.forward(text, 1.0);
Expand All @@ -39,15 +33,15 @@ All methods of `TextToSpeechModule` are explained in details here: [`TextToSpeec

## Loading the model

To initialize the module, create an instance and call the [`load`](../../06-api-reference/classes/TextToSpeechModule.md#load) method with the following parameters:
Use the static [`fromModelName`](../../06-api-reference/classes/TextToSpeechModule.md#frommodelname) factory method with the following parameters:

- [`config`](../../06-api-reference/classes/TextToSpeechModule.md#config) - Object containing:
- [`model`](../../06-api-reference/interfaces/TextToSpeechConfig.md#model) - Model configuration.
- [`voice`](../../06-api-reference/interfaces/TextToSpeechConfig.md#voice) - Voice configuration.
- [`config`](../../06-api-reference/interfaces/TextToSpeechConfig.md) - Object containing:
- [`model`](../../06-api-reference/interfaces/TextToSpeechConfig.md#model) - Model configuration (e.g. `KOKORO_MEDIUM`).
- [`voice`](../../06-api-reference/interfaces/TextToSpeechConfig.md#voice) - Voice configuration (e.g. `KOKORO_VOICE_AF_HEART`).

- [`onDownloadProgressCallback`](../../06-api-reference/classes/TextToSpeechModule.md#ondownloadprogresscallback) - Callback to track download progress.
- [`onDownloadProgress`](../../06-api-reference/classes/TextToSpeechModule.md#frommodelname) - Optional callback to track download progress (value between 0 and 1).

This method returns a promise that resolves once the assets are downloaded and loaded into memory.
This method returns a promise that resolves to a `TextToSpeechModule` instance once the assets are downloaded and loaded into memory.

For more information on resource sources, see [loading models](../../01-fundamentals/02-loading-models.md).

Expand Down Expand Up @@ -83,15 +77,13 @@ import {
} from 'react-native-executorch';
import { AudioContext } from 'react-native-audio-api';

const tts = new TextToSpeechModule();
const tts = await TextToSpeechModule.fromModelName({
model: KOKORO_MEDIUM,
voice: KOKORO_VOICE_AF_HEART,
});
const audioContext = new AudioContext({ sampleRate: 24000 });

try {
await tts.load({
model: KOKORO_MEDIUM,
voice: KOKORO_VOICE_AF_HEART,
});

const waveform = await tts.forward('Hello from ExecuTorch!', 1.0);

// Create audio buffer and play
Expand All @@ -117,11 +109,12 @@ import {
} from 'react-native-executorch';
import { AudioContext } from 'react-native-audio-api';

const tts = new TextToSpeechModule();
const tts = await TextToSpeechModule.fromModelName({
model: KOKORO_MEDIUM,
voice: KOKORO_VOICE_AF_HEART,
});
const audioContext = new AudioContext({ sampleRate: 24000 });

await tts.load({ model: KOKORO_MEDIUM, voice: KOKORO_VOICE_AF_HEART });

try {
for await (const chunk of tts.stream({
text: 'This is a streaming test, with a sample input.',
Expand Down Expand Up @@ -155,9 +148,7 @@ import {
KOKORO_VOICE_AF_HEART,
} from 'react-native-executorch';

const tts = new TextToSpeechModule();

await tts.load({
const tts = await TextToSpeechModule.fromModelName({
model: KOKORO_MEDIUM,
voice: KOKORO_VOICE_AF_HEART,
});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,9 @@ TypeScript API implementation of the [useVAD](../../03-hooks/01-natural-language
```typescript
import { VADModule, FSMN_VAD } from 'react-native-executorch';

const model = new VADModule();
await model.load(FSMN_VAD, (progress) => {
console.log(progress);
});
const model = await VADModule.fromModelName(FSMN_VAD, (progress) =>
console.log(progress)
);

await model.forward(waveform);
```
Expand All @@ -28,14 +27,15 @@ All methods of `VADModule` are explained in details here: [`VADModule` API Refer

## Loading the model

To initialize the module, create an instance and call the [`load`](../../06-api-reference/classes/VADModule.md#load) method with the following parameters:
To create a ready-to-use instance, call the static [`fromModelName`](../../06-api-reference/classes/VADModule.md#frommodelname) factory with the following parameters:

- [`model`](../../06-api-reference/classes/VADModule.md#model) - Object containing:
- [`modelSource`](../../06-api-reference/classes/VADModule.md#modelsource) - Location of the used model.
- `namedSources` - Object containing:
- `modelName` - Model name identifier.
- `modelSource` - Location of the model binary.

- [`onDownloadProgressCallback`](../../06-api-reference/classes/VADModule.md#ondownloadprogresscallback) - Callback to track download progress.
- `onDownloadProgress` - Optional callback to track download progress (value between 0 and 1).

This method returns a promise, which can resolve to an error or void.
The factory returns a promise that resolves to a loaded `VADModule` instance.

For more information on loading resources, take a look at [loading models](../../01-fundamentals/02-loading-models.md) page.

Expand Down
Loading
Loading