Defined in: modules/natural_language_processing/LLMModule.ts:17
Module for managing a Large Language Model (LLM) instance.
configure(
config):void
Defined in: modules/natural_language_processing/LLMModule.ts:132
Configures chat and tool calling and generation settings. See Configuring the model for details.
Configuration object containing chatConfig, toolsConfig, and generationConfig.
void
delete():
void
Defined in: modules/natural_language_processing/LLMModule.ts:224
Method to delete the model from memory. Note you cannot delete model while it's generating. You need to interrupt it first and make sure model stopped generation.
void
deleteMessage(
index):Message[]
Defined in: modules/natural_language_processing/LLMModule.ts:183
Deletes all messages starting with message on index position.
After deletion it will call messageHistoryCallback() containing new history.
It also returns it.
number
The index of the message to delete from history.
Message[]
- Updated message history after deletion.
forward(
input,imagePaths?):Promise<string>
Defined in: modules/natural_language_processing/LLMModule.ts:145
Runs model inference with raw input string.
You need to provide entire conversation and prompt (in correct format and with special tokens!) in input string to this method.
It doesn't manage conversation context. It is intended for users that need access to the model itself without any wrapper.
If you want a simple chat with model the consider using sendMessage
string
Raw input string containing the prompt and conversation history.
string[]
Optional array of local image paths for multimodal inference.
Promise<string>
The generated response as a string.
generate(
messages,tools?):Promise<string>
Defined in: modules/natural_language_processing/LLMModule.ts:156
Runs model to complete chat passed in messages argument. It doesn't manage conversation context.
For multimodal models, set mediaPath on user messages to include images.
Message[]
Array of messages representing the chat history. User messages may include a mediaPath field with a local image path.
Object[]
Optional array of tools that can be used during generation.
Promise<string>
The generated response as a string.
getGeneratedTokenCount():
number
Defined in: modules/natural_language_processing/LLMModule.ts:199
Returns the number of tokens generated in the last response.
number
The count of generated tokens.
getPromptTokensCount():
number
Defined in: modules/natural_language_processing/LLMModule.ts:207
Returns the number of prompt tokens in the last message.
number
The count of prompt token.
getTotalTokensCount():
number
Defined in: modules/natural_language_processing/LLMModule.ts:215
Returns the number of total tokens from the previous generation. This is a sum of prompt tokens and generated tokens.
number
The count of prompt and generated tokens.
interrupt():
void
Defined in: modules/natural_language_processing/LLMModule.ts:191
Interrupts model generation. It may return one more token after interrupt.
void
sendMessage(
message,media?):Promise<Message[]>
Defined in: modules/natural_language_processing/LLMModule.ts:168
Method to add user message to conversation.
After model responds it will call messageHistoryCallback() containing both user message and model response.
It also returns them.
string
The message string to send.
Optional media object containing a local image path for multimodal models.
string
Promise<Message[]>
- Updated message history including the new user message and model response.
setTokenCallback(
tokenCallback):void
Defined in: modules/natural_language_processing/LLMModule.ts:119
Sets new token callback invoked on every token batch.
Callback function to handle new tokens.
(token) => void
void
staticfromCustomModel(modelSource,tokenizerSource,tokenizerConfigSource,onDownloadProgress?,tokenCallback?,messageHistoryCallback?):Promise<LLMModule>
Defined in: modules/natural_language_processing/LLMModule.ts:94
Creates an LLM instance with a user-provided model binary.
Use this when working with a custom-exported LLM.
Internally uses 'custom' as the model name for telemetry.
The .pte model binary must be exported following the
ExecuTorch LLM export process.
The native runner expects the standard ExecuTorch text-generation interface — KV-cache
management, prefill/decode phases, and logit sampling are all handled by the runtime.
A fetchable resource pointing to the model binary.
A fetchable resource pointing to the tokenizer JSON file.
A fetchable resource pointing to the tokenizer config JSON file.
(progress) => void
Optional callback to monitor download progress, receiving a value between 0 and 1.
(token) => void
Optional callback invoked on every generated token.
(messageHistory) => void
Optional callback invoked when the model finishes a response, with the full message history.
Promise<LLMModule>
A Promise resolving to an LLMModule instance.
staticfromModelName(namedSources,onDownloadProgress?,tokenCallback?,messageHistoryCallback?):Promise<LLMModule>
Defined in: modules/natural_language_processing/LLMModule.ts:47
Creates an LLM instance for a built-in model.
An object specifying the model name, model source, tokenizer source, tokenizer config source, and optional capabilities.
readonly "vision"[]
(progress) => void
Optional callback to monitor download progress, receiving a value between 0 and 1.
(token) => void
Optional callback invoked on every generated token.
(messageHistory) => void
Optional callback invoked when the model finishes a response, with the full message history.
Promise<LLMModule>
A Promise resolving to an LLMModule instance.
import { LLMModule, LLAMA3_2_3B } from 'react-native-executorch';
const llm = await LLMModule.fromModelName(LLAMA3_2_3B);