Defined in: modules/natural_language_processing/TextEmbeddingsModule.ts:13
Module for generating text embeddings from input text.
BaseModule
generateFromFrame: (
frameData, ...args) =>any
Defined in: modules/BaseModule.ts:53
Process a camera frame directly for real-time inference.
This method is bound to a native JSI function after calling load(),
making it worklet-compatible and safe to call from VisionCamera's
frame processor thread.
Performance characteristics:
- Zero-copy path: When using
frame.getNativeBuffer()from VisionCamera v5, frame data is accessed directly without copying (fastest, recommended). - Copy path: When using
frame.toArrayBuffer(), pixel data is copied from native to JS, then accessed from native code (slower, fallback).
Usage with VisionCamera:
const frameOutput = useFrameOutput({
pixelFormat: 'rgb',
onFrame(frame) {
'worklet';
// Zero-copy approach (recommended)
const nativeBuffer = frame.getNativeBuffer();
const result = model.generateFromFrame(
{ nativeBuffer: nativeBuffer.pointer, width: frame.width, height: frame.height },
...args
);
nativeBuffer.release();
frame.dispose();
}
});Frame data object with either nativeBuffer (zero-copy) or data (ArrayBuffer)
...any[]
Additional model-specific arguments (e.g., threshold, options)
any
Model-specific output (e.g., detections, classifications, embeddings)
Frame for frame data format details
BaseModule.generateFromFrame
nativeModule:
any=null
Defined in: modules/BaseModule.ts:16
Internal
Native module instance (JSI Host Object)
BaseModule.nativeModule
delete():
void
Defined in: modules/BaseModule.ts:81
Unloads the model from memory and releases native resources.
Always call this method when you're done with a model to prevent memory leaks.
void
BaseModule.delete
forward(
input):Promise<Float32Array<ArrayBufferLike>>
Defined in: modules/natural_language_processing/TextEmbeddingsModule.ts:85
Executes the model's forward pass to generate an embedding for the provided text.
string
The text string to embed.
Promise<Float32Array<ArrayBufferLike>>
A Promise resolving to a Float32Array containing the embedding vector.
protectedforwardET(inputTensor):Promise<TensorPtr[]>
Defined in: modules/BaseModule.ts:62
Internal
Runs the model's forward method with the given input tensors. It returns the output tensors that mimic the structure of output from ExecuTorch.
Array of input tensors.
Promise<TensorPtr[]>
Array of output tensors.
BaseModule.forwardET
getInputShape(
methodName,index):Promise<number[]>
Defined in: modules/BaseModule.ts:72
Gets the input shape for a given method and index.
string
method name
number
index of the argument which shape is requested
Promise<number[]>
The input shape as an array of numbers.
BaseModule.getInputShape
staticfromCustomModel(modelSource,tokenizerSource,onDownloadProgress?):Promise<TextEmbeddingsModule>
Defined in: modules/natural_language_processing/TextEmbeddingsModule.ts:65
Creates a text embeddings instance with a user-provided model binary and tokenizer. Use this when working with a custom-exported model that is not one of the built-in presets.
A fetchable resource pointing to the model binary.
A fetchable resource pointing to the tokenizer file.
(progress) => void
Optional callback to monitor download progress, receiving a value between 0 and 1.
Promise<TextEmbeddingsModule>
A Promise resolving to a TextEmbeddingsModule instance.
The native model contract for this method is not formally defined and may change between releases. Refer to the native source code for the current expected tensor interface.
staticfromModelName(namedSources,onDownloadProgress?):Promise<TextEmbeddingsModule>
Defined in: modules/natural_language_processing/TextEmbeddingsModule.ts:25
Creates a text embeddings instance for a built-in model.
An object specifying which built-in model to load and where to fetch it from.
(progress) => void
Optional callback to monitor download progress, receiving a value between 0 and 1.
Promise<TextEmbeddingsModule>
A Promise resolving to a TextEmbeddingsModule instance.