Skip to content

ExecutorchModule.md

Latest commit

 

History

History
244 lines (135 loc) · 5.83 KB

ExecutorchModule.md

File metadata and controls

244 lines (135 loc) · 5.83 KB

Class: ExecutorchModule

Defined in: modules/general/ExecutorchModule.ts:13

General module for executing custom Executorch models.

Extends

  • BaseModule

Constructors

Constructor

new ExecutorchModule(): ExecutorchModule

Returns

ExecutorchModule

Inherited from

BaseModule.constructor

Properties

generateFromFrame()

generateFromFrame: (frameData, ...args) => any

Defined in: modules/BaseModule.ts:53

Process a camera frame directly for real-time inference.

This method is bound to a native JSI function after calling load(), making it worklet-compatible and safe to call from VisionCamera's frame processor thread.

Performance characteristics:

  • Zero-copy path: When using frame.getNativeBuffer() from VisionCamera v5, frame data is accessed directly without copying (fastest, recommended).
  • Copy path: When using frame.toArrayBuffer(), pixel data is copied from native to JS, then accessed from native code (slower, fallback).

Usage with VisionCamera:

const frameOutput = useFrameOutput({
  pixelFormat: 'rgb',
  onFrame(frame) {
    'worklet';
    // Zero-copy approach (recommended)
    const nativeBuffer = frame.getNativeBuffer();
    const result = model.generateFromFrame(
      { nativeBuffer: nativeBuffer.pointer, width: frame.width, height: frame.height },
      ...args
    );
    nativeBuffer.release();
    frame.dispose();
  }
});

Parameters

frameData

Frame

Frame data object with either nativeBuffer (zero-copy) or data (ArrayBuffer)

args

...any[]

Additional model-specific arguments (e.g., threshold, options)

Returns

any

Model-specific output (e.g., detections, classifications, embeddings)

See

Frame for frame data format details

Inherited from

BaseModule.generateFromFrame

nativeModule

nativeModule: any = null

Defined in: modules/BaseModule.ts:16

Internal

Native module instance (JSI Host Object)

Inherited from

BaseModule.nativeModule

Methods

delete()

delete(): void

Defined in: modules/BaseModule.ts:81

Unloads the model from memory and releases native resources.

Always call this method when you're done with a model to prevent memory leaks.

Returns

void

Inherited from

BaseModule.delete

forward()

forward(inputTensor): Promise<TensorPtr[]>

Defined in: modules/general/ExecutorchModule.ts:48

Executes the model's forward pass, where input is an array of TensorPtr objects. If the inference is successful, an array of tensor pointers is returned.

Parameters

inputTensor

TensorPtr[]

Array of input tensor pointers.

Returns

Promise<TensorPtr[]>

An array of output tensor pointers.

forwardET()

protected forwardET(inputTensor): Promise<TensorPtr[]>

Defined in: modules/BaseModule.ts:62

Internal

Runs the model's forward method with the given input tensors. It returns the output tensors that mimic the structure of output from ExecuTorch.

Parameters

inputTensor

TensorPtr[]

Array of input tensors.

Returns

Promise<TensorPtr[]>

Array of output tensors.

Inherited from

BaseModule.forwardET

getInputShape()

getInputShape(methodName, index): Promise<number[]>

Defined in: modules/BaseModule.ts:72

Gets the input shape for a given method and index.

Parameters

methodName

string

method name

index

number

index of the argument which shape is requested

Returns

Promise<number[]>

The input shape as an array of numbers.

Inherited from

BaseModule.getInputShape

load()

load(modelSource, onDownloadProgressCallback?): Promise<void>

Defined in: modules/general/ExecutorchModule.ts:20

Loads the model, where modelSource is a string, number, or object that specifies the location of the model binary. Optionally accepts a download progress callback.

Parameters

modelSource

ResourceSource

Source of the model to be loaded.

onDownloadProgressCallback?

(progress) => void

Optional callback to monitor download progress.

Returns

Promise<void>