|
1 | 1 | # react-native-nitro-onnxruntime |
2 | 2 |
|
3 | | -bob |
| 3 | +High-performance ONNX Runtime bindings for React Native, built with [Nitro Modules](https://nitro.margelo.com/) for maximum performance. |
| 4 | + |
| 5 | +## Features |
| 6 | + |
| 7 | +- ⚡ **Blazing Fast**: Built with Nitro Modules for zero-overhead JSI bindings |
| 8 | +- 🎯 **Hardware Acceleration**: Support for NNAPI (Android), CoreML (iOS), and XNNPACK |
| 9 | +- 🔄 **Modern API**: Promise-based async API with TypeScript support |
| 10 | +- 📦 **Flexible Model Loading**: Load models from files, URLs, or buffers |
| 11 | +- 🎨 **Full Type Support**: Complete TypeScript definitions |
| 12 | +- 🔧 **Configurable**: Extensive session options for optimization |
4 | 13 |
|
5 | 14 | ## Installation |
6 | 15 |
|
7 | 16 | ```sh |
8 | 17 | npm install react-native-nitro-onnxruntime react-native-nitro-modules |
9 | | - |
10 | | -> `react-native-nitro-modules` is required as this library relies on [Nitro Modules](https://nitro.margelo.com/). |
11 | 18 | ``` |
12 | 19 |
|
| 20 | +> **Note**: `react-native-nitro-modules` is required as this library relies on [Nitro Modules](https://nitro.margelo.com/). |
| 21 | +
|
13 | 22 | ## Usage |
14 | 23 |
|
15 | | -```js |
16 | | -import { multiply } from 'react-native-nitro-onnxruntime'; |
| 24 | +### Basic Example |
| 25 | + |
| 26 | +```typescript |
| 27 | +import ort from 'react-native-nitro-onnxruntime'; |
| 28 | + |
| 29 | +// Load a model |
| 30 | +const session = await ort.loadModel('path/to/model.onnx'); |
| 31 | + |
| 32 | +// Get input/output information |
| 33 | +console.log('Inputs:', session.inputNames); |
| 34 | +console.log('Outputs:', session.outputNames); |
| 35 | + |
| 36 | +// Prepare input data |
| 37 | +const inputData = new Float32Array(1 * 3 * 224 * 224); // Batch=1, Channels=3, Height=224, Width=224 |
| 38 | +// ... fill inputData with your data |
| 39 | + |
| 40 | +// Run inference |
| 41 | +const results = await session.run({ |
| 42 | + [session.inputNames[0].name]: inputData.buffer |
| 43 | +}); |
| 44 | + |
| 45 | +// Access output |
| 46 | +const outputBuffer = results[session.outputNames[0].name]; |
| 47 | +const outputData = new Float32Array(outputBuffer); |
| 48 | +console.log('Output:', outputData); |
| 49 | +``` |
| 50 | + |
| 51 | +### Loading Models from Assets |
| 52 | + |
| 53 | +Models can be loaded using `require()` for bundled assets: |
| 54 | + |
| 55 | +```typescript |
| 56 | +const session = await ort.loadModel(require('./assets/model.onnx')); |
| 57 | +``` |
| 58 | + |
| 59 | +This automatically copies the model to the device's file system on first load. |
| 60 | + |
| 61 | +### Hardware Acceleration |
| 62 | + |
| 63 | +#### Android (NNAPI) |
| 64 | + |
| 65 | +```typescript |
| 66 | +const session = await ort.loadModel('model.onnx', { |
| 67 | + executionProviders: ['nnapi'] |
| 68 | +}); |
| 69 | + |
| 70 | +// Or with options |
| 71 | +const session = await ort.loadModel('model.onnx', { |
| 72 | + executionProviders: [{ |
| 73 | + name: 'nnapi', |
| 74 | + useFP16: true, // Use FP16 precision |
| 75 | + cpuDisabled: true, // Disable CPU fallback |
| 76 | + }] |
| 77 | +}); |
| 78 | +``` |
| 79 | + |
| 80 | +#### iOS (CoreML) |
| 81 | + |
| 82 | +```typescript |
| 83 | +const session = await ort.loadModel('model.onnx', { |
| 84 | + executionProviders: ['coreml'] |
| 85 | +}); |
| 86 | + |
| 87 | +// Or with options |
| 88 | +const session = await ort.loadModel('model.onnx', { |
| 89 | + executionProviders: [{ |
| 90 | + name: 'coreml', |
| 91 | + useCPUOnly: false, |
| 92 | + onlyEnableDeviceWithANE: true, // Only use devices with Apple Neural Engine |
| 93 | + }] |
| 94 | +}); |
| 95 | +``` |
| 96 | + |
| 97 | +#### XNNPACK (Cross-platform) |
| 98 | + |
| 99 | +```typescript |
| 100 | +const session = await ort.loadModel('model.onnx', { |
| 101 | + executionProviders: ['xnnpack'] |
| 102 | +}); |
| 103 | +``` |
| 104 | + |
| 105 | +### Advanced Configuration |
| 106 | + |
| 107 | +```typescript |
| 108 | +const session = await ort.loadModel('model.onnx', { |
| 109 | + // Thread configuration |
| 110 | + intraOpNumThreads: 4, |
| 111 | + interOpNumThreads: 2, |
| 112 | + |
| 113 | + // Graph optimization |
| 114 | + graphOptimizationLevel: 'all', // 'disabled' | 'basic' | 'extended' | 'all' |
| 115 | + |
| 116 | + // Memory settings |
| 117 | + enableCpuMemArena: true, |
| 118 | + enableMemPattern: true, |
| 119 | + |
| 120 | + // Execution mode |
| 121 | + executionMode: 'sequential', // 'sequential' | 'parallel' |
| 122 | + |
| 123 | + // Logging |
| 124 | + logId: 'MyModel', |
| 125 | + logSeverityLevel: 2, // 0=Verbose, 1=Info, 2=Warning, 3=Error, 4=Fatal |
| 126 | + |
| 127 | + // Execution providers |
| 128 | + executionProviders: ['nnapi', 'cpu'] |
| 129 | +}); |
| 130 | +``` |
| 131 | + |
| 132 | +### Loading from Buffer |
| 133 | + |
| 134 | +```typescript |
| 135 | +import RNFS from 'react-native-fs'; |
| 136 | + |
| 137 | +// Load model file into buffer |
| 138 | +const modelPath = 'path/to/model.onnx'; |
| 139 | +const modelBuffer = await RNFS.readFile(modelPath, 'base64'); |
| 140 | +const arrayBuffer = Uint8Array.from(atob(modelBuffer), c => c.charCodeAt(0)).buffer; |
| 141 | + |
| 142 | +// Create session from buffer |
| 143 | +const session = await ort.loadModelFromBuffer(arrayBuffer, { |
| 144 | + executionProviders: ['nnapi'] |
| 145 | +}); |
| 146 | +``` |
| 147 | + |
| 148 | +### Memory Management |
| 149 | + |
| 150 | +```typescript |
| 151 | +// Dispose of session when done |
| 152 | +session.dispose(); |
| 153 | +``` |
| 154 | + |
| 155 | +## API Reference |
| 156 | + |
| 157 | +### `ort.getVersion()` |
17 | 158 |
|
18 | | -// ... |
| 159 | +Returns the ONNX Runtime version string. |
19 | 160 |
|
20 | | -const result = multiply(3, 7); |
| 161 | +```typescript |
| 162 | +const version = ort.getVersion(); |
| 163 | +console.log('ONNX Runtime version:', version); |
21 | 164 | ``` |
22 | 165 |
|
| 166 | +### `ort.loadModel(path, options?)` |
| 167 | + |
| 168 | +Load an ONNX model from a file path or `require()` asset. |
| 169 | + |
| 170 | +**Parameters:** |
| 171 | +- `path`: `string` | `number` (from `require()`) - Path to the model file |
| 172 | +- `options`: `SessionOptions` (optional) - Configuration options |
| 173 | + |
| 174 | +**Returns:** `Promise<InferenceSession>` |
| 175 | + |
| 176 | +### `ort.loadModelFromBuffer(buffer, options?)` |
| 177 | + |
| 178 | +Load an ONNX model from an ArrayBuffer. |
| 179 | + |
| 180 | +**Parameters:** |
| 181 | +- `buffer`: `ArrayBuffer` - Model data |
| 182 | +- `options`: `SessionOptions` (optional) - Configuration options |
| 183 | + |
| 184 | +**Returns:** `Promise<InferenceSession>` |
| 185 | + |
| 186 | +### `InferenceSession` |
| 187 | + |
| 188 | +#### `session.inputNames` |
| 189 | + |
| 190 | +Array of input tensor information: |
| 191 | +```typescript |
| 192 | +type Tensor = { |
| 193 | + name: string; |
| 194 | + dims: number[]; // Shape, negative values indicate dynamic dimensions |
| 195 | + type: string; // 'float32', 'int64', etc. |
| 196 | +}; |
| 197 | +``` |
| 198 | + |
| 199 | +#### `session.outputNames` |
| 200 | + |
| 201 | +Array of output tensor information (same format as `inputNames`). |
| 202 | + |
| 203 | +#### `session.run(feeds)` |
| 204 | + |
| 205 | +Run inference with the given inputs. |
| 206 | + |
| 207 | +**Parameters:** |
| 208 | +- `feeds`: `Record<string, ArrayBuffer>` - Map of input names to ArrayBuffers |
| 209 | + |
| 210 | +**Returns:** `Promise<Record<string, ArrayBuffer>>` - Map of output names to ArrayBuffers |
| 211 | + |
| 212 | +#### `session.dispose()` |
| 213 | + |
| 214 | +Free the session and release resources. |
| 215 | + |
| 216 | +### `SessionOptions` |
| 217 | + |
| 218 | +```typescript |
| 219 | +type SessionOptions = { |
| 220 | + intraOpNumThreads?: number; |
| 221 | + interOpNumThreads?: number; |
| 222 | + graphOptimizationLevel?: 'disabled' | 'basic' | 'extended' | 'all'; |
| 223 | + enableCpuMemArena?: boolean; |
| 224 | + enableMemPattern?: boolean; |
| 225 | + executionMode?: 'sequential' | 'parallel'; |
| 226 | + logId?: string; |
| 227 | + logSeverityLevel?: number; |
| 228 | + executionProviders?: (string | ProviderOptions)[]; |
| 229 | +}; |
| 230 | + |
| 231 | +type ProviderOptions = { |
| 232 | + name: 'nnapi' | 'coreml' | 'xnnpack'; |
| 233 | + // NNAPI options (Android) |
| 234 | + useFP16?: boolean; |
| 235 | + useNCHW?: boolean; |
| 236 | + cpuDisabled?: boolean; |
| 237 | + cpuOnly?: boolean; |
| 238 | + // CoreML options (iOS) |
| 239 | + useCPUOnly?: boolean; |
| 240 | + useCPUAndGPU?: boolean; |
| 241 | + enableOnSubgraph?: boolean; |
| 242 | + onlyEnableDeviceWithANE?: boolean; |
| 243 | +}; |
| 244 | +``` |
| 245 | + |
| 246 | +## Supported Platforms |
| 247 | + |
| 248 | +- ✅ Android (API 21+) |
| 249 | +- ✅ iOS (13.0+) |
| 250 | + |
| 251 | +## Supported Data Types |
| 252 | + |
| 253 | +- `float32` (Float) |
| 254 | +- `float64` (Double) |
| 255 | +- `int8` |
| 256 | +- `uint8` |
| 257 | +- `int16` |
| 258 | +- `int32` |
| 259 | +- `int64` |
| 260 | +- `bool` |
| 261 | + |
| 262 | +## Performance Tips |
| 263 | + |
| 264 | +1. **Use Hardware Acceleration**: Enable NNAPI (Android) or CoreML (iOS) for better performance |
| 265 | +2. **Optimize Thread Count**: Set `intraOpNumThreads` based on your device's CPU cores |
| 266 | +3. **Enable Graph Optimization**: Use `graphOptimizationLevel: 'all'` for production |
| 267 | +4. **Reuse Sessions**: Create the session once and reuse it for multiple inferences |
| 268 | +5. **Use FP16**: Enable `useFP16` on NNAPI for faster inference with acceptable accuracy loss |
| 269 | + |
| 270 | +## Troubleshooting |
| 271 | + |
| 272 | +### Android Build Issues |
| 273 | + |
| 274 | +If you encounter duplicate library errors, ensure your `android/app/build.gradle` has: |
| 275 | + |
| 276 | +```gradle |
| 277 | +android { |
| 278 | + packaging { |
| 279 | + jniLibs { |
| 280 | + pickFirsts += ['**/libonnxruntime.so'] |
| 281 | + } |
| 282 | + } |
| 283 | +} |
| 284 | +``` |
| 285 | + |
| 286 | +### Memory Issues |
| 287 | + |
| 288 | +Always call `session.dispose()` when you're done with a session to free up memory. |
| 289 | + |
| 290 | +## Example App |
| 291 | + |
| 292 | +See the [example](./example) directory for a complete working example with speed comparisons. |
| 293 | + |
23 | 294 | ## Contributing |
24 | 295 |
|
25 | 296 | See the [contributing guide](CONTRIBUTING.md) to learn how to contribute to the repository and the development workflow. |
|
30 | 301 |
|
31 | 302 | --- |
32 | 303 |
|
33 | | -Made with [create-react-native-library](https://github.com/callstack/react-native-builder-bob) |
34 | | - |
35 | | -// For the require -> It copies to file from rn to the files folder in your app. And then just checks if it already exists otherwise copy |
| 304 | +Made with [Nitro Modules](https://nitro.margelo.com/) and [create-react-native-library](https://github.com/callstack/react-native-builder-bob) |
0 commit comments