fix: add inference mutex to Text Embedding and Text-to-Image

[barhanc]

Description

Adds thread-safety to Text Embeddings and Text-to-Image models mirroring what was already done for models inheriting from VisionModel and VAD.

Introduces a breaking change?

• Yes
• No

Type of change

• Bug fix (change which fixes an issue)
• New feature (change which adds functionality)
• Documentation update (improves or adds clarity to existing documentation)
• Other (chores, tests, code style improvements etc.)

Tested on

• iOS
• Android

Testing instructions

Use the following app screen and try to trigger the race condition before fix and verify that it doesn't occur after applying the fix. You can use adb logcat | grep -E "FATAL|SIGSEGV|backtrace" to observe the error on Android.

import React, { useState } from 'react';
import { Button, ScrollView, Text, View } from 'react-native';
import {
  BK_SDM_TINY_VPRED_512,
  TextToImageModule,
} from 'react-native-executorch';
import {
  CLIP_VIT_BASE_PATCH32_TEXT,
  TextEmbeddingsModule,
} from 'react-native-executorch';
import { FSMN_VAD, VADModule } from 'react-native-executorch';

const DELAY_MS = 50; // tune this so that forward() is running when delete() is called

const MODEL_VAD = {
  name: 'VAD',
  load: (onProgress: (p: number) => void) =>
    VADModule.fromModelName(FSMN_VAD, onProgress),
  input: () => new Float32Array(16000 * 300),
};
const MODEL_TEXT_EMBEDDINGS = {
  name: 'TextEmbeddings',
  load: (onProgress: (p: number) => void) =>
    TextEmbeddingsModule.fromModelName(CLIP_VIT_BASE_PATCH32_TEXT, onProgress),
  input: () => 'hello world',
};
const MODEL_TEXT_TO_IMAGE = {
  name: 'TextToImage',
  load: (onProgress: (p: number) => void) =>
    TextToImageModule.fromModelName(BK_SDM_TINY_VPRED_512, onProgress),
  input: () => 'a red apple',
};
const MODEL = MODEL_TEXT_EMBEDDINGS;

export default function RaceTest() {
  const [lines, setLines] = useState<string[]>([]);
  const [downloadProgress, setDownloadProgress] = useState<number | null>(null);
  const log = (line: string) => setLines((prev) => [line, ...prev]);
  const run = async () => {
    setLines([]);
    setDownloadProgress(null);
    log(`model: ${MODEL.name}`);

    log('loading');
    const model = await MODEL.load((p) => setDownloadProgress(p));
    setDownloadProgress(null);

    log('running forward()');
    const result = model.forward(MODEL.input());

    log(`waiting ${DELAY_MS} ms`);
    await new Promise((r) => setTimeout(r, DELAY_MS));

    log('calling delete()');
    model.delete();

    try {
      await result;
      log('forward() completed successfully');
    } catch (e: any) {
      log('error: ' + (e?.message ?? String(e)));
    }
  };

  return (
    <View>
      <Button title="Run Race Test" onPress={run} />
      {downloadProgress !== null && (
        <Text>downloading: {Math.round(downloadProgress * 100)}%</Text>
      )}
      <ScrollView>
        {lines.map((l, i) => (
          <Text key={i}>{l}</Text>
        ))}
      </ScrollView>
    </View>
  );
}

Screenshots

Related issues

#1055

Checklist

• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have updated the documentation accordingly
• My changes generate no new warnings

Additional notes

[msluszniak]

With fix, I indeed got segfault on provided demo app code:

04-08 13:39:15.046  5901  5994 F libc    : Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x7844af9aec in tid 5994 (RN_ET_Worker2), pid 5901 (com.barern)
04-08 13:39:15.542  6059  6059 F DEBUG   : signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0000007844af9aec

when applied changes from this PR, no crashes occur.