Skip to content

Commit 2fe49d0

Browse files
authored
feat: added downloads tracking separate from huggingface (#990)
## Description Added separate request on model download to track downloads on our managed service. This would allow us to track things like geolocation and have insights on the entire timeline without the need to pay for huggingface premium. Set up so that it fails silently. Addtionally added model registry named `MODEL_REGISTRY` for better model discoverability. ### Introduces a breaking change? - [ ] Yes - [x] No ### Type of change - [ ] Bug fix (change which fixes an issue) - [x] New feature (change which adds functionality) - [x] Documentation update (improves or adds clarity to existing documentation) - [ ] Other (chores, tests, code style improvements etc.) ### Tested on - [x] iOS - [ ] Android ### Testing instructions <!-- Provide step-by-step instructions on how to test your changes. Include setup details if necessary. --> ### Screenshots <!-- Add screenshots here, if applicable --> ### Related issues <!-- Link related issues here using #issue-number --> ### Checklist - [ ] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have updated the documentation accordingly - [ ] My changes generate no new warnings ### Additional notes <!-- Include any additional information, assumptions, or context that reviewers might need to understand this PR. -->
1 parent 6d0aa95 commit 2fe49d0

File tree

9 files changed

+220
-12
lines changed

9 files changed

+220
-12
lines changed
Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
---
2+
title: Model Registry
3+
---
4+
5+
The [Model Registry](/react-native-executorch/docs/next/api-reference/variables/MODEL_REGISTRY) is a collection of all pre-configured model definitions shipped with React Native ExecuTorch. Each entry contains the model's name and all source URLs needed to download and run it, so you don't have to manage URLs manually.
6+
7+
## Usage
8+
9+
```typescript
10+
import { MODEL_REGISTRY, LLAMA3_2_1B } from 'react-native-executorch';
11+
```
12+
13+
### Accessing a model directly
14+
15+
Every model config is exported as a standalone constant:
16+
17+
```typescript
18+
import { LLAMA3_2_1B } from 'react-native-executorch';
19+
20+
const llm = useLLM({ model: LLAMA3_2_1B });
21+
```
22+
23+
### Listing all models
24+
25+
Use `MODEL_REGISTRY` to discover and enumerate all available models:
26+
27+
```typescript
28+
import { MODEL_REGISTRY } from 'react-native-executorch';
29+
30+
// Get all model names
31+
const names = Object.values(MODEL_REGISTRY.ALL_MODELS).map((m) => m.modelName);
32+
33+
// Find models by name
34+
const whisperModels = Object.values(MODEL_REGISTRY.ALL_MODELS).filter((m) =>
35+
m.modelName.includes('whisper')
36+
);
37+
```

packages/bare-resource-fetcher/src/ResourceFetcher.ts

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -300,7 +300,6 @@ export const BareResourceFetcher: BareResourceFetcherInterface = {
300300

301301
await RNFS.moveFile(extendedInfo.cacheFileUri!, extendedInfo.fileUri!);
302302
this.downloads.delete(source);
303-
ResourceFetcherUtils.triggerHuggingFaceDownloadCounter(extendedInfo.uri!);
304303

305304
const filename = extendedInfo.fileUri!.split('/').pop();
306305
if (filename) {

packages/bare-resource-fetcher/src/ResourceFetcherUtils.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ export namespace ResourceFetcherUtils {
2626
export const calculateDownloadProgress = CoreUtils.calculateDownloadProgress;
2727
export const triggerHuggingFaceDownloadCounter =
2828
CoreUtils.triggerHuggingFaceDownloadCounter;
29+
export const triggerDownloadEvent = CoreUtils.triggerDownloadEvent;
2930
export const getFilenameFromUri = CoreUtils.getFilenameFromUri;
3031

3132
export function getType(source: ResourceSource): SourceType {

packages/expo-resource-fetcher/src/ResourceFetcher.ts

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -264,9 +264,6 @@ export const ExpoResourceFetcher: ExpoResourceFetcherInterface = {
264264
to: resource.extendedInfo.fileUri,
265265
});
266266
this.downloads.delete(source);
267-
ResourceFetcherUtils.triggerHuggingFaceDownloadCounter(
268-
resource.extendedInfo.uri
269-
);
270267

271268
return this.returnOrStartNext(
272269
resource.extendedInfo,
@@ -526,7 +523,6 @@ export const ExpoResourceFetcher: ExpoResourceFetcherInterface = {
526523
to: sourceExtended.fileUri,
527524
});
528525
this.downloads.delete(source);
529-
ResourceFetcherUtils.triggerHuggingFaceDownloadCounter(uri);
530526
return ResourceFetcherUtils.removeFilePrefix(sourceExtended.fileUri);
531527
},
532528

packages/expo-resource-fetcher/src/ResourceFetcherUtils.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ export namespace ResourceFetcherUtils {
3939
export const calculateDownloadProgress = CoreUtils.calculateDownloadProgress;
4040
export const triggerHuggingFaceDownloadCounter =
4141
CoreUtils.triggerHuggingFaceDownloadCounter;
42+
export const triggerDownloadEvent = CoreUtils.triggerDownloadEvent;
4243
export const getFilenameFromUri = CoreUtils.getFilenameFromUri;
4344

4445
export function getType(source: ResourceSource): SourceType {

packages/react-native-executorch/src/constants/modelUrls.ts

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1023,3 +1023,129 @@ export const FSMN_VAD = {
10231023
modelName: 'fsmn-vad',
10241024
modelSource: FSMN_VAD_MODEL,
10251025
} as const;
1026+
1027+
/**
1028+
* Registry of all available model configurations.
1029+
*
1030+
* Use this to discover and enumerate all models shipped with the library.
1031+
* @example
1032+
* ```ts
1033+
* import { MODEL_REGISTRY } from 'react-native-executorch';
1034+
*
1035+
* // List all model names
1036+
* const names = Object.values(MODEL_REGISTRY).map(m => m.modelName);
1037+
*
1038+
* // Find models by name substring
1039+
* const whisperModels = Object.values(MODEL_REGISTRY)
1040+
* .filter(m => m.modelName.includes('whisper'));
1041+
* ```
1042+
* @category Utils
1043+
*/
1044+
export const MODEL_REGISTRY = {
1045+
ALL_MODELS: {
1046+
LLAMA3_2_3B,
1047+
LLAMA3_2_3B_QLORA,
1048+
LLAMA3_2_3B_SPINQUANT,
1049+
LLAMA3_2_1B,
1050+
LLAMA3_2_1B_QLORA,
1051+
LLAMA3_2_1B_SPINQUANT,
1052+
QWEN3_0_6B,
1053+
QWEN3_0_6B_QUANTIZED,
1054+
QWEN3_1_7B,
1055+
QWEN3_1_7B_QUANTIZED,
1056+
QWEN3_4B,
1057+
QWEN3_4B_QUANTIZED,
1058+
HAMMER2_1_0_5B,
1059+
HAMMER2_1_0_5B_QUANTIZED,
1060+
HAMMER2_1_1_5B,
1061+
HAMMER2_1_1_5B_QUANTIZED,
1062+
HAMMER2_1_3B,
1063+
HAMMER2_1_3B_QUANTIZED,
1064+
SMOLLM2_1_135M,
1065+
SMOLLM2_1_135M_QUANTIZED,
1066+
SMOLLM2_1_360M,
1067+
SMOLLM2_1_360M_QUANTIZED,
1068+
SMOLLM2_1_1_7B,
1069+
SMOLLM2_1_1_7B_QUANTIZED,
1070+
QWEN2_5_0_5B,
1071+
QWEN2_5_0_5B_QUANTIZED,
1072+
QWEN2_5_1_5B,
1073+
QWEN2_5_1_5B_QUANTIZED,
1074+
QWEN2_5_3B,
1075+
QWEN2_5_3B_QUANTIZED,
1076+
PHI_4_MINI_4B,
1077+
PHI_4_MINI_4B_QUANTIZED,
1078+
LFM2_5_1_2B_INSTRUCT,
1079+
LFM2_5_1_2B_INSTRUCT_QUANTIZED,
1080+
LFM2_VL_1_6B_QUANTIZED,
1081+
EFFICIENTNET_V2_S,
1082+
EFFICIENTNET_V2_S_QUANTIZED,
1083+
SSDLITE_320_MOBILENET_V3_LARGE,
1084+
RF_DETR_NANO,
1085+
STYLE_TRANSFER_CANDY,
1086+
STYLE_TRANSFER_CANDY_QUANTIZED,
1087+
STYLE_TRANSFER_MOSAIC,
1088+
STYLE_TRANSFER_MOSAIC_QUANTIZED,
1089+
STYLE_TRANSFER_RAIN_PRINCESS,
1090+
STYLE_TRANSFER_RAIN_PRINCESS_QUANTIZED,
1091+
STYLE_TRANSFER_UDNIE,
1092+
STYLE_TRANSFER_UDNIE_QUANTIZED,
1093+
WHISPER_TINY_EN,
1094+
WHISPER_TINY_EN_QUANTIZED,
1095+
WHISPER_BASE_EN,
1096+
WHISPER_BASE_EN_QUANTIZED,
1097+
WHISPER_SMALL_EN,
1098+
WHISPER_SMALL_EN_QUANTIZED,
1099+
WHISPER_TINY,
1100+
WHISPER_BASE,
1101+
WHISPER_SMALL,
1102+
DEEPLAB_V3_RESNET50,
1103+
DEEPLAB_V3_RESNET101,
1104+
DEEPLAB_V3_MOBILENET_V3_LARGE,
1105+
LRASPP_MOBILENET_V3_LARGE,
1106+
FCN_RESNET50,
1107+
FCN_RESNET101,
1108+
DEEPLAB_V3_RESNET50_QUANTIZED,
1109+
DEEPLAB_V3_RESNET101_QUANTIZED,
1110+
DEEPLAB_V3_MOBILENET_V3_LARGE_QUANTIZED,
1111+
LRASPP_MOBILENET_V3_LARGE_QUANTIZED,
1112+
FCN_RESNET50_QUANTIZED,
1113+
FCN_RESNET101_QUANTIZED,
1114+
SELFIE_SEGMENTATION,
1115+
YOLO26N_SEG,
1116+
YOLO26S_SEG,
1117+
YOLO26M_SEG,
1118+
YOLO26L_SEG,
1119+
YOLO26X_SEG,
1120+
RF_DETR_NANO_SEG,
1121+
CLIP_VIT_BASE_PATCH32_IMAGE,
1122+
CLIP_VIT_BASE_PATCH32_IMAGE_QUANTIZED,
1123+
ALL_MINILM_L6_V2,
1124+
ALL_MPNET_BASE_V2,
1125+
MULTI_QA_MINILM_L6_COS_V1,
1126+
MULTI_QA_MPNET_BASE_DOT_V1,
1127+
CLIP_VIT_BASE_PATCH32_TEXT,
1128+
BK_SDM_TINY_VPRED_512,
1129+
BK_SDM_TINY_VPRED_256,
1130+
FSMN_VAD,
1131+
},
1132+
} as const;
1133+
1134+
const urlToModelName = new Map<string, string>();
1135+
for (const config of Object.values(MODEL_REGISTRY.ALL_MODELS)) {
1136+
const modelName = config.modelName;
1137+
for (const [key, value] of Object.entries(config)) {
1138+
if (key !== 'modelName' && typeof value === 'string') {
1139+
urlToModelName.set(value, modelName);
1140+
}
1141+
}
1142+
}
1143+
1144+
/**
1145+
* Looks up the model name for a given source URL.
1146+
* @param url - The source URL to look up.
1147+
* @returns The model name if found, otherwise undefined.
1148+
*/
1149+
export function getModelNameForUrl(url: string): string | undefined {
1150+
return urlToModelName.get(url);
1151+
}
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
export const DOWNLOAD_EVENT_ENDPOINT =
2+
'https://ai.swmansion.com/telemetry/downloads/api/downloads';

packages/react-native-executorch/src/utils/ResourceFetcher.ts

Lines changed: 13 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,7 @@ export interface ResourceFetcherAdapter {
7676
*/
7777
export class ResourceFetcher {
7878
private static adapter: ResourceFetcherAdapter | null = null;
79+
private static reportedUrls = new Set<string>();
7980

8081
/**
8182
* Sets a custom resource fetcher adapter for resource operations.
@@ -128,16 +129,21 @@ export class ResourceFetcher {
128129
callback: (downloadProgress: number) => void = () => {},
129130
...sources: ResourceSource[]
130131
) {
131-
for (const source of sources) {
132-
if (typeof source === 'string') {
133-
try {
134-
ResourceFetcherUtils.triggerHuggingFaceDownloadCounter(source);
135-
} catch (error) {
136-
throw error;
132+
const result = await this.getAdapter().fetch(callback, ...sources);
133+
if (result) {
134+
for (const source of sources) {
135+
if (typeof source === 'string' && !this.reportedUrls.has(source)) {
136+
this.reportedUrls.add(source);
137+
try {
138+
ResourceFetcherUtils.triggerDownloadEvent(source);
139+
ResourceFetcherUtils.triggerHuggingFaceDownloadCounter(source);
140+
} catch (error) {
141+
throw error;
142+
}
137143
}
138144
}
139145
}
140-
return this.getAdapter().fetch(callback, ...sources);
146+
return result;
141147
}
142148

143149
/**

packages/react-native-executorch/src/utils/ResourceFetcherUtils.ts

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
11
import { ResourceSource } from '..';
2+
import { getModelNameForUrl } from '../constants/modelUrls';
3+
import { DOWNLOAD_EVENT_ENDPOINT } from '../constants/resourceFetcher';
24

35
/**
46
* Http status codes
@@ -193,6 +195,44 @@ export namespace ResourceFetcherUtils {
193195
}
194196
}
195197

198+
function getCountryCode(): string {
199+
try {
200+
const locale = Intl.DateTimeFormat().resolvedOptions().locale;
201+
const regionTag = locale.split('-').pop();
202+
if (regionTag && regionTag.length === 2) {
203+
return regionTag.toUpperCase();
204+
}
205+
} catch {}
206+
return 'UNKNOWN';
207+
}
208+
209+
function getModelNameFromUri(uri: string): string {
210+
const knownName = getModelNameForUrl(uri);
211+
if (knownName) {
212+
return knownName;
213+
}
214+
const pathname = new URL(uri).pathname;
215+
const filename = pathname.split('/').pop() ?? uri;
216+
return filename.replace(/\.[^.]+$/, '');
217+
}
218+
219+
/**
220+
* Sends a download event to the analytics endpoint.
221+
* @param uri - The URI of the downloaded resource.
222+
*/
223+
export function triggerDownloadEvent(uri: string) {
224+
try {
225+
fetch(DOWNLOAD_EVENT_ENDPOINT, {
226+
method: 'POST',
227+
headers: { 'Content-Type': 'application/json' },
228+
body: JSON.stringify({
229+
modelName: getModelNameFromUri(uri),
230+
countryCode: getCountryCode(),
231+
}),
232+
});
233+
} catch (e) {}
234+
}
235+
196236
/**
197237
* Generates a safe filename from a URI by removing the protocol and replacing special characters.
198238
* @param uri - The source URI.

0 commit comments

Comments
 (0)