Skip to content

Commit 33b2fab

Browse files
feat: add support openai/privacy-filter model (#1104)
## Description Adds a new **Privacy Filter** native model for on-device PII detection, with a TypeScript hook (`usePrivacyFilter`), module (`PrivacyFilterModule`), types, and a demo screen in the `apps/llm` example. Two models are wired up out-of-the-box: - **`PRIVACY_FILTER_OPENAI`** — [openai/privacy-filter](https://huggingface.co/openai/privacy-filter), 8 categories (person, email, phone, address, DOB, URL, account number, secret). - **`PRIVACY_FILTER_NEMOTRON`** — [OpenMed/privacy-filter-nemotron-base](https://huggingface.co/OpenMed/privacy-filter-nemotron-base), 55+ categories adding medical, financial, demographic, technical, and government-document spans. Both models share the same single-method `forward(input_ids, attention_mask) → logits` graph, the same o200k tokenizer (pad/eos id 199999), and 256-token banded attention. They differ only in the BIOES label space, which is supplied as `labelNames` at load time. **Highlights:** - **Constrained Viterbi decoding** with six tunable BIOES transition biases, matching OpenAI's `viterbi_calibration.json` schema. Defaults to neutral validity-only Viterbi. - **Sliding-window inference**: 256-token windows, 50% overlap, no truncation — arbitrary-length input is supported. - **`seq_len` is read from the model's `forward` input shape** at load time, so a 512-token export would Just Work without code changes. - **Typed JSI return**: `generate()` returns `PiiEntity[]` directly via a `getJsiValue` overload — no JSON bridge. - **Viterbi decoder is split** into its own `Viterbi.{h,cpp}` module under `rnexecutorch::models::privacy_filter::viterbi`, mirroring the `Utils.{h,cpp}` convention used by VAD. ### Introduces a breaking change? - [ ] Yes - [x] No ### Type of change - [ ] Bug fix (change which fixes an issue) - [x] New feature (change which adds functionality) - [ ] Documentation update (improves or adds clarity to existing documentation) - [ ] Other (chores, tests, code style improvements etc.) ### Tested on - [x] iOS - [x] Android ### Testing instructions 1. `cd apps/llm && yarn install` 2. iOS: `yarn pods && yarn ios` · Android: `yarn android` 3. From the home screen, open **"Privacy Filter"**. 4. Pick a model in the picker (OpenAI base or Nemotron fine-tune) — the model downloads on first run. 5. Tap **"Run"** to scan the bundled sample text. 6. Verify detected entity spans are highlighted inline with their label and color, and inference time is shown on the button. 7. Switch models and re-run — entity coverage should differ between the two (Nemotron picks up medical/financial/demographic categories the base model doesn't have). ### Screenshots <!-- Add screenshots here, if applicable --> ### Related issues <!-- Link related issues here using #issue-number --> ### Checklist - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have updated the documentation accordingly - [x] My changes generate no new warnings ### Additional notes - The `tokenizer.json` is bit-identical between the OpenAI base and the Nemotron fine-tune; both use the GPT-4o `o200k` BPE. - All PTE files are exported as a **single `forward` method** rather than the 18-method multimethod export — this brings RAM down from ~3.5 GB to ~1.2 GB on iPhone 16 Pro and inference from ~8s to ~1.5s, since planner buffers are reused across the full graph. - Docs are added to `docs/docs/` (next-version only). The `06-api-reference/` typedoc output regenerates automatically from the `@category Models - Privacy Filter` tags on the exported constants and the public hook/module/types — no manual entries were added there. - Benchmarks (`02-benchmarks/`) are intentionally *not* updated in this PR; we'll add measured numbers in a follow-up once we have proper multi-device timings. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent c11dbad commit 33b2fab

22 files changed

Lines changed: 1702 additions & 0 deletions

File tree

.cspell-wordlist.txt

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -188,3 +188,8 @@ stringifying
188188
hɛloʊ
189189
wɜːld
190190
bielik
191+
nemotron
192+
BIOES
193+
viterbi
194+
argmaxes
195+
unpadded

.eslintrc.js

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ const VALID_CATEGORIES = [
1313
'Models - Semantic Segmentation',
1414
'Models - Speech To Text',
1515
'Models - Style Transfer',
16+
'Models - Privacy Filter',
1617
'Models - Text Embeddings',
1718
'Models - Text to Speech',
1819
'Models - VLM',

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -103,3 +103,7 @@ packages/react-native-executorch/common/rnexecutorch/tests/integration/assets/mo
103103
Makefile
104104
*.pte
105105

106+
.agents
107+
.claude
108+
skills-lock.json
109+

apps/llm/app/_layout.tsx

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -146,6 +146,14 @@ export default function _layout() {
146146
headerTitleStyle: { color: ColorPalette.primary },
147147
}}
148148
/>
149+
<Drawer.Screen
150+
name="privacy_filter/index"
151+
options={{
152+
drawerLabel: 'Privacy Filter (PII)',
153+
title: 'Privacy Filter',
154+
headerTitleStyle: { color: ColorPalette.primary },
155+
}}
156+
/>
149157
</Drawer>
150158
</GeneratingContext>
151159
);

apps/llm/app/index.tsx

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,12 @@ export default function Home() {
4141
>
4242
<Text style={styles.buttonText}>Multimodal LLM (VLM)</Text>
4343
</TouchableOpacity>
44+
<TouchableOpacity
45+
style={styles.button}
46+
onPress={() => router.navigate('privacy_filter/')}
47+
>
48+
<Text style={styles.buttonText}>Privacy Filter (PII)</Text>
49+
</TouchableOpacity>
4450
</View>
4551
</View>
4652
);
Lines changed: 251 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,251 @@
1+
import { useMemo, useState } from 'react';
2+
import {
3+
ActivityIndicator,
4+
ScrollView,
5+
StyleSheet,
6+
Text,
7+
TouchableOpacity,
8+
View,
9+
} from 'react-native';
10+
import { useIsFocused } from '@react-navigation/native';
11+
import { useSafeAreaInsets } from 'react-native-safe-area-context';
12+
import {
13+
PiiEntity,
14+
PRIVACY_FILTER_NEMOTRON,
15+
PRIVACY_FILTER_OPENAI,
16+
PrivacyFilterModelSources,
17+
usePrivacyFilter,
18+
} from 'react-native-executorch';
19+
import ColorPalette from '../../colors';
20+
import { ModelOption, ModelPicker } from '../../components/ModelPicker';
21+
import {
22+
buildSegments,
23+
colorForLabel,
24+
matchEntities,
25+
} from '../../utils/piiMatching';
26+
27+
/* cspell:disable */
28+
// Sample tuned for the OpenAI base model — exercises the 8 entity types it
29+
// recognizes (person, email, phone, account_number, address, date, url,
30+
// secret).
31+
const OPENAI_SAMPLE = `My name is Sarah Chen and I work as a senior engineer at Acme Corp. You can reach me at sarah.chen@acmecorp.io or call my direct line at (415) 923-0847. For billing inquiries, my account number is ACC-8821-4490-3371.
32+
33+
I've been living at 17 Birchwood Lane, Portland, OR 97201 since October 3rd, 2019. Before that I was at 8 Rue de Rivoli, Paris, 75001, France. My personal website is https://sarahchen.dev and my GitHub is https://github.com/schen-eng. Feel free to connect — I usually respond within a business day.
34+
35+
My date of birth is June 12, 1991, and my backup email is s.chen.personal@gmail.com in case the primary address is unreachable. This message also contains a confidential API key: sk-T93kXpLm2NvBqR7dYwZ4. Please do not share it outside the team. You can also reach my colleague James Okonkwo at j.okonkwo@acmecorp.io or at his mobile +44 7911 123456.`;
36+
// Sample tuned for the OpenMed Nemotron model — covers categories the base
37+
// OpenAI model doesn't have (medical, financial, technical, demographic).
38+
39+
const NEMOTRON_SAMPLE = `Patient intake for Maria Lopez, female, age 47, blood type O+, born 1978-05-12. MRN 994-2210-AB; health plan beneficiary number HPBN-552-9931 with Aetna. SSN 412-55-7821, national ID DNI 88-7762-X. Primary occupation: registered nurse, currently employed full-time at Mercy General. Religion: Catholic; political view: independent.
40+
41+
Reach her at maria.lopez@example.com or +1 (415) 555-0142. Mailing address: 84 Cedar Hill Road, Apt 3B, Berkeley, CA 94703, United States. Vehicle plate 7XKL922; driver license CA-D1294883.
42+
43+
Payment for last visit: Visa ending 4992-1133-7820-4419, expires 11/28, CVV 884. Bank routing 021000089, SWIFT BIC CHASUS33. Employer EIN tax ID 47-3320118. Customer ID CUST-553201, employee ID EMP-A0093.
44+
45+
Workstation MAC 3C:22:FB:8E:01:9A, IPv4 10.0.42.118, device IMEI 359888061234560. Service account API key sk-live-Tn8x3pLm2NvBqR7dYwZ4QF, password Hunter2!Spring. Session cookie sid=eyJ1c2VyIjoiOTk0MjIxMCJ9.`;
46+
/* cspell:enable */
47+
48+
const MODEL_OPTIONS: ModelOption<PrivacyFilterModelSources>[] = [
49+
{ label: 'OpenAI Privacy Filter (8 entities)', value: PRIVACY_FILTER_OPENAI },
50+
{
51+
label: 'OpenMed Nemotron (55 entities)',
52+
value: PRIVACY_FILTER_NEMOTRON,
53+
},
54+
];
55+
56+
// Pick the right sample to display/run based on the active model.
57+
function sampleFor(model: PrivacyFilterModelSources): string {
58+
return model.modelName === PRIVACY_FILTER_NEMOTRON.modelName
59+
? NEMOTRON_SAMPLE
60+
: OPENAI_SAMPLE;
61+
}
62+
63+
function HighlightedText({
64+
source,
65+
entities,
66+
}: {
67+
source: string;
68+
entities: PiiEntity[];
69+
}) {
70+
const segments = useMemo(
71+
() => buildSegments(source, matchEntities(source, entities)),
72+
[source, entities]
73+
);
74+
return (
75+
<Text style={styles.sampleText}>
76+
{segments.map((seg, i) =>
77+
seg.label ? (
78+
<Text
79+
key={i}
80+
style={[
81+
styles.highlight,
82+
{ backgroundColor: colorForLabel(seg.label) },
83+
]}
84+
>
85+
{seg.text}
86+
</Text>
87+
) : (
88+
<Text key={i}>{seg.text}</Text>
89+
)
90+
)}
91+
</Text>
92+
);
93+
}
94+
95+
function PrivacyFilterScreen() {
96+
const { bottom } = useSafeAreaInsets();
97+
const [entities, setEntities] = useState<PiiEntity[] | null>(null);
98+
const [runError, setRunError] = useState<string | null>(null);
99+
const [inferenceMs, setInferenceMs] = useState<number | null>(null);
100+
const [selectedModel, setSelectedModel] = useState<PrivacyFilterModelSources>(
101+
PRIVACY_FILTER_OPENAI
102+
);
103+
104+
const filter = usePrivacyFilter({ model: selectedModel });
105+
const sampleText = sampleFor(selectedModel);
106+
107+
const onRun = async () => {
108+
setRunError(null);
109+
setEntities(null);
110+
setInferenceMs(null);
111+
const startedAt = Date.now();
112+
try {
113+
const result = await filter.generate(sampleText);
114+
const elapsed = Date.now() - startedAt;
115+
setInferenceMs(elapsed);
116+
setEntities(result);
117+
} catch (e) {
118+
setRunError(e instanceof Error ? e.message : String(e));
119+
}
120+
};
121+
122+
const disabled = !filter.isReady || filter.isGenerating;
123+
124+
return (
125+
<View style={[styles.container, { paddingBottom: bottom + 8 }]}>
126+
<ModelPicker
127+
models={MODEL_OPTIONS}
128+
selectedModel={selectedModel}
129+
onSelect={(m) => {
130+
setEntities(null);
131+
setRunError(null);
132+
setInferenceMs(null);
133+
setSelectedModel(m);
134+
}}
135+
label="Model"
136+
disabled={filter.isGenerating}
137+
/>
138+
139+
{filter.error && (
140+
<View style={styles.errorBanner}>
141+
<Text style={styles.errorText}>
142+
Load error: {filter.error.message}
143+
</Text>
144+
</View>
145+
)}
146+
147+
{!filter.isReady && !filter.error && (
148+
<View style={styles.centerBlock}>
149+
<ActivityIndicator color={ColorPalette.primary} />
150+
<Text style={styles.muted}>
151+
Downloading model…{' '}
152+
{Math.round((filter.downloadProgress ?? 0) * 100)}%
153+
</Text>
154+
</View>
155+
)}
156+
157+
<ScrollView style={styles.textBox}>
158+
{entities ? (
159+
<HighlightedText source={sampleText} entities={entities} />
160+
) : (
161+
<Text style={styles.sampleText}>{sampleText}</Text>
162+
)}
163+
</ScrollView>
164+
165+
<TouchableOpacity
166+
style={[styles.runButton, disabled && styles.buttonDisabled]}
167+
onPress={onRun}
168+
disabled={disabled}
169+
>
170+
{filter.isGenerating ? (
171+
<ActivityIndicator color="#fff" />
172+
) : (
173+
<Text style={styles.runButtonText}>
174+
Detect PII
175+
{inferenceMs !== null && ` · ${inferenceMs} ms`}
176+
</Text>
177+
)}
178+
</TouchableOpacity>
179+
180+
{runError && (
181+
<View style={styles.errorBanner}>
182+
<Text style={styles.errorText}>Run error: {runError}</Text>
183+
</View>
184+
)}
185+
</View>
186+
);
187+
}
188+
189+
export default function PrivacyFilterScreenWrapper() {
190+
const isFocused = useIsFocused();
191+
return isFocused ? <PrivacyFilterScreen /> : null;
192+
}
193+
194+
const styles = StyleSheet.create({
195+
container: {
196+
flex: 1,
197+
padding: 16,
198+
backgroundColor: '#fff',
199+
gap: 10,
200+
},
201+
textBox: {
202+
flex: 1,
203+
borderWidth: 1,
204+
borderColor: '#e0e0e0',
205+
borderRadius: 8,
206+
padding: 10,
207+
},
208+
sampleText: {
209+
fontSize: 13,
210+
color: '#222',
211+
lineHeight: 19,
212+
},
213+
highlight: {
214+
fontWeight: '600',
215+
borderRadius: 3,
216+
},
217+
runButton: {
218+
backgroundColor: ColorPalette.primary,
219+
borderRadius: 8,
220+
paddingVertical: 12,
221+
alignItems: 'center',
222+
},
223+
runButtonText: {
224+
color: '#fff',
225+
fontSize: 15,
226+
fontWeight: '600',
227+
},
228+
buttonDisabled: {
229+
opacity: 0.5,
230+
},
231+
centerBlock: {
232+
alignItems: 'center',
233+
gap: 6,
234+
paddingVertical: 8,
235+
},
236+
muted: {
237+
color: '#666',
238+
fontSize: 12,
239+
},
240+
errorBanner: {
241+
backgroundColor: '#fdecea',
242+
borderColor: '#f5c6cb',
243+
borderWidth: 1,
244+
borderRadius: 6,
245+
padding: 8,
246+
},
247+
errorText: {
248+
color: '#a94442',
249+
fontSize: 12,
250+
},
251+
});

0 commit comments

Comments
 (0)