Commit e8d4305
fix(llm): normalize multimodal image paths to file:// URIs (#1090)
## Description
`LLMController.forward` passed `imagePaths` straight through to
`nativeModule.generateMultimodal` with no normalization. The native side
requires the `file://` prefix; without it, native throws `"Read image
error: invalid argument"` with no further context. Callers can plausibly
arrive with either form:
- `ResourceFetcher.fetch` returns raw paths *without* `file://` (per its
own docstring on the `fetch` method).
- Platform image-picker APIs (e.g. `expo-image-picker`) typically return
`file:///...` URIs.
- The same path string passed to a vision module's `forward(...)` works
either way; the asymmetry between vision modules and multimodal LLM is
undocumented.
This PR normalizes each image path inside `LLMController.forward` so
both forms work, and updates the JSDoc on `Message.mediaPath` and
`LLMModule.forward.imagePaths` to document the new contract.
### Introduces a breaking change?
- [ ] Yes
- [x] No
Strictly additive: previously-working calls (paths with `file://`) keep
working unchanged. Previously-failing calls (paths without `file://`)
now succeed.
### Type of change
- [x] Bug fix (change which fixes an issue)
- [ ] New feature
- [ ] Documentation update
- [ ] Other
### Tested on
- [ ] iOS
- [ ] Android
The bare-path failure was reproduced on Android (Samsung Galaxy S24
Ultra) with LFM2-VL-1.6B while building a downstream consumer; both
forms tested manually post-fix on the same device. Re-verification of
both forms on iOS is recommended.
### Testing instructions
```ts
import { LLMModule, LFM2_VL_1_6B_QUANTIZED } from 'react-native-executorch';
const llm = await LLMModule.fromModelName(LFM2_VL_1_6B_QUANTIZED);
// Both should now work; previously only the first did.
await llm.generate([
{ role: 'user', content: 'Describe.', mediaPath: 'file:///absolute/path/to/img.jpg' },
]);
await llm.generate([
{ role: 'user', content: 'Describe.', mediaPath: '/absolute/path/to/img.jpg' },
]);
```
### Related issues
Addresses item 3 of #1086.
### Checklist
- [ ] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have updated the documentation accordingly
- [ ] My changes generate no new warnings
### Additional notes
The normalizer is module-scope (matching `messagesForChatTemplate` from
#1089) rather than a class method because it doesn't depend on
controller state.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent da22171 commit e8d4305
3 files changed
Lines changed: 16 additions & 2 deletions
File tree
- packages/react-native-executorch/src
- controllers
- modules/natural_language_processing
- types
Lines changed: 13 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
254 | 254 | | |
255 | 255 | | |
256 | 256 | | |
257 | | - | |
| 257 | + | |
258 | 258 | | |
259 | 259 | | |
260 | 260 | | |
| |||
456 | 456 | | |
457 | 457 | | |
458 | 458 | | |
| 459 | + | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
139 | 139 | | |
140 | 140 | | |
141 | 141 | | |
142 | | - | |
| 142 | + | |
143 | 143 | | |
144 | 144 | | |
145 | 145 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
270 | 270 | | |
271 | 271 | | |
272 | 272 | | |
| 273 | + | |
| 274 | + | |
273 | 275 | | |
274 | 276 | | |
275 | 277 | | |
| |||
0 commit comments