Skip to content

Commit 7b452e6

Browse files
authored
fix(llm): surface tool-call errors via sendMessage (#1169)
## Description `useLLM` with `toolsConfig` silently dropped errors from the tool-execution path: `parseToolCall` swallowed every parse failure with `Logger.error`, and `LLMController.sendMessage` fired `executeToolCallback` without `await` or `.catch`. Neither `llm.error` nor the caller's `await llm.sendMessage(...)` `try/catch` ever saw them. This PR makes those errors propagate: - `parseToolCall` now distinguishes "no tool call in output" (returns `[]` — the model legitimately chose not to call a tool) from "malformed tool-call JSON" (throws `RnExecutorchError(InvalidModelOutput)`). - `LLMController.sendMessage` `await`s `executeToolCallback` inside `try/catch` and rethrows via `parseUnknownError`, so failures reach the caller's `try/catch` (and the hook's `error` state if wired). ### Introduces a breaking change? - [x] Yes - [ ] No Callers of `parseToolCall` that previously received `[]` for malformed tool-call JSON now get a thrown `RnExecutorchError(InvalidModelOutput)`. Same applies to `sendMessage` when `toolsConfig` is set and the tool path throws — the error now surfaces instead of being silently swallowed. ### Type of change - [x] Bug fix (change which fixes an issue) - [ ] New feature (change which adds functionality) - [ ] Documentation update (improves or adds clarity to existing documentation) - [ ] Other (chores, tests, code style improvements etc.) ### Tested on - [x] iOS - [ ] Android ### Testing instructions 1. Open the `llm` demo app → **Tool calling** screen. 2. Switch the picker to a non-tool-trained model (e.g. Llama 3.2 1B QLoRA / SpinQuant). 3. Send a tool-shaped prompt (`What events do I have today?`). 4. If the tool-execution path throws (e.g. "max recursion depth reached" on tool-naive models, malformed tool-call JSON), the on-screen `ErrorBanner` now displays the error instead of staying empty. 5. Sanity check on a tool-trained model (Hammer 2.1 1.5B): tool calls still execute normally and no error banner appears. ### Related issues Fixes #1157 ### Checklist - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have updated the documentation accordingly - [x] My changes generate no new warnings
1 parent 00422ef commit 7b452e6

2 files changed

Lines changed: 51 additions & 38 deletions

File tree

packages/react-native-executorch/src/controllers/LLMController.ts

Lines changed: 14 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -416,18 +416,20 @@ export class LLMController {
416416
}
417417

418418
if (this.toolsConfig) {
419-
const toolCalls = parseToolCall(response);
420-
for (const toolCall of toolCalls) {
421-
this.toolsConfig
422-
.executeToolCallback(toolCall)
423-
.then((toolResponse: string | null) => {
424-
if (toolResponse) {
425-
this.messageHistoryCallback([
426-
...this._messageHistory,
427-
{ content: toolResponse, role: 'assistant' },
428-
]);
429-
}
430-
});
419+
try {
420+
const toolCalls = parseToolCall(response);
421+
for (const toolCall of toolCalls) {
422+
const toolResponse =
423+
await this.toolsConfig.executeToolCallback(toolCall);
424+
if (toolResponse != null) {
425+
this.messageHistoryCallback([
426+
...this._messageHistory,
427+
{ content: toolResponse, role: 'assistant' },
428+
]);
429+
}
430+
}
431+
} catch (e) {
432+
throw parseUnknownError(e);
431433
}
432434
}
433435

packages/react-native-executorch/src/utils/llm.ts

Lines changed: 37 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -4,45 +4,56 @@ import { Schema, Validator } from 'jsonschema';
44
import { jsonrepair } from 'jsonrepair';
55
import { DEFAULT_STRUCTURED_OUTPUT_PROMPT } from '../constants/llmDefaults';
66
import * as zCore from 'zod/v4/core';
7-
import { Logger } from '../common/Logger';
7+
import { RnExecutorchError } from '../errors/errorUtils';
8+
import { RnExecutorchErrorCode } from '../errors/ErrorCodes';
89

910
/**
1011
* Parses tool calls from a given message string.
1112
* @category Utilities - LLM
1213
* @param message - The message string containing tool calls in JSON format.
13-
* @returns An array of `ToolCall` objects extracted from the message.
14+
* @returns An array of `ToolCall` objects extracted from the message. Returns
15+
* an empty array if the model did not emit a `[...]` block (i.e. chose not
16+
* to call any tool).
17+
* @throws {RnExecutorchError} `InvalidModelOutput` when a `[...]` block is
18+
* present but cannot be parsed as JSON — distinct from the model
19+
* legitimately deciding not to invoke a tool.
1420
*/
1521
export const parseToolCall: (message: string) => ToolCall[] = (
1622
message: string
1723
) => {
24+
const unparsedToolCalls = message.match('\\[(.|\\s)*\\]');
25+
if (!unparsedToolCalls) {
26+
return [];
27+
}
28+
29+
let parsedMessage: LLMTool[];
1830
try {
19-
const unparsedToolCalls = message.match('\\[(.|\\s)*\\]');
20-
if (!unparsedToolCalls) {
21-
throw Error('Regex did not match array.');
22-
}
23-
const parsedMessage: LLMTool[] = JSON.parse(unparsedToolCalls[0]);
24-
const results = [];
31+
parsedMessage = JSON.parse(unparsedToolCalls[0]);
32+
} catch (e) {
33+
throw new RnExecutorchError(
34+
RnExecutorchErrorCode.InvalidModelOutput,
35+
`Failed to parse tool call JSON from model output: ${unparsedToolCalls[0]}`,
36+
e
37+
);
38+
}
2539

26-
for (const tool of parsedMessage) {
27-
if (
28-
'name' in tool &&
29-
typeof tool.name === 'string' &&
30-
'arguments' in tool &&
31-
tool.arguments !== null &&
32-
typeof tool.arguments === 'object'
33-
) {
34-
results.push({
35-
toolName: tool.name,
36-
arguments: tool.arguments,
37-
});
38-
}
40+
const results: ToolCall[] = [];
41+
for (const tool of parsedMessage) {
42+
if (
43+
'name' in tool &&
44+
typeof tool.name === 'string' &&
45+
'arguments' in tool &&
46+
tool.arguments !== null &&
47+
typeof tool.arguments === 'object'
48+
) {
49+
results.push({
50+
toolName: tool.name,
51+
arguments: tool.arguments,
52+
});
3953
}
40-
41-
return results;
42-
} catch (e) {
43-
Logger.error(e);
44-
return [];
4554
}
55+
56+
return results;
4657
};
4758

4859
const filterObjectKeys = (obj: object, keysToRemove: string[]) => {

0 commit comments

Comments
 (0)