Skip to content

Commit 89bcf67

Browse files
chmjkbchmjkb
authored
fix: return accumulated response from native to TS (#770)
1 parent aea37ea commit 89bcf67

8 files changed

Lines changed: 86 additions & 122 deletions

File tree

‎docs/docs/03-hooks/01-natural-language-processing/useLLM.md‎

Lines changed: 22 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -60,21 +60,21 @@ For more information on loading resources, take a look at [loading models](../..
6060

6161
### Returns
6262

63-
| Field | Type | Description |
64-
| ------------------------ | -------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
65-
| `generate()` | `(messages: Message[], tools?: LLMTool[]) => Promise<void>` | Runs model to complete chat passed in `messages` argument. It doesn't manage conversation context. |
66-
| `interrupt()` | `() => void` | Function to interrupt the current inference. |
67-
| `response` | `string` | State of the generated response. This field is updated with each token generated by the model. |
68-
| `token` | `string` | The most recently generated token. |
69-
| `isReady` | `boolean` | Indicates whether the model is ready. |
70-
| `isGenerating` | `boolean` | Indicates whether the model is currently generating a response. |
71-
| `downloadProgress` | `number` | Represents the download progress as a value between 0 and 1, indicating the extent of the model file retrieval. |
72-
| `error` | <code>string &#124; null</code> | Contains the error message if the model failed to load. |
73-
| `configure` | `({chatConfig?: Partial<ChatConfig>, toolsConfig?: ToolsConfig, generationConfig?: GenerationConfig}) => void` | Configures chat and tool calling. See more details in [configuring the model](#configuring-the-model). |
74-
| `sendMessage` | `(message: string) => Promise<void>` | Function to add user message to conversation. After model responds, `messageHistory` will be updated with both user message and model response. |
75-
| `deleteMessage` | `(index: number) => void` | Deletes all messages starting with message on `index` position. After deletion `messageHistory` will be updated. |
76-
| `messageHistory` | `Message[]` | History containing all messages in conversation. This field is updated after model responds to `sendMessage`. |
77-
| `getGeneratedTokenCount` | `() => number` | Returns the number of tokens generated in the last response. |
63+
| Field | Type | Description |
64+
| ------------------------ | -------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
65+
| `generate()` | `(messages: Message[], tools?: LLMTool[]) => Promise<string>` | Runs model to complete chat passed in `messages` argument. Returns the generated response. It doesn't manage conversation context. |
66+
| `interrupt()` | `() => void` | Function to interrupt the current inference. |
67+
| `response` | `string` | State of the generated response. This field is updated with each token generated by the model. |
68+
| `token` | `string` | The most recently generated token. |
69+
| `isReady` | `boolean` | Indicates whether the model is ready. |
70+
| `isGenerating` | `boolean` | Indicates whether the model is currently generating a response. |
71+
| `downloadProgress` | `number` | Represents the download progress as a value between 0 and 1, indicating the extent of the model file retrieval. |
72+
| `error` | <code>string &#124; null</code> | Contains the error message if the model failed to load. |
73+
| `configure` | `({chatConfig?: Partial<ChatConfig>, toolsConfig?: ToolsConfig, generationConfig?: GenerationConfig}) => void` | Configures chat and tool calling. See more details in [configuring the model](#configuring-the-model). |
74+
| `sendMessage` | `(message: string) => Promise<string>` | Function to add user message to conversation. Returns the generated response. After model responds, `messageHistory` will be updated with both user message and model response. |
75+
| `deleteMessage` | `(index: number) => void` | Deletes all messages starting with message on `index` position. After deletion `messageHistory` will be updated. |
76+
| `messageHistory` | `Message[]` | History containing all messages in conversation. This field is updated after model responds to `sendMessage`. |
77+
| `getGeneratedTokenCount` | `() => number` | Returns the number of tokens generated in the last response. |
7878

7979
<details>
8080
<summary>Type definitions</summary>
@@ -110,8 +110,8 @@ interface LLMType {
110110
generationConfig?: GenerationConfig;
111111
}) => void;
112112
getGeneratedTokenCount: () => number;
113-
generate: (messages: Message[], tools?: LLMTool[]) => Promise<void>;
114-
sendMessage: (message: string) => Promise<void>;
113+
generate: (messages: Message[], tools?: LLMTool[]) => Promise<string>;
114+
sendMessage: (message: string) => Promise<string>;
115115
deleteMessage: (index: number) => void;
116116
interrupt: () => void;
117117
}
@@ -166,21 +166,22 @@ You can use functions returned from this hooks in two manners:
166166

167167
### Simple generation
168168

169-
To perform chat completion you can use the `generate` function. There is no return value. Instead, the `response` value is updated with each token.
169+
To perform chat completion you can use the `generate` function. The `response` value is updated with each token as it's generated, and the function returns a promise that resolves to the complete response when generation finishes.
170170

171171
```tsx
172172
const llm = useLLM({ model: LLAMA3_2_1B });
173173

174-
const handleGenerate = () => {
174+
const handleGenerate = async () => {
175175
const chat: Message[] = [
176176
{ role: 'system', content: 'You are a helpful assistant' },
177177
{ role: 'user', content: 'Hi!' },
178178
{ role: 'assistant', content: 'Hi!, how can I help you?' },
179179
{ role: 'user', content: 'What is the meaning of life?' },
180180
];
181181

182-
// Chat completion
183-
llm.generate(chat);
182+
// Chat completion - returns the generated response
183+
const response = await llm.generate(chat);
184+
console.log('Complete response:', response);
184185
};
185186

186187
return (

0 commit comments

Comments
 (0)