Skip to content

Commit 180a40d

Browse files
authored
docs: update API reference with getPromptTokenCount and getTotalTokenCount (#789)
## Description <!-- Provide a concise and descriptive summary of the changes implemented in this PR. --> ### Introduces a breaking change? - [ ] Yes - [ ] No ### Type of change - [ ] Bug fix (change which fixes an issue) - [ ] New feature (change which adds functionality) - [ ] Documentation update (improves or adds clarity to existing documentation) - [ ] Other (chores, tests, code style improvements etc.) ### Tested on - [ ] iOS - [ ] Android ### Testing instructions <!-- Provide step-by-step instructions on how to test your changes. Include setup details if necessary. --> ### Screenshots <!-- Add screenshots here, if applicable --> ### Related issues <!-- Link related issues here using #issue-number --> ### Checklist - [ ] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have updated the documentation accordingly - [ ] My changes generate no new warnings ### Additional notes <!-- Include any additional information, assumptions, or context that reviewers might need to understand this PR. -->
1 parent 345f048 commit 180a40d

File tree

2 files changed

+90
-26
lines changed

2 files changed

+90
-26
lines changed

docs/docs/06-api-reference/classes/LLMModule.md

Lines changed: 44 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Class: LLMModule
22

3-
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:10](https://github.com/software-mansion/react-native-executorch/blob/4ee3121e1a18c982703726f1f72421920ed523a4/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L10)
3+
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:10](https://github.com/software-mansion/react-native-executorch/blob/345f048951557e9c3ca349383a9fe6e94974f3df/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L10)
44

55
Module for managing a Large Language Model (LLM) instance.
66

@@ -10,7 +10,7 @@ Module for managing a Large Language Model (LLM) instance.
1010

1111
> **new LLMModule**(`optionalCallbacks`): `LLMModule`
1212
13-
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:19](https://github.com/software-mansion/react-native-executorch/blob/4ee3121e1a18c982703726f1f72421920ed523a4/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L19)
13+
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:19](https://github.com/software-mansion/react-native-executorch/blob/345f048951557e9c3ca349383a9fe6e94974f3df/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L19)
1414

1515
Creates a new instance of `LLMModule` with optional callbacks.
1616

@@ -45,7 +45,7 @@ A new LLMModule instance.
4545

4646
> **configure**(`configuration`): `void`
4747
48-
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:81](https://github.com/software-mansion/react-native-executorch/blob/4ee3121e1a18c982703726f1f72421920ed523a4/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L81)
48+
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:81](https://github.com/software-mansion/react-native-executorch/blob/345f048951557e9c3ca349383a9fe6e94974f3df/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L81)
4949

5050
Configures chat and tool calling and generation settings.
5151
See [Configuring the model](https://docs.swmansion.com/react-native-executorch/docs/hooks/natural-language-processing/useLLM#configuring-the-model) for details.
@@ -68,7 +68,7 @@ Configuration object containing `chatConfig`, `toolsConfig`, and `generationConf
6868

6969
> **delete**(): `void`
7070
71-
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:156](https://github.com/software-mansion/react-native-executorch/blob/4ee3121e1a18c982703726f1f72421920ed523a4/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L156)
71+
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:174](https://github.com/software-mansion/react-native-executorch/blob/345f048951557e9c3ca349383a9fe6e94974f3df/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L174)
7272

7373
Method to delete the model from memory.
7474
Note you cannot delete model while it's generating.
@@ -84,7 +84,7 @@ You need to interrupt it first and make sure model stopped generation.
8484

8585
> **deleteMessage**(`index`): [`Message`](../interfaces/Message.md)[]
8686
87-
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:130](https://github.com/software-mansion/react-native-executorch/blob/4ee3121e1a18c982703726f1f72421920ed523a4/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L130)
87+
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:130](https://github.com/software-mansion/react-native-executorch/blob/345f048951557e9c3ca349383a9fe6e94974f3df/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L130)
8888

8989
Deletes all messages starting with message on `index` position.
9090
After deletion it will call `messageHistoryCallback()` containing new history.
@@ -110,7 +110,7 @@ The index of the message to delete from history.
110110

111111
> **forward**(`input`): `Promise`\<`string`\>
112112
113-
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:94](https://github.com/software-mansion/react-native-executorch/blob/4ee3121e1a18c982703726f1f72421920ed523a4/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L94)
113+
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:94](https://github.com/software-mansion/react-native-executorch/blob/345f048951557e9c3ca349383a9fe6e94974f3df/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L94)
114114

115115
Runs model inference with raw input string.
116116
You need to provide entire conversation and prompt (in correct format and with special tokens!) in input string to this method.
@@ -137,7 +137,7 @@ The generated response as a string.
137137

138138
> **generate**(`messages`, `tools?`): `Promise`\<`string`\>
139139
140-
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:105](https://github.com/software-mansion/react-native-executorch/blob/4ee3121e1a18c982703726f1f72421920ed523a4/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L105)
140+
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:105](https://github.com/software-mansion/react-native-executorch/blob/345f048951557e9c3ca349383a9fe6e94974f3df/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L105)
141141

142142
Runs model to complete chat passed in `messages` argument. It doesn't manage conversation context.
143143

@@ -167,7 +167,7 @@ The generated response as a string.
167167

168168
> **getGeneratedTokenCount**(): `number`
169169
170-
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:147](https://github.com/software-mansion/react-native-executorch/blob/4ee3121e1a18c982703726f1f72421920ed523a4/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L147)
170+
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:147](https://github.com/software-mansion/react-native-executorch/blob/345f048951557e9c3ca349383a9fe6e94974f3df/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L147)
171171

172172
Returns the number of tokens generated in the last response.
173173

@@ -179,11 +179,43 @@ The count of generated tokens.
179179

180180
---
181181

182+
### getPromptTokensCount()
183+
184+
> **getPromptTokensCount**(): `number`
185+
186+
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:156](https://github.com/software-mansion/react-native-executorch/blob/345f048951557e9c3ca349383a9fe6e94974f3df/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L156)
187+
188+
Returns the number of prompt tokens in the last message.
189+
190+
#### Returns
191+
192+
`number`
193+
194+
The count of prompt token.
195+
196+
---
197+
198+
### getTotalTokensCount()
199+
200+
> **getTotalTokensCount**(): `number`
201+
202+
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:165](https://github.com/software-mansion/react-native-executorch/blob/345f048951557e9c3ca349383a9fe6e94974f3df/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L165)
203+
204+
Returns the number of total tokens from the previous generation. This is a sum of prompt tokens and generated tokens.
205+
206+
#### Returns
207+
208+
`number`
209+
210+
The count of prompt and generated tokens.
211+
212+
---
213+
182214
### interrupt()
183215

184216
> **interrupt**(): `void`
185217
186-
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:138](https://github.com/software-mansion/react-native-executorch/blob/4ee3121e1a18c982703726f1f72421920ed523a4/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L138)
218+
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:138](https://github.com/software-mansion/react-native-executorch/blob/345f048951557e9c3ca349383a9fe6e94974f3df/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L138)
187219

188220
Interrupts model generation. It may return one more token after interrupt.
189221

@@ -197,7 +229,7 @@ Interrupts model generation. It may return one more token after interrupt.
197229

198230
> **load**(`model`, `onDownloadProgressCallback`): `Promise`\<`void`\>
199231
200-
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:48](https://github.com/software-mansion/react-native-executorch/blob/4ee3121e1a18c982703726f1f72421920ed523a4/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L48)
232+
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:48](https://github.com/software-mansion/react-native-executorch/blob/345f048951557e9c3ca349383a9fe6e94974f3df/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L48)
201233

202234
Loads the LLM model and tokenizer.
203235

@@ -241,7 +273,7 @@ Optional callback to track download progress (value between 0 and 1).
241273

242274
> **sendMessage**(`message`): `Promise`\<[`Message`](../interfaces/Message.md)[]\>
243275
244-
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:117](https://github.com/software-mansion/react-native-executorch/blob/4ee3121e1a18c982703726f1f72421920ed523a4/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L117)
276+
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:117](https://github.com/software-mansion/react-native-executorch/blob/345f048951557e9c3ca349383a9fe6e94974f3df/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L117)
245277

246278
Method to add user message to conversation.
247279
After model responds it will call `messageHistoryCallback()` containing both user message and model response.
@@ -267,7 +299,7 @@ The message string to send.
267299

268300
> **setTokenCallback**(`tokenCallback`): `void`
269301
270-
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:67](https://github.com/software-mansion/react-native-executorch/blob/4ee3121e1a18c982703726f1f72421920ed523a4/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L67)
302+
Defined in: [packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts:67](https://github.com/software-mansion/react-native-executorch/blob/345f048951557e9c3ca349383a9fe6e94974f3df/packages/react-native-executorch/src/modules/natural_language_processing/LLMModule.ts#L67)
271303

272304
Sets new token callback invoked on every token batch.
273305

0 commit comments

Comments
 (0)