Skip to content

Commit 6831fe4

Browse files
julien-cjulien-agent
andauthored
docs: document usage object in server timings response (ggml-org#23110)
* docs: document `usage` object in server timings response Co-Authored-By: julien-agent <Agents+cyolo@huggingface.co> * Apply suggestion from @julien-c --------- Co-authored-by: julien-agent <Agents+cyolo@huggingface.co>
1 parent 72e60f5 commit 6831fe4

1 file changed

Lines changed: 16 additions & 0 deletions

File tree

tools/server/README.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1322,6 +1322,22 @@ This provides information on the performance of the server. It also allows calcu
13221322

13231323
The total number of tokens in context is equal to `prompt_n + cache_n + predicted_n`
13241324

1325+
The response also includes a standard `usage` object:
1326+
1327+
```js
1328+
{
1329+
// ...
1330+
"usage": {
1331+
"completion_tokens": 48,
1332+
"prompt_tokens": 44,
1333+
"total_tokens": 92,
1334+
"prompt_tokens_details": {
1335+
"cached_tokens": 0
1336+
}
1337+
}
1338+
}
1339+
```
1340+
13251341
*Reasoning support*
13261342

13271343
The server supports parsing and returning reasoning via the `reasoning_content` field, similar to Deepseek API.

0 commit comments

Comments
 (0)