Model: Fix usage stats in non-streaming gens

kingbri1 · kingbri1 · commit e4bb70930594 · 2024-05-12T22:44:50.000-04:00
The wrong key was being returned from the model to the API.

Signed-off-by: kingbri &lt;bdashore3@proton.me&gt;
diff --git a/backends/exllamav2/model.py b/backends/exllamav2/model.py
@@ -656,7 +656,7 @@ async def generate(self, prompt: str, **kwargs):
             joined_generation["prompt_tokens"] = unwrap(
                 generations[-1].get("prompt_tokens"), 0
             )
-            joined_generation["generation_tokens"] = unwrap(
+            joined_generation["generated_tokens"] = unwrap(
                 generations[-1].get("generated_tokens"), 0
             )
 

Original file line number	Diff line number	Diff line change
`@@ -656,7 +656,7 @@ async def generate(self, prompt: str, **kwargs):`
`656`	`656`	`joined_generation["prompt_tokens"] = unwrap(`
`657`	`657`	`generations[-1].get("prompt_tokens"), 0`
`658`	`658`	`)`
`659`		`- joined_generation["generation_tokens"] = unwrap(`
	`659`	`+ joined_generation["generated_tokens"] = unwrap(`
`660`	`660`	`generations[-1].get("generated_tokens"), 0`
`661`	`661`	`)`
`662`	`662`