Skip to content

Add usage to final chunk for deepinfra#1041

Merged
VisargD merged 2 commits into
Portkey-AI:mainfrom
francescov1:add-deepinfra-usage-to-streaming
Apr 22, 2025
Merged

Add usage to final chunk for deepinfra#1041
VisargD merged 2 commits into
Portkey-AI:mainfrom
francescov1:add-deepinfra-usage-to-streaming

Conversation

@francescov1
Copy link
Copy Markdown
Contributor

@francescov1 francescov1 commented Apr 15, 2025

Code Quality type: new feature

Author Description

(optional)

  • Detailed change 1
  • Detailed change 2

Title: Add usage to final chunk for deepinfra

🔄 What Changed

  • Added a usage field to the DeepInfraStreamChunk interface
  • Modified the DeepInfraChatCompleteStreamChunkTransform function to include usage data in the transformed output when available

🔍 Impact of the Change

  • Enables clients to receive token usage statistics in the final streaming chunk from DeepInfra API calls
  • Provides valuable information for monitoring, billing, and optimizing token usage in applications

📁 Total Files Changed

  • 1 file modified: src/providers/deepinfra/chatComplete.ts (+12 lines, -0 lines)

🧪 Test Added

  • N/A

🔒 Security Vulnerabilities

  • No security vulnerabilities identified in this change

Related Issues: N/A

Quality Recommendations

  1. Add unit tests to verify the correct handling of usage information in streaming responses

  2. Consider adding documentation comments to explain the purpose of the usage field and how it's used

  3. Add error handling for potential malformed usage data from the API

finish_reason: parsedChunk.choices[0].finish_reason,
},
],
usage: parsedChunk.usage
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this could simply be parsedChunk.usage imo

@VisargD VisargD merged commit dbcb0f1 into Portkey-AI:main Apr 22, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants