feat: enhance LiteLLM metadata extraction by adding cost retrieval from response headers#567
Merged
gustavocidornelas merged 1 commit intomainfrom Dec 23, 2025
Conversation
…om response headers
gustavocidornelas
approved these changes
Dec 23, 2025
gustavocidornelas
deleted the
vini/open-8556-bug-litellm-tracer-isnt-capturing-the-cost
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
Summary
Adds a fallback mechanism to capture cost from LiteLLM response headers when
response_costis zero or unavailable.Changes
extract_litellm_metadatato track response headers and check for cost fallback_extract_cost_from_headershelper function to parsex-litellm-response-costandx-litellm-response-cost-originalheadersContext
In some cases, the
response_costfrom LiteLLM's hidden params is returned as0, but the actual cost is available in the response headers underx-litellm-response-cost. This fix ensures we capture cost data from headers as a fallback when the primary source fails.Applies to both streaming and non-streaming completions.
Testing