[Feature] Add Anthropic and Google (Gemini) response format support to requestcostmetadata extractor#234
Merged
github-actions[bot] merged 1 commit intoJul 15, 2026
Conversation
github-actions
Bot
added
kind/feature
abdallahsamabd
marked this pull request as draft
July 12, 2026 14:40
abdallahsamabd
marked this pull request as ready for review
July 13, 2026 10:16
abdallahsamabd
force-pushed
the
feature/issue-230-multi-format-extractor
branch
from
July 13, 2026 10:16
d2259a9 to
1fbae59
Compare
github-actions
Bot
added
size/XL
abdallahsamabd
force-pushed
the
feature/issue-230-multi-format-extractor
branch
from
July 13, 2026 10:18
1fbae59 to
4aae3db
Compare
github-actions
Bot
added
size/L
abdallahsamabd
force-pushed
the
feature/issue-230-multi-format-extractor
branch
from
July 13, 2026 10:21
4aae3db to
8491ce2
Compare
Contributor
Author
davidbreitgand
left a comment
davidbreitgand
left a comment
Contributor
There was a problem hiding this comment.
@abdallahsamabd : Nice! Thanks for this PR.
| } | ||
|
|
||
| func extractOpenAI(body map[string]any) (prompt, completion float64, ok bool) { | ||
| usage := body["usage"].(map[string]any) |
Contributor
There was a problem hiding this comment.
Nit: potentially confusing usage of unchecked type assertion
Suggested change
| usage := body["usage"].(map[string]any) | |
| // Safe: detectFormat() verified "usage" key exists and is map[string]any | |
| usage := body["usage"].(map[string]any) |
Contributor
There was a problem hiding this comment.
@abdallahsamabd , actually, thinking more about this... I'm not sure why you need this type assertion at all. detectFormat() already validated the structure, the extract functions seem to re-validate the exact same structure. No mutation occurs between detectFormat() and extractOpenAI()/extractAnthropic()/extractGoogle() calls. Why do we need this additional assertion?
The same question for Lines 290 and 303.
| } | ||
|
|
||
| func extractAnthropic(body map[string]any) (prompt, completion float64, ok bool) { | ||
| usage := body["usage"].(map[string]any) |
| } | ||
|
|
||
| func extractGoogle(body map[string]any) (prompt, completion float64, ok bool) { | ||
| usageMetadata := body["usageMetadata"].(map[string]any) |
Contributor
Author
There was a problem hiding this comment.
I refactored to inline the detection and pass the validated sub-map directly to each extractor, eliminating both the redundant assertions and the separate detectFormat function
…metadata extractor The extractTokenCounts function now auto-detects the response format by inspecting the response body structure and dispatches to format-specific extraction helpers for OpenAI, Anthropic, and Google/Gemini. This enables CostGuard to observe actual inference costs from all major LLM providers without any user configuration. Fixes llm-d#230 Signed-off-by: Abdallah Samara <abdallahsamabd@gmail.com>
abdallahsamabd
force-pushed
the
feature/issue-230-multi-format-extractor
branch
from
July 14, 2026 21:48
8491ce2 to
e229a5b
Compare
Collaborator
|
/lgtm |
github-actions
Bot
added
the
lgtm
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
/kind feature
/kind test
What this PR does / why we need it:
The
requestcostmetadataextractor currently only parses OpenAI-formatted responses (usage.prompt_tokens/usage.completion_tokens). This PR adds auto-detection and extraction support for Anthropic (usage.input_tokens/usage.output_tokens) and Google/Gemini (usageMetadata.promptTokenCount/usageMetadata.candidatesTokenCount) response formats.The extractor inspects the response body structure to determine the format — no user configuration needed. This ensures CostGuard can observe actual inference costs from all major LLM providers.
Which issue(s) this PR fixes:
Fixes #230
Release note (write
NONEif no user-facing change):