(Openinference Migration: Langchain) - Add support for cache and reasoning token counts (PR #1)#272
Open
rads-1996 wants to merge 11 commits into
Open
(Openinference Migration: Langchain) - Add support for cache and reasoning token counts (PR #1)#272rads-1996 wants to merge 11 commits into
rads-1996 wants to merge 11 commits into
Conversation
rads-1996
force-pushed
the
pr-token-counts
branch
2 times, most recently
from
July 16, 2026 22:11
fb0a489 to
1e5b733
Compare
rads-1996
force-pushed
the
pr-token-counts
branch
from
July 16, 2026 22:12
1e5b733 to
6007a22
Compare
rads-1996
marked this pull request as ready for review
July 16, 2026 22:16
Contributor
There was a problem hiding this comment.
Pull request overview
Adds support for LangChain-provided cache token counts and reasoning (“thinking”) token counts, ensuring util-genai’s token metrics treat reasoning tokens as part of overall output token usage.
Changes:
- Update util-genai inference token metric accounting to include
thinking_tokensin OUTPUT token metrics. - Extract cache/reasoning token breakdowns from LangChain
usage_metadataand apply them to the inference invocation (including adjustingoutput_tokensso util-genai can re-sum totals correctly). - Add unit tests covering token breakdown extraction and
on_llm_endbehavior for cache/reasoning and ignored audio tokens.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| util/opentelemetry-util-genai/src/opentelemetry/util/genai/_inference_invocation.py | Includes thinking_tokens in OUTPUT token metrics to match span attribute behavior. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/src/opentelemetry/instrumentation/genai/langchain/callback_handler.py | Parses LangChain usage metadata into cache/reasoning fields and adjusts output token handling. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/src/opentelemetry/instrumentation/genai/langchain/utils.py | Adds helper to extract cache/reasoning token details from LangChain metadata. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/tests/test_callback_handler.py | Adds tests for new token breakdown behavior and helper extraction. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/.changelog/272.added | Adds a towncrier fragment for the new feature. |
Pull request dashboard status
|
11 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
cache_creation_input_tokens, cache_read_input_tokens, and thinking_tokensfrom LangChain usage_metadata.additional_kwargs.function_call shapeinto a ToolCallRequest on both the input path (_ai_message_parts) and the output path (on_llm_end), matching the modern tool_calls handling.Type of change
Please delete options that are not relevant.
How has this been tested?
Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. List any relevant details for your test
configuration.
Checklist
See CONTRIBUTING.md
for the style guide, changelog guidance, and more.