Enhance LangChain GenAI semconv tracing#4389
Open
nagkumar91 wants to merge 4 commits intoopen-telemetry:mainfrom
Open
Enhance LangChain GenAI semconv tracing#4389nagkumar91 wants to merge 4 commits intoopen-telemetry:mainfrom
nagkumar91 wants to merge 4 commits intoopen-telemetry:mainfrom
Conversation
Add on_llm_new_token callback method to track streaming chunk timing: - Records time_to_first_chunk (TTFC) on first token arrival - Records time_per_output_chunk (TPOC) between subsequent tokens - Uses timeit.default_timer() consistent with TelemetryHandler timing - Builds metric attributes matching InvocationMetricsRecorder pattern - Cleans up streaming state in on_llm_end and on_llm_error Creates two new histograms using semconv metric names: - gen_ai.client.operation.time_to_first_chunk - gen_ai.client.operation.time_per_output_chunk Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Major enhancement to the LangChain instrumentor bringing it to parity with the GenAI semantic conventions for spans, metrics, and events. New callback handlers: - on_chain_start/end/error: invoke_agent and invoke_workflow spans - on_tool_start/end/error: execute_tool spans with tool attributes - on_retriever_start/end/error: execute_tool spans with retriever semantics - on_llm_start: text_completion spans for non-chat models - on_llm_new_token: streaming timing metrics (TTFC, TPOC) - on_agent_action/finish: agent lifecycle support New infrastructure: - operation_mapping.py: safe callback-to-semconv classification - semconv_attributes.py: per-operation attribute matrix - span_manager.py: full span lifecycle with hierarchy tracking - utils.py: provider detection, server address, W3C propagation - message_formatting.py: structured message serialization with redaction - content_recording.py: content capture policy integration - event_emitter.py: semconv-aligned events for non-LLM spans Key capabilities: - Removed ChatOpenAI/ChatBedrock restriction, supports any LLM - Full request/response attribute extraction per semconv - Parent-child hierarchy with LangGraph support (filtering, goto, stacks) - Token usage accumulation from LLM to agent spans - W3C trace context propagation (incoming traceparent/tracestate) - Cache token attributes (cache_read, cache_creation) - Streaming metrics (time_to_first_chunk, time_per_output_chunk) - 276 unit and e2e tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Apply the follow-up LangChain review fixes, wire non-LLM event emission, correct experimental content capture behavior, clean ignored-run state, and repair the package's VCR-based integration tests under the current pytest-recording setup. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Member
|
Thanks for the PR @nagkumar91. At over 7k lines of code, this is nearly impossible to approve with confidence. Could you break this up in to smaller chunks? For example, your PR desc breaks down the changes into 4 distint areas, I'd suggest trying to use those as areas to focus on per PR. |
MikeGoldsmith
requested changes
Apr 8, 2026
Member
MikeGoldsmith
left a comment
MikeGoldsmith
left a comment
There was a problem hiding this comment.
Forgot to add a review yesterday. I'd really like to see this PR broken down into smaller parts.
github-project-automation
bot
moved this from Ready for review
to Reviewed PRs that need fixes
in Python PR digest
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR expands the LangChain instrumentor toward semconv-first GenAI coverage and folds in the review/test follow-up fixes needed to make the package consistent with contrib expectations.
The main changes are:
pytest-recordingType of change
How Has This Been Tested?
python3 -m pytest instrumentation-genai/opentelemetry-instrumentation-langchain/testsuv tool run ruff check src tests(frominstrumentation-genai/opentelemetry-instrumentation-langchain)tox -e typechecktox -e spellcheckDoes This PR Require a Core Repo Change?
Checklist:
Notes
instrumentation-genai/opentelemetry-instrumentation-langchain/CHANGELOG.md.test_geminiis skipped whentests/cassettes/test_gemini.yamlis absent and no real Google API key is available for recording.