Adding support for ChatAnthropic to the langchain instrumentation#4496
Adding support for ChatAnthropic to the langchain instrumentation#4496salaboy wants to merge 2 commits intoopen-telemetry:mainfrom
Conversation
|
This would be my first contribution here, so I would appreciate any guidance from experienced maintainers :) I am happy to change and update stuff as needed. Also, I am a real person not a bot, just in case :) |
| "ChatOpenAI", | ||
| "ChatBedrock", | ||
| "ChatAnthropic", | ||
| ): |
There was a problem hiding this comment.
Why not use constants to define the chat's names?
There was a problem hiding this comment.
Pull request overview
Adds LangChain instrumentation coverage for Anthropic’s ChatAnthropic, extending the existing GenAI LangChain callback handler and accompanying VCR-based tests.
Changes:
- Extend
OpenTelemetryLangChainCallbackHandlerto recognizeChatAnthropicand extract Anthropic-specific invocation parameters (e.g.,model,max_tokens,stop_sequences). - Add a new VCR-backed test + cassette covering a
ChatAnthropicinvoke call and validating span attributes. - Update package docstring to include
ChatAnthropicas a supported chat model.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| instrumentation-genai/opentelemetry-instrumentation-langchain/src/opentelemetry/instrumentation/langchain/callback_handler.py | Adds ChatAnthropic recognition and parameter extraction for proper invocation capture. |
| instrumentation-genai/opentelemetry-instrumentation-langchain/src/opentelemetry/instrumentation/langchain/init.py | Updates module docstring to reflect Anthropic support. |
| instrumentation-genai/opentelemetry-instrumentation-langchain/tests/conftest.py | Adds ChatAnthropic fixture and test env/VCR header filtering for Anthropic API key usage. |
| instrumentation-genai/opentelemetry-instrumentation-langchain/tests/test_llm_call.py | Adds Anthropic chat test and span-attribute assertion helper. |
| instrumentation-genai/opentelemetry-instrumentation-langchain/tests/cassettes/test_chat_anthropic_claude_sonnet_llm_call.yaml | Recorded Anthropic /v1/messages interaction for deterministic testing. |
| assert span.attributes["gen_ai.provider.name"] == "anthropic" | ||
| assert span.attributes[gen_ai_attributes.GEN_AI_REQUEST_MAX_TOKENS] == 100 | ||
| assert span.attributes[gen_ai_attributes.GEN_AI_REQUEST_TEMPERATURE] == 0.1 |
There was a problem hiding this comment.
The provider attribute is accessed via the literal key "gen_ai.provider.name". Prefer using the generated semantic convention constant (gen_ai_attributes.GEN_AI_PROVIDER_NAME) to avoid typos and keep these assertions aligned with the semconv module API (this file already uses constants for the same attribute in metric assertions).
| # span_exporter, start_instrumentation, chat_anthropic_claude_sonnet are coming from fixtures defined in conftest.py | ||
| @pytest.mark.vcr() | ||
| def test_chat_anthropic_claude_sonnet_llm_call( | ||
| span_exporter, start_instrumentation, chat_anthropic_claude_sonnet | ||
| ): | ||
| messages = [ | ||
| SystemMessage(content="You are a helpful assistant!"), | ||
| HumanMessage(content="What is the capital of France?"), | ||
| ] | ||
|
|
||
| result = chat_anthropic_claude_sonnet.invoke(messages) | ||
|
|
||
| assert result.content.find("The capital of France is Paris") != -1 | ||
|
|
||
| # verify spans | ||
| spans = span_exporter.get_finished_spans() | ||
| assert_anthropic_completion_attributes(spans[0], result) | ||
|
|
There was a problem hiding this comment.
For new ChatAnthropic support, there’s only a non-streaming “happy path” VCR test. Per the GenAI instrumentation test guidelines, provider additions should also include recorded tests for streaming and for error scenarios (at least: provider error/endpoint unavailable, and a stream interrupted/closed early). Adding those tests here will prevent regressions in callback handling across ChatAnthropic variants.
Description
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
Fixes #4495
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
test_chat_anthropic_claude_sonnet_llm_callshows testing the ChatAnthropic endpoint and the spans creationsDoes This PR Require a Core Repo Change?
Checklist:
See contributing.md for styleguide, changelog guidelines, and more.