Skip to content

fix(langchain): resolve request model from ls_model_name metadata for LangGraph tool spans#4316

Open
leorivastech wants to merge 1 commit into
traceloop:mainfrom
leorivastech:fix/langchain-request-model-from-metadata
Open

fix(langchain): resolve request model from ls_model_name metadata for LangGraph tool spans#4316
leorivastech wants to merge 1 commit into
traceloop:mainfrom
leorivastech:fix/langchain-request-model-from-metadata

Conversation

@leorivastech

@leorivastech leorivastech commented Jun 20, 2026

Copy link
Copy Markdown

When you run a LangChain chat model through LangGraph with a provider that
doesn't surface the model in its serialized kwargs — for example
init_chat_model(..., model_provider="bedrock_converse") bound with tools — the
resulting spans come out with gen_ai.request.model and gen_ai.response.model
set to unknown. The span dump in #3098 shows it nicely: everything else is
there (prompts, tool calls), but the model name is gone.

I traced it down to set_request_params. It looks for the model in two spots:
the callback's invocation_params, and serialized["kwargs"] (via
set_chat_request). For something like ChatOpenAI the name lives in
serialized["kwargs"]["model_name"], so it resolves fine and nobody notices.
But for LangGraph tool-orchestrated models neither spot has it, so we just give
up and write "unknown".

The thing is, LangChain does hand us the model name — it's in the callback
metadata as ls_model_name, which _get_ls_params fills in automatically for
every chat model. We were already receiving that metadata in
on_chat_model_start and passing it to _create_llm_span, just never down to
set_request_params.

So the fix is small: thread the metadata through and, when the usual lookups come
up empty, fall back to ls_model_name before defaulting to unknown. It's the
same key we already trust on the response side (_extract_model_name_from_association_metadata,
from #3237), so this just brings the request side in line. When the model is
resolvable the normal way, nothing changes — it's purely a fallback.

For testing I added tests/test_request_model_extraction.py. It drives the
callback handler directly against an in-memory exporter, so it needs no cassettes
and no API keys: one test reproduces the LangGraph/bedrock_converse case (it fails
on main with assert 'unknown' == 'us.anthropic...' and passes here), and
another makes sure an explicit model in serialized["kwargs"] still takes
precedence over the metadata fallback. The full langchain suite stays green (151
passed) and ruff is clean.

Fixes #3098


  • I have added tests that cover my changes.
  • If adding a new instrumentation or changing an existing one, I've added screenshots from some observability platform showing the change. — this is an attribute-mapping fix, not a UI/instrumentation addition, so I went with a span-asserting test instead of screenshots.
  • PR name follows conventional commits format.
  • (If applicable) I have updated the documentation accordingly.

Summary by CodeRabbit

  • Improvements

    • Enhanced model identification in OpenTelemetry traces for tool-orchestrated LangChain chat models by supporting metadata-based model attribution when model information is not available through default resolution paths.
  • Tests

    • Added test coverage for model extraction logic in tracing scenarios.

… LangGraph tool spans

LangGraph tool-orchestrated chat models (e.g. init_chat_model with model_provider=bedrock_converse) don't expose the model in serialized kwargs or invocation_params, so set_request_params fell back to "unknown". LangChain still provides it via the callback metadata (ls_model_name), so thread that through and use it as a fallback, matching what the response side already does. Pure fallback; normal resolution is unchanged.

Fixes traceloop#3098
@coderabbitai

coderabbitai Bot commented Jun 20, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 339cc9d9-9810-4570-8924-89b7d6dec8b0

📥 Commits

Reviewing files that changed from the base of the PR and between fc33f1c and b7d941d.

📒 Files selected for processing (3)
  • packages/opentelemetry-instrumentation-langchain/opentelemetry/instrumentation/langchain/callback_handler.py
  • packages/opentelemetry-instrumentation-langchain/opentelemetry/instrumentation/langchain/span_utils.py
  • packages/opentelemetry-instrumentation-langchain/tests/test_request_model_extraction.py

📝 Walkthrough

Walkthrough

Adds an optional metadata parameter to set_request_params, set_llm_request, and set_chat_request in span_utils.py. When model derivation from kwargs/invocation_params fails, the code falls back to metadata["ls_model_name"]. The callback_handler.py threads the existing metadata argument through both on_chat_model_start and on_llm_start. Two unit tests cover the fallback and serialized-payload precedence behaviors.

Changes

Metadata-based model name fallback

Layer / File(s) Summary
Model name fallback in span_utils
packages/opentelemetry-instrumentation-langchain/opentelemetry/instrumentation/langchain/span_utils.py
set_request_params gains an optional metadata parameter and reads ls_model_name from it when kwargs/invocation_params yield no model name; set_llm_request and set_chat_request expose the same optional metadata parameter and forward it.
metadata threading in callback_handler
packages/opentelemetry-instrumentation-langchain/opentelemetry/instrumentation/langchain/callback_handler.py
on_chat_model_start and on_llm_start now pass the existing metadata argument to set_request_params, set_chat_request, and set_llm_request.
Regression and precedence tests
packages/opentelemetry-instrumentation-langchain/tests/test_request_model_extraction.py
New test file drives on_chat_model_start + on_llm_end against an InMemorySpanExporter-backed handler and asserts that gen_ai.request.model resolves from metadata["ls_model_name"] when invocation_params are empty, and that serialized["kwargs"]["model_name"] takes precedence when present.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐇 A model named "unknown" hid in the trace,
No kwargs or params to show its face.
But metadata whispered, "ls_model_name here!"
The rabbit patched it, the span became clear.
Now LangGraph spans bloom with names bright and true —
No more mystery models, hurrah and woo-hoo! 🎉

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 61.54% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title clearly and specifically describes the main change: resolving the request model from ls_model_name metadata for LangGraph tool spans, which directly addresses the core issue.
Linked Issues check ✅ Passed The PR fully addresses issue #3098 by implementing a metadata fallback mechanism to populate gen_ai.request.model and gen_ai.response.model with the actual model name for LangGraph-orchestrated chat models instead of defaulting to unknown.
Out of Scope Changes check ✅ Passed All changes are directly scoped to resolving the reported issue: threading metadata through callback handler and span utilities, adding model resolution fallback logic, and including comprehensive test cases with no unrelated modifications.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🐛 Bug Report: gen_ai.request.model set to unknown for LangGraph/LangChain

1 participant