Description
When using LangChain with non-OpenAI providers (e.g., ChatSnowflake, ChatAnthropic), the gen_ai.request.model span attribute shows unknown_model instead of the actual model name. This makes it impossible to track which model was used for cost attribution, performance analysis, etc.
Environment
splunk-otel-instrumentation-langchain: 0.1.x
langchain-snowflake: 0.2.x (or langchain-anthropic)
- Provider: Snowflake Cortex (
ChatSnowflake)
Steps to Reproduce
from langchain_snowflake import ChatSnowflake
llm = ChatSnowflake(model="claude-3-5-sonnet", ...)
response = llm.invoke("Hello")
# Check span attributes in trace
Expected Behavior
Span name: chat claude-3-5-sonnet
gen_ai.request.model: claude-3-5-sonnet
Actual Behavior
Span name: chat unknown_model
gen_ai.request.model: unknown_model
Evidence
Observed in production trace from Snowflake SPCS deployment:
- Trace ID:
90a9d9e471e7ba2a1c88f96e2bc46293
- Multiple
chat unknown_model spans despite using claude-3-5-sonnet
📊 Trace Span Evidence
LLM span showing the bug:
{
"spanId": "ac8f798541d3812c",
"operationName": "chat unknown_model",
"tags": {
"gen_ai.request.model": "unknown_model",
"gen_ai.response.model": "claude-3-5-sonnet",
"gen_ai.provider.name": "snowflake",
"gen_ai.usage.input_tokens": 1108,
"gen_ai.usage.output_tokens": 99
}
}
Key observation:
gen_ai.request.model: "unknown_model" ❌ (should be claude-3-5-sonnet)
gen_ai.response.model: "claude-3-5-sonnet" ✅ (correct from response)
- The model is
claude-3-5-sonnet but request model shows unknown_model
Full trace JSON: trace_flat_hierarchy_bug.json
Root Cause Analysis
The callback handler only checks invocation_params.model_name:
# Current code - only checks model_name (OpenAI convention)
model = invocation_params.get("model_name", "unknown_model")
But ChatSnowflake and ChatAnthropic use invocation_params.model:
# What ChatSnowflake/ChatAnthropic provides
invocation_params = {"model": "claude-3-5-sonnet", ...}
Provider Compatibility
| Provider |
Field Used |
Current Status |
OpenAI (ChatOpenAI) |
model_name |
✅ Works |
Snowflake (ChatSnowflake) |
model |
❌ Shows unknown_model |
Impact
- ❌ Cost tracking broken - can't attribute costs to specific models
- ❌ Performance analysis impossible - can't compare model latencies
- ❌ Span names misleading -
chat unknown_model provides no context
- ❌ Dashboard filters broken - can't filter by model in Splunk O11y
Description
When using LangChain with non-OpenAI providers (e.g.,
ChatSnowflake,ChatAnthropic), thegen_ai.request.modelspan attribute showsunknown_modelinstead of the actual model name. This makes it impossible to track which model was used for cost attribution, performance analysis, etc.Environment
splunk-otel-instrumentation-langchain: 0.1.xlangchain-snowflake: 0.2.x (orlangchain-anthropic)ChatSnowflake)Steps to Reproduce
Expected Behavior
Actual Behavior
Evidence
Observed in production trace from Snowflake SPCS deployment:
90a9d9e471e7ba2a1c88f96e2bc46293chat unknown_modelspans despite usingclaude-3-5-sonnet📊 Trace Span Evidence
LLM span showing the bug:
{ "spanId": "ac8f798541d3812c", "operationName": "chat unknown_model", "tags": { "gen_ai.request.model": "unknown_model", "gen_ai.response.model": "claude-3-5-sonnet", "gen_ai.provider.name": "snowflake", "gen_ai.usage.input_tokens": 1108, "gen_ai.usage.output_tokens": 99 } }Key observation:
gen_ai.request.model: "unknown_model"❌ (should beclaude-3-5-sonnet)gen_ai.response.model: "claude-3-5-sonnet"✅ (correct from response)claude-3-5-sonnetbut request model showsunknown_modelFull trace JSON: trace_flat_hierarchy_bug.json
Root Cause Analysis
The callback handler only checks
invocation_params.model_name:But
ChatSnowflakeandChatAnthropicuseinvocation_params.model:Provider Compatibility
ChatOpenAI)model_nameChatSnowflake)modelunknown_modelImpact
chat unknown_modelprovides no context