Skip to content

[Bug] gen_ai.request.model shows 'unknown_model' for non-OpenAI providers #124

Description

@NikitaVoitov

Description

When using LangChain with non-OpenAI providers (e.g., ChatSnowflake, ChatAnthropic), the gen_ai.request.model span attribute shows unknown_model instead of the actual model name. This makes it impossible to track which model was used for cost attribution, performance analysis, etc.

Environment

  • splunk-otel-instrumentation-langchain: 0.1.x
  • langchain-snowflake: 0.2.x (or langchain-anthropic)
  • Provider: Snowflake Cortex (ChatSnowflake)

Steps to Reproduce

from langchain_snowflake import ChatSnowflake

llm = ChatSnowflake(model="claude-3-5-sonnet", ...)
response = llm.invoke("Hello")
# Check span attributes in trace

Expected Behavior

Span name: chat claude-3-5-sonnet
gen_ai.request.model: claude-3-5-sonnet

Actual Behavior

Span name: chat unknown_model
gen_ai.request.model: unknown_model

Evidence

Observed in production trace from Snowflake SPCS deployment:

  • Trace ID: 90a9d9e471e7ba2a1c88f96e2bc46293
  • Multiple chat unknown_model spans despite using claude-3-5-sonnet
📊 Trace Span Evidence

LLM span showing the bug:

{
  "spanId": "ac8f798541d3812c",
  "operationName": "chat unknown_model",
  "tags": {
    "gen_ai.request.model": "unknown_model",
    "gen_ai.response.model": "claude-3-5-sonnet",
    "gen_ai.provider.name": "snowflake",
    "gen_ai.usage.input_tokens": 1108,
    "gen_ai.usage.output_tokens": 99
  }
}

Key observation:

  • gen_ai.request.model: "unknown_model" ❌ (should be claude-3-5-sonnet)
  • gen_ai.response.model: "claude-3-5-sonnet" ✅ (correct from response)
  • The model is claude-3-5-sonnet but request model shows unknown_model

Full trace JSON: trace_flat_hierarchy_bug.json

Image

Root Cause Analysis

The callback handler only checks invocation_params.model_name:

# Current code - only checks model_name (OpenAI convention)
model = invocation_params.get("model_name", "unknown_model")

But ChatSnowflake and ChatAnthropic use invocation_params.model:

# What ChatSnowflake/ChatAnthropic provides
invocation_params = {"model": "claude-3-5-sonnet", ...}

Provider Compatibility

Provider Field Used Current Status
OpenAI (ChatOpenAI) model_name ✅ Works
Snowflake (ChatSnowflake) model ❌ Shows unknown_model

Impact

  • Cost tracking broken - can't attribute costs to specific models
  • Performance analysis impossible - can't compare model latencies
  • Span names misleading - chat unknown_model provides no context
  • Dashboard filters broken - can't filter by model in Splunk O11y

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions