fix: capture Claude SDK stream model metadata#237
Conversation
ralf0131
left a comment
There was a problem hiding this comment.
Review
Fixes Claude Agent SDK model name capture from stream metadata when client options/env don't provide an explicit model. Also:
- Fail-open LLM spans when SDK streams raise exceptions (preserves parent context cleanup)
- Tool-only assistant messages now create tool spans without fabricating LLM spans
- finish_reason correctly set to tool_calls vs stop
The _resolve_assistant_model() / _apply_assistant_model() late-fill pattern is clean -- it doesn't override an explicit model, only fills in when the SDK reveals its default.
CI: all relevant checks passing.
Verdict: APPROVED
ralf0131
left a comment
There was a problem hiding this comment.
Summary
LGTM. Fixes model metadata capture for Claude Agent SDK streams: resolves default model names from stream metadata (SystemMessage/AssistantMessage/ResultMessage), adds fail-open LLM span handling on stream exceptions, and preserves tool-only assistant messages without fabricating LLM spans.
Findings
No blocking issues. Key improvements:
- Model resolution chain is well-designed: SystemMessage.data.model → AssistantMessage.model → ResultMessage.model_usage, with
_is_unknown_modelguard preventing stale unknown values from overwriting real ones. _extract_model_from_result_messagesafely returnsNonewhenmodel_usagehas multiple entries — avoids ambiguity. Good defensive coding.finish_reasonfix — previously hardcoded to "stop" for all assistant outputs; now correctly uses "tool_calls" when tool calls are present. Aligns with GenAI semantic conventions.- Fail-open behavior —
close_agent_successfully()/fail_agent()closures ensure single-close semantics;fail_llm_turn()is called before agent failure to preserve span hierarchy cleanup. Correct ordering. - Tool-only messages — closing the LLM turn before creating tool spans prevents context leakage. The comment explains the rationale well.
- Tests cover model resolution from all three sources, fail-open on exceptions, and tool-only message handling.
Suggestions
- [Info]
_process_system_messagereturn type changed fromOptional[str]totuple[Optional[str], Optional[str]]. Thetuple[...]syntax requires Python 3.9+ (PEP 585). The repo config says Python 3.9, so this is fine, but worth noting for awareness.
Automated review by github-manager-bot
Description
This PR fixes Claude Agent SDK stream span lifecycle and model attribution:
ResultMessageis received so a caller that stops consuming after the terminal message does not leak the previous agent context into the next querygen_ai.request.modelfrom real SDK stream metadata when options/environment do not provide a model:SystemMessage.data.model,AssistantMessage.model, or a singleResultMessage.model_usagekeyunknownwhen no real model source exists orResultMessage.model_usageis ambiguousFixes # (N/A)
Type of change
How Has This Been Tested?
python -m pytest instrumentation-loongsuite/loongsuite-instrumentation-claude-agent-sdk/tests/test_session_capture.py -qtox -c tox-loongsuite.ini -e py312-test-loongsuite-instrumentation-claude-agent-sdk-latest -- instrumentation-loongsuite/loongsuite-instrumentation-claude-agent-sdk/tests/test_session_capture.py -qtox -e precommitweaver registry live-check -r /tmp/loongsuite-semantic-conventions/model --input-source <local spans-only sample> --input-format json --format json --no-stream true --advice-profile loongsuite-genai --skip-policies trueValidation Evidence
Spec and Scope
loongsuite-instrumentation-claude-agent-sdkLocal Checks
tox -e precommitpython -m pytest instrumentation-loongsuite/loongsuite-instrumentation-claude-agent-sdk/tests/test_session_capture.py -qtox -c tox-loongsuite.ini -e py312-test-loongsuite-instrumentation-claude-agent-sdk-latest -- instrumentation-loongsuite/loongsuite-instrumentation-claude-agent-sdk/tests/test_session_capture.py -qReal E2E Matrix
unknownResultMessagecloses the agent before iterator exhaustionTelemetry and Weaver
SPAN_ONLYweaver registry live-check -r /tmp/loongsuite-semantic-conventions/model --input-source <local spans-only sample> --input-format json --format json --no-stream true --advice-profile loongsuite-genai --skip-policies trueCI
Does This PR Require a Core Repo Change?
Checklist:
See contributing.md for styleguide, changelog guidelines, and more.