Skip to content

fix: capture Claude SDK stream model metadata#237

Merged
sipercai merged 2 commits into
mainfrom
fix/claude-agent-stream-context
Jul 7, 2026
Merged

fix: capture Claude SDK stream model metadata#237
sipercai merged 2 commits into
mainfrom
fix/claude-agent-stream-context

Conversation

@sipercai

@sipercai sipercai commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Description

This PR fixes Claude Agent SDK stream span lifecycle and model attribution:

  • close the agent span as soon as ResultMessage is received so a caller that stops consuming after the terminal message does not leak the previous agent context into the next query
  • preserve the caller's active parent span instead of clearing OpenTelemetry context
  • fill gen_ai.request.model from real SDK stream metadata when options/environment do not provide a model: SystemMessage.data.model, AssistantMessage.model, or a single ResultMessage.model_usage key
  • keep unknown when no real model source exists or ResultMessage.model_usage is ambiguous
  • fail open LLM spans when stream processing raises, and preserve tool-only assistant messages as TOOL spans without fabricating LLM spans

Fixes # (N/A)

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

  • python -m pytest instrumentation-loongsuite/loongsuite-instrumentation-claude-agent-sdk/tests/test_session_capture.py -q
  • tox -c tox-loongsuite.ini -e py312-test-loongsuite-instrumentation-claude-agent-sdk-latest -- instrumentation-loongsuite/loongsuite-instrumentation-claude-agent-sdk/tests/test_session_capture.py -q
  • tox -e precommit
  • weaver registry live-check -r /tmp/loongsuite-semantic-conventions/model --input-source <local spans-only sample> --input-format json --format json --no-stream true --advice-profile loongsuite-genai --skip-policies true

Validation Evidence

Spec and Scope

  • Linked issue/spec: N/A
  • Approved spec/comment: local investigation of Claude Agent SDK stream context chaining and model attribution
  • Changed surface: loongsuite-instrumentation-claude-agent-sdk

Local Checks

Check Command Result Notes
Static readiness LoongSuite PR readiness checker pass local pipeline checker exited 0
Precommit tox -e precommit pass fresh-cache run passed after ruff-format changes were applied
Focused tests python -m pytest instrumentation-loongsuite/loongsuite-instrumentation-claude-agent-sdk/tests/test_session_capture.py -q pass 24 passed
Plugin tests tox -c tox-loongsuite.ini -e py312-test-loongsuite-instrumentation-claude-agent-sdk-latest -- instrumentation-loongsuite/loongsuite-instrumentation-claude-agent-sdk/tests/test_session_capture.py -q pass 108 passed, 9 skipped
Claude review qoder deep review pass round 2 found 0 findings
Privacy scan cassette/token/path scan for Claude Agent SDK cassettes pass no matches

Real E2E Matrix

Scenario Status Command or Demo Evidence
non-streaming pass focused in-memory stream tests and bounded live SDK smoke model remains real when SDK emits it; otherwise remains unknown
streaming pass focused stream lifecycle tests ResultMessage closes the agent before iterator exhaustion
concurrency pass focused parallel stream test session ids and parent context remain isolated
agent/tool/ReAct pass focused tool-call stream test and bounded live SDK smoke AGENT/LLM/TOOL spans keep valid parent-child links
tool-heavy pass focused two-tool stream test tool-only assistant messages do not create synthetic LLM spans
error path pass focused failing-stream tests open LLM span is failed and parent context is restored

Telemetry and Weaver

Check Status Command or Artifact Notes
Span tree / span kinds pass focused in-memory exporter sample plus cloud readback smoke AGENT, LLM, TOOL tree validated locally; private cloud trace details kept out of PR body
Content capture modes pass focused tests with SPAN_ONLY content attributes are present in test spans
Concurrency isolation pass focused parallel stream test no cross-call parent chaining
Weaver live-check pass weaver registry live-check -r /tmp/loongsuite-semantic-conventions/model --input-source <local spans-only sample> --input-format json --format json --no-stream true --advice-profile loongsuite-genai --skip-policies true exit 0; no violations

CI

  • GitHub checks: pending until PR is opened
  • Known unrelated failures: none known
  • Follow-up needed: monitor CI after PR creation

Does This PR Require a Core Repo Change?

  • Yes. - Link to PR:
  • No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

  • Followed the style guidelines of this project
  • Changelogs have been updated
  • Unit tests have been added
  • Documentation has been updated

@ralf0131 ralf0131 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review

Fixes Claude Agent SDK model name capture from stream metadata when client options/env don't provide an explicit model. Also:

  • Fail-open LLM spans when SDK streams raise exceptions (preserves parent context cleanup)
  • Tool-only assistant messages now create tool spans without fabricating LLM spans
  • finish_reason correctly set to tool_calls vs stop

The _resolve_assistant_model() / _apply_assistant_model() late-fill pattern is clean -- it doesn't override an explicit model, only fills in when the SDK reveals its default.

CI: all relevant checks passing.

Verdict: APPROVED

@ralf0131 ralf0131 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

LGTM. Fixes model metadata capture for Claude Agent SDK streams: resolves default model names from stream metadata (SystemMessage/AssistantMessage/ResultMessage), adds fail-open LLM span handling on stream exceptions, and preserves tool-only assistant messages without fabricating LLM spans.

Findings

No blocking issues. Key improvements:

  • Model resolution chain is well-designed: SystemMessage.data.model → AssistantMessage.model → ResultMessage.model_usage, with _is_unknown_model guard preventing stale unknown values from overwriting real ones.
  • _extract_model_from_result_message safely returns None when model_usage has multiple entries — avoids ambiguity. Good defensive coding.
  • finish_reason fix — previously hardcoded to "stop" for all assistant outputs; now correctly uses "tool_calls" when tool calls are present. Aligns with GenAI semantic conventions.
  • Fail-open behaviorclose_agent_successfully()/fail_agent() closures ensure single-close semantics; fail_llm_turn() is called before agent failure to preserve span hierarchy cleanup. Correct ordering.
  • Tool-only messages — closing the LLM turn before creating tool spans prevents context leakage. The comment explains the rationale well.
  • Tests cover model resolution from all three sources, fail-open on exceptions, and tool-only message handling.

Suggestions

  • [Info] _process_system_message return type changed from Optional[str] to tuple[Optional[str], Optional[str]]. The tuple[...] syntax requires Python 3.9+ (PEP 585). The repo config says Python 3.9, so this is fine, but worth noting for awareness.

Automated review by github-manager-bot

@sipercai sipercai merged commit d6baaec into main Jul 7, 2026
37 checks passed
@sipercai sipercai deleted the fix/claude-agent-stream-context branch July 7, 2026 15:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants