Skip to content

fix(hermes-agent): create entry span from session source#216

Open
sipercai wants to merge 2 commits into
mainfrom
liuyu/fix-hermes-entry-source
Open

fix(hermes-agent): create entry span from session source#216
sipercai wants to merge 2 commits into
mainfrom
liuyu/fix-hermes-entry-source

Conversation

@sipercai

@sipercai sipercai commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Keep Hermes CLI, IM/gateway, TUI, and API Server ENTRY behavior unchanged when AIAgent.platform is present.
  • Match Hermes session-source fallback for platform-less AIAgent calls: platform, then HERMES_SESSION_SOURCE, then cli.
  • Fix no-platform top-level Hermes runs so they emit ENTRY -> AGENT instead of AGENT only.

Validation Evidence

Spec and Scope

  • Linked issue/spec: customer report that ENTRY spans disappeared after Hermes upgrade.
  • Approved spec/comment: direct user request to support latest Hermes no-platform/session-source behavior.
  • Changed surface: only loongsuite-instrumentation-hermes-agent code, docs, changelog, and tests.

Local Checks

Check Command Result Notes
Static readiness python "$PIPELINE_SKILL_DIR/scripts/check_loongsuite_pr_readiness.py" --repo . pass LoongSuite readiness passed
Precommit tox -e precommit pass ruff, format, uv-lock, rstcheck passed
Focused tests PYTHONPATH="instrumentation-loongsuite/loongsuite-instrumentation-hermes-agent/src:util/opentelemetry-util-genai/src:opentelemetry-api/src:opentelemetry-sdk/src:opentelemetry-semantic-conventions/src:opentelemetry-instrumentation/src" python -m pytest instrumentation-loongsuite/loongsuite-instrumentation-hermes-agent/tests/test_telemetry_spec.py -q pass 34 passed
Claude review local review loop pass Accepted fixes applied; no remaining P0/P1 for this Hermes scope
Privacy scan `git diff origin/main...HEAD rg -n "(AKIA ASIA

Real E2E Matrix

Scenario Status Command or Demo Evidence
non-streaming pass latest Hermes offline smoke plus focused wrapper tests no-platform AIAgent.platform=None produced ENTRY -> AGENT
streaming pass focused Hermes streaming tests streaming TTFT and nested LLM suppression covered
concurrency pass test_state_is_isolated_across_concurrent_invocations isolated ENTRY/AGENT state across concurrent runs
agent/tool/ReAct pass focused Hermes telemetry tests ReAct, tool dispatch, delegate tool, nested agent covered
tool-heavy pass focused Hermes tool tests memory/tool dispatch and skill tool spans covered
error path pass test_entry_span_fails_when_agent_fail_cleanup_raises ENTRY cleanup/error behavior covered

Telemetry and Weaver

Check Status Command or Artifact Notes
Span tree / span kinds pass focused tests + latest Hermes smoke no-platform case emits ENTRY -> AGENT
Content capture modes pass test_entry_span_omits_messages_when_content_capture_disabled capture disabled omits message attributes
Concurrency isolation pass test_state_is_isolated_across_concurrent_invocations no state leak across runs
Weaver live-check pass weaver registry live-check -r /tmp/loongsuite-semantic-conventions/model --input-source /tmp/hermes-pr216-weaver-entry-sample.json --input-format json --format json --no-stream true --advice-profile loongsuite-genai --skip-policies true ENTRY-only sample passed with no violations; full ENTRY+AGENT sample exposes pre-existing gen_ai.agent.system registry gap outside this PR

CI

  • GitHub checks: pending after rebased branch update.
  • Known unrelated failures: local tox -e typecheck on the rebased head currently fails in util/opentelemetry-util-genai/.../extended_handler.py; this file is not changed by this PR. Current main checks for de65b325 were queued when verified.
  • Follow-up needed: keep PR draft until GitHub CI for the rebased commit finishes.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@sipercai sipercai force-pushed the liuyu/fix-hermes-entry-source branch from dc0d39f to c438bc4 Compare June 17, 2026 07:02
@sipercai sipercai marked this pull request as ready for review June 17, 2026 07:57

@ralf0131 ralf0131 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review by github-manager-bot

Summary

Fixes missing ENTRY spans for platform-less AIAgent calls by mirroring Hermes's own session source resolution: AIAgent.platform, then HERMES_SESSION_SOURCE env var, then cli default. Previously, platform-less runs emitted only AGENT; now they correctly emit ENTRY -> AGENT. Also adds hermes-agent to CI lint/test workflows.

Findings

  • [Info] helpers.py — The resolve_entry_platform() fallback chain (platform, env, cli) correctly mirrors Hermes's session source logic. Renaming from _entry_platform to the public resolve_entry_platform and removing should_create_entry_for_agent() is clean since the function now always returns a truthy value.
  • [Info] wrappers.py — The entry_platform variable is resolved once and used consistently for the entry-creation guard. The TODO comment about passing entry_platform into EntryInvocation when the upstream util adds a session source field is appropriately forward-looking.
  • [Info] Tests — pytest.importorskip for hermes_state and run_agent in test_agent_features.py / test_live_llm_calls.py is the right approach to avoid hard failures in CI without the Hermes runtime source.
  • [Info] test_telemetry_spec.py — Comprehensive new tests for all resolution paths: TUI platform, env session source preference, env over platform, CLI default, and the behavioral change for platform-less agents.

Tests

Excellent coverage: the renamed test test_agent_without_platform_or_env_uses_cli_default_entry_source now asserts ENTRY -> AGENT instead of AGENT-only, matching the intended behavioral change.

LGTM. Well-documented behavioral change with thorough test coverage and clean CI integration.


Automated review by github-manager-bot

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants