Skip to content

Genai utils | Add AgentInvocation type#4274

Open
etserend wants to merge 20 commits intoopen-telemetry:mainfrom
etserend:genai-utils/agent-invocation
Open

Genai utils | Add AgentInvocation type#4274
etserend wants to merge 20 commits intoopen-telemetry:mainfrom
etserend:genai-utils/agent-invocation

Conversation

@etserend
Copy link
Copy Markdown

Description

This PR adds the AgentInvocation type extending _BaseAgent for agent invocation (invoke_agent) spans. It provides TelemetryHandler lifecycle methods and an agent() context manager — aligned with the GenAI agent span semantic conventions.

Builds on the _BaseAgent base class and AgentCreation type added in #4217. Agent metrics and events to come in follow-up PRs.

Type of change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

  • Basic unit tests for starting and stopping agent invocation spans
  • Unit tests for agent invocation with all attributes (request + response)
  • Unit tests for context manager happy path and error path
  • Unit tests for AgentInvocation dataclass defaults and messages

Does This PR Require a Core Repo Change?

  • No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

  • Followed the style guidelines of this project
  • Changelogs have been updated
  • Unit tests have been added
  • Documentation has been updated

@etserend etserend force-pushed the genai-utils/agent-invocation branch from 1b5cd9f to 08e2b07 Compare February 27, 2026 17:45
@etserend etserend changed the title Genai utils/agent invocation Genai utils/Add AgentInvocation type Mar 2, 2026
@etserend etserend changed the title Genai utils/Add AgentInvocation type Genai utils | Add AgentInvocation type Mar 2, 2026
@etserend
Copy link
Copy Markdown
Author

etserend commented Mar 2, 2026

Sample trace from demo app

Trace (8d30ba6647c92e8f3c471a1c7041bcb2)
└── invoke_agent Currency Exchange Agent (7.07s, Client)
    ├── gen_ai.agent.description: Currency exchange agent demo
    ├── gen_ai.agent.name: Currency Exchange Agent
    ├── gen_ai.operation.name: invoke_agent
    ├── gen_ai.provider.name: gcp_vertex_ai
    ├── gen_ai.request.model: gemini-2.5-flash
    ├── gen_ai.response.finish_reasons: ["stop"]
    ├── gen_ai.response.model: gemini-2.5-flash
    ├── gen_ai.usage.input_tokens: 9
    ├── gen_ai.usage.output_tokens: 102
    └── server.address: generativelanguage.googleapis.com

Trace (3c0caf7d81d59ee14d084ff14d3a971d)
└── invoke_agent Manual Agent (0.58s, Client)
    ├── gen_ai.agent.description: Manual lifecycle demo
    ├── gen_ai.agent.name: Manual Agent
    ├── gen_ai.operation.name: invoke_agent
    ├── gen_ai.provider.name: gcp_vertex_ai
    ├── gen_ai.request.model: gemini-2.5-flash
    ├── gen_ai.response.finish_reasons: ["stop"]
    ├── gen_ai.response.model: gemini-2.5-flash
    ├── gen_ai.usage.input_tokens: 8
    ├── gen_ai.usage.output_tokens: 7
    └── server.address: generativelanguage.googleapis.com

@etserend etserend marked this pull request as ready for review March 2, 2026 22:23
@etserend etserend requested a review from a team as a code owner March 2, 2026 22:23
@tammy-baylis-swi tammy-baylis-swi moved this to Ready for review in Python PR digest Mar 3, 2026
Comment thread util/opentelemetry-util-genai/src/opentelemetry/util/genai/types.py Outdated
Comment thread util/opentelemetry-util-genai/src/opentelemetry/util/genai/span_utils.py Outdated
@etserend etserend force-pushed the genai-utils/agent-invocation branch from e0ee125 to 25411ca Compare March 6, 2026 16:28
Copy link
Copy Markdown
Contributor

@eternalcuriouslearner eternalcuriouslearner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM!! I left few questions to understand:

  1. If we need to improve overall robustness of error handling.
  2. early capture of few sem conv.

I used this doc for reviewing if sem conv are captured correctly or not. Additionally does it make sense to make this pr specific for agent invocation and removing agent creation specific code.

Comment thread util/opentelemetry-util-genai/src/opentelemetry/util/genai/handler.py Outdated
Comment thread util/opentelemetry-util-genai/src/opentelemetry/util/genai/handler.py Outdated
Comment thread util/opentelemetry-util-genai/src/opentelemetry/util/genai/span_utils.py Outdated
Comment thread util/opentelemetry-util-genai/src/opentelemetry/util/genai/handler.py Outdated
Comment thread util/opentelemetry-util-genai/src/opentelemetry/util/genai/types.py Outdated
Copy link
Copy Markdown
Contributor

@wrisa wrisa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Though would need refactoring in handler.py in case other PRs are merged first.

Copy link
Copy Markdown
Contributor

@eternalcuriouslearner eternalcuriouslearner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: Do we have GEN_AI_AGENT_VERSION, GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS, GEN_AI_TOOL_DEFINITIONS and GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS in opentelemetry.semconv._incubating.attributes?

@etserend
Copy link
Copy Markdown
Author

question: Do we have GEN_AI_AGENT_VERSION, GEN_AI_USAGE_CACHE_CREATION_INPUT_TOKENS, GEN_AI_TOOL_DEFINITIONS and GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS in opentelemetry.semconv._incubating.attributes?

Yes @eternalcuriouslearner , all four constants in semconv package and referenced them via the GenAI. alias.

@etserend
Copy link
Copy Markdown
Author

etserend commented Mar 17, 2026

Verified the AgentInvocation type and TelemetryHandler.agent() context manager end-to-end using the Vertex AI SDK with VertexAIInstrumentor auto-instrumentation.

Demo PR: #4340

Steps exercised:

  • invoke_agent span wrapping a generate_content LLM call
  • Auto-instrumented chat span appears as child of invoke_agent (same trace)
  • Token usage populated from response metadata
Trace (60e99da09ed18470d275be708544362d)
└── invoke_agent Currency Exchange Agent (6.70s, Client)
    ├── gen_ai.operation.name: invoke_agent
    ├── gen_ai.agent.name: Currency Exchange Agent
    ├── gen_ai.agent.description: Currency exchange agent demo
    ├── gen_ai.provider.name: gcp_vertex_ai
    ├── gen_ai.request.model: gemini-2.5-flash
    ├── gen_ai.response.finish_reasons: ["stop"]
    ├── gen_ai.usage.input_tokens: 12
    ├── gen_ai.usage.output_tokens: 163
    ├── gen_ai.input.messages: [{"role":"user","parts":[{"content":"What is the..."}]}]
    ├── gen_ai.output.messages: [{"role":"assistant","parts":[{"content":"As of..."}]}]
    ├── server.address: us-central1-aiplatform.googleapis.com
    │
    └── chat gemini-2.5-flash (6.67s, Client)
        ├── gen_ai.operation.name: chat
        ├── gen_ai.request.model: gemini-2.5-flash
        ├── gen_ai.response.model: gemini-2.5-flash
        ├── gen_ai.response.finish_reasons: ["stop"]
        ├── gen_ai.usage.input_tokens: 12
        ├── gen_ai.usage.output_tokens: 163
        ├── gen_ai.input.messages: [{"role":"user","parts":[{"content":"What is the..."}]}]
        ├── gen_ai.output.messages: [{"role":"model","parts":[{"content":"As of..."}]}]
        ├── server.address: us-central1-aiplatform.googleapis.com
        └── server.port: 443

@keith-decker
Copy link
Copy Markdown
Contributor

@etserend Note: based on the discussion in SIG today, we should not be introducing new start_, stop_ and fail_* methods.

see #4219 for the discussion and examples.

@keith-decker keith-decker self-requested a review March 19, 2026 16:23
@etserend etserend force-pushed the genai-utils/agent-invocation branch from a3a0f9e to c320ca6 Compare March 26, 2026 21:41
Comment thread util/opentelemetry-util-genai/src/opentelemetry/util/genai/span_utils.py Outdated
Comment thread util/opentelemetry-util-genai/src/opentelemetry/util/genai/_agent_invocation.py Outdated
Comment thread util/opentelemetry-util-genai/src/opentelemetry/util/genai/_agent_invocation.py Outdated
Comment thread util/opentelemetry-util-genai/src/opentelemetry/util/genai/handler.py Outdated
Comment thread util/opentelemetry-util-genai/src/opentelemetry/util/genai/handler.py Outdated
Comment thread util/opentelemetry-util-genai/src/opentelemetry/util/genai/_agent_invocation.py Outdated
Comment thread docs/nitpick-exceptions.ini Outdated
@etserend etserend force-pushed the genai-utils/agent-invocation branch from 54a896f to 928069a Compare April 17, 2026 15:15
…nvocation

# Conflicts:
#	util/opentelemetry-util-genai/CHANGELOG.md
Copy link
Copy Markdown
Member

@lmolkova lmolkova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with some minor comments!

Comment thread util/opentelemetry-util-genai/tests/test_handler_agent.py
Comment thread util/opentelemetry-util-genai/src/opentelemetry/util/genai/_content.py Outdated
@etserend etserend force-pushed the genai-utils/agent-invocation branch from 2926372 to a7c6e63 Compare April 22, 2026 17:06
ToolDefinition,
)

_GEN_AI_AGENT_VERSION: str = getattr(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why using getattr on these instead of just direct access ? It looks weird to me

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we usually just use string literal when attr is not yet in semconv package, doesn't seem to be necessary to getattr

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied suggested changes.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's more instances of getattr() still. Can you also maybe just stick a TODO comment so we know to migrate to the conventions once they are generated?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @aabmass, replaced all remaining getattr() instances with string literals across the PR. Added TODO comments to track migration once the constants land in the semconv.

@etserend etserend force-pushed the genai-utils/agent-invocation branch from 7147979 to 6ed56b6 Compare April 23, 2026 17:26
@etserend etserend force-pushed the genai-utils/agent-invocation branch from 180869b to 274a14b Compare April 23, 2026 20:44
Comment thread util/opentelemetry-util-genai/CHANGELOG.md Outdated
@etserend etserend force-pushed the genai-utils/agent-invocation branch from 1fdcee5 to e525aa3 Compare April 23, 2026 21:02
Copy link
Copy Markdown
Member

@Cirilla-zmh Cirilla-zmh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Ready for review

Development

Successfully merging this pull request may close these issues.

10 participants