Skip to content

GenAI Utils | Adding Embedding metrics #4377

Merged
aabmass merged 19 commits into
open-telemetry:mainfrom
shuningc:addingEmbeddingMetric
Apr 16, 2026
Merged

GenAI Utils | Adding Embedding metrics #4377
aabmass merged 19 commits into
open-telemetry:mainfrom
shuningc:addingEmbeddingMetric

Conversation

@shuningc

@shuningc shuningc commented Mar 30, 2026

Copy link
Copy Markdown
Contributor

Description

This PR adds metrics and events telemetry for [EmbeddingInvocation], aligning with OpenTelemetry semantic conventions. Previously, embedding invocations only emitted spans - now they also emit duration metrics.

Type of change

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Added 3 new tests for embedding metrics:
[test_stop_embedding_records_duration_and_tokens] - verifies duration is recorded but token metrics are NOT
[test_stop_embedding_records_duration_with_additional_attributes] - verifies server address, port, custom attributes, and response model are included
[test_fail_embedding_records_error_and_duration] - verifies error path records
error.type and duration

Does This PR Require a Core Repo Change?

  • No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

  • Followed the style guidelines of this project
  • Changelogs have been updated
  • Unit tests have been added

@shuningc shuningc requested a review from a team as a code owner March 30, 2026 12:52
@shuningc shuningc marked this pull request as draft March 30, 2026 12:52
@shuningc shuningc marked this pull request as ready for review March 30, 2026 12:53
Comment thread util/opentelemetry-util-genai/src/opentelemetry/util/genai/handler.py Outdated
@xrmx xrmx added the gen-ai Related to generative AI label Apr 1, 2026
@xrmx xrmx moved this to Reviewed PRs that need fixes in Python PR digest Apr 1, 2026
@shuningc shuningc changed the title GenAI Utils | Adding Embedding metrics and events GenAI Utils | Adding Embedding metrics Apr 2, 2026
Comment thread util/opentelemetry-util-genai/src/opentelemetry/util/genai/metrics.py Outdated
Comment thread util/opentelemetry-util-genai/src/opentelemetry/util/genai/metrics.py Outdated
Comment thread util/opentelemetry-util-genai/src/opentelemetry/util/genai/metrics.py Outdated
Comment thread util/opentelemetry-util-genai/src/opentelemetry/util/genai/metrics.py Outdated
@shuningc shuningc force-pushed the addingEmbeddingMetric branch from e6216e4 to 68d328f Compare April 8, 2026 00:12
shuningc and others added 10 commits April 9, 2026 08:58
---
updated-dependencies:
- dependency-name: aiohttp
  dependency-version: 3.13.4
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…try#4326)" (open-telemetry#4413)

This reverts commit 22879d6.

Now that we have just one job to check we don't need to build the list anymore.
* Drop Python 3.9 support

Signed-off-by: emdneto <9735060+emdneto@users.noreply.github.com>

* generate-workflows

Signed-off-by: emdneto <9735060+emdneto@users.noreply.github.com>

* fixes

Signed-off-by: emdneto <9735060+emdneto@users.noreply.github.com>

* remove extra reference to pypy310

Signed-off-by: emdneto <9735060+emdneto@users.noreply.github.com>

* changelog

Signed-off-by: emdneto <9735060+emdneto@users.noreply.github.com>

* fix flask tests

Signed-off-by: emdneto <9735060+emdneto@users.noreply.github.com>

* fix google-genai tests

Signed-off-by: emdneto <9735060+emdneto@users.noreply.github.com>

* fix google-genai tests

Signed-off-by: emdneto <9735060+emdneto@users.noreply.github.com>

* fix

Signed-off-by: emdneto <9735060+emdneto@users.noreply.github.com>

* remove unused _ensure_gzip_single_response

Signed-off-by: emdneto <9735060+emdneto@users.noreply.github.com>

---------

Signed-off-by: emdneto <9735060+emdneto@users.noreply.github.com>
Co-authored-by: Riccardo Magliocchetti <riccardo.magliocchetti@gmail.com>
* Add CLAUDE.md with project structure and commands

* Add AGENTS.md symlink to CLAUDE.md

* Move guidance to AGENTS.md and address review feedback

- Move main content from CLAUDE.md to AGENTS.md so all AI agents
  (not only Claude) pick up the guidance; CLAUDE.md now just references
  it via `@AGENTS.md`.
- Add general rules, PR scoping, and `Assisted-by:` commit trailer
  guidance (inspired by the Collector's AGENTS.md).
- Clarify that only instrumentation packages live under
  `src/opentelemetry/instrumentation/{name}/`; other package types use
  their own namespace.

---------

Co-authored-by: Riccardo Magliocchetti <riccardo.magliocchetti@gmail.com>
* Bump pylint to 4.0.5 to fix Python 3.14 concurrent.futures false positives

* Fix too-many-positional-arguments pylint failures - Add max-positional-arguments=10, Add pylint: disable=too-many-positional-arguments to functions that legitimately exceed the limit

* Bump max-positional-arguments to 12 and remove unnecessary disable comments

* Address review comments: fix CHANGELOG, remove stale pylintrc comment, add openai-agents disable

* Fix formatting: restore blank line in CHANGELOG and remove extra blank line in .pylintrc

* Update processor.py

* Update test_botocore_bedrock.py

---------

Co-authored-by: Riccardo Magliocchetti <riccardo.magliocchetti@gmail.com>
open-telemetry#4391)

* Refactor public API on GenAI utils

* more lint

* review feedback

* update tests to use named params

* address some of the comments

* up

* fix failing checks and clean up imports

* lint

* lint

* fix lint

* replace @deprecated with docstring info to avoid warnings for users

* up

* common code for context manager

@MikeGoldsmith MikeGoldsmith left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me - thanks @shuningc

@github-project-automation github-project-automation Bot moved this from Reviewed PRs that need fixes to Approved PRs in Python PR digest Apr 16, 2026
@aabmass aabmass merged commit b8ca943 into open-telemetry:main Apr 16, 2026
751 checks passed
@github-project-automation github-project-automation Bot moved this from Approved PRs to Done in Python PR digest Apr 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gen-ai Related to generative AI

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.