Skip to content

Commit c6e316a

Browse files
committed
Refactor public API on GenAI utils
1 parent f1b6ea9 commit c6e316a

24 files changed

Lines changed: 1285 additions & 1205 deletions

File tree

.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,3 +64,5 @@ target
6464

6565
# opentelemetry-admin jobs
6666
opentelemetry-admin-jobs.txt
67+
68+
.claude*/

CLAUDE.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# OpenTelemetry Python Contrib — Claude Guidelines
2+
3+
**Before writing any code in this repo, you must have read the nearest `AGENTS.md` file
4+
in that directory or its parents (up to this root).** This is a precondition, not a suggestion.
5+
Follow the guidelines there in addition to these.

instrumentation-genai/AGENTS.md

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# GenAI Instrumentation — Agent and Contributor Guidelines
2+
3+
Instrumentation packages here wrap specific libraries (OpenAI, Anthropic, etc.) and bridge
4+
them to the shared telemetry layer in `util/opentelemetry-util-genai`.
5+
6+
## 1. Instrumentation Layer Boundary
7+
8+
Do not call OpenTelemetry APIs (`tracer`, `meter`, `span`, event APIs) directly.
9+
Always go through `TelemetryHandler` and the invocation objects it returns.
10+
11+
This layer is responsible only for:
12+
13+
- Patching the library
14+
- Parsing library-specific input/output into invocation fields
15+
16+
Everything else (span creation, metric recording, event emission, context propagation)
17+
belongs in `util/opentelemetry-util-genai`.
18+
19+
## 2. Invocation Pattern
20+
21+
Use `start_*()` and control span lifetime manually:
22+
23+
```python
24+
invocation = handler.start_inference(provider, request_model, server_address=..., server_port=...)
25+
invocation.temperature = ...
26+
try:
27+
response = client.call(...)
28+
invocation.response_model_name = response.model
29+
invocation.finish_reasons = response.finish_reasons
30+
invocation.stop()
31+
except Exception as exc:
32+
invocation.fail(exc)
33+
raise
34+
```
35+
36+
## 3. Exception Handling
37+
38+
- Do not add `raise` statements in instrumentation/telemetry code — validation belongs in
39+
tests and callers, not in the instrumentation layer.
40+
- When catching exceptions from the underlying library to record telemetry, always re-raise
41+
the original exception unmodified.
42+
- Do not wrap, replace, or suppress exceptions — telemetry must be transparent to callers.
43+

instrumentation-genai/opentelemetry-instrumentation-openai-agents-v2/pyproject.toml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -26,10 +26,10 @@ classifiers = [
2626
"Programming Language :: Python :: 3.14",
2727
]
2828
dependencies = [
29-
"opentelemetry-api >= 1.37",
29+
"opentelemetry-api >= 1.39",
3030
"opentelemetry-instrumentation >= 0.58b0",
31-
"opentelemetry-semantic-conventions >= 0.58b0",
32-
"opentelemetry-util-genai"
31+
"opentelemetry-semantic-conventions >= 0.60b0",
32+
"opentelemetry-util-genai >= 0.4b0.dev"
3333
]
3434

3535
[project.optional-dependencies]

instrumentation-genai/opentelemetry-instrumentation-openai-agents-v2/tests/requirements.latest.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,4 +50,5 @@ grpcio>=1.60.0; implementation_name != "pypy"
5050
grpcio<1.60.0; implementation_name == "pypy"
5151

5252
-e opentelemetry-instrumentation
53+
-e util/opentelemetry-util-genai
5354
-e instrumentation-genai/opentelemetry-instrumentation-openai-agents-v2

instrumentation-genai/opentelemetry-instrumentation-openai-agents-v2/tests/requirements.oldest.txt

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -26,10 +26,11 @@ pytest-asyncio==0.21.0
2626
wrapt==1.16.0
2727
opentelemetry-exporter-otlp-proto-grpc~=1.30
2828
opentelemetry-exporter-otlp-proto-http~=1.30
29-
opentelemetry-api==1.37 # when updating, also update in pyproject.toml
30-
opentelemetry-sdk==1.37 # when updating, also update in pyproject.toml
31-
opentelemetry-semantic-conventions==0.58b0 # when updating, also update in pyproject.toml
29+
opentelemetry-api==1.39 # when updating, also update in pyproject.toml
30+
opentelemetry-sdk==1.39 # when updating, also update in pyproject.toml
31+
opentelemetry-semantic-conventions==0.60b0 # when updating, also update in pyproject.toml
3232
grpcio>=1.60.0; implementation_name != "pypy"
3333
grpcio<1.60.0; implementation_name == "pypy"
3434

35+
-e util/opentelemetry-util-genai
3536
-e instrumentation-genai/opentelemetry-instrumentation-openai-agents-v2

instrumentation-genai/opentelemetry-instrumentation-openai-v2/pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ dependencies = [
2929
"opentelemetry-api ~= 1.39",
3030
"opentelemetry-instrumentation ~= 0.60b0",
3131
"opentelemetry-semantic-conventions ~= 0.60b0",
32-
"opentelemetry-util-genai",
32+
"opentelemetry-util-genai >= 0.4b0.dev",
3333
]
3434

3535
[project.optional-dependencies]

tox.ini

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -789,8 +789,8 @@ deps =
789789

790790
util-genai: {[testenv]test_deps}
791791
util-genai: -r {toxinidir}/util/opentelemetry-util-genai/test-requirements.txt
792-
util-genai: {toxinidir}/util/opentelemetry-util-genai
793-
lint-util-genai: {toxinidir}/util/opentelemetry-util-genai
792+
util-genai: -e {toxinidir}/util/opentelemetry-util-genai
793+
lint-util-genai: -e {toxinidir}/util/opentelemetry-util-genai
794794

795795
; FIXME: add coverage testing
796796
allowlist_externals =
Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# GenAI Utils — Agent and Contributor Guidelines
2+
3+
This package provides shared telemetry utilities for OpenTelemetry GenAI instrumentation.
4+
5+
## 1. Semantic Convention Compliance
6+
7+
All attributes, operation names, and span names must match the
8+
[OpenTelemetry GenAI semantic conventions](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/)
9+
exactly.
10+
11+
Use the appropriate semconv attribute modules — do not hardcode attribute name strings:
12+
13+
- `gen_ai.*` attributes: `opentelemetry.semconv._incubating.attributes.gen_ai_attributes`
14+
- `server.*` attributes: `opentelemetry.semconv.attributes.server_attributes`
15+
- `error.*` attributes: `opentelemetry.semconv.attributes.error_attributes`
16+
- Other namespaces: use the corresponding module from `opentelemetry.semconv`
17+
18+
## 2. Invocation Lifecycle Pattern
19+
20+
Every new operation type must follow this pattern:
21+
22+
```python
23+
invocation = handler.start_inference(provider, request_model, server_address=..., server_port=...)
24+
invocation.temperature = ...
25+
try:
26+
response = client.call(...)
27+
invocation.response_model_name = response.model
28+
invocation.finish_reasons = response.finish_reasons
29+
invocation.stop()
30+
except Exception as exc:
31+
invocation.fail(exc)
32+
raise
33+
```
34+
35+
Factory methods on `TelemetryHandler` (`handler.py`):
36+
37+
- `start_inference(provider, request_model, *, server_address, server_port)``InferenceInvocation`
38+
- `start_embedding(provider, request_model, *, server_address, server_port)``EmbeddingInvocation`
39+
- `start_workflow(name)``WorkflowInvocation`
40+
- For tool calls, construct `ToolInvocation` directly and call `invocation.stop()` / `invocation.fail(exc)`
41+
42+
Context manager equivalents (`handler.inference()`, `handler.embedding()`, `handler.workflow()`) are
43+
available when the span lifetime maps cleanly to a `with` block.
44+
45+
### Anti-patterns
46+
47+
**Never construct invocation types directly** (`WorkflowInvocation(...)`, etc.) in
48+
instrumentation or production code — direct construction skips span creation and context
49+
propagation, so all telemetry calls become no-ops. Always use `handler.start_*()`.
50+
51+
## 3. Exception Handling
52+
53+
- Do not add `raise {Error}` statements to `handler.py` or `types.py` — validation belongs in
54+
tests and callers, not telemetry internals.
55+
- When catching exceptions from the underlying library to record telemetry, always re-raise
56+
the original exception unmodified.
57+
- Do not wrap, replace, or suppress exceptions — telemetry must be transparent to callers.
58+
59+
## 3. Documentation
60+
61+
- Docstrings for invocation types and span/event helpers must include a link to the
62+
corresponding operation in the semconv spec.
63+
- When adding or changing attributes, update the docstring to describe what is set and under
64+
what conditions (e.g., "set only when `server_address` is provided").
65+
66+
## 4. Tests
67+
68+
- Every new operation type or attribute change must have tests verifying the exact attribute
69+
names and values produced, checked against the semconv spec.
70+
- Cover all paths: success (`invocation.stop()`), failure (`invocation.fail(exc)`), and any
71+
conditional attribute logic (e.g., attributes set only when optional fields are populated).
72+
- Tests live in `tests/` — follow existing patterns there.
73+
- Don't call internal API in testswhen the public API is available.
74+
75+
## 5. Python API Conventions
76+
77+
- Mark internal class methods, instance fields, and module-level helpers with a `_` prefix;
78+
anything without it is considered public API.
79+
- Use `field(init=False)` for fields set internally after construction.
80+
- Before removing or renaming a public symbol, deprecate it first with
81+
`warnings.warn(..., DeprecationWarning, stacklevel=2)` pointing to the replacement;
Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# Copyright The OpenTelemetry Authors
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
from __future__ import annotations
16+
17+
from typing import TYPE_CHECKING, Any
18+
19+
from opentelemetry.semconv._incubating.attributes import (
20+
gen_ai_attributes as GenAI,
21+
)
22+
from opentelemetry.semconv.attributes import server_attributes
23+
24+
from opentelemetry.trace import SpanKind
25+
26+
from opentelemetry.util.genai.types import Error, GenAIInvocation
27+
28+
if TYPE_CHECKING:
29+
from opentelemetry.util.genai.handler import TelemetryHandler
30+
31+
32+
class EmbeddingInvocation(GenAIInvocation):
33+
"""Represents a single embedding model invocation.
34+
35+
Use handler.start_embedding(provider) or the handler.embedding(provider)
36+
context manager rather than constructing this directly.
37+
"""
38+
39+
@property
40+
def operation_name(self) -> str:
41+
return GenAI.GenAiOperationNameValues.EMBEDDINGS.value
42+
43+
def __init__(
44+
self,
45+
handler: TelemetryHandler,
46+
provider: str,
47+
*,
48+
request_model: str | None = None,
49+
server_address: str | None = None,
50+
server_port: int | None = None,
51+
encoding_formats: list[str] | None = None,
52+
input_tokens: int | None = None,
53+
dimension_count: int | None = None,
54+
response_model_name: str | None = None,
55+
attributes: dict[str, Any] | None = None,
56+
metric_attributes: dict[str, Any] | None = None,
57+
) -> None:
58+
"""Use handler.start_embedding(provider) or handler.embedding(provider) instead of calling this directly."""
59+
super().__init__(handler, attributes=attributes, metric_attributes=metric_attributes)
60+
self.provider = provider # e.g., azure.ai.openai, openai, aws.bedrock
61+
self.request_model = request_model
62+
self.server_address = server_address
63+
self.server_port = server_port
64+
# encoding_formats can be multi-value -> combinational cardinality risk.
65+
# Keep on spans/events only.
66+
self.encoding_formats = encoding_formats
67+
self.input_tokens = input_tokens
68+
self.dimension_count = dimension_count
69+
self.response_model_name = response_model_name
70+
self._span_name = f"{self.operation_name} {request_model}" if request_model else self.operation_name
71+
self._span_kind = SpanKind.CLIENT
72+
handler._start(self)
73+
74+
def _apply_finish(self, error: Error | None = None) -> None:
75+
optional_attrs = (
76+
(GenAI.GEN_AI_PROVIDER_NAME, self.provider),
77+
(server_attributes.SERVER_ADDRESS, self.server_address),
78+
(server_attributes.SERVER_PORT, self.server_port),
79+
(GenAI.GEN_AI_REQUEST_MODEL, self.request_model),
80+
(GenAI.GEN_AI_EMBEDDINGS_DIMENSION_COUNT, self.dimension_count),
81+
(GenAI.GEN_AI_REQUEST_ENCODING_FORMATS, self.encoding_formats),
82+
(GenAI.GEN_AI_RESPONSE_MODEL, self.response_model_name),
83+
(GenAI.GEN_AI_USAGE_INPUT_TOKENS, self.input_tokens),
84+
)
85+
attributes: dict[str, Any] = {
86+
GenAI.GEN_AI_OPERATION_NAME: self.operation_name,
87+
**{key: value for key, value in optional_attrs if value is not None},
88+
}
89+
if error is not None:
90+
self._apply_error_attributes(error)
91+
attributes.update(self.attributes)
92+
self.span.set_attributes(attributes)
93+
# Metrics recorder currently supports InferenceInvocation fields only.
94+
# No-op until dedicated embedding metric support is added.

0 commit comments

Comments
 (0)