From f3d0b4cf13d65705c0b0b682518f29a2dfa65b8e Mon Sep 17 00:00:00 2001 From: Keith Decker Date: Fri, 27 Mar 2026 09:24:33 -0600 Subject: [PATCH 1/3] update readme --- util/opentelemetry-util-genai/README.rst | 130 +++++++++++++++++------ 1 file changed, 96 insertions(+), 34 deletions(-) diff --git a/util/opentelemetry-util-genai/README.rst b/util/opentelemetry-util-genai/README.rst index bc4b348fcf..ed6de8247f 100644 --- a/util/opentelemetry-util-genai/README.rst +++ b/util/opentelemetry-util-genai/README.rst @@ -1,46 +1,104 @@ OpenTelemetry Util for GenAI ============================ +The GenAI Utils package provides boilerplate and helpers to standardize instrumentation for Generative AI. +It offers APIs to minimize the work needed to instrument GenAI libraries, +while providing standardization for generating spans, metrics, and events. -The GenAI Utils package will include boilerplate and helpers to standardize instrumentation for Generative AI. -This package will provide APIs and decorators to minimize the work needed to instrument genai libraries, -while providing standardization for generating both types of otel, "spans and metrics" and "spans, metrics and events" -This package relies on environment variables to configure capturing of message content. +Key Components +-------------- + +- ``TelemetryHandler`` -- manages LLM invocation lifecycles (spans, metrics, events) +- ``LLMInvocation`` and message types (``Text``, ``Reasoning``, ``Blob``, etc.) -- structured data model for GenAI interactions +- ``CompletionHook`` -- protocol for uploading content to external storage (built-in ``fsspec`` support) +- Metrics -- ``gen_ai.client.operation.duration`` and ``gen_ai.client.token.usage`` histograms + + +Usage +----- + +See the module docstring in ``opentelemetry.util.genai.handler`` for usage examples, +including context manager and manual lifecycle patterns. + + +Environment Variables +--------------------- + +This package relies on environment variables to configure capturing of message content. By default, message content will not be captured. -Set the environment variable `OTEL_SEMCONV_STABILITY_OPT_IN` to `gen_ai_latest_experimental` to enable experimental features. -Set the environment variable `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT` to one of: -- `NO_CONTENT`: Do not capture message content (default). -- `SPAN_ONLY`: Capture message content in spans only. -- `EVENT_ONLY`: Capture message content in events only. -- `SPAN_AND_EVENT`: Capture message content in both spans and events. - -To control event emission, you can optionally set `OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT` to `true` or `false` (case-insensitive). -This variable controls whether to emit `gen_ai.client.inference.operation.details` events. -If not explicitly set, the default value is automatically determined by `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT`: -- When `NO_CONTENT` or `SPAN_ONLY` is set: defaults to `false` -- When `EVENT_ONLY` or `SPAN_AND_EVENT` is set: defaults to `true` +Set the environment variable ``OTEL_SEMCONV_STABILITY_OPT_IN`` to ``gen_ai_latest_experimental`` to enable experimental features. +Set the environment variable ``OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT`` to one of: + +- ``NO_CONTENT``: Do not capture message content (default). +- ``SPAN_ONLY``: Capture message content in spans only. +- ``EVENT_ONLY``: Capture message content in events only. +- ``SPAN_AND_EVENT``: Capture message content in both spans and events. + +To control event emission, you can optionally set ``OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT`` to ``true`` or ``false`` (case-insensitive). +This variable controls whether to emit ``gen_ai.client.inference.operation.details`` events. +If not explicitly set, the default value is automatically determined by ``OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT``: + +- When ``NO_CONTENT`` or ``SPAN_ONLY`` is set: defaults to ``false`` +- When ``EVENT_ONLY`` or ``SPAN_AND_EVENT`` is set: defaults to ``true`` + If explicitly set, the user's value takes precedence over the default. -This package provides these span attributes: +When ``EVENT_ONLY`` or ``SPAN_AND_EVENT`` mode is enabled and a LoggerProvider is configured, +the package also emits ``gen_ai.client.inference.operation.details`` events with structured +message content (as dictionaries instead of JSON strings). Note that when using ``EVENT_ONLY`` +or ``SPAN_AND_EVENT``, the ``OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT`` environment variable defaults +to ``true``, so events will be emitted automatically unless explicitly set to ``false``. -- `gen_ai.provider.name`: Str(openai) -- `gen_ai.operation.name`: Str(chat) -- `gen_ai.request.model`: Str(gpt-3.5-turbo) -- `gen_ai.response.finish_reasons`: Slice(["stop"]) -- `gen_ai.response.model`: Str(gpt-3.5-turbo-0125) -- `gen_ai.response.id`: Str(chatcmpl-Bz8yrvPnydD9pObv625n2CGBPHS13) -- `gen_ai.usage.input_tokens`: Int(24) -- `gen_ai.usage.output_tokens`: Int(7) -- `gen_ai.input.messages`: Str('[{"role": "Human", "parts": [{"content": "hello world", "type": "text"}]}]') -- `gen_ai.output.messages`: Str('[{"role": "AI", "parts": [{"content": "hello back", "type": "text"}], "finish_reason": "stop"}]') -- `gen_ai.system_instructions`: Str('[{"content": "You are a helpful assistant.", "type": "text"}]') (when system instruction is provided) +**Completion Hook / Upload** -When `EVENT_ONLY` or `SPAN_AND_EVENT` mode is enabled and a LoggerProvider is configured, -the package also emits `gen_ai.client.inference.operation.details` events with structured -message content (as dictionaries instead of JSON strings). Note that when using `EVENT_ONLY` -or `SPAN_AND_EVENT`, the `OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT` environment variable defaults -to `true`, so events will be emitted automatically unless explicitly set to `false`. +- ``OTEL_INSTRUMENTATION_GENAI_COMPLETION_HOOK``: Name of the completion hook entry point to load (e.g. ``upload``). +- ``OTEL_INSTRUMENTATION_GENAI_UPLOAD_BASE_PATH``: An ``fsspec``-compatible URI/path for uploading prompts and responses + (e.g. ``/path/to/prompts`` or ``gs://my_bucket``). Required when using the ``upload`` hook. +- ``OTEL_INSTRUMENTATION_GENAI_UPLOAD_FORMAT``: Format for uploaded data -- ``json`` (default) or ``jsonl``. +- ``OTEL_INSTRUMENTATION_GENAI_UPLOAD_MAX_QUEUE_SIZE``: Maximum number of concurrent uploads to queue (default: ``20``). + + +Span Attributes +--------------- + +This package sets the following span attributes on LLM invocations: + +**Common attributes:** + +- ``gen_ai.operation.name``: Str(chat) +- ``gen_ai.provider.name``: Str(openai) +- ``gen_ai.request.model``: Str(gpt-4o) +- ``server.address``: Str(api.openai.com) +- ``server.port``: Int(443) + +**Response attributes:** + +- ``gen_ai.response.finish_reasons``: Slice(["stop"]) +- ``gen_ai.response.model``: Str(gpt-4o-2024-05-13) +- ``gen_ai.response.id``: Str(chatcmpl-Bz8yrvPnydD9pObv625n2CGBPHS13) +- ``gen_ai.usage.input_tokens``: Int(24) +- ``gen_ai.usage.output_tokens``: Int(7) + +**Request parameter attributes (when provided):** + +- ``gen_ai.request.temperature``: Float(0.7) +- ``gen_ai.request.top_p``: Float(1.0) +- ``gen_ai.request.frequency_penalty``: Float(0.0) +- ``gen_ai.request.presence_penalty``: Float(0.0) +- ``gen_ai.request.max_tokens``: Int(1024) +- ``gen_ai.request.stop_sequences``: Slice(["\\n"]) +- ``gen_ai.request.seed``: Int(42) + +**Content attributes (experimental, requires content capturing enabled):** + +- ``gen_ai.input.messages``: Str('[{"role": "user", "parts": [{"content": "hello world", "type": "text"}]}]') +- ``gen_ai.output.messages``: Str('[{"role": "assistant", "parts": [{"content": "hello back", "type": "text"}], "finish_reason": "stop"}]') +- ``gen_ai.system_instructions``: Str('[{"content": "You are a helpful assistant.", "type": "text"}]') + +**Error attributes:** + +- ``error.type``: Str(TimeoutError) Installation @@ -50,11 +108,15 @@ Installation pip install opentelemetry-util-genai +For upload support (requires ``fsspec``):: + + pip install opentelemetry-util-genai[upload] + Design Document --------------- -The design document for the OpenTelemetry GenAI Utils can be found at: `Design Document `_ +The design document for the OpenTelemetry GenAI Utils can be found at: `Design Document `_ References ---------- From 6b8c71e5f21e94b754045e96f4d4dd39ecd0f7a4 Mon Sep 17 00:00:00 2001 From: Keith Decker <47755047+keith-decker@users.noreply.github.com> Date: Mon, 30 Mar 2026 15:39:25 -0600 Subject: [PATCH 2/3] Update util/opentelemetry-util-genai/README.rst Co-authored-by: Tammy Baylis <96076570+tammy-baylis-swi@users.noreply.github.com> --- util/opentelemetry-util-genai/README.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/util/opentelemetry-util-genai/README.rst b/util/opentelemetry-util-genai/README.rst index 97af028018..24a02dc748 100644 --- a/util/opentelemetry-util-genai/README.rst +++ b/util/opentelemetry-util-genai/README.rst @@ -50,7 +50,8 @@ message content (as dictionaries instead of JSON strings). Note that when using or ``SPAN_AND_EVENT``, the ``OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT`` environment variable defaults to ``true``, so events will be emitted automatically unless explicitly set to ``false``. -**Completion Hook / Upload** +Completion Hook / Upload +^^^^^^^^^^^^^^^^^^^^^^^^ - ``OTEL_INSTRUMENTATION_GENAI_COMPLETION_HOOK``: Name of the completion hook entry point to load (e.g. ``upload``). - ``OTEL_INSTRUMENTATION_GENAI_UPLOAD_BASE_PATH``: An ``fsspec``-compatible URI/path for uploading prompts and responses From 1d4279e7c47de7a406b949a2093f02d57055ed0a Mon Sep 17 00:00:00 2001 From: Keith Decker <47755047+keith-decker@users.noreply.github.com> Date: Tue, 31 Mar 2026 09:14:17 -0600 Subject: [PATCH 3/3] Update util/opentelemetry-util-genai/README.rst Co-authored-by: Liudmila Molkova --- util/opentelemetry-util-genai/README.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/util/opentelemetry-util-genai/README.rst b/util/opentelemetry-util-genai/README.rst index 24a02dc748..1394b8160a 100644 --- a/util/opentelemetry-util-genai/README.rst +++ b/util/opentelemetry-util-genai/README.rst @@ -91,7 +91,7 @@ This package sets the following span attributes on LLM invocations: - ``gen_ai.request.stop_sequences``: Slice(["\\n"]) - ``gen_ai.request.seed``: Int(42) -**Content attributes (experimental, requires content capturing enabled):** +**Content attributes (sensitive, requires content capturing enabled):** - ``gen_ai.input.messages``: Str('[{"role": "user", "parts": [{"content": "hello world", "type": "text"}]}]') - ``gen_ai.output.messages``: Str('[{"role": "assistant", "parts": [{"content": "hello back", "type": "text"}], "finish_reason": "stop"}]')