Skip to content

Commit f3d0b4c

Browse files
keith-deckerkeith-decker
committed
update readme
1 parent 822cd77 commit f3d0b4c

1 file changed

Lines changed: 96 additions & 34 deletions

File tree

‎util/opentelemetry-util-genai/README.rst‎

Lines changed: 96 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -1,46 +1,104 @@
11
OpenTelemetry Util for GenAI
22
============================
33

4+
The GenAI Utils package provides boilerplate and helpers to standardize instrumentation for Generative AI.
5+
It offers APIs to minimize the work needed to instrument GenAI libraries,
6+
while providing standardization for generating spans, metrics, and events.
47

5-
The GenAI Utils package will include boilerplate and helpers to standardize instrumentation for Generative AI.
6-
This package will provide APIs and decorators to minimize the work needed to instrument genai libraries,
7-
while providing standardization for generating both types of otel, "spans and metrics" and "spans, metrics and events"
88

9-
This package relies on environment variables to configure capturing of message content.
9+
Key Components
10+
--------------
11+
12+
- ``TelemetryHandler`` -- manages LLM invocation lifecycles (spans, metrics, events)
13+
- ``LLMInvocation`` and message types (``Text``, ``Reasoning``, ``Blob``, etc.) -- structured data model for GenAI interactions
14+
- ``CompletionHook`` -- protocol for uploading content to external storage (built-in ``fsspec`` support)
15+
- Metrics -- ``gen_ai.client.operation.duration`` and ``gen_ai.client.token.usage`` histograms
16+
17+
18+
Usage
19+
-----
20+
21+
See the module docstring in ``opentelemetry.util.genai.handler`` for usage examples,
22+
including context manager and manual lifecycle patterns.
23+
24+
25+
Environment Variables
26+
---------------------
27+
28+
This package relies on environment variables to configure capturing of message content.
1029
By default, message content will not be captured.
11-
Set the environment variable `OTEL_SEMCONV_STABILITY_OPT_IN` to `gen_ai_latest_experimental` to enable experimental features.
12-
Set the environment variable `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT` to one of:
13-
- `NO_CONTENT`: Do not capture message content (default).
14-
- `SPAN_ONLY`: Capture message content in spans only.
15-
- `EVENT_ONLY`: Capture message content in events only.
16-
- `SPAN_AND_EVENT`: Capture message content in both spans and events.
17-
18-
To control event emission, you can optionally set `OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT` to `true` or `false` (case-insensitive).
19-
This variable controls whether to emit `gen_ai.client.inference.operation.details` events.
20-
If not explicitly set, the default value is automatically determined by `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT`:
21-
- When `NO_CONTENT` or `SPAN_ONLY` is set: defaults to `false`
22-
- When `EVENT_ONLY` or `SPAN_AND_EVENT` is set: defaults to `true`
30+
Set the environment variable ``OTEL_SEMCONV_STABILITY_OPT_IN`` to ``gen_ai_latest_experimental`` to enable experimental features.
31+
Set the environment variable ``OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT`` to one of:
32+
33+
- ``NO_CONTENT``: Do not capture message content (default).
34+
- ``SPAN_ONLY``: Capture message content in spans only.
35+
- ``EVENT_ONLY``: Capture message content in events only.
36+
- ``SPAN_AND_EVENT``: Capture message content in both spans and events.
37+
38+
To control event emission, you can optionally set ``OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT`` to ``true`` or ``false`` (case-insensitive).
39+
This variable controls whether to emit ``gen_ai.client.inference.operation.details`` events.
40+
If not explicitly set, the default value is automatically determined by ``OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT``:
41+
42+
- When ``NO_CONTENT`` or ``SPAN_ONLY`` is set: defaults to ``false``
43+
- When ``EVENT_ONLY`` or ``SPAN_AND_EVENT`` is set: defaults to ``true``
44+
2345
If explicitly set, the user's value takes precedence over the default.
2446

25-
This package provides these span attributes:
47+
When ``EVENT_ONLY`` or ``SPAN_AND_EVENT`` mode is enabled and a LoggerProvider is configured,
48+
the package also emits ``gen_ai.client.inference.operation.details`` events with structured
49+
message content (as dictionaries instead of JSON strings). Note that when using ``EVENT_ONLY``
50+
or ``SPAN_AND_EVENT``, the ``OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT`` environment variable defaults
51+
to ``true``, so events will be emitted automatically unless explicitly set to ``false``.
2652

27-
- `gen_ai.provider.name`: Str(openai)
28-
- `gen_ai.operation.name`: Str(chat)
29-
- `gen_ai.request.model`: Str(gpt-3.5-turbo)
30-
- `gen_ai.response.finish_reasons`: Slice(["stop"])
31-
- `gen_ai.response.model`: Str(gpt-3.5-turbo-0125)
32-
- `gen_ai.response.id`: Str(chatcmpl-Bz8yrvPnydD9pObv625n2CGBPHS13)
33-
- `gen_ai.usage.input_tokens`: Int(24)
34-
- `gen_ai.usage.output_tokens`: Int(7)
35-
- `gen_ai.input.messages`: Str('[{"role": "Human", "parts": [{"content": "hello world", "type": "text"}]}]')
36-
- `gen_ai.output.messages`: Str('[{"role": "AI", "parts": [{"content": "hello back", "type": "text"}], "finish_reason": "stop"}]')
37-
- `gen_ai.system_instructions`: Str('[{"content": "You are a helpful assistant.", "type": "text"}]') (when system instruction is provided)
53+
**Completion Hook / Upload**
3854

39-
When `EVENT_ONLY` or `SPAN_AND_EVENT` mode is enabled and a LoggerProvider is configured,
40-
the package also emits `gen_ai.client.inference.operation.details` events with structured
41-
message content (as dictionaries instead of JSON strings). Note that when using `EVENT_ONLY`
42-
or `SPAN_AND_EVENT`, the `OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT` environment variable defaults
43-
to `true`, so events will be emitted automatically unless explicitly set to `false`.
55+
- ``OTEL_INSTRUMENTATION_GENAI_COMPLETION_HOOK``: Name of the completion hook entry point to load (e.g. ``upload``).
56+
- ``OTEL_INSTRUMENTATION_GENAI_UPLOAD_BASE_PATH``: An ``fsspec``-compatible URI/path for uploading prompts and responses
57+
(e.g. ``/path/to/prompts`` or ``gs://my_bucket``). Required when using the ``upload`` hook.
58+
- ``OTEL_INSTRUMENTATION_GENAI_UPLOAD_FORMAT``: Format for uploaded data -- ``json`` (default) or ``jsonl``.
59+
- ``OTEL_INSTRUMENTATION_GENAI_UPLOAD_MAX_QUEUE_SIZE``: Maximum number of concurrent uploads to queue (default: ``20``).
60+
61+
62+
Span Attributes
63+
---------------
64+
65+
This package sets the following span attributes on LLM invocations:
66+
67+
**Common attributes:**
68+
69+
- ``gen_ai.operation.name``: Str(chat)
70+
- ``gen_ai.provider.name``: Str(openai)
71+
- ``gen_ai.request.model``: Str(gpt-4o)
72+
- ``server.address``: Str(api.openai.com)
73+
- ``server.port``: Int(443)
74+
75+
**Response attributes:**
76+
77+
- ``gen_ai.response.finish_reasons``: Slice(["stop"])
78+
- ``gen_ai.response.model``: Str(gpt-4o-2024-05-13)
79+
- ``gen_ai.response.id``: Str(chatcmpl-Bz8yrvPnydD9pObv625n2CGBPHS13)
80+
- ``gen_ai.usage.input_tokens``: Int(24)
81+
- ``gen_ai.usage.output_tokens``: Int(7)
82+
83+
**Request parameter attributes (when provided):**
84+
85+
- ``gen_ai.request.temperature``: Float(0.7)
86+
- ``gen_ai.request.top_p``: Float(1.0)
87+
- ``gen_ai.request.frequency_penalty``: Float(0.0)
88+
- ``gen_ai.request.presence_penalty``: Float(0.0)
89+
- ``gen_ai.request.max_tokens``: Int(1024)
90+
- ``gen_ai.request.stop_sequences``: Slice(["\\n"])
91+
- ``gen_ai.request.seed``: Int(42)
92+
93+
**Content attributes (experimental, requires content capturing enabled):**
94+
95+
- ``gen_ai.input.messages``: Str('[{"role": "user", "parts": [{"content": "hello world", "type": "text"}]}]')
96+
- ``gen_ai.output.messages``: Str('[{"role": "assistant", "parts": [{"content": "hello back", "type": "text"}], "finish_reason": "stop"}]')
97+
- ``gen_ai.system_instructions``: Str('[{"content": "You are a helpful assistant.", "type": "text"}]')
98+
99+
**Error attributes:**
100+
101+
- ``error.type``: Str(TimeoutError)
44102

45103

46104
Installation
@@ -50,11 +108,15 @@ Installation
50108

51109
pip install opentelemetry-util-genai
52110

111+
For upload support (requires ``fsspec``)::
112+
113+
pip install opentelemetry-util-genai[upload]
114+
53115

54116
Design Document
55117
---------------
56118

57-
The design document for the OpenTelemetry GenAI Utils can be found at: `Design Document <https://docs.google.com/document/d/1w9TbtKjuRX_wymS8DRSwPA03_VhrGlyx65hHAdNik1E/edit?tab=t.qneb4vabc1wc#heading=h.kh4j6stirken>`_
119+
The design document for the OpenTelemetry GenAI Utils can be found at: `Design Document <https://docs.google.com/document/d/1LzNGylxot5zaIV1goOJZ2mz3LI0weFu1SgaMnyDv7gg/edit?usp=sharing>`_
58120

59121
References
60122
----------

0 commit comments

Comments
 (0)