Skip to content

HYBIM 665 Converting stop_llm calls from sync to async #319

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
shuningc wants to merge 6 commits into
base: main
Choose a base branch
from
Open

HYBIM 665 Converting stop_llm calls from sync to async #319

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
ef65901
Converting stop_llm call from sync to async
shuningc May 5, 2026
4035db9
revert(example): remove debug Console exporters from multi_agent_trav…
shuningc May 5, 2026
3130298
Linting fix
shuningc May 5, 2026
fadd509
Ruff format fix
shuningc May 5, 2026
9b35e3c
Dealing with comments
shuningc May 5, 2026
ed9abd4
Removing evaluation error attribute from span emission
shuningc May 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 0 additions & 7 deletions util/opentelemetry-util-genai/src/opentelemetry/util/genai/emitters/span.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -224,19 +224,12 @@ def _apply_evaluation_attributes(
span.set_attribute(
"gen_ai.evaluation.sampled", invocation.sample_for_evaluation
)
span.set_attribute(
"gen_ai.evaluation.error",
str(invocation.evaluation_error),
)
elif span is not None and hasattr(span, "_attributes"):
# Fallback for ReadableSpan: directly mutate _attributes
try:
span._attributes["gen_ai.evaluation.sampled"] = str(
invocation.sample_for_evaluation
).lower()
span._attributes["gen_ai.evaluation.error"] = str(
invocation.evaluation_error
)

except Exception:
pass
Expand Down
37 changes: 37 additions & 0 deletions util/opentelemetry-util-genai/src/opentelemetry/util/genai/environment_variables.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -374,6 +374,40 @@
export OTEL_INSTRUMENTATION_GENAI_CONTEXT_PROPAGATION=true
"""

OTEL_INSTRUMENTATION_GENAI_ASYNC_FINALIZATION = (
"OTEL_INSTRUMENTATION_GENAI_ASYNC_FINALIZATION"
)
"""
.. envvar:: OTEL_INSTRUMENTATION_GENAI_ASYNC_FINALIZATION

Offload ``stop_*`` / ``fail_*`` completion callbacks and metric
``force_flush`` to a background thread pool so the caller's thread returns
immediately. Default: ``false``.

``end_time``, OTel context cleanup (``_pop_current_span``), and ``span.end()``
always run inline before the handoff so timing accuracy, span parent/child
relationships, and framework span-lifecycle expectations are preserved.

If the background queue is full the finalization runs inline as a fallback —
telemetry is never dropped.

Examples::

export OTEL_INSTRUMENTATION_GENAI_ASYNC_FINALIZATION=true
"""

OTEL_INSTRUMENTATION_GENAI_ASYNC_FINALIZATION_QUEUE_SIZE = (
"OTEL_INSTRUMENTATION_GENAI_ASYNC_FINALIZATION_QUEUE_SIZE"
)
"""
.. envvar:: OTEL_INSTRUMENTATION_GENAI_ASYNC_FINALIZATION_QUEUE_SIZE

Maximum number of finalization tasks that can be queued in the background
thread pool when :envvar:`OTEL_INSTRUMENTATION_GENAI_ASYNC_FINALIZATION` is
enabled. Default: ``128``. When the queue is full the next finalization runs
inline rather than being dropped.
"""

__all__ = [
# existing
"OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT",
Expand Down Expand Up @@ -410,4 +444,7 @@
# genai context
"OTEL_INSTRUMENTATION_GENAI_CONTEXT_INCLUDE_IN_METRICS",
"OTEL_INSTRUMENTATION_GENAI_CONTEXT_PROPAGATION",
# async finalization
"OTEL_INSTRUMENTATION_GENAI_ASYNC_FINALIZATION",
"OTEL_INSTRUMENTATION_GENAI_ASYNC_FINALIZATION_QUEUE_SIZE",
]
Loading
Loading