Skip to content

Commit b15a8e8

Browse files
author
流屿
committed
feat(fara): add LoongSuite Fara (Computer Use Agent) instrumentation
Implements GenAI auto-instrumentation for Microsoft Fara per llm-dev/fara/investigate/execute.md (Plan C: hybrid). Span hierarchy: ENTRY enter_ai_application_system (run_fara_agent) └── AGENT invoke_agent FaraAgent (FaraAgent.run) └── STEP react step (round=N) (generate_model_call) ├── LLM chat {model} (OpenAI instr — reused, not re-wrapped) └── TOOL execute_tool {action} (execute_action) Design: - LLM spans owned by opentelemetry-instrumentation-openai-v2 (Fara calls AsyncOpenAI.chat.completions.create in _make_model_call — exact method already wrapped upstream). - STEP rotation via generate_model_call patch (one call per ReAct round) — mirrors WebArena NextActionWrapper pattern; avoids re-implementing FaraAgent.run's loop body. - ContextVar state for async-safe per-task STEP/session tracking. - All spans built via util/opentelemetry-util-genai ExtendedTelemetryHandler. Tests: 20 unit tests covering ENTRY/AGENT/STEP/TOOL emission, parent/ child hierarchy, finish_reason semantics (terminate/action_complete/ max_rounds/<exception type>), content-capture gating, error paths, and instrument/uninstrument lifecycle. All pass locally with python3.11.
1 parent bced622 commit b15a8e8

14 files changed

Lines changed: 1682 additions & 0 deletions

File tree

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
[build-system]
2+
requires = ["hatchling"]
3+
build-backend = "hatchling.build"
4+
5+
[project]
6+
name = "loongsuite-instrumentation-fara"
7+
dynamic = ["version"]
8+
description = "LoongSuite Fara (Computer Use Agent) instrumentation"
9+
license = "Apache-2.0"
10+
requires-python = ">=3.10,<4"
11+
authors = [
12+
{ name = "OpenTelemetry Authors", email = "cncf-opentelemetry-contributors@lists.cncf.io" },
13+
]
14+
classifiers = [
15+
"Development Status :: 4 - Beta",
16+
"Intended Audience :: Developers",
17+
"License :: OSI Approved :: Apache Software License",
18+
"Programming Language :: Python",
19+
"Programming Language :: Python :: 3",
20+
"Programming Language :: Python :: 3.10",
21+
"Programming Language :: Python :: 3.11",
22+
"Programming Language :: Python :: 3.12",
23+
"Programming Language :: Python :: 3.13",
24+
]
25+
dependencies = [
26+
"opentelemetry-api >= 1.37.0",
27+
"opentelemetry-instrumentation >= 0.58b0",
28+
"opentelemetry-semantic-conventions >= 0.58b0",
29+
"wrapt >= 1.17.3, < 2.0.0",
30+
]
31+
32+
[project.optional-dependencies]
33+
instruments = [
34+
"fara >= 0.1",
35+
]
36+
37+
[project.entry-points.opentelemetry_instrumentor]
38+
fara = "opentelemetry.instrumentation.fara:FaraInstrumentor"
39+
40+
[project.urls]
41+
Homepage = "https://github.com/alibaba/loongsuite-python-agent/tree/main/instrumentation-loongsuite/loongsuite-instrumentation-fara"
42+
Repository = "https://github.com/alibaba/loongsuite-python-agent"
43+
44+
[tool.hatch.version]
45+
path = "src/opentelemetry/instrumentation/fara/version.py"
46+
47+
[tool.hatch.build.targets.sdist]
48+
include = [
49+
"/src",
50+
"/tests",
51+
]
52+
53+
[tool.hatch.build.targets.wheel]
54+
packages = ["src/opentelemetry"]
Lines changed: 177 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,177 @@
1+
# Copyright The OpenTelemetry Authors
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
"""OpenTelemetry Fara Instrumentation.
16+
17+
Automatic instrumentation for the
18+
`Microsoft Fara <https://github.com/microsoft/fara>`_ Computer Use Agent
19+
framework.
20+
21+
Span hierarchy
22+
--------------
23+
24+
::
25+
26+
ENTRY enter_ai_application_system (run_fara_agent)
27+
└── AGENT invoke_agent FaraAgent (FaraAgent.run)
28+
├── STEP react step (round=1) (generate_model_call)
29+
│ ├── LLM chat {model} (AsyncCompletions.create)
30+
│ │ * emitted by opentelemetry-
31+
│ │ instrumentation-openai-v2
32+
│ └── TOOL execute_tool {action} (execute_action)
33+
├── STEP react step (round=2)
34+
│ ├── LLM chat {model}
35+
│ └── TOOL execute_tool {action}
36+
└── ...
37+
38+
Design principles
39+
-----------------
40+
41+
* **LLM spans are owned by the OpenAI instrumentation.** Fara calls
42+
``AsyncOpenAI.chat.completions.create`` in ``FaraAgent._make_model_call``;
43+
``opentelemetry-instrumentation-openai-v2`` already wraps that exact
44+
method, so we rely on it for token usage / model / request attrs and
45+
let its LLM span attach naturally as a child of the current STEP.
46+
* **STEP rotation via ``generate_model_call``.``FaraAgent.run`` is a
47+
single async method with an internal for-loop; we can't hook code
48+
blocks. ``generate_model_call`` is called exactly once per round at
49+
the top of each iteration, so wrapping it is equivalent to "start of
50+
round" and mirrors the WebArena ``NextActionWrapper`` pattern.
51+
* **ContextVar isolation.`` ``FaraAgent.run`` is async, so all per-task
52+
state lives in ContextVars for safe concurrent execution.
53+
54+
Usage
55+
-----
56+
57+
.. code:: python
58+
59+
from opentelemetry.instrumentation.fara import FaraInstrumentor
60+
61+
FaraInstrumentor().instrument()
62+
63+
# Then run Fara as normal (e.g. ``python -m fara.run_fara --task "..."``).
64+
"""
65+
66+
from __future__ import annotations
67+
68+
import logging
69+
from typing import Any, Collection
70+
71+
from wrapt import wrap_function_wrapper
72+
73+
from opentelemetry.instrumentation.fara.package import _instruments
74+
from opentelemetry.instrumentation.fara.version import __version__
75+
from opentelemetry.instrumentation.instrumentor import BaseInstrumentor
76+
from opentelemetry.instrumentation.utils import unwrap
77+
from opentelemetry.util.genai.extended_handler import ExtendedTelemetryHandler
78+
79+
logger = logging.getLogger(__name__)
80+
81+
__all__ = ["FaraInstrumentor", "__version__"]
82+
83+
84+
# (module, qualname, wrapper_attr_name)
85+
_PATCH_TARGETS = (
86+
("fara.run_fara", "run_fara_agent", "_entry_wrapper"),
87+
("fara.fara_agent", "FaraAgent.run", "_agent_run_wrapper"),
88+
("fara.fara_agent", "FaraAgent.generate_model_call", "_generate_model_call_wrapper"),
89+
("fara.fara_agent", "FaraAgent.execute_action", "_tool_wrapper"),
90+
)
91+
92+
93+
class FaraInstrumentor(BaseInstrumentor):
94+
"""An ``opentelemetry-instrumentation`` plugin for Microsoft Fara.
95+
96+
Spans (see module docstring) are emitted via ``wrapt`` hooks on four
97+
Fara functions. LLM spans are intentionally **not** emitted here (the
98+
OpenAI SDK probe handles them).
99+
"""
100+
101+
_patched: list[tuple[str, str]] = []
102+
_handler: ExtendedTelemetryHandler | None = None
103+
104+
def instrumentation_dependencies(self) -> Collection[str]:
105+
return _instruments
106+
107+
def _instrument(self, **kwargs: Any) -> None:
108+
tracer_provider = kwargs.get("tracer_provider")
109+
meter_provider = kwargs.get("meter_provider")
110+
logger_provider = kwargs.get("logger_provider")
111+
112+
self._handler = ExtendedTelemetryHandler(
113+
tracer_provider=tracer_provider,
114+
meter_provider=meter_provider,
115+
logger_provider=logger_provider,
116+
)
117+
118+
from opentelemetry.instrumentation.fara.internal._wrappers import (
119+
AgentRunWrapper,
120+
EntryWrapper,
121+
GenerateModelCallWrapper,
122+
ToolWrapper,
123+
)
124+
125+
wrappers = {
126+
"_entry_wrapper": EntryWrapper(self._handler),
127+
"_agent_run_wrapper": AgentRunWrapper(self._handler),
128+
"_generate_model_call_wrapper": GenerateModelCallWrapper(self._handler),
129+
"_tool_wrapper": ToolWrapper(self._handler),
130+
}
131+
132+
type(self)._patched = []
133+
for module, qualname, wrapper_key in _PATCH_TARGETS:
134+
try:
135+
wrap_function_wrapper(
136+
module=module,
137+
name=qualname,
138+
wrapper=wrappers[wrapper_key],
139+
)
140+
type(self)._patched.append((module, qualname))
141+
except Exception as exc: # noqa: BLE001
142+
logger.warning(
143+
"FaraInstrumentor: could not wrap %s.%s: %s",
144+
module,
145+
qualname,
146+
exc,
147+
)
148+
149+
def _uninstrument(self, **kwargs: Any) -> None:
150+
import importlib # noqa: PLC0415
151+
152+
for module, qualname in list(type(self)._patched):
153+
try:
154+
mod = importlib.import_module(module)
155+
except Exception as exc: # noqa: BLE001
156+
logger.debug(
157+
"FaraInstrumentor: could not import %s for unwrap: %s",
158+
module,
159+
exc,
160+
)
161+
continue
162+
parts = qualname.split(".")
163+
try:
164+
target = mod
165+
for p in parts[:-1]:
166+
target = getattr(target, p)
167+
if hasattr(target, parts[-1]):
168+
unwrap(target, parts[-1])
169+
except Exception as exc: # noqa: BLE001
170+
logger.debug(
171+
"FaraInstrumentor: could not unwrap %s.%s: %s",
172+
module,
173+
qualname,
174+
exc,
175+
)
176+
type(self)._patched = []
177+
self._handler = None
Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Copyright The OpenTelemetry Authors
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
"""Configuration via environment variables."""
16+
17+
from __future__ import annotations
18+
19+
import os
20+
21+
22+
def _int_env(name: str, default: str) -> int:
23+
try:
24+
return int(os.getenv(name, default))
25+
except ValueError:
26+
return int(default)
27+
28+
29+
def _bool_env(name: str, default: bool = False) -> bool:
30+
raw = os.getenv(name)
31+
if raw is None:
32+
return default
33+
return raw.strip().lower() in {"1", "true", "yes", "on"}
34+
35+
36+
# Cap on non-content string attribute values (URLs, tool names, etc.)
37+
FARA_OTEL_MAX_ATTR_LENGTH = _int_env("FARA_OTEL_MAX_ATTR_LENGTH", "1024")
38+
39+
# Cap on prompt / message preview length when capture-message-content is on
40+
FARA_OTEL_PROMPT_PREVIEW_MAX_LEN = _int_env(
41+
"FARA_OTEL_PROMPT_PREVIEW_MAX_LEN", "4096"
42+
)
43+
44+
45+
def capture_message_content() -> bool:
46+
"""Whether to record prompt / completion / tool argument bodies.
47+
48+
Honours the standard semantic-conventions opt-in flag.
49+
Accepts SPAN_ONLY / SPAN_AND_EVENT / EVENT_ONLY as truthy values.
50+
"""
51+
val = os.getenv("OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT")
52+
if val is None:
53+
return False
54+
return val.strip().upper() in {
55+
"TRUE",
56+
"1",
57+
"YES",
58+
"ON",
59+
"SPAN_ONLY",
60+
"SPAN_AND_EVENT",
61+
"EVENT_ONLY",
62+
}
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Copyright The OpenTelemetry Authors
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.

0 commit comments

Comments
 (0)