feat: integrate AgentScope v2, MAF, and bootstrap registry updates by sipercai · Pull Request #233 · alibaba/loongsuite-python

sipercai · 2026-07-01T10:04:03Z

Description

This PR supersedes and integrates:

feat(agentscope): support AgentScope v2 #208: AgentScope v2 instrumentation support.
feat(instrumentation): add instrumentation for microsoft-agent-framework #229: Microsoft Agent Framework instrumentation support.
fix: split bootstrap registry and stabilize CrewAI test #232: bootstrap registry split and CrewAI CI stabilization.

The combined branch resolves the generated workflow / bootstrap registry hotspot once, then keeps the three feature surfaces in one reviewable PR:

add AgentScope v2 middleware, tests, and recorded e2e cassettes;
add loongsuite-instrumentation-microsoft-agent-framework with semantic processor, ReAct step patching, util-genai bridge, tests, README, and changelog;
split LoongSuite bootstrap metadata into generated bootstrap_registry/* fragments and update change detection / workflow generation tests;
keep CrewAI error-scenario stabilization and changelog coverage;
fix MAF executor.process AgentExecutor spans to set gen_ai.provider.name=microsoft.agent_framework when reclassified as invoke_agent / create_agent.

Supersedes #208, #229, and #232.

Type of change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

How Has This Been Tested?

Local validation:

tox -e generate-workflows
tox -c tox-loongsuite.ini -e py311-test-detect-loongsuite-changes
tox -c tox-loongsuite.ini -e py311-test-loongsuite-instrumentation-microsoft-agent-framework
tox -c tox-loongsuite.ini -e py311-test-loongsuite-instrumentation-crewai
tox -c tox-loongsuite.ini -e py311-test-loongsuite-instrumentation-agentscope-latest -- instrumentation-loongsuite/loongsuite-instrumentation-agentscope/tests/test_v2_instrumentation.py
Fresh-cache PRE_COMMIT_HOME="$(mktemp -d)" tox -e precommit
git diff --check
LoongSuite static PR readiness checker
Secret-like scan over changed framework tests and cassettes

Real runtime / telemetry validation:

Microsoft Agent Framework agent-framework-core 1.10.0: real Agent, AgentSession, WorkflowBuilder, AgentExecutor, local deterministic tool, streaming, multi-turn, concurrency, and error flows against a DashScope OpenAI-compatible model. ARMS/CMS readback covered 8 representative traces.
AgentScope 2.0.3: real v2 Agent.reply, Agent.reply_stream, Toolkit, FunctionTool, multi-turn, concurrency, and error flows against a DashScope model. ARMS/CMS readback covered 6 representative traces.
CrewAI 1.15.1 + LiteLLM 1.90.2: multi-agent incident workflow, streaming crew, concurrent crews with OTel context propagation, deterministic tools, and error path. ARMS/CMS readback covered 4 representative traces.
Weaver standard GenAI live-check passed for the combined emitted-span sample. Strict registry mode only reports known extension-namespace gaps for gen_ai.crewai.*.

Validation Evidence

Spec and Scope

Linked issue/spec: supersedes existing PRs feat(agentscope): support AgentScope v2 #208, feat(instrumentation): add instrumentation for microsoft-agent-framework #229, and fix: split bootstrap registry and stabilize CrewAI test #232.
Approved spec/comment: integrate the three PRs into one branch and validate CrewAI, Microsoft Agent Framework, and AgentScope v2 with real LLM/runtime scenarios before publication.
Changed surface: loongsuite-instrumentation-agentscope, loongsuite-instrumentation-microsoft-agent-framework, loongsuite-instrumentation-crewai, loongsuite-distro bootstrap registry generation, LoongSuite workflow generation.

Local Checks

Check	Command	Result	Notes
Static readiness	`python "$PIPELINE_SKILL_DIR/scripts/check_loongsuite_pr_readiness.py" --repo . --base origin/main`	pass	`OK: LoongSuite static PR readiness checks passed`
Review matrix	`python "$PIPELINE_SKILL_DIR/scripts/plan_review_matrix.py" --repo . --format markdown`	pass	GenAI, agent, streaming, concurrency, telemetry, and privacy scenarios planned
Precommit	`PRE_COMMIT_HOME="$(mktemp -d)" tox -e precommit`	pass	Fresh hook cache
Diff whitespace	`git diff --check`	pass	No whitespace errors
Workflow generation	`tox -e generate-workflows`	pass	Generated workflows are committed
Change detector tests	`tox -c tox-loongsuite.ini -e py311-test-detect-loongsuite-changes`	pass	`45 passed`
MAF focused tests	`tox -c tox-loongsuite.ini -e py311-test-loongsuite-instrumentation-microsoft-agent-framework`	pass	`58 passed`
CrewAI focused tests	`tox -c tox-loongsuite.ini -e py311-test-loongsuite-instrumentation-crewai`	pass	`48 passed`
AgentScope v2 focused tests	`tox -c tox-loongsuite.ini -e py311-test-loongsuite-instrumentation-agentscope-latest -- instrumentation-loongsuite/loongsuite-instrumentation-agentscope/tests/test_v2_instrumentation.py`	pass	`13 passed`
External review	qoder deep review against `origin/main`	pass	0 blocking findings; 3 advisory findings deferred
Privacy scan	Secret-like scan over changed framework tests and cassettes	pass	No credentials, bearer tokens, local paths, or private trace data found

Real E2E Matrix

Scenario	Status	Command or Demo	Evidence
non-streaming	pass	Bounded real-provider smokes for MAF, AgentScope v2, and CrewAI	Expected `LLM` / `AGENT` trees exported and read back from ARMS/CMS
streaming	pass	MAF streaming agent call, AgentScope v2 `reply_stream`, CrewAI streaming workflow	Streaming spans finalized and exported
concurrency	pass	Concurrent real-provider smokes with unique run/session ids	Trace isolation validated; no cross-run parent/context bleed observed
agent/tool/ReAct	pass	MAF `AgentExecutor` workflow, AgentScope v2 `Toolkit` / `FunctionTool`, CrewAI multi-agent workflow	`AGENT` / `STEP` / `TOOL` / `LLM` parent-child trees observed
tool-heavy	pass	CrewAI multi-tool workflow plus MAF / AgentScope deterministic tool paths	Tool spans and following LLM spans exported under the expected parent spans
error path	pass	Framework-specific failing tool/model path smokes	Error spans retained parent-child context and were exported

Telemetry and Weaver

Check	Status	Command or Artifact	Notes
Span tree / span kinds	pass	ARMS/CMS trace readback plus local emitted-span samples	MAF, AgentScope v2, and CrewAI emitted expected `ENTRY` / `WORKFLOW` / `AGENT` / `STEP` / `TOOL` / `LLM` shapes where applicable
Required provider attributes	pass	MAF real workflow trace + Weaver standard sample	Reclassified MAF agent executor spans include `gen_ai.provider.name=microsoft.agent_framework`
Content capture modes	pass	Focused framework tests and privacy scan	Content-capture behavior covered by tests; public PR body intentionally omits private prompts, responses, and trace ids
Concurrency isolation	pass	Real-provider concurrent smokes	Separate traces and run/session ids remained isolated
Weaver live-check	pass	`weaver registry live-check ... --advice-profile loongsuite-genai ...`	Standard combined sample passed; strict mode only reports known extension-namespace gaps for `gen_ai.crewai.*`

CI

GitHub checks: pending until this integrated PR is opened.
Known unrelated failures: none known from local gates.
Follow-up needed: monitor GitHub Actions after PR creation.

Does This PR Require a Core Repo Change?

Yes. - Link to PR:
No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

Followed the style guidelines of this project
Changelogs have been updated
Unit tests have been added
Documentation has been updated

CLAassistant · 2026-07-01T10:04:31Z

All committers have signed the CLA.

…te.md Implements the hybrid "SpanProcessor + optional ReAct step patch" plan documented in llm-dev/microsoft-agent-framework/investigate/execute.md. - MicrosoftAgentFrameworkInstrumentor: enables MAF native OTel layers (force=True) and registers MAFSemanticProcessor. - MAFSemanticProcessor (span_processor.py): injects gen_ai.span.kind, gen_ai.operation.name, renames MAF private-prefix attributes to gen_ai.*, normalizes provider.name (azure_openai -> openai), backfills TTFT from streaming events, sets StatusCode.OK on success, aggregates the 6 ARMS gauges. Reuses opentelemetry.util.genai.utils.gen_ai_json_dumps (aligned with openai-agents-v2/span_processor.py:27) to coerce dict/list attribute values into JSON strings. - react_step_patch.py (opt-in via ARMS_MAF_REACT_STEP_ENABLED): emits one react step span per LLM round-trip inside FunctionInvocationLayer via ExtendedTelemetryHandler.react_step() from opentelemetry-util-genai. - config.py: env switches (master, sensitive data, react step, slow threshold). - tests: 23 passing unit tests covering span classification, metric aggregation, provider normalization, TTFT backfill, dict coercion, and react_step handler behavior.

[M1] MCP span classification: detect mcp.method.name attribute (or SpanKind.CLIENT + mcp.* fallback) in _classify_span and return (CLIENT, MCP). MAF_SPAN_NAME_PREFIXES documents that MCP is detected via attribute rather than prefix (method names are unbounded). [M2] revert_react_step_patch: capture originals BEFORE wrapping (via __wrapped__ unwrap chain), and switch from broken decorator form to wrap_function_wrapper(class, name, wrapper) + @wrapt.decorator. revert now restores the original; apply->revert->apply does not stack wrappers. [L1] _safe_dumps: cap output at 4096 chars (execute.md single-field cap) since gen_ai_json_dumps only serializes. Docstring + module docstring updated to match actual behavior. Tests: +6 (test_mcp_span_classified_as_client, mcp client-kind fallback, non-mcp client negative, _safe_dumps truncation, apply->revert->apply round-trip, _unwrap_to_function). 29 passed.

- P0: react_step_patch wrappers now return coroutines from sync wrappers so MAF's await layer.get_response no longer raises TypeError. ContextVar tokens are set inside the coroutine body so set/reset share the same asyncio task context. - P1: extend op_name override condition to include CLIENT span kind so MCP tools/call inner spans get gen_ai.operation.name=mcp even when MAF pre-wrote execute_tool. - P3: provider.name normalization now handles sequence values (MAF emits list-wrapped values on AGENT spans) and falls back to case-insensitive matching so microsoft.agent_framework -> openai on AGENT spans. - P5: instrument() prepends MAFSemanticProcessor to the SDK processor tuple so on_end enrichments run before exporter processors registered earlier in bootstrap. Adds 7 unit tests; pytest tests/ -> 36 passed.

Add license header to all 11 files in the microsoft-agent-framework instrumentation package. Resolves the check-license-header CI failure.

sipercai mentioned this pull request Jul 1, 2026

feat(agentscope): support AgentScope v2 #208

Closed

16 tasks

This was referenced Jul 1, 2026

feat(instrumentation): add instrumentation for microsoft-agent-framework #229

Closed

fix: split bootstrap registry and stabilize CrewAI test #232

Closed

sipercai force-pushed the liuyu/feat-integrate-pr208-229-232 branch from 85ac7e7 to 5509517 Compare July 1, 2026 11:24

github-actions Bot assigned 123liuziming, Cirilla-zmh and ralf0131 Jul 1, 2026

sipercai mentioned this pull request Jul 2, 2026

test: PR 233 on ARC runners sipercai/loongsuite-python-agent#1

Draft

sipercai and others added 21 commits July 3, 2026 14:02

fix: split bootstrap registry fragments

fda9cd1

chore: add CrewAI changelog entry

e9b8285

fix(maf): align telemetry with genai conventions

f5f771e

fix(maf): address review comments and ci

4be6a79

fix(maf): move instrumentation into loongsuite matrix

29c3fdc

fix: add missing Apache 2.0 license headers

db33cac

Add license header to all 11 files in the microsoft-agent-framework instrumentation package. Resolves the check-license-header CI failure.

fix(maf): finalize spans through util genai bridge

603c6c7

fix(maf): address util genai review comments

7e262e4

fix(maf): align workflow and mcp genai semantics

c9a3503

fix(maf): align workflow telemetry with genai util

0b1fa90

style(maf): apply ruff formatting

de7e88c

feat(agentscope): support AgentScope v2

385a33d

Support AgentScope v2 instrumentation

9da8655

fix(agentscope): serialize v2 tool result content

7976aa1

docs(agentscope): clarify v2 handler factory

b940fa6

chore: clean integration whitespace

02c0c31

fix: set MAF framework provider for agent executor spans

c77e2f3

chore: refresh loongsuite generated files after rebase

3f63471

sipercai force-pushed the liuyu/feat-integrate-pr208-229-232 branch from 5509517 to 3f63471 Compare July 3, 2026 06:25

123liuziming approved these changes Jul 3, 2026

View reviewed changes

sipercai merged commit 6a99a8a into main Jul 3, 2026
211 checks passed

sipercai deleted the liuyu/feat-integrate-pr208-229-232 branch July 3, 2026 08:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: integrate AgentScope v2, MAF, and bootstrap registry updates#233

feat: integrate AgentScope v2, MAF, and bootstrap registry updates#233
sipercai merged 21 commits into
mainfrom
liuyu/feat-integrate-pr208-229-232

sipercai commented Jul 1, 2026

Uh oh!

CLAassistant commented Jul 1, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

sipercai commented Jul 1, 2026

Description

Type of change

How Has This Been Tested?

Validation Evidence

Spec and Scope

Local Checks

Real E2E Matrix

Telemetry and Weaver

CI

Does This PR Require a Core Repo Change?

Checklist:

Uh oh!

CLAassistant commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

CLAassistant commented Jul 1, 2026 •

edited

Loading