[Feature] WFGY problem codes as tags in AgentOps traces for multi-agent debugging

[onestardao]

Hi, thanks for AgentOps and the MCP server. This kind of observability is essential when debugging complex agent runs.

I work on WFGY, an open semantic “firewall” and diagnostic layer for LLM pipelines, based on a 16-problem failure map for RAG and agents:

ProblemMap and docs:
https://github.com/onestardao/WFGY/blob/main/ProblemMap/README.md

The same taxonomy is now used or referenced in Harvard MIMS ToolUniverse, QCRI Multimodal RAG Survey and the Innsbruck Rankify project.

Idea

When reading traces in AgentOps, users often see a failure but have no compact vocabulary to describe it.

I propose an optional “WFGY problem tag” feature:

allow users to tag a trace or span with one or more WFGY problem codes (No.1–No.16),

surface these tags in the UI and export,

optionally include a link to the relevant ProblemMap doc for each tag.

This keeps AgentOps neutral, but gives teams a shared language for what went wrong.

If this aligns with your roadmap, I can help provide the mapping text and short descriptions for each problem.

[joaquinariasco-lab]

Hi @onestardao,

This is a very thoughtful approach, a shared vocabulary for LLM agent trace failures is exactly the kind of semantic layer that makes observability actionable.

In a separate project (Flowing), I’ve been building structured execution trace capture for multi-agent LLM workflows, focusing on parent-child relationships, decision spans, and reproducible traces. Your “WFGY problem tag” concept aligns closely: it overlays semantic meaning on top of raw execution spans.

It’s interesting because structured traces + problem tagging form a natural stack:

Execution traces → semantic problem tagging → actionable debugging & reproducibility

I’d be curious how you envision:

• Integrating these tags with multi-agent workflows
• Propagating problem semantics across chained agent calls
• Supporting deterministic replay of tagged failures

If useful, I’d be happy to explore schema alignment or discuss how Flowing’s traces could serve as the base layer for WFGY-style problem tagging.

Project context:
https://github.com/joaquinariasco-lab/Flowing.git

Best,
Joaquin

[onestardao]

Thanks for the thoughtful follow-up. What you describe around structured traces, parent-child relationships, and reproducible execution paths is very much the kind of base layer I had in mind.

For a first step, I would keep the MVP very small and practical: not a full automatic classifier, but an opt-in semantic tag layer on top of the existing trace/span model.

In AgentOps terms, the simplest version could be attaching WFGY problem codes as span metadata using the current tracing flow, starting with the multi-agent relevant cases such as No.3, No.7, and No.13. That keeps the first iteration lightweight while still making traces more semantically actionable.

If that direction fits, I’d be happy to sketch a tiny MVP proposal, or even open a small PR that demonstrates the tagging pattern first. If it makes more sense later, I’d also be very interested in exploring schema alignment with a trace-first project like Flowing as a deeper next step.

[joaquinariasco-lab]

Hi @onestardao,

Thanks for clarifying, that lightweight MVP approach makes perfect sense. Starting with just a few key problem codes (No.3, No.7, No.13) as span metadata is exactly the kind of pragmatic first step that balances immediate usefulness with minimal overhead.

If you like, I can help sketch a minimal Flowing-compatible tagging interface for spans, so that even the MVP traces could later align seamlessly with a full trace-schema integration. That way, the initial PR or demo doesn’t need any heavy refactoring later.

Would you like me to draft a small example implementation showing how these WFGY problem codes could attach to spans and propagate through parent-child relationships in a multi-agent run? That could serve as a concrete starting point for the MVP.

Best,
Joaquin

[onestardao]

That sounds like a great direction.

If you’d like to explore a Flowing-compatible tagging interface, it might make sense to open a dedicated issue in the Flowing repo and tag me there.

I’d be happy to discuss how WFGY problem codes could align with a trace-first schema on your side.

For now, I’ll keep the AgentOps MVP focused and minimal, and we can treat Flowing alignment as a next step.