Suggestion: add WFGY 3.0 · open source LLM debugging toolkit (16 Problem Map)

[onestardao]

Hi, thanks for putting this awesome AI toolkit together.

I would like to suggest adding WFGY as a developer-facing toolkit for debugging and stress-testing LLM applications:

• Repo: https://github.com/onestardao/WFGY
• License: MIT
• Stars: 1k+ (actively maintained)

Very short description:

• An open source framework that uses a 16 Problem Map of failure modes to debug RAG and LLM apps.
• Includes concrete checklists for vector stores, retrieval logic, prompt wiring, and long-horizon reasoning.
• Designed so that a developer can paste their failing prompt / code context and quickly see which problem type they are hitting, plus suggested fixes.

Why it might fit this list:

• It is not a general “AI toy”, it is specifically built for LLM reliability engineering.
• Pairs well with other tools in this repo that focus on training and deployment, by covering the “why is my LLM app broken in production” side.
• Fully open source, already used in the wild as a free “clinic” for RAG and LLM bugs.

If you feel it fits, I am happy to send a PR and place it in a section like “LLM tools for developers” or “Debugging and evaluation”.

Thanks for considering it.

[onestardao]

Hi, just a brief follow-up on this suggestion.

Since opening this issue, the WFGY 2.0 “RAG 16 Problem Map” has been adopted or referenced by several RAG and LLM ecosystem projects, including:

• RAGFlow
• LlamaIndex
• ToolUniverse (Harvard MIMS Lab)
• Rankify (University of Innsbruck)
• Multimodal RAG Survey (QCRI LLM Lab)

In these cases, WFGY is not positioned as a competing framework, but as a structured diagnostic layer for debugging and evaluating RAG / LLM pipelines.

Given that this list curates tools and resources around LLM systems, I wanted to check again whether adding WFGY as a debugging / evaluation resource would make sense here.

If aligned with your scope, I’m happy to submit a minimal MVP PR — just a concise one-line entry formatted exactly to your style.

If it’s outside the intended focus, no worries at all. Just wanted to confirm before proceeding.

Thanks again for maintaining the list.