|
| 1 | +# VMind OpenAPI |
| 2 | + |
| 3 | +Built on the VMind chart generation tool, the VisActor team has launched VMind Agent, which improves the accuracy and efficiency of intelligent generation and adds multi‑turn editing capabilities. VMind Agent currently provides OpenAPI interfaces that you can call to achieve more advanced intelligent generation. The [VMind Agent Playground](/vmind/playground) offers a simple testing environment to experience VMind Agent’s capabilities. |
| 4 | + |
| 5 | +## VSeed |
| 6 | + |
| 7 | +VSeed is a domain‑specific language (DSL) for analytics. It encapsulates commonly used charting capabilities, greatly improving the efficiency of building analytic reporting platforms. In most cases, given a dataset and chart type, it can render directly; each chart type comes with extensive default conventions. |
| 8 | + |
| 9 | +VSeed is a subset of VChart and VTable. Its configuration is greatly simplified compared with VChart and VTable, making the options easier for large models to understand. It is a chart library specifically packaged for LLM‑driven generation. |
| 10 | + |
| 11 | +VSeed repository: [https://github.com/VisActor/VSeed](https://github.com/VisActor/VSeed) |
| 12 | + |
| 13 | +VSeed showcase: [https://visactor.github.io/VSeed/galley/chartType/line.html](https://visactor.github.io/VSeed/galley/chartType/line.html) |
| 14 | + |
| 15 | +VMind Agent prioritizes using VSeed for code generation for better results, while also supporting native code generation for VChart and VTable. |
| 16 | + |
| 17 | +## VMind Agent |
| 18 | + |
| 19 | +The overall technical approach of VMind Agent is as follows: |
| 20 | + |
| 21 | +When the user provides instructions and code blocks to VMind Agent, the agent uses the LLM to actively call tools and proceeds step by step through Generate/Edit → Check → Edit → Check … until the user’s requirements are met, outputting the corresponding code blocks and explanatory text. |
| 22 | + |
| 23 | +Compared with traditional chart generation tools, VMind Agent offers advantages in the following areas: |
| 24 | +- 🤖 ReAct architecture: Reasoning–action loop implemented with LangGraph |
| 25 | +- 🛠️ Diverse tools: Supports multiple chart generation and editing tools |
| 26 | +- 📊 Chart types: Supports many chart and table types |
| 27 | +- 🎨 Template system: Prebuilt chart templates |
| 28 | +- 🔧 Chart editing: Supports modifying existing charts |
| 29 | +- 💬 Natural language: Supports generating charts from pure natural‑language input |
| 30 | + |
| 31 | +### VMind Agent Architecture |
| 32 | + |
| 33 | +VMind Agent is composed of a Coordinator (scheduler) and a set of tools, and is implemented with the `@langchain/langgraph` library. The main modules are: |
| 34 | + |
| 35 | +<img src="https://lf9-dp-fe-cms-tos.byteorg.com/obj/bit-cloud/vmind/tutorials/VMind_Agent_Design_1.png" > |
| 36 | + |
| 37 | +#### Coordinator |
| 38 | + |
| 39 | +The scheduler’s main job is to determine, based on the current state, which tool to call next or whether to end the task. |
| 40 | + |
| 41 | +#### Generate Tool |
| 42 | + |
| 43 | +The generation tool is invoked when the user has not provided code or explicitly asks to regenerate code. If no chart type is specified, the tool recommends a suitable chart type based on the data. Each chart or table type provides a basic template; the LLM uses the user’s data and the template to generate the simplest code block for subsequent editing. |
| 44 | + |
| 45 | +#### Edit Tool |
| 46 | + |
| 47 | +The editing tool is the core of VMind Agent. Based on the current code and the user’s goal, the LLM breaks the task into multiple simple steps and completes them one by one. Before executing each edit step, the tool retrieves relevant materials from the knowledge base according to the step’s requirements to assist the LLM. |
| 48 | + |
| 49 | +#### Check Tool |
| 50 | + |
| 51 | +After each edit operation, the check tool verifies whether the change introduces serious errors or fails to achieve the step’s goal. If the check fails, the previous edit is rolled back. |
| 52 | + |
| 53 | +#### Short‑term Memory |
| 54 | + |
| 55 | +Each conversation records the current `sessionId`, allowing the same user to recall the state saved in previous turns and continue multi‑turn editing. |
| 56 | + |
| 57 | +### VMind Agent Knowledge Base |
| 58 | + |
| 59 | +To prevent hallucinations during editing, VMind Agent strictly limits the edit scope to the knowledge base fragments provided for each operation. The knowledge base consists of two parts: |
| 60 | + |
| 61 | +#### RAG |
| 62 | + |
| 63 | +Tutorial documents, demo examples, and Q&A content have strong independence and are well suited to a RAG approach. Relevant fragments are vectorized and stored in a vector database. For each edit operation, the system retrieves a set of related texts and code snippets for the model’s reference. When editing, if highly similar content is retrieved, the model can accurately perform the corresponding edits on the matched code snippets. |
| 64 | + |
| 65 | +#### Type Docs |
| 66 | + |
| 67 | +Configuration schemas for charts and tables are split by chart/table type and by module (e.g., axes, legends) into separate typed documentation fragments. Each fragment includes property names, types, and descriptions, and different fragments are indexed by different TopKeys. The LLM summarizes each fragment into Markdown and stores it in the database. For each TopKey, the LLM also produces a concise summary describing the scope covered by that TopKey. |
| 68 | + |
| 69 | +For each edit, the system first retrieves relevant TopKeys based on the instruction, then injects the associated Markdown fragments into the prompt. The LLM’s edits are strictly constrained to the range covered by the retrieved knowledge to avoid hallucination‑driven errors. |
| 70 | + |
| 71 | +## VMind Agent Applications |
| 72 | + |
| 73 | +### Playground |
| 74 | + |
| 75 | +VMind Agent Playground provides a simple testing environment to experience the agent’s capabilities. You can enter instructions and code blocks to see the generation and editing results. |
| 76 | + |
| 77 | +[VMind Agent Playground](/vmind/playground) |
| 78 | + |
| 79 | +### Douyin Mini Program |
| 80 | + |
| 81 | +Based on VMind Agent, we have developed a sample mini program. In Douyin mini programs, search for “数据智能洞察助手”. Users can input text for intelligent insights and experience the agent’s generation and editing results. |
| 82 | + |
| 83 | +<img src="https://lf9-dp-fe-cms-tos.byteorg.com/obj/bit-cloud/vmind/tutorials/VMind_Agent_Demo.jpeg" > |
| 84 | + |
| 85 | +## VMind OpenAPI Support |
| 86 | + |
| 87 | +Join our Feishu group to contact us and consult on VMind OpenAPI usage. |
| 88 | + |
| 89 | +<img src="https://lf9-dp-fe-cms-tos.byteorg.com/obj/bit-cloud/vmind/tutorials/open_api_lark_group.png" > |
| 90 | + |
0 commit comments