Skip to content

Commit 3327891

Browse files
authored
Merge pull request #293 from VisActor/doc/vmind-agent
docs: add VMind OpenAPI documentation and menu entries
2 parents 39c37b6 + 3475400 commit 3327891

6 files changed

Lines changed: 195 additions & 7 deletions

File tree

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# VMind OpenAPI
2+
3+
Built on the VMind chart generation tool, the VisActor team has launched VMind Agent, which improves the accuracy and efficiency of intelligent generation and adds multi‑turn editing capabilities. VMind Agent currently provides OpenAPI interfaces that you can call to achieve more advanced intelligent generation. The [VMind Agent Playground](/vmind/playground) offers a simple testing environment to experience VMind Agent’s capabilities.
4+
5+
## VSeed
6+
7+
VSeed is a domain‑specific language (DSL) for analytics. It encapsulates commonly used charting capabilities, greatly improving the efficiency of building analytic reporting platforms. In most cases, given a dataset and chart type, it can render directly; each chart type comes with extensive default conventions.
8+
9+
VSeed is a subset of VChart and VTable. Its configuration is greatly simplified compared with VChart and VTable, making the options easier for large models to understand. It is a chart library specifically packaged for LLM‑driven generation.
10+
11+
VSeed repository: [https://github.com/VisActor/VSeed](https://github.com/VisActor/VSeed)
12+
13+
VSeed showcase: [https://visactor.github.io/VSeed/galley/chartType/line.html](https://visactor.github.io/VSeed/galley/chartType/line.html)
14+
15+
VMind Agent prioritizes using VSeed for code generation for better results, while also supporting native code generation for VChart and VTable.
16+
17+
## VMind Agent
18+
19+
The overall technical approach of VMind Agent is as follows:
20+
21+
When the user provides instructions and code blocks to VMind Agent, the agent uses the LLM to actively call tools and proceeds step by step through Generate/Edit → Check → Edit → Check … until the user’s requirements are met, outputting the corresponding code blocks and explanatory text.
22+
23+
Compared with traditional chart generation tools, VMind Agent offers advantages in the following areas:
24+
- 🤖 ReAct architecture: Reasoning–action loop implemented with LangGraph
25+
- 🛠️ Diverse tools: Supports multiple chart generation and editing tools
26+
- 📊 Chart types: Supports many chart and table types
27+
- 🎨 Template system: Prebuilt chart templates
28+
- 🔧 Chart editing: Supports modifying existing charts
29+
- 💬 Natural language: Supports generating charts from pure natural‑language input
30+
31+
### VMind Agent Architecture
32+
33+
VMind Agent is composed of a Coordinator (scheduler) and a set of tools, and is implemented with the `@langchain/langgraph` library. The main modules are:
34+
35+
<img src="https://lf9-dp-fe-cms-tos.byteorg.com/obj/bit-cloud/vmind/tutorials/VMind_Agent_Design_1.png" >
36+
37+
#### Coordinator
38+
39+
The scheduler’s main job is to determine, based on the current state, which tool to call next or whether to end the task.
40+
41+
#### Generate Tool
42+
43+
The generation tool is invoked when the user has not provided code or explicitly asks to regenerate code. If no chart type is specified, the tool recommends a suitable chart type based on the data. Each chart or table type provides a basic template; the LLM uses the user’s data and the template to generate the simplest code block for subsequent editing.
44+
45+
#### Edit Tool
46+
47+
The editing tool is the core of VMind Agent. Based on the current code and the user’s goal, the LLM breaks the task into multiple simple steps and completes them one by one. Before executing each edit step, the tool retrieves relevant materials from the knowledge base according to the step’s requirements to assist the LLM.
48+
49+
#### Check Tool
50+
51+
After each edit operation, the check tool verifies whether the change introduces serious errors or fails to achieve the step’s goal. If the check fails, the previous edit is rolled back.
52+
53+
#### Short‑term Memory
54+
55+
Each conversation records the current `sessionId`, allowing the same user to recall the state saved in previous turns and continue multi‑turn editing.
56+
57+
### VMind Agent Knowledge Base
58+
59+
To prevent hallucinations during editing, VMind Agent strictly limits the edit scope to the knowledge base fragments provided for each operation. The knowledge base consists of two parts:
60+
61+
#### RAG
62+
63+
Tutorial documents, demo examples, and Q&A content have strong independence and are well suited to a RAG approach. Relevant fragments are vectorized and stored in a vector database. For each edit operation, the system retrieves a set of related texts and code snippets for the model’s reference. When editing, if highly similar content is retrieved, the model can accurately perform the corresponding edits on the matched code snippets.
64+
65+
#### Type Docs
66+
67+
Configuration schemas for charts and tables are split by chart/table type and by module (e.g., axes, legends) into separate typed documentation fragments. Each fragment includes property names, types, and descriptions, and different fragments are indexed by different TopKeys. The LLM summarizes each fragment into Markdown and stores it in the database. For each TopKey, the LLM also produces a concise summary describing the scope covered by that TopKey.
68+
69+
For each edit, the system first retrieves relevant TopKeys based on the instruction, then injects the associated Markdown fragments into the prompt. The LLM’s edits are strictly constrained to the range covered by the retrieved knowledge to avoid hallucination‑driven errors.
70+
71+
## VMind Agent Applications
72+
73+
### Playground
74+
75+
VMind Agent Playground provides a simple testing environment to experience the agent’s capabilities. You can enter instructions and code blocks to see the generation and editing results.
76+
77+
[VMind Agent Playground](/vmind/playground)
78+
79+
### Douyin Mini Program
80+
81+
Based on VMind Agent, we have developed a sample mini program. In Douyin mini programs, search for “数据智能洞察助手”. Users can input text for intelligent insights and experience the agent’s generation and editing results.
82+
83+
<img src="https://lf9-dp-fe-cms-tos.byteorg.com/obj/bit-cloud/vmind/tutorials/VMind_Agent_Demo.jpeg" >
84+
85+
## VMind OpenAPI Support
86+
87+
Join our Feishu group to contact us and consult on VMind OpenAPI usage.
88+
89+
<img src="https://lf9-dp-fe-cms-tos.byteorg.com/obj/bit-cloud/vmind/tutorials/open_api_lark_group.png" >
90+

docs/assets/guide/en/VMind_Website_Guide.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,10 @@
33

44
The purpose of this article is to help you use the VMind website more effectively and quickly find the information you need.
55

6+
## VMind OpenAPI
7+
8+
VMind provides OpenAPI interfaces that allow you to achieve more advanced intelligent generation capabilities by calling these interfaces. You can learn more about OpenAPI in the [VMind OpenAPI](./VMind_OpenAPI) section.
9+
610
## Quick Start
711

812
### Getting Started

docs/assets/guide/menu.json

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,13 @@
88
"en": "VMind Website Guide"
99
}
1010
},
11+
{
12+
"path": "VMind_OpenAPI",
13+
"title": {
14+
"zh": "VMind OpenAPI",
15+
"en": "VMind OpenAPI"
16+
}
17+
},
1118
{
1219
"path": "Intro_to_VMind",
1320
"title": {
Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# VMind OpenAPI
2+
3+
在VMind图表生成工具的基础上,VisActor团队全新推出了VMind Agent,提升了智能生成的准确性和效率,并新增了多轮编辑能力。目前VMind Agent提供了OpenAPI接口,你可以通过调用这些接口来实现更高级的智能生成能力。[VMind Agent Playground](/vmind/playground)提供了一个简单的测试环境,方便体验VMind Agent的能力。
4+
5+
## VSeed
6+
7+
VSeed是一个面向分析领域的DSL, 封装了该领域常用的图表能力, 极大地提升分析式报表平台的搭建效率。一般情况下, 给定数据集和图表类型, 即可直接出图,每类图表类型存在大量的默认约定逻辑。
8+
9+
VSeed是VChart和VTable的子集,配置相比VChart和VTable大幅简化,配置项更适合大模型理解,是专为LLM生成封装的图表库。
10+
11+
VSeed 项目仓库:[https://github.com/VisActor/VSeed](https://github.com/VisActor/VSeed)
12+
13+
VSeed 能力展示:[https://visactor.github.io/VSeed/galley/chartType/line.html](https://visactor.github.io/VSeed/galley/chartType/line.html)
14+
15+
VMind Agent优先推荐使用VSeed进行代码生成,生成效果更加优秀,也同时支持VChart和VTable的原生代码生成。
16+
17+
## VMind Agent
18+
19+
VMind Agent的整体技术方案如下:
20+
21+
用户输入的指令和代码块,输入VMind Agent后,Agent会基于LLM主动调用工具,开始逐步进行 生成/编辑 -> 检查 -> 编辑 -> 检查 ...... 任务流,最终完成用户的要求,输出对应的代码块和文本解释说明。
22+
23+
相比传统的图表生成工具,VMind Agent在一下方面有优势:
24+
- 🤖 ReAct 架构: 基于 LangGraph 实现的推理-行动循环
25+
- 🛠️ 多样化工具: 支持多种图表生成和编辑工具
26+
- 📊 图表类型: 支持多种图表类型和表格类型
27+
- 🎨 模板系统: 预置多种图表模板
28+
- 🔧 图表编辑: 支持对现有图表进行修改
29+
- 💬 自然语言: 支持纯自然语言输入生成图表
30+
31+
### VMind Agent 架构
32+
33+
VMind Agent 主要由Coordinator调度器和若干工具组成,基于`@langchain/langgraph库`完成Agent的实现。主要模块如下:
34+
35+
<img src="https://lf9-dp-fe-cms-tos.byteorg.com/obj/bit-cloud/vmind/tutorials/VMind_Agent_Design_1.png" >
36+
37+
#### Coordinator
38+
39+
调度器主要工作是:基于当前状态,判断下一步应该调用哪一个工具,或结束任务。
40+
41+
#### Generate Tool
42+
43+
生成工具在用户没有输入代码,或指令要求重新生成代码时调用。如果没有指定图表类型,工具会基于数据推荐适合的图表类型。每一种图表/表格类型都会提供基础的模版,LLM会基于用户的数据和模版生成最简单的代码块,供进一步编辑使用。
44+
45+
#### Edit Tool
46+
47+
编辑工具是VMind Agent的核心部分,LLM会基于当前的代码和用户的目标,将任务拆分为多个简单的步骤并逐一完成。在执行每次编辑任务前,工具会基于本次步骤的任务要求,在知识库中召回对应的资料,辅助LLM进行编辑。
48+
49+
#### Check Tool
50+
51+
检查工具在每一步编辑操作后执行,用来检查当次编辑操作是否会引起恶性错误,或是否没有达到当次步骤的目标。如果检查不通过,会回退上一次编辑操作。
52+
53+
#### 短期记忆
54+
55+
每一次对话会记录当前的sessionId,方便同一个用户进行多轮对话时,找回之前轮次的对话所记录的状态,方便用户进行多轮编辑操作。
56+
57+
### VMind Agent 知识库
58+
59+
为了避免LLM在编辑代码时出现幻觉,VMind Agent将编辑范围严格限制在每次编辑操作提供的知识库片段中。知识库主要分为两部分:
60+
61+
#### RAG
62+
63+
教程文档、示例Demo和Q&A问答内容,内容独立性强,适合使用RAG方案。相应片段进行向量化后,存入向量数据库。每次进行编辑操作时,会召回对应的若干文本及代码片段,供大模型参考。
64+
大模型进行编辑时,如果召回有非常接近的内容,会已经对应的代码片段准确进行相应的编辑。
65+
66+
#### Type Docs
67+
68+
图表和表格的配置Schema,按照不同的图表/表格类型,以及不同的模块(例如坐标轴、图例等)进行拆分,组织成一个个类型说明片段。每个片段中有该部分对应的属性名称、属性类型和属性说明,不同的片段使用不同的TopKey进行索引。使用LLM将每个每个片段进行归纳总结,生成对应的说明Markdown片段,存储在数据库中。每一个TopKey也依据对应的Markdown片段,使用LLM总结一段简单的说明,介绍这个TopKey下的功能范围。
69+
70+
每次编辑时,首先依据编辑指令召回对应的若干TopKey,之后将TopKey对应的Markdown文本片段,作为知识库加入prompt中,大模型的编辑操作会被严格限制在本次召回的知识库的范围,避免幻觉影响编辑结果。
71+
72+
## VMind Agent 应用
73+
74+
### Playground
75+
76+
VMind Agent Playground提供了一个简单的测试环境,方便体验VMind Agent的能力。你可以在Playground中输入指令和代码块,查看Agent的生成和编辑结果。
77+
78+
[VMind Agent Playground](/vmind/playground)
79+
80+
### 抖音小程序
81+
82+
基于VMind Agent的,我们开发了一个示例小程序,抖音小程序搜索 数据智能洞察助手 对话小程序,用户可以在小程序中输入需要进行智能洞察的文本,体验Agent的生成和编辑结果。
83+
84+
<img src="https://lf9-dp-fe-cms-tos.byteorg.com/obj/bit-cloud/vmind/tutorials/VMind_Agent_Demo.jpeg" >
85+
86+
## VMind OpenAPI 使用咨询
87+
88+
加入飞书群欢迎联系我们,咨询VMind OpenAPI的使用方案。
89+
90+
<img src="https://lf9-dp-fe-cms-tos.byteorg.com/obj/bit-cloud/vmind/tutorials/open_api_lark_group.png" >

docs/assets/guide/zh/VMind_Website_Guide.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,10 @@
33

44
本篇文章的目的是为了帮助你更有效地使用 VMind 网站,让你能更快地找到你需要的信息。
55

6+
## VMind OpenAPI
7+
8+
VMind 提供了 OpenAPI 接口,你可以通过调用这些接口来实现更高级的智能生成能力。你可以在[VMind OpenAPI](./VMind_OpenAPI)章节了解更多关于 OpenAPI 的信息。
9+
610
## 快速开始
711

812
### 快速上手

docs/assets/scenario/menu.json

Lines changed: 0 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -42,13 +42,6 @@
4242
"zh": "数据开发",
4343
"en": "Data Development"
4444
}
45-
},
46-
{
47-
"path": "Open_API",
48-
"title": {
49-
"zh": "VMind Open API",
50-
"en": "VMind Open API"
51-
}
5245
}
5346
]
5447
}

0 commit comments

Comments
 (0)