doc: add docs

Author: AsterDY

README.md

@@ -1,34 +1,22 @@
-<!--
- Copyright 2025 CloudWeGo Authors
- 
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
- 
-     https://www.apache.org/licenses/LICENSE-2.0
- 
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
 # ABCoder: AI-Based Coder(AKA: A Brand-new Coder)
 
 ![ABCoder](images/ABCoder.png)
 
-ABCoder, an AI-oriented code-processing tool, is designed to enhance coding-context for Large-Language-Model (LLM), simplify AI-assisted-coding process.
+ABCoder, an AI-oriented code-processing SDK, is designed to enhance coding context for Large-Language-Model (LLM), and boost developing AI-assisted-coding workflow.
 
 ## Features
 
--  Universal Abstract Syntax Tree (UniAST), an language-independent and AI-friendly  coding-context AST specfication, providing ample and recursive code information for both AI and hunman.
+-  Universal Abstract Syntax Tree (UniAST), an language-independent, AI-friendly code-struct specfication, providing flexible and structrual coding-context for both AI and hunman.
 
 -  Universal Parser, parses abitary languages to UniAST.
 
 -  Univeral Writer, transforms UniAST back to codes.
+  
+- (Comming Soon) Univeral Iterator, provides a set of interfaces and tools to help developers to implement their agents without deep knowledge of the UniAST structure.
+
+- (Comming Soon) Code RAG, provides a set of tools to help the LLM understand your codes much deeper than ever.
 
-Based on these features, developers can easily implement or enhance their AI-assisted-coding agent or workflow, such as reviewing, optimizing, translating...
+Based on these features, developers can easily implement or enhance their AI-assisted-coding workflows (or agents), such as reviewing, optimizing, translating...
 
 ## Getting Started
 
@@ -67,7 +55,6 @@ ABCoder currently supports the following languages:
 
 We encourage developers to contribute and make this tool more powerful. If you are interested in contributing to ABCoder
 project, kindly check out our Getting Involved Guide:
-- [Parser Extension](docs/parser_extension-zh.md)
-- [Writer Extension](docs/writer_extension-zh.md)
+- [Parser Extension](docs/parser-zh.md)
 
 > Note: This is a dynamic README and is subject to changes as the project evolves.

docs/parser-zh.md

@@ -0,0 +1,103 @@
+# ABCoder - Language Parser 介绍
+
+当前 ABCoder 基于 [LSP](https://microsoft.github.io/language-server-protocol/) 协议实现 Parser ，以达到精确依赖收集，并方便后续多语言扩展
+
+## 代码结构
+
+位于 [lang](/lang) 包下，包括：
+
+- uniast：统一 AST 结构的 golang 定义
+- lsp：LSP 协议处理 client，提供了 文件解析、引用查找、语法树解析、定义查找等接口，以及**通用的语言规范 LanguageSpec 接口**
+- collect：负责基于 LSP 符号收集和导出 UniAST，是核心运算逻辑
+- rust：主体是对 lsp#Spec 接口的 rust 语言规范实现。此外还有具体 lsp（rust-analyzer）的一些具体调用逻辑，以及针对按 id 查找的实现
+- go: go 的 parser 和 writer 实现
+
+## 运算过程
+
+![lang-parser](../images/lang-parser.png)
+
+1. 通过命令行参数识别语言启动对应 lsp server，并传入初始化参数
+2. 遍历仓库文件，调用 `textDocument/documentSymbol` 方法获取每个文件的所有符号。对于每个符号
+   1. 调用 `textDocument/semanticTokens/range` 方法获取符号代码中的 tokens
+   2. 识别出有效实体的 token，调用 `textDocument/definition` 跳转到对应符号位置，从而建立节点依赖关系
+3. 循环 2 直文件结束。最终将收集到的 lsp symbols 转换为 UniAST 格式并输出
+
+## 扩展其它语言实现
+
+由于 UniAST 并不完全等价 LSP， 因此需要实现一些特定语言专属的行为接口才能进行转换。参考 lang/rust 包，大体需要实现以下能力：
+
+- GetDefaultLSP()：映射用户输入 language 到具体的 lsp.Language，以及对应的 lsp 名称
+- CheckRepo()：检查用户仓库情况，根据各语言规范额处理工具链等问题，并返回默认打开的第一个文件（用于触发 lsp-server），以及等候 sever 初始化完成的时间（根据仓库大小来决定）
+- **LanguageSpec interface**: 核心模块，用于处理非 LSP 通用的语法信息、比如判断一个 token 是否是标准库的符号、函数签名解析等：
+- ModulePatcher: 后处理模块，用于处理语言特殊的信息收集。比如 rust 的 use 符号收集（lsp 不收集）。可以不实现
+
+### LaunguageSpec
+
+```
+用于在 lsp 符号收集过程中转换为 UniAST 所需信息，并且这些信息非 LSP 通用定义
+
+```go
+
+// Detailed implementation used for collect LSP symbols and transform them to UniAST
+type LanguageSpec interface {
+    // initialize a root workspace, and return all modules [modulename=>abs-path] inside
+    WorkSpace(root string) (map[string]string, error)
+
+    // give an absolute file path and returns its module name and package path
+    // external path should alse be supported
+    // FIXEM: some language (like rust) may have sub-mods inside a file, but we still consider it as a unity mod here
+    NameSpace(path string) (string, string, error)
+
+    // tells if a file belang to language AST
+    ShouldSkip(path string) bool
+
+    // return the first declaration token of a symbol, as Type-Name
+    DeclareTokenOfSymbol(sym DocumentSymbol) int
+
+    // tells if a token is an AST entity
+    IsEntityToken(tok Token) bool
+
+    // tells if a token is a std token
+    IsStdToken(tok Token) bool
+
+    // return the SymbolKind of a token
+    TokenKind(tok Token) SymbolKind
+
+    // tells if a symbol is a main function
+    IsMainFunction(sym DocumentSymbol) bool
+
+    // tells if a symbol is a language symbol (func, type, variable, etc) in workspace
+    IsEntitySymbol(sym DocumentSymbol) bool
+
+    // tells if a symbol is public in workspace
+    IsPublicSymbol(sym DocumentSymbol) bool
+
+    // declare if the language has impl symbol
+    // if it return true, the ImplSymbol() will be called
+    HasImplSymbol() bool
+    // if a symbol is an impl symbol, return the token index of interface type, receiver type and first-method start (-1 means not found)
+    // ortherwise the collector will use FunctionSymbol() as receiver type token index (-1 means not found)
+    ImplSymbol(sym DocumentSymbol) (int, int, int)
+
+    // if a symbol is a Function or Method symbol,  return the token index of Receiver (-1 means not found),TypeParameters, InputParameters and Outputs
+    FunctionSymbol(sym DocumentSymbol) (int, []int, []int, []int)
+}
+```
+
+- Rust-parser 实现位置：[RustSpec](/lang/rust/spec.go)
+
+```
+
+### ModulePatcher
+
+用于后处理收集完成的模块信息
+
+```go
+// ModulePatcher supplements some information for module
+type ModulePatcher interface {
+    // Patch is called after collect all symbols
+    Patch(ast *parse.Module)
+}
+```
+
+- Rust-parser 实现: [RustModulePatcher](/lang/rust/patch.go)