Skip to content

Commit 5971bde

Browse files
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
1 parent 8202313 commit 5971bde

30 files changed

Lines changed: 229 additions & 158 deletions

EdgeCraftRAG/docker_compose/intel/gpu/arc/README.md

Lines changed: 16 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -22,19 +22,22 @@ This section describes how to quickly deploy and test the EdgeCraftRAG service m
2222

2323
### 1. Prerequisites
2424

25-
EC-RAG supports vLLM deployment(default method) and local OpenVINO deployment for Intel Arc GPU and Core Ultra Platform. Prerequisites are shown as below:
25+
EC-RAG supports vLLM deployment(default method) and local OpenVINO deployment for Intel Arc GPU and Core Ultra Platform. Prerequisites are shown as below:
2626

2727
#### Core Ultra
28+
2829
**OS**: Ubuntu 24.04 or newer
2930
**Driver & libraries**: Please refer to [Installing Client GPUs on Ubuntu Desktop](https://dgpu-docs.intel.com/driver/client/overview.html#installing-client-gpus-on-ubuntu-desktop)
3031
**Available Inferencing Framework**: openVINO
3132

3233
#### Intel Arc B60
33-
**OS**: Ubuntu 25.04 Desktop (for Core Ultra and Xeon-W), Ubuntu 25.04 Server (for Xeon-SP).
34+
35+
**OS**: Ubuntu 25.04 Desktop (for Core Ultra and Xeon-W), Ubuntu 25.04 Server (for Xeon-SP).
3436
**Driver & libraries**: Please refer to [Install Bare Metal Environment](https://github.com/intel/llm-scaler/tree/main/vllm#11-install-bare-metal-environment) for detailed setup
3537
**Available Inferencing Framework**: openVINO, vLLM
3638

3739
#### Intel Arc A770
40+
3841
**OS**: Ubuntu Server 22.04.1 or newer (at least 6.2 LTS kernel)
3942
**Driver & libraries**: Please refer to [Installing GPUs Drivers](https://dgpu-docs.intel.com/driver/installation-rolling.html#installing-gpu-drivers) for detailed driver & libraries setup
4043
**Available Inferencing Framework**: openVINO, vLLM
@@ -50,9 +53,9 @@ cd GenAIExamples/EdgeCraftRAG
5053

5154
> **NOTE**: If you want to checkout a released version, such as v1.5:
5255
>
53-
>```
54-
>git checkout v1.5
55-
>```
56+
> ```
57+
> git checkout v1.5
58+
> ```
5659
5760
### 3. Prepare models
5861
@@ -75,6 +78,7 @@ optimum-cli export openvino -m BAAI/bge-reranker-large ${MODEL_PATH}/BAAI/bge-re
7578
#### LLM
7679

7780
##### openVINO
81+
7882
If you have Core Ultra platform only, please prepare openVINO models:
7983
You can also run openVINO models on discrete GPU.
8084

@@ -84,6 +88,7 @@ optimum-cli export openvino --model Qwen/Qwen3-8B ${MODEL_PATH}/Qwen/Qwen3-8B/IN
8488
```
8589

8690
##### vLLM
91+
8792
Alternatively, if you have discrete GPU and want to use vLLM, please prepare models for vLLM:
8893

8994
```bash
@@ -141,20 +146,23 @@ export MILVUS_ENABLED=0
141146

142147
#### Option a. Deploy openVINO LLM based EC-RAG for Core Ultra, Arc B60, Arc A770
143148

144-
Make sure you have prepared [openVINO models](#openvino)
149+
Make sure you have prepared [openVINO models](#openvino)
150+
145151
```bash
146152
docker compose -f docker_compose/intel/gpu/arc/compose.yaml up -d
147153
```
148154

149155
#### Option b.1. Deploy vLLM based EC-RAG for Arc B60
150-
Make sure you have prepared [vLLM models](#vllm)
156+
157+
Make sure you have prepared [vLLM models](#vllm)
151158

152159
```bash
153160
docker compose --profile b60 -f docker_compose/intel/gpu/arc/compose.yaml up -d
154161
```
155162

156163
#### Option b.2. Deploy vLLM based EC-RAG for Arc A770
157-
Make sure you have prepared [vLLM models](#vllm)
164+
165+
Make sure you have prepared [vLLM models](#vllm)
158166

159167
```bash
160168
docker compose --profile a770 -f docker_compose/intel/gpu/arc/compose.yaml up -d

EdgeCraftRAG/docker_compose/intel/gpu/arc/README_zh.md

Lines changed: 16 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -25,16 +25,19 @@
2525
EC-RAG 支持 vLLM 部署(默认方式)以及面向 Intel Arc GPU 和 Core Ultra 平台的本地 OpenVINO 部署。前置条件如下:
2626

2727
#### Core Ultra
28+
2829
**操作系统**:Ubuntu 24.04 或更高版本
2930
**驱动与库**:请参考 [Installing Client GPUs on Ubuntu Desktop](https://dgpu-docs.intel.com/driver/client/overview.html#installing-client-gpus-on-ubuntu-desktop)
3031
**可用推理框架**:openVINO
3132

3233
#### Intel Arc B60
34+
3335
**操作系统**:Ubuntu 25.04 Desktop(适用于 Core Ultra 和 Xeon-W),Ubuntu 25.04 Server(适用于 Xeon-SP)。
3436
**驱动与库**:详细安装请参考 [Install Bare Metal Environment](https://github.com/intel/llm-scaler/tree/main/vllm#11-install-bare-metal-environment)
3537
**可用推理框架**:openVINO、vLLM
3638

3739
#### Intel Arc A770
40+
3841
**操作系统**:Ubuntu Server 22.04.1 或更高版本(至少 6.2 LTS 内核)
3942
**驱动与库**:详细驱动与库安装请参考 [Installing GPUs Drivers](https://dgpu-docs.intel.com/driver/installation-rolling.html#installing-gpu-drivers)
4043
**可用推理框架**:openVINO、vLLM
@@ -50,9 +53,9 @@ cd GenAIExamples/EdgeCraftRAG
5053

5154
> **注意**:如果你想切换到某个发布版本,例如 v1.5:
5255
>
53-
>```
54-
>git checkout v1.5
55-
>```
56+
> ```
57+
> git checkout v1.5
58+
> ```
5659
5760
### 3. 准备模型
5861
@@ -75,6 +78,7 @@ optimum-cli export openvino -m BAAI/bge-reranker-large ${MODEL_PATH}/BAAI/bge-re
7578
#### LLM
7679

7780
##### openVINO
81+
7882
如果你只有 Core Ultra 平台,请准备 openVINO 模型:
7983
你也可以在独立 GPU 上运行 openVINO 模型。
8084

@@ -84,6 +88,7 @@ optimum-cli export openvino --model Qwen/Qwen3-8B ${MODEL_PATH}/Qwen/Qwen3-8B/IN
8488
```
8589

8690
##### vLLM
91+
8792
另外,如果你有独立显卡,可以为 vLLM 准备模型:
8893

8994
```bash
@@ -199,11 +204,11 @@ docker compose -f docker_compose/intel/gpu/arc/compose.yaml down
199204

200205
下表全面概述了示例 Docker Compose 文件中各类部署所使用的 EdgeCraftRAG 服务。表中每一行代表一个独立服务,详细说明了可用镜像及其在部署架构中的功能描述。
201206

202-
| 服务名称 | 可选镜像名称 | 可选 | 描述 |
203-
| ------------------- | ---------------------------------------- | ---- | ------------------------------------------------------------------------------------------------ |
204-
| etcd | quay.io/coreos/etcd:v3.5.5 || 提供分布式键值存储,用于服务发现和配置管理。 |
205-
| minio | minio/minio:RELEASE.2023-03-20T20-16-18Z || 提供对象存储服务,用于存储文档和模型文件。 |
206-
| milvus-standalone | milvusdb/milvus:v2.4.6 || 提供向量数据库能力,用于管理 embedding 和相似度检索。 |
207-
| edgecraftrag-server | opea/edgecraftrag-server:latest || 作为 EdgeCraftRAG 服务后端,具体形态随部署方式不同而变化。 |
208-
| edgecraftrag-ui | opea/edgecraftrag-ui:latest || 提供 EdgeCraftRAG 服务的用户界面。 |
209-
| ecrag | opea/edgecraftrag:latest || 作为反向代理,管理 UI 与后端服务之间的流量。 |
207+
| 服务名称 | 可选镜像名称 | 可选 | 描述 |
208+
| ------------------- | ---------------------------------------- | ---- | ---------------------------------------------------------- |
209+
| etcd | quay.io/coreos/etcd:v3.5.5 || 提供分布式键值存储,用于服务发现和配置管理。 |
210+
| minio | minio/minio:RELEASE.2023-03-20T20-16-18Z || 提供对象存储服务,用于存储文档和模型文件。 |
211+
| milvus-standalone | milvusdb/milvus:v2.4.6 || 提供向量数据库能力,用于管理 embedding 和相似度检索。 |
212+
| edgecraftrag-server | opea/edgecraftrag-server:latest || 作为 EdgeCraftRAG 服务后端,具体形态随部署方式不同而变化。 |
213+
| edgecraftrag-ui | opea/edgecraftrag-ui:latest || 提供 EdgeCraftRAG 服务的用户界面。 |
214+
| ecrag | opea/edgecraftrag:latest || 作为反向代理,管理 UI 与后端服务之间的流量。 |

EdgeCraftRAG/edgecraftrag/api/v1/data.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,13 @@
55
import os
66
from typing import List
77

8+
from edgecraftrag.api.v1.knowledge_base import add_file_to_knowledge_base
89
from edgecraftrag.api_schema import DataIn, FilesIn
910
from edgecraftrag.config_repository import MilvusConfigRepository
1011
from edgecraftrag.context import ctx
1112
from edgecraftrag.env import UI_DIRECTORY
1213
from fastapi import FastAPI, File, HTTPException, UploadFile, status
13-
from edgecraftrag.api.v1.knowledge_base import add_file_to_knowledge_base
14+
1415
data_app = FastAPI()
1516

1617

@@ -24,7 +25,7 @@ async def get_nodes_with_kb(kb_name=None):
2425
kb = ctx.get_knowledge_mgr().get_active_knowledge_base()
2526
if kb.indexer.comp_subtype == "faiss_vector":
2627
return kb.indexer.docstore.docs
27-
elif kb.indexer.comp_subtype == "milvus_vector":
28+
elif kb.indexer.comp_subtype == "milvus_vector":
2829
collection_name = kb.name
2930
Milvus_node_list = MilvusConfigRepository.create_connection(collection_name, 1, kb.indexer.vector_url)
3031
results = Milvus_node_list.get_configs(output_fields=["text", "_node_content", "doc_id"])

EdgeCraftRAG/edgecraftrag/api/v1/knowledge_base.py

Lines changed: 30 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -8,34 +8,33 @@
88
from typing import Dict, List, Union
99

1010
from edgecraftrag.api_schema import DataIn, ExperienceIn, KnowledgeBaseCreateIn
11-
from edgecraftrag.components.query_preprocess import query_search
12-
from edgecraftrag.components.indexer import get_kbs_info
13-
from edgecraftrag.config_repository import (
14-
MilvusConfigRepository,
15-
save_knowledge_configurations,
16-
)
17-
from edgecraftrag.context import ctx
18-
from edgecraftrag.env import (
19-
KNOWLEDGEBASE_FILE,
20-
SEARCH_CONFIG_PATH,
21-
SEARCH_DIR,
22-
UI_DIRECTORY,
23-
)
2411
from edgecraftrag.base import (
2512
IndexerType,
2613
ModelType,
2714
NodeParserType,
2815
)
2916
from edgecraftrag.components.benchmark import Benchmark
30-
from edgecraftrag.components.indexer import KBADMINIndexer, VectorIndexer
17+
from edgecraftrag.components.indexer import KBADMINIndexer, VectorIndexer, get_kbs_info
3118
from edgecraftrag.components.node_parser import (
3219
HierarchyNodeParser,
3320
KBADMINParser,
3421
SimpleNodeParser,
3522
SWindowNodeParser,
3623
UnstructedNodeParser,
3724
)
38-
from fastapi import FastAPI, HTTPException, status, Query
25+
from edgecraftrag.components.query_preprocess import query_search
26+
from edgecraftrag.config_repository import (
27+
MilvusConfigRepository,
28+
save_knowledge_configurations,
29+
)
30+
from edgecraftrag.context import ctx
31+
from edgecraftrag.env import (
32+
KNOWLEDGEBASE_FILE,
33+
SEARCH_CONFIG_PATH,
34+
SEARCH_DIR,
35+
UI_DIRECTORY,
36+
)
37+
from fastapi import FastAPI, HTTPException, Query, status
3938

4039
kb_app = FastAPI()
4140

@@ -51,7 +50,9 @@ async def get_all_knowledge_bases():
5150

5251
# Get knowledge base files in a certain range.
5352
@kb_app.get("/v1/knowledge/{knowledge_name}/filemap")
54-
async def get_knowledge_base_filemap(knowledge_name: str, page_num: int = Query(1, ge=1), page_size: int = Query(20, ge=1)):
53+
async def get_knowledge_base_filemap(
54+
knowledge_name: str, page_num: int = Query(1, ge=1), page_size: int = Query(20, ge=1)
55+
):
5556
kb = ctx.knowledgemgr.get_knowledge_base_by_name_or_id(knowledge_name)
5657
if kb and kb.file_map:
5758
file_map = kb.file_map
@@ -61,7 +62,7 @@ async def get_knowledge_base_filemap(knowledge_name: str, page_num: int = Query(
6162
if start >= filemap_len:
6263
return None
6364
file_map_subset = itertools.islice(file_map.items(), start, end)
64-
return {"file_map": dict(file_map_subset),"total": kb.calculate_totals()}
65+
return {"file_map": dict(file_map_subset), "total": kb.calculate_totals()}
6566
else:
6667
return None
6768

@@ -101,7 +102,7 @@ async def create_knowledge_base(knowledge: KnowledgeBaseCreateIn):
101102
active_pl.update_retriever_list(ctx.knowledgemgr.get_active_knowledge_base())
102103
except Exception as e:
103104
ctx.knowledgemgr.delete_knowledge_base(knowledge.name)
104-
raise HTTPException(status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, detail=str(e))
105+
raise HTTPException(status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, detail=str(e))
105106
await save_knowledge_configurations("add", kb)
106107
return "Create knowledge base successfully"
107108
except Exception as e:
@@ -156,7 +157,7 @@ async def update_knowledge_base(knowledge: KnowledgeBaseCreateIn):
156157
raise HTTPException(status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, detail=str(e))
157158

158159
# reload data for knowledge base
159-
node_parser_changed = (kb_node_parser != kb.node_parser)
160+
node_parser_changed = kb_node_parser != kb.node_parser
160161
if node_parser_changed or kb_indexer != kb.indexer:
161162
await handle_reload_data(kb, node_parser_changed)
162163
elif kb.comp_subtype == "kbadmin_kb":
@@ -176,10 +177,10 @@ async def update_knowledge_base(knowledge: KnowledgeBaseCreateIn):
176177
@kb_app.post(path="/v1/knowledge/{knowledge_name}/files")
177178
async def add_file_to_knowledge_base(knowledge_name, file_path: DataIn):
178179
"""
179-
1. Parse file into Llamaindex Document and add file to filemgr
180-
2. Add file path to knowledge base
181-
3. Update nodes and vector store for knowledge base
182-
4. Update pipeline retriever if active knowledge base's indexer changed
180+
1. Parse file into Llamaindex Document and add file to filemgr
181+
2. Add file path to knowledge base
182+
3. Update nodes and vector store for knowledge base
183+
4. Update pipeline retriever if active knowledge base's indexer changed
183184
"""
184185
try:
185186
kb = ctx.knowledgemgr.get_knowledge_base_by_name_or_id(knowledge_name)
@@ -222,7 +223,7 @@ async def add_file_to_knowledge_base(knowledge_name, file_path: DataIn):
222223
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Error uploading file.")
223224

224225
# update retriever with indexer since indexer updated
225-
if kb.active:
226+
if kb.active:
226227
active_pl = ctx.get_pipeline_mgr().get_active_pipeline()
227228
if active_pl:
228229
active_pl.update_retriever(kb, prev_indexer)
@@ -258,7 +259,7 @@ async def remove_file_from_knowledge_base(knowledge_name, file_path: DataIn):
258259
)
259260
await remove_document_handler(document_list, kb)
260261
# update retriever with indexer since indexer updated
261-
if kb.active:
262+
if kb.active:
262263
active_pl = ctx.get_pipeline_mgr().get_active_pipeline()
263264
if active_pl:
264265
active_pl.update_retriever(kb, prev_indexer)
@@ -459,6 +460,7 @@ async def handle_reload_data(kb, node_parser_changed: bool = False):
459460
# update indexer
460461
await kb.update_nodes_to_indexer()
461462

463+
462464
async def update_kb_handler(kb, knowledge):
463465
if kb.enable_benchmark:
464466
kb.benchmark = Benchmark(True, "")
@@ -493,7 +495,9 @@ async def update_kb_handler(kb, knowledge):
493495
ctx.get_node_parser_mgr().add(kb.node_parser)
494496
if knowledge.indexer is not None:
495497
ind = knowledge.indexer
496-
found_indexer = ctx.get_indexer_mgr().search_indexer(ind) if ind.indexer_type != IndexerType.MILVUS_VECTOR else None
498+
found_indexer = (
499+
ctx.get_indexer_mgr().search_indexer(ind) if ind.indexer_type != IndexerType.MILVUS_VECTOR else None
500+
)
497501
if found_indexer is not None:
498502
kb.indexer = found_indexer
499503
else:

EdgeCraftRAG/edgecraftrag/api/v1/pipeline.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,6 @@
1717
from edgecraftrag.components.benchmark import Benchmark
1818
from edgecraftrag.components.generator import FreeChatGenerator, QnAGenerator
1919
from edgecraftrag.components.postprocessor import MetadataReplaceProcessor, RerankProcessor
20-
2120
from edgecraftrag.config_repository import MilvusConfigRepository, save_pipeline_configurations
2221
from edgecraftrag.context import ctx
2322
from edgecraftrag.env import PIPELINE_FILE
@@ -217,7 +216,9 @@ async def update_pipeline_handler(pl, req):
217216
raise Exception("Inference Type Not Supported")
218217

219218
if pl.status.active != req.active:
220-
ctx.get_pipeline_mgr().activate_pipeline(pl.name, req.active, ctx.get_knowledge_mgr().get_active_knowledge_base())
219+
ctx.get_pipeline_mgr().activate_pipeline(
220+
pl.name, req.active, ctx.get_knowledge_mgr().get_active_knowledge_base()
221+
)
221222
return pl
222223

223224

EdgeCraftRAG/edgecraftrag/components/indexer.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,14 +6,15 @@
66
import faiss
77
from edgecraftrag.base import BaseComponent, CompType, IndexerType
88
from edgecraftrag.context import ctx
9+
from langchain_milvus import Milvus
910
from langchain_openai import OpenAIEmbeddings
1011
from llama_index.core import StorageContext, VectorStoreIndex
1112
from llama_index.vector_stores.faiss import FaissVectorStore
1213
from llama_index.vector_stores.milvus import MilvusVectorStore
1314
from pydantic import model_serializer
14-
from langchain_milvus import Milvus
1515
from pymilvus import Collection, MilvusException, connections, utility
1616

17+
1718
class VectorIndexer(BaseComponent, VectorStoreIndex):
1819
def __init__(self, embed_model, vector_type, vector_url="http://localhost:19530", kb_name="default_kb"):
1920
BaseComponent.__init__(

EdgeCraftRAG/edgecraftrag/components/knowledge_base.py

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@
33

44
import json
55
import os
6-
import uuid
76
import time
7+
import uuid
88
from typing import Any, Dict, List, Optional, Union
99

1010
from edgecraftrag.base import BaseComponent, BenchType, CompType
@@ -16,6 +16,7 @@
1616
from llama_index.core.schema import Document
1717
from pydantic import Field, model_serializer
1818

19+
1920
class Knowledge(BaseComponent):
2021

2122
node_parser: Optional[BaseComponent] = Field(default=None)
@@ -347,7 +348,7 @@ def calculate_totals(self):
347348
else:
348349
total = None
349350
return total
350-
351+
351352
def update_nodes(self, nodes: List[Document]):
352353
self.nodes = nodes
353354

@@ -370,7 +371,7 @@ async def run_node_parser(self, docs: List[Document]) -> Any:
370371
self.benchmark.update_benchmark_data(benchmark_index, BenchType.CHUNK_NUM, benchmark_data)
371372
self.add_nodes(nodes)
372373
return nodes
373-
374+
374375
async def update_nodes_to_indexer(self) -> Any:
375376
if self.indexer is not None:
376377
self.indexer.insert_nodes(self.nodes)

0 commit comments

Comments
 (0)