Use COS in production and require PGroonga

Author: wang-h

README.md

@@ -59,13 +59,13 @@ DeepTrans Studio is described in our CSCW '26 Companion Demo (CCF-A) paper. If y
 - **Multi-Engine Support**: Integration with OpenAI and custom AI models
 - **Terminology Extraction**: Automated domain-specific term extraction
 - **Quality Assessment**: AI-driven grammar, syntax, and discourse evaluation
-- **Translation Memory**: Vector-based semantic search using PostgreSQL + pgvector
+- **Translation Memory**: Vector search with pgvector and CJK keyword search with PGroonga
 
 ### 📚 Knowledge Management
 
 - **Project Dictionaries**: Project-specific terminology databases
 - **Translation Memory**: Import/export translation memory in TMX, CSV, XLSX formats
-- **Semantic Search**: Vector similarity search powered by pgvector
+- **Semantic Search**: Vector similarity search powered by pgvector, with PGroonga keyword retrieval
 
 ### 🔄 Workflow Automation
 
@@ -76,7 +76,7 @@ DeepTrans Studio is described in our CSCW '26 Companion Demo (CCF-A) paper. If y
 
 ### 🔌 Extensibility
 
-- **Open Architecture**: Modular design with PostgreSQL, MinIO, Valkey integration
+- **Open Architecture**: Modular design with PostgreSQL, Valkey, and pluggable object storage
 - **API Gateway**: RESTful APIs for external integration
 - **Custom Agents**: Extensible AI agent framework
 - **Plugin System**: Support for custom translation engines and processing pipelines
@@ -95,20 +95,20 @@ graph TD
     Worker[Worker Service] -->|Consume Tasks| Valkey
     Worker -->|ORM| Postgres
     Worker -->|Vector Ops| Postgres
-    Worker -->|Object Storage| MinIO[(MinIO)]
+    Worker -->|Object Storage Interface| Storage[(MinIO / Tencent COS)]
 ```
 
 ### Core Components
 
-| Component    | Technology                                            | Purpose                                           |
-| ------------ | ----------------------------------------------------- | ------------------------------------------------- |
-| **Studio**   | Next.js 15, React 19, TypeScript                      | Frontend UI, Server Actions, Authentication       |
-| **Worker**   | Node.js, BullMQ                                       | Background job processing, batch operations       |
-| **Database** | PostgreSQL 18, pgvector, Prisma 6                     | Relational data, vector search, and ORM           |
-| **Cache**    | Valkey                                                | Redis-protocol cache, session state, task queues  |
-| **Storage**  | MinIO (S3-compatible)                                 | Document and asset storage                        |
-| **Parser**   | DOCX XML parser, pdf-parse, OCR fallback, text parser | Document parsing for DOCX, PDF, TXT, and Markdown |
-| **Gateway**  | Traefik                                               | Reverse proxy, SSL/TLS termination                |
+| Component    | Technology                                            | Purpose                                                     |
+| ------------ | ----------------------------------------------------- | ----------------------------------------------------------- |
+| **Studio**   | Next.js 15, React 19, TypeScript                      | Frontend UI, Server Actions, Authentication                 |
+| **Worker**   | Node.js, BullMQ                                       | Background job processing, batch operations                 |
+| **Database** | PostgreSQL 18, pgvector, PGroonga, Prisma 6           | Relational data, vector search, CJK keyword search, and ORM |
+| **Cache**    | Valkey                                                | Redis-protocol cache, session state, task queues            |
+| **Storage**  | StorageService interface, MinIO, Tencent COS          | Document and asset storage                                  |
+| **Parser**   | DOCX XML parser, pdf-parse, OCR fallback, text parser | Document parsing for DOCX, PDF, TXT, and Markdown           |
+| **Gateway**  | Traefik                                               | Reverse proxy, SSL/TLS termination                          |
 
 ## 🚀 Quick Start
 
@@ -228,7 +228,7 @@ cp .env.example .env.production
 docker compose build app app_worker
 
 # Deploy services
-docker compose up -d traefik app app_worker db valkey minio
+docker compose up -d traefik app app_worker db valkey
 
 # Services will be available on configured domain with SSL via Traefik
 ```
@@ -337,8 +337,9 @@ Built with modern technologies:
 - [Next.js](https://nextjs.org/) - React framework
 - [Prisma](https://www.prisma.io/) - Database ORM
 - [pgvector](https://github.com/pgvector/pgvector) - PostgreSQL vector search
+- [PGroonga](https://pgroonga.github.io/) - CJK-capable PostgreSQL full-text search
 - [BullMQ](https://docs.bullmq.io/) - Job queues
-- [MinIO](https://min.io/) - Object storage
+- [MinIO](https://min.io/) / Tencent COS - Object storage
 - [Traefik](https://traefik.io/) - Reverse proxy
 
 ## 📞 Support

README_ZH.md

@@ -59,13 +59,13 @@ DeepTrans Studio 已在 CSCW '26 Companion Demo（CCF-A）论文中介绍。如
 - **多引擎支持**：集成 OpenAI 及自定义 AI 模型
 - **术语提取**：自动提取领域专业术语
 - **质量评估**：AI 驱动的语法、句法和语篇评估
-- **翻译记忆**：基于 PostgreSQL + pgvector 的向量语义搜索
+- **翻译记忆**：基于 pgvector 的向量检索和基于 PGroonga 的 CJK 关键词检索
 
 ### 📚 知识管理
 
 - **项目词典**：项目专属术语数据库
 - **翻译记忆**：支持 TMX、CSV、XLSX 格式导入导出
-- **语义搜索**：基于 pgvector 的向量相似度搜索
+- **语义搜索**：基于 pgvector 的向量相似度搜索，结合 PGroonga 关键词检索
 
 ### 🔄 工作流自动化
 
@@ -76,7 +76,7 @@ DeepTrans Studio 已在 CSCW '26 Companion Demo（CCF-A）论文中介绍。如
 
 ### 🔌 可扩展性
 
-- **开放架构**：模块化设计，集成 PostgreSQL、MinIO、Valkey
+- **开放架构**：模块化设计，集成 PostgreSQL、Valkey 与可插拔对象存储
 - **API 网关**：提供 RESTful API 用于外部集成
 - **自定义智能体**：可扩展的 AI 智能体框架
 - **插件系统**：支持自定义翻译引擎和处理流程
@@ -95,20 +95,20 @@ graph TD
     Worker[Worker 服务] -->|消费任务| Valkey
     Worker -->|ORM| Postgres
     Worker -->|向量操作| Postgres
-    Worker -->|对象存储| MinIO[(MinIO)]
+    Worker -->|对象存储接口| Storage[(MinIO / 腾讯云 COS)]
 ```
 
 ### 核心组件
 
-| 组件       | 技术栈                                           | 用途                              |
-| ---------- | ------------------------------------------------ | --------------------------------- |
-| **Studio** | Next.js 15, React 19, TypeScript                 | 前端 UI、Server Actions、身份验证 |
-| **Worker** | Node.js, BullMQ                                  | 后台任务处理、批量操作            |
-| **数据库** | PostgreSQL 18, pgvector, Prisma 6                | 关系型数据、向量检索与 ORM        |
-| **缓存**   | Valkey                                           | Redis 协议缓存、任务队列          |
-| **存储**   | MinIO (S3 兼容)                                  | 文档和资源存储                    |
-| **解析器** | DOCX XML 解析器、pdf-parse、OCR 兜底、文本解析器 | 解析 DOCX、PDF、TXT 与 Markdown   |
-| **网关**   | Traefik                                          | 反向代理、SSL/TLS 终端            |
+| 组件       | 技术栈                                           | 用途                                       |
+| ---------- | ------------------------------------------------ | ------------------------------------------ |
+| **Studio** | Next.js 15, React 19, TypeScript                 | 前端 UI、Server Actions、身份验证          |
+| **Worker** | Node.js, BullMQ                                  | 后台任务处理、批量操作                     |
+| **数据库** | PostgreSQL 18, pgvector, PGroonga, Prisma 6      | 关系型数据、向量检索、CJK 关键词检索与 ORM |
+| **缓存**   | Valkey                                           | Redis 协议缓存、任务队列                   |
+| **存储**   | StorageService 接口、MinIO、腾讯云 COS           | 文档和资源存储                             |
+| **解析器** | DOCX XML 解析器、pdf-parse、OCR 兜底、文本解析器 | 解析 DOCX、PDF、TXT 与 Markdown            |
+| **网关**   | Traefik                                          | 反向代理、SSL/TLS 终端                     |
 
 ## 🚀 快速开始
 
@@ -228,7 +228,7 @@ cp .env.example .env.production
 docker compose build app app_worker
 
 # 部署服务
-docker compose up -d traefik app app_worker db valkey minio
+docker compose up -d traefik app app_worker db valkey
 
 # 服务将在配置的域名上通过 Traefik 提供 SSL 访问
 ```
@@ -337,8 +337,9 @@ DeepTrans Studio 使用 [next-intl](https://next-intl-docs.vercel.app/) 进行
 - [Next.js](https://nextjs.org/) - React 框架
 - [Prisma](https://www.prisma.io/) - 数据库 ORM
 - [pgvector](https://github.com/pgvector/pgvector) - PostgreSQL 向量检索
+- [PGroonga](https://pgroonga.github.io/) - 支持 CJK 的 PostgreSQL 全文检索
 - [BullMQ](https://docs.bullmq.io/) - 任务队列
-- [MinIO](https://min.io/) - 对象存储
+- [MinIO](https://min.io/) / 腾讯云 COS - 对象存储
 - [Traefik](https://traefik.io/) - 反向代理
 
 ## 📞 支持

docker-compose-prod.yml

@@ -51,45 +51,6 @@ services:
             - 'traefik.http.routers.traefik.entrypoints=websecure'
             - 'traefik.http.routers.traefik.tls=true'
             - 'traefik.http.routers.traefik.tls.certresolver=le'
-    minio:
-        image: minio/minio:latest
-        container_name: deeptrans-minio
-        restart: always
-        environment:
-            MINIO_ROOT_USER: ${MINIO_ACCESS_KEY:-minioadmin}
-            MINIO_ROOT_PASSWORD: ${MINIO_SECRET_KEY:-minioadmin}
-            MINIO_BUCKET_NAME: ${MINIO_BUCKET:-deeptrans}
-        volumes:
-            - minio-data:/data
-        ports:
-            - '${MINIO_PORT:-9000}:9000' # API 端口）
-            - '${MINIO_CONSOLE_PORT:-9001}:9001' # 控制台端口
-        command: minio server /data --console-address ":9001"
-        healthcheck:
-            test: ['CMD', 'mc', 'ready', 'local']
-            interval: 30s
-            timeout: 10s
-            retries: 3
-        networks:
-            - app_local
-            - web # 连接到web网络
-        labels:
-            - 'traefik.enable=true'
-            # 1. 控制台路由 (Console) - 用于浏览器访问
-            - 'traefik.http.routers.minio-console.rule=Host(`minio-admin.deeptrans.studio`)'
-            - 'traefik.http.routers.minio-console.entrypoints=websecure'
-            - 'traefik.http.routers.minio-console.tls=true'
-            - 'traefik.http.routers.minio-console.tls.certresolver=le'
-            - 'traefik.http.routers.minio-console.service=minio-console-svc'
-            - 'traefik.http.services.minio-console-svc.loadbalancer.server.port=9001'
-
-            # 2. API 路由 (S3 API) - 用于程序和 SDK 访问
-            - 'traefik.http.routers.minio-api.rule=Host(`minio.deeptrans.studio`)'
-            - 'traefik.http.routers.minio-api.entrypoints=websecure'
-            - 'traefik.http.routers.minio-api.tls=true'
-            - 'traefik.http.routers.minio-api.tls.certresolver=le'
-            - 'traefik.http.routers.minio-api.service=minio-api-svc'
-            - 'traefik.http.services.minio-api-svc.loadbalancer.server.port=9000'
     valkey:
         image: valkey/valkey:8-alpine
         container_name: deeptrans-valkey
@@ -107,6 +68,16 @@ services:
         container_name: deeptrans-app
         env_file:
             - .env
+        environment:
+            STORAGE_TYPE: cos
+            STORAGE_USE_SSL: 'true'
+            COS_SECRET_ID: ${COS_SECRET_ID}
+            COS_SECRET_KEY: ${COS_SECRET_KEY}
+            COS_BUCKET: ${COS_BUCKET}
+            COS_REGION: ${COS_REGION}
+            COS_APP_ID: ${COS_APP_ID:-}
+            COS_DOMAIN: ${COS_DOMAIN:-}
+            COS_USE_SSL: ${COS_USE_SSL:-true}
         depends_on:
             - db
             - valkey
@@ -127,6 +98,16 @@ services:
         container_name: deeptrans-app_worker
         env_file:
             - .env
+        environment:
+            STORAGE_TYPE: cos
+            STORAGE_USE_SSL: 'true'
+            COS_SECRET_ID: ${COS_SECRET_ID}
+            COS_SECRET_KEY: ${COS_SECRET_KEY}
+            COS_BUCKET: ${COS_BUCKET}
+            COS_REGION: ${COS_REGION}
+            COS_APP_ID: ${COS_APP_ID:-}
+            COS_DOMAIN: ${COS_DOMAIN:-}
+            COS_USE_SSL: ${COS_USE_SSL:-true}
         depends_on:
             - db
             - valkey
@@ -142,6 +123,5 @@ networks:
 
 volumes:
     postgres_data:
-    minio-data:
     traefik_data:
     valkey-data:

prisma/migrations/20260628093000_pgvector_translation_memory/migration.sql

@@ -1,15 +1,7 @@
 CREATE EXTENSION IF NOT EXISTS vector;
 CREATE EXTENSION IF NOT EXISTS pg_trgm;
 CREATE EXTENSION IF NOT EXISTS unaccent;
-
-DO $$
-BEGIN
-    CREATE EXTENSION IF NOT EXISTS pgroonga;
-EXCEPTION
-    WHEN OTHERS THEN
-        RAISE NOTICE 'pgroonga extension is not installed; continuing with pgvector and pg_trgm';
-END
-$$;
+CREATE EXTENSION IF NOT EXISTS pgroonga;
 
 ALTER TABLE "TranslationMemoryEntry"
     ADD COLUMN IF NOT EXISTS embedding vector(1536);
@@ -27,15 +19,10 @@ CREATE INDEX IF NOT EXISTS "TranslationMemoryEntry_targetText_trgm_idx"
     ON "TranslationMemoryEntry"
     USING gin ("targetText" gin_trgm_ops);
 
-DO $$
-BEGIN
-    IF EXISTS (SELECT 1 FROM pg_extension WHERE extname = 'pgroonga') THEN
-        CREATE INDEX IF NOT EXISTS "TranslationMemoryEntry_sourceText_pgroonga_idx"
-            ON "TranslationMemoryEntry"
-            USING pgroonga ("sourceText");
-        CREATE INDEX IF NOT EXISTS "TranslationMemoryEntry_targetText_pgroonga_idx"
-            ON "TranslationMemoryEntry"
-            USING pgroonga ("targetText");
-    END IF;
-END
-$$;
+CREATE INDEX IF NOT EXISTS "TranslationMemoryEntry_sourceText_pgroonga_idx"
+    ON "TranslationMemoryEntry"
+    USING pgroonga ("sourceText");
+
+CREATE INDEX IF NOT EXISTS "TranslationMemoryEntry_targetText_pgroonga_idx"
+    ON "TranslationMemoryEntry"
+    USING pgroonga ("targetText");

src/actions/upload.ts

@@ -4,14 +4,17 @@ import { createLogger } from '@/lib/logger';
 import { requireOwnedProject, requireUser, requireWritableProject } from '@/lib/guards';
 import { getStorageConfigFromEnv } from '@/lib/storage/config';
 import { createStorageService } from '@/lib/storage/factory';
-const logger = createLogger({
-    type: 'actions:upload',
-}, {
-    json: false,// 开启json格式输出
-    pretty: false, // 关闭开发环境美化输出
-    colors: true, // 仅当json：false时启用颜色输出可用
-    includeCaller: false, // 日志不包含调用者
-});
+const logger = createLogger(
+    {
+        type: 'actions:upload',
+    },
+    {
+        json: false, // 开启json格式输出
+        pretty: false, // 关闭开发环境美化输出
+        colors: true, // 仅当json：false时启用颜色输出可用
+        includeCaller: false, // 日志不包含调用者
+    }
+);
 // 创建存储服务实例
 const storageService = createStorageService(getStorageConfigFromEnv());
 

src/app/(app)/docs/i18n/pages-en/installation-en.json

@@ -24,7 +24,7 @@
         "envConfig": {
             "title": "Configure Environment Variables",
             "description": "Copy or create .env.local and fill in according to your actual environment:",
-            "note": "💡 For production environments, please change the database, Valkey, pgvector, and object storage to dedicated instances and keep credentials secure.",
+            "note": "💡 For production environments, please use Tencent COS for object storage and change the database, Valkey, and pgvector to dedicated instances and keep credentials secure.",
             "example": "# Database & Cache\nDATABASE_URL=\"postgresql://postgres:123456@localhost:5432/deeptrans\"\nREDIS_URL=\"redis://valkey:6379\"\n\n# Authentication & Site Config\nAUTH_SECRET=\"Please generate a random string\"\nNEXTAUTH_URL=\"http://localhost:3000\"\nNODE_ENV=development\n\n# LLM / AI Services\nOPENAI_API_KEY=\"sk-xxxx\"\nOPENAI_BASE_URL=\"https://api.openai.com/v1\"\nOPENAI_API_MODEL=\"gpt-4o-mini\"\n\n# Object Storage: MinIO or Tencent COS\nSTORAGE_TYPE=minio\nSTORAGE_ENDPOINT=localhost\nSTORAGE_PORT=9000\nSTORAGE_USE_SSL=false\nSTORAGE_ACCESS_KEY=minioadmin\nSTORAGE_SECRET_KEY=minioadmin\nSTORAGE_BUCKET=deeptrans\n\n# Tencent COS example\n# STORAGE_TYPE=cos\n# COS_SECRET_ID=AKIDxxxxxxxx\n# COS_SECRET_KEY=xxxxxxxx\n# COS_BUCKET=deeptrans-1250000000\n# COS_REGION=ap-guangzhou\n\n# Other Services\nSTUDIO_HOST=localhost        # Used for Traefik/HTTPS deployment\n\n# Optional: GitHub OAuth, SMTP, pgvector access info, etc."
         },
         "initDb": {
@@ -42,7 +42,7 @@
             "step1": {
                 "title": "Start Dependency Services",
                 "description": "Recommended to use services from Docker Compose:",
-                "command": "docker compose up -d db valkey minio\n# If you need Traefik, append traefik"
+                "command": "docker compose up -d db valkey minio\n# Development uses MinIO locally. Production uses Tencent COS and does not start MinIO."
             },
             "step2": {
                 "title": "Start Next.js Application",
@@ -73,7 +73,7 @@
             },
             "step3": {
                 "title": "Start Main Services",
-                "command": "docker compose up -d traefik app app_worker db valkey minio",
+                "command": "docker compose up -d traefik app app_worker db valkey",
                 "note": "Traefik will map ports 80/443 to Studio. Prisma Studio can be started as needed."
             }
         },

src/app/(app)/docs/i18n/pages-zh/installation-zh.json

@@ -24,7 +24,7 @@
         "envConfig": {
             "title": "配置环境变量",
             "description": "复制或创建 .env.local 并依据实际环境填写：",
-            "note": "💡 生产环境请将数据库、Valkey、pgvector、对象存储改为专用实例，并妥善保管密钥。",
+            "note": "💡 生产环境使用腾讯云 COS 作为对象存储，并将数据库、Valkey、pgvector 改为专用实例，并妥善保管密钥。",
             "example": "# 数据库与缓存\nDATABASE_URL=\"postgresql://postgres:123456@localhost:5432/deeptrans\"\nREDIS_URL=\"redis://valkey:6379\"\n\n# 鉴权 & 站点配置\nAUTH_SECRET=\"请生成随机字符串\"\nNEXTAUTH_URL=\"http://localhost:3000\"\nNODE_ENV=development\n\n# LLM / AI 服务\nOPENAI_API_KEY=\"sk-xxxx\"\nOPENAI_BASE_URL=\"https://api.openai.com/v1\"\nOPENAI_API_MODEL=\"gpt-4o-mini\"\n\n# 对象存储：MinIO 或腾讯云 COS\nSTORAGE_TYPE=minio\nSTORAGE_ENDPOINT=localhost\nSTORAGE_PORT=9000\nSTORAGE_USE_SSL=false\nSTORAGE_ACCESS_KEY=minioadmin\nSTORAGE_SECRET_KEY=minioadmin\nSTORAGE_BUCKET=deeptrans\n\n# 腾讯云 COS 示例\n# STORAGE_TYPE=cos\n# COS_SECRET_ID=AKIDxxxxxxxx\n# COS_SECRET_KEY=xxxxxxxx\n# COS_BUCKET=deeptrans-1250000000\n# COS_REGION=ap-guangzhou\n\n# 其他服务\nSTUDIO_HOST=localhost        # Traefik/HTTPS 部署时使用\n\n# 可选：GitHub OAuth、SMTP、pgvector 访问信息等"
         },
         "initDb": {
@@ -42,7 +42,7 @@
             "step1": {
                 "title": "启动依赖服务",
                 "description": "推荐使用 Docker Compose 中的服务：",
-                "command": "docker compose up -d db valkey minio\n# 如需 Traefik，请追加 traefik"
+                "command": "docker compose up -d db valkey minio\n# 开发环境本地使用 MinIO；生产环境使用腾讯云 COS，不启动 MinIO。"
             },
             "step2": {
                 "title": "启动 Next.js 应用",
@@ -73,7 +73,7 @@
             },
             "step3": {
                 "title": "启动主要服务",
-                "command": "docker compose up -d traefik app app_worker db valkey minio",
+                "command": "docker compose up -d traefik app app_worker db valkey",
                 "note": "Traefik 会将 80/443 端口映射到 Studio，Prisma Studio 可按需启动。"
             }
         },