English / [简体中文](./README_CN.md)
-One-Click to get a well-designed cross-platform ChatGPT web UI, with GPT3, GPT4 & Gemini Pro support.
+
-一键免费部署你的跨平台私人 ChatGPT 应用, 支持 GPT3, GPT4 & Gemini Pro 模型。
+✨ Light and Fast AI Assistant,with Claude, DeepSeek, GPT4 & Gemini Pro support.
[![Saas][Saas-image]][saas-url]
[![Web][Web-image]][web-url]
@@ -18,28 +18,49 @@ One-Click to get a well-designed cross-platform ChatGPT web UI, with GPT3, GPT4
[![MacOS][MacOS-image]][download-url]
[![Linux][Linux-image]][download-url]
-[NextChatAI](https://nextchat.dev/chat?utm_source=readme) / [Web App](https://app.nextchat.dev) / [Desktop App](https://github.com/Yidadaa/ChatGPT-Next-Web/releases) / [Discord](https://discord.gg/YCkeafCafC) / [Enterprise Edition](#enterprise-edition) / [Twitter](https://twitter.com/NextChatDev)
+[NextChatAI](https://nextchat.club?utm_source=readme) / [iOS APP](https://apps.apple.com/us/app/nextchat-ai/id6743085599) / [Web App Demo](https://app.nextchat.club) / [Desktop App](https://github.com/Yidadaa/ChatGPT-Next-Web/releases) / [Enterprise Edition](#enterprise-edition)
-[NextChatAI](https://nextchat.dev/chat) / [网页版](https://app.nextchat.dev) / [客户端](https://github.com/Yidadaa/ChatGPT-Next-Web/releases) / [企业版](#%E4%BC%81%E4%B8%9A%E7%89%88) / [反馈](https://github.com/Yidadaa/ChatGPT-Next-Web/issues)
-
-[saas-url]: https://nextchat.dev/chat?utm_source=readme
+[saas-url]: https://nextchat.club?utm_source=readme
[saas-image]: https://img.shields.io/badge/NextChat-Saas-green?logo=microsoftedge
-[web-url]: https://app.nextchat.dev/
+[web-url]: https://app.nextchat.club/
[download-url]: https://github.com/Yidadaa/ChatGPT-Next-Web/releases
[Web-image]: https://img.shields.io/badge/Web-PWA-orange?logo=microsoftedge
[Windows-image]: https://img.shields.io/badge/-Windows-blue?logo=windows
[MacOS-image]: https://img.shields.io/badge/-MacOS-black?logo=apple
[Linux-image]: https://img.shields.io/badge/-Linux-333?logo=ubuntu
-[](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2FChatGPTNextWeb%2FChatGPT-Next-Web&env=OPENAI_API_KEY&env=CODE&project-name=nextchat&repository-name=NextChat) [](https://zeabur.com/templates/ZBUEFA) [](https://gitpod.io/#https://github.com/Yidadaa/ChatGPT-Next-Web)
+[](https://zeabur.com/templates/ZBUEFA) [](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2FChatGPTNextWeb%2FChatGPT-Next-Web&env=OPENAI_API_KEY&env=CODE&project-name=nextchat&repository-name=NextChat) [](https://gitpod.io/#https://github.com/ChatGPTNextWeb/NextChat) [](https://opendeploy.dev/github/ChatGPTNextWeb/NextChat)
-[](https://monica.im/?utm=nxcrp)
+[](https://monica.im/?utm=nxcrp)
+## ❤️ Sponsor AI API
+
+
+
+
+
+[302.AI](https://302.ai/) is a pay-as-you-go AI application platform that offers the most comprehensive AI APIs and online applications available.
+
+## 🥳 Cheer for NextChat iOS Version Online!
+
+> [👉 Click Here to Install Now](https://apps.apple.com/us/app/nextchat-ai/id6743085599)
+
+> [❤️ Source Code Coming Soon](https://github.com/ChatGPTNextWeb/NextChat-iOS)
+
+
+
+## 🫣 NextChat Support MCP !
+
+> Before build, please set env ENABLE_MCP=true
+
+
+
## Enterprise Edition
Meeting Your Company's Privatization and Customization Deployment Requirements:
+
- **Brand Customization**: Tailored VI/UI to seamlessly align with your corporate brand image.
- **Resource Integration**: Unified configuration and management of dozens of AI resources by company administrators, ready for use by team members.
- **Permission Control**: Clearly defined member permissions, resource permissions, and knowledge base permissions, all controlled via a corporate-grade Admin Panel.
@@ -50,20 +71,11 @@ Meeting Your Company's Privatization and Customization Deployment Requirements:
For enterprise inquiries, please contact: **business@nextchat.dev**
-## 企业版
-
-满足企业用户私有化部署和个性化定制需求：
-- **品牌定制**：企业量身定制 VI/UI，与企业品牌形象无缝契合
-- **资源集成**：由企业管理人员统一配置和管理数十种 AI 资源，团队成员开箱即用
-- **权限管理**：成员权限、资源权限、知识库权限层级分明，企业级 Admin Panel 统一控制
-- **知识接入**：企业内部知识库与 AI 能力相结合，比通用 AI 更贴近企业自身业务需求
-- **安全审计**：自动拦截敏感提问，支持追溯全部历史对话记录，让 AI 也能遵循企业信息安全规范
-- **私有部署**：企业级私有部署，支持各类主流私有云部署，确保数据安全和隐私保护
-- **持续更新**：提供多模态、智能体等前沿能力持续更新升级服务，常用常新、持续先进
+## Screenshots
-企业版咨询: **business@nextchat.dev**
+
-
+
## Features
@@ -96,62 +108,23 @@ For enterprise inquiries, please contact: **business@nextchat.dev**
- [x] Artifacts: Easily preview, copy and share generated content/webpages through a separate window [#5092](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/pull/5092)
- [x] Plugins: support network search, calculator, any other apis etc. [#165](https://github.com/Yidadaa/ChatGPT-Next-Web/issues/165) [#5353](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/issues/5353)
- [x] network search, calculator, any other apis etc. [#165](https://github.com/Yidadaa/ChatGPT-Next-Web/issues/165) [#5353](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/issues/5353)
+- [x] Supports Realtime Chat [#5672](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/issues/5672)
- [ ] local knowledge base
## What's New
+- 🚀 v2.15.8 Now supports Realtime Chat [#5672](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/issues/5672)
- 🚀 v2.15.4 The Application supports using Tauri fetch LLM API, MORE SECURITY! [#5379](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/issues/5379)
- 🚀 v2.15.0 Now supports Plugins! Read this: [NextChat-Awesome-Plugins](https://github.com/ChatGPTNextWeb/NextChat-Awesome-Plugins)
-- 🚀 v2.14.0 Now supports Artifacts & SD
+- 🚀 v2.14.0 Now supports Artifacts & SD
- 🚀 v2.10.1 support Google Gemini Pro model.
- 🚀 v2.9.11 you can use azure endpoint now.
- 🚀 v2.8 now we have a client that runs across all platforms!
- 🚀 v2.7 let's share conversations as image, or share to ShareGPT!
- 🚀 v2.0 is released, now you can create prompt templates, turn your ideas into reality! Read this: [ChatGPT Prompt Engineering Tips: Zero, One and Few Shot Prompting](https://www.allabtai.com/prompt-engineering-tips-zero-one-and-few-shot-prompting/).
-## 主要功能
-
-- 在 1 分钟内使用 Vercel **免费一键部署**
-- 提供体积极小（~5MB）的跨平台客户端（Linux/Windows/MacOS）, [下载地址](https://github.com/Yidadaa/ChatGPT-Next-Web/releases)
-- 完整的 Markdown 支持：LaTex 公式、Mermaid 流程图、代码高亮等等
-- 精心设计的 UI，响应式设计，支持深色模式，支持 PWA
-- 极快的首屏加载速度（~100kb），支持流式响应
-- 隐私安全，所有数据保存在用户浏览器本地
-- 预制角色功能（面具），方便地创建、分享和调试你的个性化对话
-- 海量的内置 prompt 列表，来自[中文](https://github.com/PlexPt/awesome-chatgpt-prompts-zh)和[英文](https://github.com/f/awesome-chatgpt-prompts)
-- 自动压缩上下文聊天记录，在节省 Token 的同时支持超长对话
-- 多国语言支持：English, 简体中文, 繁体中文, 日本語, Español, Italiano, Türkçe, Deutsch, Tiếng Việt, Русский, Čeština, 한국어, Indonesia
-- 拥有自己的域名？好上加好，绑定后即可在任何地方**无障碍**快速访问
-
-## 开发计划
-
-- [x] 为每个对话设置系统 Prompt [#138](https://github.com/Yidadaa/ChatGPT-Next-Web/issues/138)
-- [x] 允许用户自行编辑内置 Prompt 列表
-- [x] 预制角色：使用预制角色快速定制新对话 [#993](https://github.com/Yidadaa/ChatGPT-Next-Web/issues/993)
-- [x] 分享为图片，分享到 ShareGPT 链接 [#1741](https://github.com/Yidadaa/ChatGPT-Next-Web/pull/1741)
-- [x] 使用 tauri 打包桌面应用
-- [x] 支持自部署的大语言模型：开箱即用 [RWKV-Runner](https://github.com/josStorer/RWKV-Runner) ，服务端部署 [LocalAI 项目](https://github.com/go-skynet/LocalAI) llama / gpt4all / rwkv / vicuna / koala / gpt4all-j / cerebras / falcon / dolly 等等，或者使用 [api-for-open-llm](https://github.com/xusenlinzy/api-for-open-llm)
-- [x] Artifacts: 通过独立窗口，轻松预览、复制和分享生成的内容/可交互网页 [#5092](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/pull/5092)
-- [x] 插件机制，支持`联网搜索`、`计算器`、调用其他平台 api [#165](https://github.com/Yidadaa/ChatGPT-Next-Web/issues/165) [#5353](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/issues/5353)
- - [x] 支持联网搜索、计算器、调用其他平台 api [#165](https://github.com/Yidadaa/ChatGPT-Next-Web/issues/165) [#5353](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/issues/5353)
- - [ ] 本地知识库
-
-## 最新动态
-
-- 🚀 v2.15.4 客户端支持Tauri本地直接调用大模型API，更安全！[#5379](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/issues/5379)
-- 🚀 v2.15.0 现在支持插件功能了！了解更多：[NextChat-Awesome-Plugins](https://github.com/ChatGPTNextWeb/NextChat-Awesome-Plugins)
-- 🚀 v2.14.0 现在支持 Artifacts & SD 了。
-- 🚀 v2.10.1 现在支持 Gemini Pro 模型。
-- 🚀 v2.9.11 现在可以使用自定义 Azure 服务了。
-- 🚀 v2.8 发布了横跨 Linux/Windows/MacOS 的体积极小的客户端。
-- 🚀 v2.7 现在可以将会话分享为图片了，也可以分享到 ShareGPT 的在线链接。
-- 🚀 v2.0 已经发布，现在你可以使用面具功能快速创建预制对话了！ 了解更多： [ChatGPT 提示词高阶技能：零次、一次和少样本提示](https://github.com/Yidadaa/ChatGPT-Next-Web/issues/138)。
-- 💡 想要更方便地随时随地使用本项目？可以试下这款桌面插件：https://github.com/mushan0x0/AI0x0.com
-
## Get Started
-> [简体中文 > 如何开始使用](./README_CN.md#开始使用)
-
1. Get [OpenAI API Key](https://platform.openai.com/account/api-keys);
2. Click
[](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2FYidadaa%2FChatGPT-Next-Web&env=OPENAI_API_KEY&env=CODE&project-name=chatgpt-next-web&repository-name=ChatGPT-Next-Web), remember that `CODE` is your page password;
@@ -159,14 +132,10 @@ For enterprise inquiries, please contact: **business@nextchat.dev**
## FAQ
-[简体中文 > 常见问题](./docs/faq-cn.md)
-
[English > FAQ](./docs/faq-en.md)
## Keep Updated
-> [简体中文 > 如何保持代码更新](./README_CN.md#保持更新)
-
If you have deployed your own project with just one click following the steps above, you may encounter the issue of "Updates Available" constantly showing up. This is because Vercel will create a new project for you by default instead of forking this project, resulting in the inability to detect updates correctly.
We recommend that you follow the steps below to re-deploy:
@@ -193,8 +162,6 @@ You can star or watch this project or follow author to get release notifications
## Access Password
-> [简体中文 > 如何增加访问密码](./README_CN.md#配置页面访问密码)
-
This project provides limited access control. Please add an environment variable named `CODE` on the vercel environment variables page. The value should be passwords separated by comma like this:
```
@@ -205,8 +172,6 @@ After adding or modifying this environment variable, please redeploy the project
## Environment Variables
-> [简体中文 > 如何配置 api key、访问密码、接口代理](./README_CN.md#环境变量)
-
### `CODE` (optional)
Access password, separated by comma.
@@ -301,6 +266,22 @@ iflytek Api Key.
iflytek Api Secret.
+### `CHATGLM_API_KEY` (optional)
+
+ChatGLM Api Key.
+
+### `CHATGLM_URL` (optional)
+
+ChatGLM Api Url.
+
+### `DEEPSEEK_API_KEY` (optional)
+
+DeepSeek Api Key.
+
+### `DEEPSEEK_URL` (optional)
+
+DeepSeek Api Url.
+
### `HIDE_USER_API_KEY` (optional)
> Default: Empty
@@ -335,21 +316,31 @@ To control custom models, use `+` to add a custom model, use `-` to hide a model
User `-all` to disable all default models, `+all` to enable all default models.
For Azure: use `modelName@Azure=deploymentName` to customize model name and deployment name.
+
> Example: `+gpt-3.5-turbo@Azure=gpt35` will show option `gpt35(Azure)` in model list.
> If you only can use Azure model, `-all,+gpt-3.5-turbo@Azure=gpt35` will `gpt35(Azure)` the only option in model list.
For ByteDance: use `modelName@bytedance=deploymentName` to customize model name and deployment name.
+
> Example: `+Doubao-lite-4k@bytedance=ep-xxxxx-xxx` will show option `Doubao-lite-4k(ByteDance)` in model list.
### `DEFAULT_MODEL` （optional）
Change default model
+### `VISION_MODELS` (optional)
+
+> Default: Empty
+> Example: `gpt-4-vision,claude-3-opus,my-custom-model` means add vision capabilities to these models in addition to the default pattern matches (which detect models containing keywords like "vision", "claude-3", "gemini-1.5", etc).
+
+Add additional models to have vision capabilities, beyond the default pattern matching. Multiple models should be separated by commas.
+
### `WHITE_WEBDAV_ENDPOINTS` (optional)
You can use this option if you want to increase the number of webdav service addresses you are allowed to access, as required by the format：
-- Each address must be a complete endpoint
-> `https://xxxx/yyy`
+
+- Each address must be a complete endpoint
+ > `https://xxxx/yyy`
- Multiple addresses are connected by ', '
### `DEFAULT_INPUT_TEMPLATE` (optional)
@@ -364,14 +355,32 @@ Stability API key.
Customize Stability API url.
+### `ENABLE_MCP` (optional)
+
+Enable MCP（Model Context Protocol）Feature
+
+### `SILICONFLOW_API_KEY` (optional)
+
+SiliconFlow API Key.
+
+### `SILICONFLOW_URL` (optional)
+
+SiliconFlow API URL.
+
+### `AI302_API_KEY` (optional)
+
+302.AI API Key.
+
+### `AI302_URL` (optional)
+
+302.AI API URL.
+
## Requirements
NodeJS >= 18, Docker >= 20
## Development
-> [简体中文 > 如何进行二次开发](./README_CN.md#开发)
-
[](https://gitpod.io/#https://github.com/Yidadaa/ChatGPT-Next-Web)
Before starting development, you must create a new `.env.local` file at project root, and place your api key into it:
@@ -395,8 +404,6 @@ yarn dev
## Deployment
-> [简体中文 > 如何部署到私人服务器](./README_CN.md#部署)
-
### Docker (Recommended)
```shell
@@ -424,6 +431,16 @@ If your proxy needs password, use:
-e PROXY_URL="http://127.0.0.1:7890 user pass"
```
+If enable MCP, use：
+
+```
+docker run -d -p 3000:3000 \
+ -e OPENAI_API_KEY=sk-xxxx \
+ -e CODE=your-password \
+ -e ENABLE_MCP=true \
+ yidadaa/chatgpt-next-web
+```
+
### Shell
```shell
@@ -444,12 +461,6 @@ bash <(curl -s https://raw.githubusercontent.com/Yidadaa/ChatGPT-Next-Web/main/s
- [How to use Vercel (No English)](./docs/vercel-cn.md)
- [User Manual (Only Chinese, WIP)](./docs/user-manual-cn.md)
-## Screenshots
-
-
-
-
-
## Translation
If you want to add a new translation, read this [document](./docs/translation.md).
@@ -460,38 +471,6 @@ If you want to add a new translation, read this [document](./docs/translation.md
## Special Thanks
-### Sponsor
-
-> 仅列出捐赠金额 >= 100RMB 的用户。
-
-[@mushan0x0](https://github.com/mushan0x0)
-[@ClarenceDan](https://github.com/ClarenceDan)
-[@zhangjia](https://github.com/zhangjia)
-[@hoochanlon](https://github.com/hoochanlon)
-[@relativequantum](https://github.com/relativequantum)
-[@desenmeng](https://github.com/desenmeng)
-[@webees](https://github.com/webees)
-[@chazzhou](https://github.com/chazzhou)
-[@hauy](https://github.com/hauy)
-[@Corwin006](https://github.com/Corwin006)
-[@yankunsong](https://github.com/yankunsong)
-[@ypwhs](https://github.com/ypwhs)
-[@fxxxchao](https://github.com/fxxxchao)
-[@hotic](https://github.com/hotic)
-[@WingCH](https://github.com/WingCH)
-[@jtung4](https://github.com/jtung4)
-[@micozhu](https://github.com/micozhu)
-[@jhansion](https://github.com/jhansion)
-[@Sha1rholder](https://github.com/Sha1rholder)
-[@AnsonHyq](https://github.com/AnsonHyq)
-[@synwith](https://github.com/synwith)
-[@piksonGit](https://github.com/piksonGit)
-[@ouyangzhiping](https://github.com/ouyangzhiping)
-[@wenjiavv](https://github.com/wenjiavv)
-[@LeXwDeX](https://github.com/LeXwDeX)
-[@Licoy](https://github.com/Licoy)
-[@shangmin2009](https://github.com/shangmin2009)
-
### Contributors
diff --git a/README_CN.md b/README_CN.md
index 3f339ea61f6..f4c441ad006 100644
--- a/README_CN.md
+++ b/README_CN.md
@@ -6,17 +6,26 @@
+
+## ❤️ AI API 후원사
+
+
+
+
+
+[302.AI](https://302.ai/)는 사용한 만큼만 비용을 지불하는 AI 애플리케이션 플랫폼으로, 다양한 AI API 및 온라인 애플리케이션을 제공합니다.
+
+## 🥳 NextChat iOS 버전 출시!
+
+> 👉 [지금 설치하기](https://apps.apple.com/us/app/nextchat-ai/id6743085599)
+
+> ❤️ [소스 코드 곧 공개 예정](https://github.com/ChatGPTNextWeb/NextChat-iOS)
+
+
+
+## 🫣 NextChat, MCP 지원!
+
+> 빌드 전 환경 변수(env) `ENABLE_MCP=true` 설정 필요
+
+
+
+## 엔터프라이즈 버전
+
+회사 내부 시스템에 맞춘 프라이빗 배포 및 맞춤형 커스터마이징 지원:
+
+- **브랜드 커스터마이징**: 기업 이미지에 맞는 UI/UX 테마 적용
+- **리소스 통합 관리**: 다양한 AI 모델을 통합하여 팀원이 손쉽게 사용 가능
+- **권한 제어**: 관리자 패널을 통한 멤버·리소스·지식 베이스 권한 설정
+- **지식 통합**: 사내 문서 및 데이터와 AI를 결합한 맞춤형 답변 제공
+- **보안 감사**: 민감한 질문 차단 및 모든 기록 추적 가능
+- **프라이빗 배포 지원**: 주요 클라우드 서비스에 맞춘 배포 옵션
+- **지속적 업데이트**: 멀티모달 등 최신 AI 기능 지속 반영
+
+엔터프라이즈 문의: **business@nextchat.dev**
+
+## 🖼️ 스크린샷
+
+
+
+
+## 주요 기능 소개
+
+- Vercel에서 원클릭 무료 배포 (1분 내 완성)
+- 모든 OS(Linux/Windows/MacOS)에서 사용 가능한 클라이언트 (~5MB) [지금 다운 받기](https://github.com/Yidadaa/ChatGPT-Next-Web/releases)
+- 자체 LLM 서버와 완벽 호환. [RWKV-Runner](https://github.com/josStorer/RWKV-Runner) 또는 [LocalAI](https://github.com/go-skynet/LocalAI)와 함께 사용하는 것을 추천
+- 개인 정보 보호: 모든 대화 기록은 브라우저에만 저장
+- Markdown 지원: LaTex, Mermaid, 코드 하이라이팅 등
+- 반응형 디자인, 다크 모드, PWA 지원
+- 빠른 초기 로딩 속도 (~100kb), 스트리밍 응답
+- 프롬프트 템플릿 생성/공유/디버깅 지원 (v2)
+- v2: 프롬프트 템플릿 기반 도구 생성, 공유, 디버깅 가능
+- 고급 프롬프트 내장 [awesome-chatgpt-prompts-zh](https://github.com/PlexPt/awesome-chatgpt-prompts-zh) and [awesome-chatgpt-prompts](https://github.com/f/awesome-chatgpt-prompts)
+- 긴 대화 내용 자동 압축 저장으로 토큰 절약
+- I18n: English, 简体中文, 繁体中文, 日本語, Français, Español, Italiano, Türkçe, Deutsch, Tiếng Việt, Русский, Čeština, 한국어, Indonesia
+
+
+
+
+
+
+
+## 개발 로드맵
+
+- [x] 시스템 프롬프트: 사용자가 정의한 프롬프트를 시스템 프롬프트로 고정하기 [#138](https://github.com/Yidadaa/ChatGPT-Next-Web/issues/138)
+- [x] 사용자 프롬프트: 사용자 정의 프롬프트를 편집 및 저장하여 리스트로 관리 가능
+- [x] 프롬프트 템플릿: 사전 정의된 인컨텍스트 프롬프트로 새 채팅 생성 [#993](https://github.com/Yidadaa/ChatGPT-Next-Web/issues/993)
+- [x] 이미지로 공유하거나 ShareGPT로 공유 [#1741](https://github.com/Yidadaa/ChatGPT-Next-Web/pull/1741)
+- [x] Tauri 기반 데스크톱 앱
+- [x] 자체 모델 호스팅: [RWKV-Runner](https://github.com/josStorer/RWKV-Runner), [LocalAI](https://github.com/go-skynet/LocalAI) 등 서버 배포 모델들과 완벽 호환 (llama, gpt4all, rwkv, vicuna, koala, gpt4all-j, cerebras, falcon, dolly 등)
+- [x] 아티팩트: 생성된 콘텐츠 및 웹페이지를 별도 창으로 미리보기, 복사, 공유 가능 [#5092](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/pull/5092)
+- [x] 플러그인: 웹 검색, 계산기, 기타 외부 API 기능 지원 [#165](https://github.com/Yidadaa/ChatGPT-Next-Web/issues/165) [#5353](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/issues/5353)
+- [x] 실시간 채팅 지원 [#5672](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/issues/5672)
+- [ ] 로컬 지식 베이스 지원 예정
+
+## 🚀 최근 업데이트
+
+- 🚀 v2.15.8 실시간 채팅 지원 [#5672](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/issues/5672)
+- 🚀 v2.15.4 Tauri 기반 LLM API 호출 기능 추가 → 보안 강화 [#5379](https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/issues/5379)
+- 🚀 v2.15.0 플러그인 기능 추가 → [NextChat-Awesome-Plugins](https://github.com/ChatGPTNextWeb/NextChat-Awesome-Plugins)
+- 🚀 v2.14.0 아티팩트 및 Stable Diffusion 기능 추가
+- 🚀 v2.10.1 Google Gemini Pro 모델 지원
+- 🚀 v2.9.11 Azure Endpoint 사용 가능
+- 🚀 v2.8 모든 플랫폼에서 실행 가능한 클라이언트 출시
+- 🚀 v2.7 대화 내용을 이미지로, 또는 ShareGPT로 공유 가능
+- 🚀 v2.0 릴리즈: 프롬프트 템플릿 생성 및 아이디어 구현 가능! → [ChatGPT Prompt Engineering Tips](https://www.allabtai.com/prompt-engineering-tips-zero-one-and-few-shot-prompting/)
+
+## 시작하기
+
+1. [OpenAI API 키](https://platform.openai.com/account/api-keys)를 발급받습니다.
+2.
+ [](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2FYidadaa%2FChatGPT-Next-Web&env=OPENAI_API_KEY&env=CODE&project-name=chatgpt-next-web&repository-name=ChatGPT-Next-Web) 버튼을 클릭해 Vercel에 배포합니다. `CODE`는 페이지 비밀번호라는 점을 기억하세요.
+
+3. Enjoy :)
+
+## FAQ
+
+[FAQ](./docs/faq-ko.md)
+
+## 최신 상태 유지 (Keep Updated)
+
+Vercel로 배포한 경우, "Updates Available" 메시지가 계속 나타날 수 있습니다. 이는 프로젝트를 포크하지 않고 새로 생성했기 때문입니다.
+
+다음 절차에 따라 다시 배포를 권장합니다:
+
+1. 기존 레포 삭제
+2. 우측 상단 "Fork" 버튼 클릭 → 포크 생성
+3. 포크된 프로젝트를 다시 Vercel에 배포
+ → [자세한 튜토리얼 보기](./docs/vercel-ko.md)
+
+### 자동 업데이트 활성화 (Enable Automatic Updates)
+
+> Upstream Sync 오류 발생 시, [수동으로 코드 업데이트](./README_KO.md#manually-updating-code)하세요.
+
+프로젝트 포크 후에는 GitHub의 제약으로 인해 Actions 페이지에서 아래 항목들을 수동으로 활성화해야 합니다:
+
+- `Workflows`
+- `Upstream Sync Action`
+
+이후 매 시간 자동으로 업데이트됩니다:
+
+
+
+
+### 수동 업데이트 방법 (Manually Updating Code)
+
+즉시 업데이트가 필요한 경우, [깃헙 문서](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork)를 참고해 포크된 프로젝트를 upstream code와 동기화하세요.
+
+릴리스 알림을 원하시면 star 또는 watch를 눌러주세요.
+
+## 접근 비밀번호 설정 (Access Password)
+
+이 프로젝트는 제한된 접근 제어를 제공합니다.
+Vercel 환경 변수에 `CODE`를 다음 형식으로 추가하세요. value는 ,를 통해 구분된 비밀번호여야 합니다.:
+
+```
+code1,code2,code3
+```
+
+수정 후 반드시 다시 배포해야 적용됩니다.
+
+## 환경 변수 (Environment Variables)
+
+### `CODE` (선택 사항)
+
+접속 비밀번호. 쉼표로 구분합니다.
+
+### `OPENAI_API_KEY` (필수)
+
+당신의 OpenAI API 키, 여러 개를 사용하려면 쉼표로 연결합니다.
+
+### `BASE_URL` (선택 사항)
+
+> 기본값: `https://api.openai.com`
+
+> 예시: `http://your-openai-proxy.com`
+
+OpenAI API 요청의 기본 URL을 재정의합니다.
+
+### `OPENAI_ORG_ID` (선택 사항)
+
+OpenAI organization ID를 지정합니다.
+
+### `AZURE_URL` (선택 사항)
+
+> 예시: https://{azure-resource-url}/openai
+
+Azure 배포 URL입니다.
+
+### `AZURE_API_KEY` (선택 사항)
+
+Azure API 키입니다.
+
+### `AZURE_API_VERSION` (선택 사항)
+
+Azure API 버전입니다. [Azure 문서](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#chat-completions)에서 확인할 수 있습니다.
+
+### `GOOGLE_API_KEY` (선택 사항)
+
+Google Gemini Pro API 키입니다.
+
+### `GOOGLE_URL` (선택 사항)
+
+Google Gemini Pro API URL입니다.
+
+### `ANTHROPIC_API_KEY` (선택 사항)
+
+Anthropic Claude API 키입니다.
+
+### `ANTHROPIC_API_VERSION` (선택 사항)
+
+Anthropic Claude API 버전입니다.
+
+### `ANTHROPIC_URL` (선택 사항)
+
+Anthropic Claude API URL입니다.
+
+### `BAIDU_API_KEY` (선택 사항)
+
+Baidu API 키입니다.
+
+### `BAIDU_SECRET_KEY` (선택 사항)
+
+Baidu Secret 키입니다.
+
+### `BAIDU_URL` (선택 사항)
+
+Baidu API URL입니다.
+
+### `BYTEDANCE_API_KEY` (선택 사항)
+
+ByteDance API 키입니다.
+
+### `BYTEDANCE_URL` (선택 사항)
+
+ByteDance API URL입니다.
+
+### `ALIBABA_API_KEY` (선택 사항)
+
+Alibaba Cloud API 키입니다.
+
+### `ALIBABA_URL` (선택 사항)
+
+Alibaba Cloud API URL입니다.
+
+### `IFLYTEK_URL` (선택 사항)
+
+iflytek API URL입니다.
+
+### `IFLYTEK_API_KEY` (선택 사항)
+
+iflytek API 키입니다.
+
+### `IFLYTEK_API_SECRET` (선택 사항)
+
+iflytek API 시크릿입니다.
+
+### `CHATGLM_API_KEY` (선택 사항)
+
+ChatGLM API 키입니다.
+
+### `CHATGLM_URL` (선택 사항)
+
+ChatGLM API URL입니다.
+
+### `DEEPSEEK_API_KEY` (선택 사항)
+
+DeepSeek API 키입니다.
+
+### `DEEPSEEK_URL` (선택 사항)
+
+DeepSeek API URL입니다.
+
+### `HIDE_USER_API_KEY` (선택 사항)
+
+> 기본값: 비어 있음
+
+사용자가 자신의 API 키를 입력하지 못하게 하려면 이 값을 1로 설정하세요.
+
+### `DISABLE_GPT4` (선택 사항)
+
+> 기본값: 비어 있음
+
+사용자가 GPT-4를 사용하지 못하게 하려면 이 값을 1로 설정하세요.
+
+### `ENABLE_BALANCE_QUERY` (선택 사항)
+
+> 기본값: 비어 있음
+
+사용자가 쿼리 잔액을 조회할 수 있도록 하려면 이 값을 1로 설정하세요.
+
+### `DISABLE_FAST_LINK` (선택 사항)
+
+> 기본값: 비어 있음
+
+URL에서 설정을 파싱하는 기능을 비활성화하려면 이 값을 1로 설정하세요.
+
+### `CUSTOM_MODELS` (선택 사항)
+
+> 기본값: 비어 있음
+> 예시: `+llama,+claude-2,-gpt-3.5-turbo,gpt-4-1106-preview=gpt-4-turbo`
+이는 `llama`, `claude-2`를 모델 리스트에 추가하고, `gpt-3.5-turbo`를 제거하며, `gpt-4-1106-preview`를 `gpt-4-turbo`로 표시합니다.
+
+사용자 지정 모델 제어 시 `+`는 추가, `-`는 제거, `이름=표시이름`은 모델명 커스터마이징을 의미합니다. 쉼표로 구분하세요.
+
+- `-all`은 기본 모델을 모두 비활성화
+- `+all`은 기본 모델을 모두 활성화
+
+Azure 용법 예시: `modelName@Azure=deploymentName` → 배포 이름을 커스터마이징 가능
+> 예시: `+gpt-3.5-turbo@Azure=gpt35` → 리스트에 `gpt35(Azure)` 표시됨
+> Azure 모델만 사용할 경우: `-all,+gpt-3.5-turbo@Azure=gpt35`
+
+ByteDance 용법 예시: `modelName@bytedance=deploymentName`
+> 예시: `+Doubao-lite-4k@bytedance=ep-xxxxx-xxx` → `Doubao-lite-4k(ByteDance)`로 표시됨
+
+### `DEFAULT_MODEL` (선택 사항)
+
+기본 모델을 변경합니다.
+
+### `VISION_MODELS` (선택 사항)
+
+> 기본값: 비어 있음
+> 예시: `gpt-4-vision,claude-3-opus,my-custom-model`
+위의 모델들에 시각 기능을 부여합니다 (기본적으로 `"vision"`, `"claude-3"`, `"gemini-1.5"` 키워드를 포함한 모델은 자동 인식됨). 기본 모델 외에도 모델을 추가할 수 있습니다. 쉼표로 구분하세요.
+
+### `WHITE_WEBDAV_ENDPOINTS` (선택 사항)
+
+접속 허용할 WebDAV 서비스 주소를 늘리고자 할 때 사용합니다.
+
+- 각 주소는 완전한 endpoint 여야 함: `https://xxxx/yyy`
+- 여러 주소는 `,`로 구분
+
+### `DEFAULT_INPUT_TEMPLATE` (선택 사항)
+
+설정 메뉴의 사용자 입력 전처리 구성 항목 초기화 시 사용할 기본 템플릿을 설정합니다.
+
+### `STABILITY_API_KEY` (선택 사항)
+
+Stability API 키입니다.
+
+### `STABILITY_URL` (선택 사항)
+
+Stability API URL을 커스터마이징합니다.
+
+### `ENABLE_MCP` (선택 사항)
+
+MCP (Model Context Protocol) 기능을 활성화합니다.
+
+### `SILICONFLOW_API_KEY` (선택 사항)
+
+SiliconFlow API 키입니다.
+
+### `SILICONFLOW_URL` (선택 사항)
+
+SiliconFlow API URL입니다.
+
+### `AI302_API_KEY` (선택 사항)
+
+302.AI API 키입니다.
+
+### `AI302_URL` (선택 사항)
+
+302.AI API URL입니다.
+
+## 요구 사항 (Requirements)
+
+NodeJS >= 18, Docker >= 20
+
+## 개발 (Development)
+
+[](https://gitpod.io/#https://github.com/Yidadaa/ChatGPT-Next-Web)
+
+개발을 시작하기 전에 프로젝트 루트에 `.env.local` 파일을 만들고, 아래와 같이 API 키를 입력하세요:
+
+```
+OPENAI_API_KEY=<여기에 API 키 입력>
+
+# OpenAI 서비스를 사용할 수 없는 경우 아래 BASE_URL 사용
+BASE_URL=https://chatgpt1.nextweb.fun/api/proxy
+```
+
+### 로컬 개발 실행
+
+```shell
+# 1. Node.js와 Yarn을 먼저 설치
+# 2. `.env.local` 파일에 환경 변수 설정
+# 3. 실행
+yarn install
+yarn dev
+```
+
+## 배포 (Deployment)
+
+### Docker (권장)
+
+```shell
+docker pull yidadaa/chatgpt-next-web
+
+docker run -d -p 3000:3000 \
+ -e OPENAI_API_KEY=sk-xxxx \
+ -e CODE=your-password \
+ yidadaa/chatgpt-next-web
+```
+
+서비스에 프록시를 사용하려면:
+
+```shell
+docker run -d -p 3000:3000 \
+ -e OPENAI_API_KEY=sk-xxxx \
+ -e CODE=your-password \
+ -e PROXY_URL=http://localhost:7890 \
+ yidadaa/chatgpt-next-web
+```
+
+프록시에 인증이 필요한 경우:
+
+```shell
+-e PROXY_URL="http://127.0.0.1:7890 user pass"
+```
+
+MCP를 활성화하려면:
+
+```
+docker run -d -p 3000:3000 \
+ -e OPENAI_API_KEY=sk-xxxx \
+ -e CODE=your-password \
+ -e ENABLE_MCP=true \
+ yidadaa/chatgpt-next-web
+```
+
+### 로컬 배포
+
+콘솔에서 다음 명령을 실행하세요.
+
+```shell
+bash <(curl -s https://raw.githubusercontent.com/Yidadaa/ChatGPT-Next-Web/main/scripts/setup.sh)
+```
+
+⚠️ 참고: 설치 중에 문제가 발생하면 Docker 배포를 사용하세요.
+
+## 채팅 기록 동기화 (UpStash)
+
+| [简体中文](./docs/synchronise-chat-logs-cn.md) | [English](./docs/synchronise-chat-logs-en.md) | [Italiano](./docs/synchronise-chat-logs-es.md) | [日本語](./docs/synchronise-chat-logs-ja.md) | [한국어](./docs/synchronise-chat-logs-ko.md)
+
+## 문서 (Documentation)
+
+> 더 많은 문서는 [docs](./docs) 디렉토리를 참고하세요.
+
+- [Cloudflare 배포 가이드 (폐기됨)](./docs/cloudflare-pages-ko.md)
+- [자주 묻는 질문](./docs/faq-ko.md)
+- [새 번역 추가 방법](./docs/translation.md)
+- [Vercel 사용법 (중문)](./docs/vercel-cn.md)
+- [사용자 매뉴얼 (중문, 작성 중)](./docs/user-manual-cn.md)
+
+## 번역 (Translation)
+
+새로운 번역을 추가하고 싶다면, [이 문서](./docs/translation.md)를 읽어보세요.
+
+## 후원 (Donation)
+
+[Buy Me a Coffee](https://www.buymeacoffee.com/yidadaa)
+
+## 특별 감사 (Special Thanks)
+
+### 기여자 (Contributors)
+
+
+
+
+
+## 라이선스 (LICENSE)
+
+[MIT](https://opensource.org/license/mit/)
diff --git a/app/api/302ai.ts b/app/api/302ai.ts
new file mode 100644
index 00000000000..b4636bcd7be
--- /dev/null
+++ b/app/api/302ai.ts
@@ -0,0 +1,128 @@
+import { getServerSideConfig } from "@/app/config/server";
+import {
+ AI302_BASE_URL,
+ ApiPath,
+ ModelProvider,
+ ServiceProvider,
+} from "@/app/constant";
+import { prettyObject } from "@/app/utils/format";
+import { NextRequest, NextResponse } from "next/server";
+import { auth } from "@/app/api/auth";
+import { isModelNotavailableInServer } from "@/app/utils/model";
+
+const serverConfig = getServerSideConfig();
+
+export async function handle(
+ req: NextRequest,
+ { params }: { params: { path: string[] } },
+) {
+ console.log("[302.AI Route] params ", params);
+
+ if (req.method === "OPTIONS") {
+ return NextResponse.json({ body: "OK" }, { status: 200 });
+ }
+
+ const authResult = auth(req, ModelProvider["302.AI"]);
+ if (authResult.error) {
+ return NextResponse.json(authResult, {
+ status: 401,
+ });
+ }
+
+ try {
+ const response = await request(req);
+ return response;
+ } catch (e) {
+ console.error("[302.AI] ", e);
+ return NextResponse.json(prettyObject(e));
+ }
+}
+
+async function request(req: NextRequest) {
+ const controller = new AbortController();
+
+ // alibaba use base url or just remove the path
+ let path = `${req.nextUrl.pathname}`.replaceAll(ApiPath["302.AI"], "");
+
+ let baseUrl = serverConfig.ai302Url || AI302_BASE_URL;
+
+ if (!baseUrl.startsWith("http")) {
+ baseUrl = `https://${baseUrl}`;
+ }
+
+ if (baseUrl.endsWith("/")) {
+ baseUrl = baseUrl.slice(0, -1);
+ }
+
+ console.log("[Proxy] ", path);
+ console.log("[Base Url]", baseUrl);
+
+ const timeoutId = setTimeout(
+ () => {
+ controller.abort();
+ },
+ 10 * 60 * 1000,
+ );
+
+ const fetchUrl = `${baseUrl}${path}`;
+ const fetchOptions: RequestInit = {
+ headers: {
+ "Content-Type": "application/json",
+ Authorization: req.headers.get("Authorization") ?? "",
+ },
+ method: req.method,
+ body: req.body,
+ redirect: "manual",
+ // @ts-ignore
+ duplex: "half",
+ signal: controller.signal,
+ };
+
+ // #1815 try to refuse some request to some models
+ if (serverConfig.customModels && req.body) {
+ try {
+ const clonedBody = await req.text();
+ fetchOptions.body = clonedBody;
+
+ const jsonBody = JSON.parse(clonedBody) as { model?: string };
+
+ // not undefined and is false
+ if (
+ isModelNotavailableInServer(
+ serverConfig.customModels,
+ jsonBody?.model as string,
+ ServiceProvider["302.AI"] as string,
+ )
+ ) {
+ return NextResponse.json(
+ {
+ error: true,
+ message: `you are not allowed to use ${jsonBody?.model} model`,
+ },
+ {
+ status: 403,
+ },
+ );
+ }
+ } catch (e) {
+ console.error(`[302.AI] filter`, e);
+ }
+ }
+ try {
+ const res = await fetch(fetchUrl, fetchOptions);
+
+ // to prevent browser prompt for credentials
+ const newHeaders = new Headers(res.headers);
+ newHeaders.delete("www-authenticate");
+ // to disable nginx buffering
+ newHeaders.set("X-Accel-Buffering", "no");
+
+ return new Response(res.body, {
+ status: res.status,
+ statusText: res.statusText,
+ headers: newHeaders,
+ });
+ } finally {
+ clearTimeout(timeoutId);
+ }
+}
diff --git a/app/api/[provider]/[...path]/route.ts b/app/api/[provider]/[...path]/route.ts
index 5ac248d0c87..e8af34f29f8 100644
--- a/app/api/[provider]/[...path]/route.ts
+++ b/app/api/[provider]/[...path]/route.ts
@@ -10,8 +10,12 @@ import { handle as alibabaHandler } from "../../alibaba";
import { handle as moonshotHandler } from "../../moonshot";
import { handle as stabilityHandler } from "../../stability";
import { handle as iflytekHandler } from "../../iflytek";
+import { handle as deepseekHandler } from "../../deepseek";
+import { handle as siliconflowHandler } from "../../siliconflow";
import { handle as xaiHandler } from "../../xai";
+import { handle as chatglmHandler } from "../../glm";
import { handle as proxyHandler } from "../../proxy";
+import { handle as ai302Handler } from "../../302ai";
async function handle(
req: NextRequest,
@@ -39,10 +43,18 @@ async function handle(
return stabilityHandler(req, { params });
case ApiPath.Iflytek:
return iflytekHandler(req, { params });
+ case ApiPath.DeepSeek:
+ return deepseekHandler(req, { params });
case ApiPath.XAI:
return xaiHandler(req, { params });
+ case ApiPath.ChatGLM:
+ return chatglmHandler(req, { params });
+ case ApiPath.SiliconFlow:
+ return siliconflowHandler(req, { params });
case ApiPath.OpenAI:
return openaiHandler(req, { params });
+ case ApiPath["302.AI"]:
+ return ai302Handler(req, { params });
default:
return proxyHandler(req, { params });
}
diff --git a/app/api/alibaba.ts b/app/api/alibaba.ts
index 894b1ae4c04..20f6caefa8d 100644
--- a/app/api/alibaba.ts
+++ b/app/api/alibaba.ts
@@ -8,7 +8,7 @@ import {
import { prettyObject } from "@/app/utils/format";
import { NextRequest, NextResponse } from "next/server";
import { auth } from "@/app/api/auth";
-import { isModelAvailableInServer } from "@/app/utils/model";
+import { isModelNotavailableInServer } from "@/app/utils/model";
const serverConfig = getServerSideConfig();
@@ -89,7 +89,7 @@ async function request(req: NextRequest) {
// not undefined and is false
if (
- isModelAvailableInServer(
+ isModelNotavailableInServer(
serverConfig.customModels,
jsonBody?.model as string,
ServiceProvider.Alibaba as string,
diff --git a/app/api/anthropic.ts b/app/api/anthropic.ts
index 7a44443710f..b96637b2c8c 100644
--- a/app/api/anthropic.ts
+++ b/app/api/anthropic.ts
@@ -9,7 +9,7 @@ import {
import { prettyObject } from "@/app/utils/format";
import { NextRequest, NextResponse } from "next/server";
import { auth } from "./auth";
-import { isModelAvailableInServer } from "@/app/utils/model";
+import { isModelNotavailableInServer } from "@/app/utils/model";
import { cloudflareAIGatewayUrl } from "@/app/utils/cloudflare";
const ALLOWD_PATH = new Set([Anthropic.ChatPath, Anthropic.ChatPath1]);
@@ -122,7 +122,7 @@ async function request(req: NextRequest) {
// not undefined and is false
if (
- isModelAvailableInServer(
+ isModelNotavailableInServer(
serverConfig.customModels,
jsonBody?.model as string,
ServiceProvider.Anthropic as string,
diff --git a/app/api/auth.ts b/app/api/auth.ts
index d4ac66a113b..8c78c70c865 100644
--- a/app/api/auth.ts
+++ b/app/api/auth.ts
@@ -92,9 +92,18 @@ export function auth(req: NextRequest, modelProvider: ModelProvider) {
systemApiKey =
serverConfig.iflytekApiKey + ":" + serverConfig.iflytekApiSecret;
break;
+ case ModelProvider.DeepSeek:
+ systemApiKey = serverConfig.deepseekApiKey;
+ break;
case ModelProvider.XAI:
systemApiKey = serverConfig.xaiApiKey;
break;
+ case ModelProvider.ChatGLM:
+ systemApiKey = serverConfig.chatglmApiKey;
+ break;
+ case ModelProvider.SiliconFlow:
+ systemApiKey = serverConfig.siliconFlowApiKey;
+ break;
case ModelProvider.GPT:
default:
if (req.nextUrl.pathname.includes("azure/deployments")) {
diff --git a/app/api/baidu.ts b/app/api/baidu.ts
index 0408b43c5bc..0f4e05ee86c 100644
--- a/app/api/baidu.ts
+++ b/app/api/baidu.ts
@@ -8,7 +8,7 @@ import {
import { prettyObject } from "@/app/utils/format";
import { NextRequest, NextResponse } from "next/server";
import { auth } from "@/app/api/auth";
-import { isModelAvailableInServer } from "@/app/utils/model";
+import { isModelNotavailableInServer } from "@/app/utils/model";
import { getAccessToken } from "@/app/utils/baidu";
const serverConfig = getServerSideConfig();
@@ -104,7 +104,7 @@ async function request(req: NextRequest) {
// not undefined and is false
if (
- isModelAvailableInServer(
+ isModelNotavailableInServer(
serverConfig.customModels,
jsonBody?.model as string,
ServiceProvider.Baidu as string,
diff --git a/app/api/bytedance.ts b/app/api/bytedance.ts
index cb65b106109..51b39ceb7cb 100644
--- a/app/api/bytedance.ts
+++ b/app/api/bytedance.ts
@@ -8,7 +8,7 @@ import {
import { prettyObject } from "@/app/utils/format";
import { NextRequest, NextResponse } from "next/server";
import { auth } from "@/app/api/auth";
-import { isModelAvailableInServer } from "@/app/utils/model";
+import { isModelNotavailableInServer } from "@/app/utils/model";
const serverConfig = getServerSideConfig();
@@ -88,7 +88,7 @@ async function request(req: NextRequest) {
// not undefined and is false
if (
- isModelAvailableInServer(
+ isModelNotavailableInServer(
serverConfig.customModels,
jsonBody?.model as string,
ServiceProvider.ByteDance as string,
diff --git a/app/api/common.ts b/app/api/common.ts
index b4c792d6ff0..b7e41fa2647 100644
--- a/app/api/common.ts
+++ b/app/api/common.ts
@@ -1,8 +1,8 @@
import { NextRequest, NextResponse } from "next/server";
import { getServerSideConfig } from "../config/server";
import { OPENAI_BASE_URL, ServiceProvider } from "../constant";
-import { isModelAvailableInServer } from "../utils/model";
import { cloudflareAIGatewayUrl } from "../utils/cloudflare";
+import { getModelProvider, isModelNotavailableInServer } from "../utils/model";
const serverConfig = getServerSideConfig();
@@ -71,7 +71,7 @@ export async function requestOpenai(req: NextRequest) {
.filter((v) => !!v && !v.startsWith("-") && v.includes(modelName))
.forEach((m) => {
const [fullName, displayName] = m.split("=");
- const [_, providerName] = fullName.split("@");
+ const [_, providerName] = getModelProvider(fullName);
if (providerName === "azure" && !displayName) {
const [_, deployId] = (serverConfig?.azureUrl ?? "").split(
"deployments/",
@@ -118,15 +118,14 @@ export async function requestOpenai(req: NextRequest) {
// not undefined and is false
if (
- isModelAvailableInServer(
+ isModelNotavailableInServer(
serverConfig.customModels,
jsonBody?.model as string,
- ServiceProvider.OpenAI as string,
- ) ||
- isModelAvailableInServer(
- serverConfig.customModels,
- jsonBody?.model as string,
- ServiceProvider.Azure as string,
+ [
+ ServiceProvider.OpenAI,
+ ServiceProvider.Azure,
+ jsonBody?.model as string, // support provider-unspecified model
+ ],
)
) {
return NextResponse.json(
diff --git a/app/api/config/route.ts b/app/api/config/route.ts
index b0d9da03103..855a5db017e 100644
--- a/app/api/config/route.ts
+++ b/app/api/config/route.ts
@@ -14,6 +14,7 @@ const DANGER_CONFIG = {
disableFastLink: serverConfig.disableFastLink,
customModels: serverConfig.customModels,
defaultModel: serverConfig.defaultModel,
+ visionModels: serverConfig.visionModels,
};
declare global {
diff --git a/app/api/deepseek.ts b/app/api/deepseek.ts
new file mode 100644
index 00000000000..a9879ecedeb
--- /dev/null
+++ b/app/api/deepseek.ts
@@ -0,0 +1,128 @@
+import { getServerSideConfig } from "@/app/config/server";
+import {
+ DEEPSEEK_BASE_URL,
+ ApiPath,
+ ModelProvider,
+ ServiceProvider,
+} from "@/app/constant";
+import { prettyObject } from "@/app/utils/format";
+import { NextRequest, NextResponse } from "next/server";
+import { auth } from "@/app/api/auth";
+import { isModelNotavailableInServer } from "@/app/utils/model";
+
+const serverConfig = getServerSideConfig();
+
+export async function handle(
+ req: NextRequest,
+ { params }: { params: { path: string[] } },
+) {
+ console.log("[DeepSeek Route] params ", params);
+
+ if (req.method === "OPTIONS") {
+ return NextResponse.json({ body: "OK" }, { status: 200 });
+ }
+
+ const authResult = auth(req, ModelProvider.DeepSeek);
+ if (authResult.error) {
+ return NextResponse.json(authResult, {
+ status: 401,
+ });
+ }
+
+ try {
+ const response = await request(req);
+ return response;
+ } catch (e) {
+ console.error("[DeepSeek] ", e);
+ return NextResponse.json(prettyObject(e));
+ }
+}
+
+async function request(req: NextRequest) {
+ const controller = new AbortController();
+
+ // alibaba use base url or just remove the path
+ let path = `${req.nextUrl.pathname}`.replaceAll(ApiPath.DeepSeek, "");
+
+ let baseUrl = serverConfig.deepseekUrl || DEEPSEEK_BASE_URL;
+
+ if (!baseUrl.startsWith("http")) {
+ baseUrl = `https://${baseUrl}`;
+ }
+
+ if (baseUrl.endsWith("/")) {
+ baseUrl = baseUrl.slice(0, -1);
+ }
+
+ console.log("[Proxy] ", path);
+ console.log("[Base Url]", baseUrl);
+
+ const timeoutId = setTimeout(
+ () => {
+ controller.abort();
+ },
+ 10 * 60 * 1000,
+ );
+
+ const fetchUrl = `${baseUrl}${path}`;
+ const fetchOptions: RequestInit = {
+ headers: {
+ "Content-Type": "application/json",
+ Authorization: req.headers.get("Authorization") ?? "",
+ },
+ method: req.method,
+ body: req.body,
+ redirect: "manual",
+ // @ts-ignore
+ duplex: "half",
+ signal: controller.signal,
+ };
+
+ // #1815 try to refuse some request to some models
+ if (serverConfig.customModels && req.body) {
+ try {
+ const clonedBody = await req.text();
+ fetchOptions.body = clonedBody;
+
+ const jsonBody = JSON.parse(clonedBody) as { model?: string };
+
+ // not undefined and is false
+ if (
+ isModelNotavailableInServer(
+ serverConfig.customModels,
+ jsonBody?.model as string,
+ ServiceProvider.DeepSeek as string,
+ )
+ ) {
+ return NextResponse.json(
+ {
+ error: true,
+ message: `you are not allowed to use ${jsonBody?.model} model`,
+ },
+ {
+ status: 403,
+ },
+ );
+ }
+ } catch (e) {
+ console.error(`[DeepSeek] filter`, e);
+ }
+ }
+ try {
+ const res = await fetch(fetchUrl, fetchOptions);
+
+ // to prevent browser prompt for credentials
+ const newHeaders = new Headers(res.headers);
+ newHeaders.delete("www-authenticate");
+ // to disable nginx buffering
+ newHeaders.set("X-Accel-Buffering", "no");
+
+ return new Response(res.body, {
+ status: res.status,
+ statusText: res.statusText,
+ headers: newHeaders,
+ });
+ } finally {
+ clearTimeout(timeoutId);
+ }
+}
diff --git a/app/api/glm.ts b/app/api/glm.ts
new file mode 100644
index 00000000000..8431c5db5b0
--- /dev/null
+++ b/app/api/glm.ts
@@ -0,0 +1,129 @@
+import { getServerSideConfig } from "@/app/config/server";
+import {
+ CHATGLM_BASE_URL,
+ ApiPath,
+ ModelProvider,
+ ServiceProvider,
+} from "@/app/constant";
+import { prettyObject } from "@/app/utils/format";
+import { NextRequest, NextResponse } from "next/server";
+import { auth } from "@/app/api/auth";
+import { isModelNotavailableInServer } from "@/app/utils/model";
+
+const serverConfig = getServerSideConfig();
+
+export async function handle(
+ req: NextRequest,
+ { params }: { params: { path: string[] } },
+) {
+ console.log("[GLM Route] params ", params);
+
+ if (req.method === "OPTIONS") {
+ return NextResponse.json({ body: "OK" }, { status: 200 });
+ }
+
+ const authResult = auth(req, ModelProvider.ChatGLM);
+ if (authResult.error) {
+ return NextResponse.json(authResult, {
+ status: 401,
+ });
+ }
+
+ try {
+ const response = await request(req);
+ return response;
+ } catch (e) {
+ console.error("[GLM] ", e);
+ return NextResponse.json(prettyObject(e));
+ }
+}
+
+async function request(req: NextRequest) {
+ const controller = new AbortController();
+
+ // alibaba use base url or just remove the path
+ let path = `${req.nextUrl.pathname}`.replaceAll(ApiPath.ChatGLM, "");
+
+ let baseUrl = serverConfig.chatglmUrl || CHATGLM_BASE_URL;
+
+ if (!baseUrl.startsWith("http")) {
+ baseUrl = `https://${baseUrl}`;
+ }
+
+ if (baseUrl.endsWith("/")) {
+ baseUrl = baseUrl.slice(0, -1);
+ }
+
+ console.log("[Proxy] ", path);
+ console.log("[Base Url]", baseUrl);
+
+ const timeoutId = setTimeout(
+ () => {
+ controller.abort();
+ },
+ 10 * 60 * 1000,
+ );
+
+ const fetchUrl = `${baseUrl}${path}`;
+ console.log("[Fetch Url] ", fetchUrl);
+ const fetchOptions: RequestInit = {
+ headers: {
+ "Content-Type": "application/json",
+ Authorization: req.headers.get("Authorization") ?? "",
+ },
+ method: req.method,
+ body: req.body,
+ redirect: "manual",
+ // @ts-ignore
+ duplex: "half",
+ signal: controller.signal,
+ };
+
+ // #1815 try to refuse some request to some models
+ if (serverConfig.customModels && req.body) {
+ try {
+ const clonedBody = await req.text();
+ fetchOptions.body = clonedBody;
+
+ const jsonBody = JSON.parse(clonedBody) as { model?: string };
+
+ // not undefined and is false
+ if (
+ isModelNotavailableInServer(
+ serverConfig.customModels,
+ jsonBody?.model as string,
+ ServiceProvider.ChatGLM as string,
+ )
+ ) {
+ return NextResponse.json(
+ {
+ error: true,
+ message: `you are not allowed to use ${jsonBody?.model} model`,
+ },
+ {
+ status: 403,
+ },
+ );
+ }
+ } catch (e) {
+ console.error(`[GLM] filter`, e);
+ }
+ }
+ try {
+ const res = await fetch(fetchUrl, fetchOptions);
+
+ // to prevent browser prompt for credentials
+ const newHeaders = new Headers(res.headers);
+ newHeaders.delete("www-authenticate");
+ // to disable nginx buffering
+ newHeaders.set("X-Accel-Buffering", "no");
+
+ return new Response(res.body, {
+ status: res.status,
+ statusText: res.statusText,
+ headers: newHeaders,
+ });
+ } finally {
+ clearTimeout(timeoutId);
+ }
+}
diff --git a/app/api/iflytek.ts b/app/api/iflytek.ts
index 8b8227dce1f..6624f74e9ab 100644
--- a/app/api/iflytek.ts
+++ b/app/api/iflytek.ts
@@ -8,7 +8,7 @@ import {
import { prettyObject } from "@/app/utils/format";
import { NextRequest, NextResponse } from "next/server";
import { auth } from "@/app/api/auth";
-import { isModelAvailableInServer } from "@/app/utils/model";
+import { isModelNotavailableInServer } from "@/app/utils/model";
// iflytek
const serverConfig = getServerSideConfig();
@@ -89,7 +89,7 @@ async function request(req: NextRequest) {
// not undefined and is false
if (
- isModelAvailableInServer(
+ isModelNotavailableInServer(
serverConfig.customModels,
jsonBody?.model as string,
ServiceProvider.Iflytek as string,
diff --git a/app/api/moonshot.ts b/app/api/moonshot.ts
index 5bf4807e3e6..792d14d3334 100644
--- a/app/api/moonshot.ts
+++ b/app/api/moonshot.ts
@@ -8,7 +8,7 @@ import {
import { prettyObject } from "@/app/utils/format";
import { NextRequest, NextResponse } from "next/server";
import { auth } from "@/app/api/auth";
-import { isModelAvailableInServer } from "@/app/utils/model";
+import { isModelNotavailableInServer } from "@/app/utils/model";
const serverConfig = getServerSideConfig();
@@ -88,7 +88,7 @@ async function request(req: NextRequest) {
// not undefined and is false
if (
- isModelAvailableInServer(
+ isModelNotavailableInServer(
serverConfig.customModels,
jsonBody?.model as string,
ServiceProvider.Moonshot as string,
diff --git a/app/api/openai.ts b/app/api/openai.ts
index bbba69e569c..5bb3a5dcb64 100644
--- a/app/api/openai.ts
+++ b/app/api/openai.ts
@@ -14,8 +14,12 @@ function getModels(remoteModelRes: OpenAIListModelResponse) {
if (config.disableGPT4) {
remoteModelRes.data = remoteModelRes.data.filter(
(m) =>
- !(m.id.startsWith("gpt-4") || m.id.startsWith("chatgpt-4o")) ||
- m.id.startsWith("gpt-4o-mini"),
+ !(
+ m.id.startsWith("gpt-4") ||
+ m.id.startsWith("chatgpt-4o") ||
+ m.id.startsWith("o1") ||
+ m.id.startsWith("o3")
+ ) || m.id.startsWith("gpt-4o-mini"),
);
}
diff --git a/app/api/proxy.ts b/app/api/proxy.ts
index 731003aa1ea..b3e5e7b7b93 100644
--- a/app/api/proxy.ts
+++ b/app/api/proxy.ts
@@ -1,4 +1,5 @@
import { NextRequest, NextResponse } from "next/server";
+import { getServerSideConfig } from "@/app/config/server";
export async function handle(
req: NextRequest,
@@ -9,6 +10,7 @@ export async function handle(
if (req.method === "OPTIONS") {
return NextResponse.json({ body: "OK" }, { status: 200 });
}
+ const serverConfig = getServerSideConfig();
// remove path params from searchParams
req.nextUrl.searchParams.delete("path");
@@ -31,6 +33,18 @@ export async function handle(
return true;
}),
);
+ // if dalle3 use openai api key
+ const baseUrl = req.headers.get("x-base-url");
+ if (baseUrl?.includes("api.openai.com")) {
+ if (!serverConfig.apiKey) {
+ return NextResponse.json(
+ { error: "OpenAI API key not configured" },
+ { status: 500 },
+ );
+ }
+ headers.set("Authorization", `Bearer ${serverConfig.apiKey}`);
+ }
+
const controller = new AbortController();
const fetchOptions: RequestInit = {
headers,
diff --git a/app/api/siliconflow.ts b/app/api/siliconflow.ts
new file mode 100644
index 00000000000..e298a21d43c
--- /dev/null
+++ b/app/api/siliconflow.ts
@@ -0,0 +1,128 @@
+import { getServerSideConfig } from "@/app/config/server";
+import {
+ SILICONFLOW_BASE_URL,
+ ApiPath,
+ ModelProvider,
+ ServiceProvider,
+} from "@/app/constant";
+import { prettyObject } from "@/app/utils/format";
+import { NextRequest, NextResponse } from "next/server";
+import { auth } from "@/app/api/auth";
+import { isModelNotavailableInServer } from "@/app/utils/model";
+
+const serverConfig = getServerSideConfig();
+
+export async function handle(
+ req: NextRequest,
+ { params }: { params: { path: string[] } },
+) {
+ console.log("[SiliconFlow Route] params ", params);
+
+ if (req.method === "OPTIONS") {
+ return NextResponse.json({ body: "OK" }, { status: 200 });
+ }
+
+ const authResult = auth(req, ModelProvider.SiliconFlow);
+ if (authResult.error) {
+ return NextResponse.json(authResult, {
+ status: 401,
+ });
+ }
+
+ try {
+ const response = await request(req);
+ return response;
+ } catch (e) {
+ console.error("[SiliconFlow] ", e);
+ return NextResponse.json(prettyObject(e));
+ }
+}
+
+async function request(req: NextRequest) {
+ const controller = new AbortController();
+
+ // alibaba use base url or just remove the path
+ let path = `${req.nextUrl.pathname}`.replaceAll(ApiPath.SiliconFlow, "");
+
+ let baseUrl = serverConfig.siliconFlowUrl || SILICONFLOW_BASE_URL;
+
+ if (!baseUrl.startsWith("http")) {
+ baseUrl = `https://${baseUrl}`;
+ }
+
+ if (baseUrl.endsWith("/")) {
+ baseUrl = baseUrl.slice(0, -1);
+ }
+
+ console.log("[Proxy] ", path);
+ console.log("[Base Url]", baseUrl);
+
+ const timeoutId = setTimeout(
+ () => {
+ controller.abort();
+ },
+ 10 * 60 * 1000,
+ );
+
+ const fetchUrl = `${baseUrl}${path}`;
+ const fetchOptions: RequestInit = {
+ headers: {
+ "Content-Type": "application/json",
+ Authorization: req.headers.get("Authorization") ?? "",
+ },
+ method: req.method,
+ body: req.body,
+ redirect: "manual",
+ // @ts-ignore
+ duplex: "half",
+ signal: controller.signal,
+ };
+
+ // #1815 try to refuse some request to some models
+ if (serverConfig.customModels && req.body) {
+ try {
+ const clonedBody = await req.text();
+ fetchOptions.body = clonedBody;
+
+ const jsonBody = JSON.parse(clonedBody) as { model?: string };
+
+ // not undefined and is false
+ if (
+ isModelNotavailableInServer(
+ serverConfig.customModels,
+ jsonBody?.model as string,
+ ServiceProvider.SiliconFlow as string,
+ )
+ ) {
+ return NextResponse.json(
+ {
+ error: true,
+ message: `you are not allowed to use ${jsonBody?.model} model`,
+ },
+ {
+ status: 403,
+ },
+ );
+ }
+ } catch (e) {
+ console.error(`[SiliconFlow] filter`, e);
+ }
+ }
+ try {
+ const res = await fetch(fetchUrl, fetchOptions);
+
+ // to prevent browser prompt for credentials
+ const newHeaders = new Headers(res.headers);
+ newHeaders.delete("www-authenticate");
+ // to disable nginx buffering
+ newHeaders.set("X-Accel-Buffering", "no");
+
+ return new Response(res.body, {
+ status: res.status,
+ statusText: res.statusText,
+ headers: newHeaders,
+ });
+ } finally {
+ clearTimeout(timeoutId);
+ }
+}
diff --git a/app/api/xai.ts b/app/api/xai.ts
index a4ee8b39731..4aad5e5fb3e 100644
--- a/app/api/xai.ts
+++ b/app/api/xai.ts
@@ -8,7 +8,7 @@ import {
import { prettyObject } from "@/app/utils/format";
import { NextRequest, NextResponse } from "next/server";
import { auth } from "@/app/api/auth";
-import { isModelAvailableInServer } from "@/app/utils/model";
+import { isModelNotavailableInServer } from "@/app/utils/model";
const serverConfig = getServerSideConfig();
@@ -88,7 +88,7 @@ async function request(req: NextRequest) {
// not undefined and is false
if (
- isModelAvailableInServer(
+ isModelNotavailableInServer(
serverConfig.customModels,
jsonBody?.model as string,
ServiceProvider.XAI as string,
diff --git a/app/client/api.ts b/app/client/api.ts
index 4238c2a264b..f60b0e2ad71 100644
--- a/app/client/api.ts
+++ b/app/client/api.ts
@@ -20,7 +20,11 @@ import { QwenApi } from "./platforms/alibaba";
import { HunyuanApi } from "./platforms/tencent";
import { MoonshotApi } from "./platforms/moonshot";
import { SparkApi } from "./platforms/iflytek";
+import { DeepSeekApi } from "./platforms/deepseek";
import { XAIApi } from "./platforms/xai";
+import { ChatGLMApi } from "./platforms/glm";
+import { SiliconflowApi } from "./platforms/siliconflow";
+import { Ai302Api } from "./platforms/ai302";
export const ROLES = ["system", "user", "assistant"] as const;
export type MessageRole = (typeof ROLES)[number];
@@ -37,6 +41,11 @@ export interface MultimodalContent {
};
}
+export interface MultimodalContentForAlibaba {
+ text?: string;
+ image?: string;
+}
+
export interface RequestMessage {
role: MessageRole;
content: string | MultimodalContent[];
@@ -69,7 +78,7 @@ export interface ChatOptions {
config: LLMConfig;
onUpdate?: (message: string, chunk: string) => void;
- onFinish: (message: string) => void;
+ onFinish: (message: string, responseRes: Response) => void;
onError?: (err: Error) => void;
onController?: (controller: AbortController) => void;
onBeforeTool?: (tool: ChatMessageTool) => void;
@@ -153,9 +162,21 @@ export class ClientApi {
case ModelProvider.Iflytek:
this.llm = new SparkApi();
break;
+ case ModelProvider.DeepSeek:
+ this.llm = new DeepSeekApi();
+ break;
case ModelProvider.XAI:
this.llm = new XAIApi();
break;
+ case ModelProvider.ChatGLM:
+ this.llm = new ChatGLMApi();
+ break;
+ case ModelProvider.SiliconFlow:
+ this.llm = new SiliconflowApi();
+ break;
+ case ModelProvider["302.AI"]:
+ this.llm = new Ai302Api();
+ break;
default:
this.llm = new ChatGPTApi();
}
@@ -243,7 +264,12 @@ export function getHeaders(ignoreHeaders: boolean = false) {
const isAlibaba = modelConfig.providerName === ServiceProvider.Alibaba;
const isMoonshot = modelConfig.providerName === ServiceProvider.Moonshot;
const isIflytek = modelConfig.providerName === ServiceProvider.Iflytek;
+ const isDeepSeek = modelConfig.providerName === ServiceProvider.DeepSeek;
const isXAI = modelConfig.providerName === ServiceProvider.XAI;
+ const isChatGLM = modelConfig.providerName === ServiceProvider.ChatGLM;
+ const isSiliconFlow =
+ modelConfig.providerName === ServiceProvider.SiliconFlow;
+ const isAI302 = modelConfig.providerName === ServiceProvider["302.AI"];
const isEnabledAccessControl = accessStore.enabledAccessControl();
const apiKey = isGoogle
? accessStore.googleApiKey
@@ -259,10 +285,18 @@ export function getHeaders(ignoreHeaders: boolean = false) {
? accessStore.moonshotApiKey
: isXAI
? accessStore.xaiApiKey
+ : isDeepSeek
+ ? accessStore.deepseekApiKey
+ : isChatGLM
+ ? accessStore.chatglmApiKey
+ : isSiliconFlow
+ ? accessStore.siliconflowApiKey
: isIflytek
? accessStore.iflytekApiKey && accessStore.iflytekApiSecret
? accessStore.iflytekApiKey + ":" + accessStore.iflytekApiSecret
: ""
+ : isAI302
+ ? accessStore.ai302ApiKey
: accessStore.openaiApiKey;
return {
isGoogle,
@@ -273,7 +307,11 @@ export function getHeaders(ignoreHeaders: boolean = false) {
isAlibaba,
isMoonshot,
isIflytek,
+ isDeepSeek,
isXAI,
+ isChatGLM,
+ isSiliconFlow,
+ isAI302,
apiKey,
isEnabledAccessControl,
};
@@ -294,6 +332,15 @@ export function getHeaders(ignoreHeaders: boolean = false) {
isAzure,
isAnthropic,
isBaidu,
+ isByteDance,
+ isAlibaba,
+ isMoonshot,
+ isIflytek,
+ isDeepSeek,
+ isXAI,
+ isChatGLM,
+ isSiliconFlow,
+ isAI302,
apiKey,
isEnabledAccessControl,
} = getConfig();
@@ -336,8 +383,16 @@ export function getClientApi(provider: ServiceProvider): ClientApi {
return new ClientApi(ModelProvider.Moonshot);
case ServiceProvider.Iflytek:
return new ClientApi(ModelProvider.Iflytek);
+ case ServiceProvider.DeepSeek:
+ return new ClientApi(ModelProvider.DeepSeek);
case ServiceProvider.XAI:
return new ClientApi(ModelProvider.XAI);
+ case ServiceProvider.ChatGLM:
+ return new ClientApi(ModelProvider.ChatGLM);
+ case ServiceProvider.SiliconFlow:
+ return new ClientApi(ModelProvider.SiliconFlow);
+ case ServiceProvider["302.AI"]:
+ return new ClientApi(ModelProvider["302.AI"]);
default:
return new ClientApi(ModelProvider.GPT);
}
diff --git a/app/client/platforms/ai302.ts b/app/client/platforms/ai302.ts
new file mode 100644
index 00000000000..7bd18f96d4c
--- /dev/null
+++ b/app/client/platforms/ai302.ts
@@ -0,0 +1,287 @@
+"use client";
+
+import {
+ ApiPath,
+ AI302_BASE_URL,
+ DEFAULT_MODELS,
+ AI302,
+} from "@/app/constant";
+import {
+ useAccessStore,
+ useAppConfig,
+ useChatStore,
+ ChatMessageTool,
+ usePluginStore,
+} from "@/app/store";
+import { preProcessImageContent, streamWithThink } from "@/app/utils/chat";
+import {
+ ChatOptions,
+ getHeaders,
+ LLMApi,
+ LLMModel,
+ SpeechOptions,
+} from "../api";
+import { getClientConfig } from "@/app/config/client";
+import {
+ getMessageTextContent,
+ getMessageTextContentWithoutThinking,
+ isVisionModel,
+ getTimeoutMSByModel,
+} from "@/app/utils";
+import { RequestPayload } from "./openai";
+
+import { fetch } from "@/app/utils/stream";
+export interface Ai302ListModelResponse {
+ object: string;
+ data: Array<{
+ id: string;
+ object: string;
+ root: string;
+ }>;
+}
+
+export class Ai302Api implements LLMApi {
+ private disableListModels = false;
+
+ path(path: string): string {
+ const accessStore = useAccessStore.getState();
+
+ let baseUrl = "";
+
+ if (accessStore.useCustomConfig) {
+ baseUrl = accessStore.ai302Url;
+ }
+
+ if (baseUrl.length === 0) {
+ const isApp = !!getClientConfig()?.isApp;
+ const apiPath = ApiPath["302.AI"];
+ baseUrl = isApp ? AI302_BASE_URL : apiPath;
+ }
+
+ if (baseUrl.endsWith("/")) {
+ baseUrl = baseUrl.slice(0, baseUrl.length - 1);
+ }
+ if (
+ !baseUrl.startsWith("http") &&
+ !baseUrl.startsWith(ApiPath["302.AI"])
+ ) {
+ baseUrl = "https://" + baseUrl;
+ }
+
+ console.log("[Proxy Endpoint] ", baseUrl, path);
+
+ return [baseUrl, path].join("/");
+ }
+
+ extractMessage(res: any) {
+ return res.choices?.at(0)?.message?.content ?? "";
+ }
+
+ speech(options: SpeechOptions): Promise {
+ throw new Error("Method not implemented.");
+ }
+
+ async chat(options: ChatOptions) {
+ const visionModel = isVisionModel(options.config.model);
+ const messages: ChatOptions["messages"] = [];
+ for (const v of options.messages) {
+ if (v.role === "assistant") {
+ const content = getMessageTextContentWithoutThinking(v);
+ messages.push({ role: v.role, content });
+ } else {
+ const content = visionModel
+ ? await preProcessImageContent(v.content)
+ : getMessageTextContent(v);
+ messages.push({ role: v.role, content });
+ }
+ }
+
+ const modelConfig = {
+ ...useAppConfig.getState().modelConfig,
+ ...useChatStore.getState().currentSession().mask.modelConfig,
+ ...{
+ model: options.config.model,
+ providerName: options.config.providerName,
+ },
+ };
+
+ const requestPayload: RequestPayload = {
+ messages,
+ stream: options.config.stream,
+ model: modelConfig.model,
+ temperature: modelConfig.temperature,
+ presence_penalty: modelConfig.presence_penalty,
+ frequency_penalty: modelConfig.frequency_penalty,
+ top_p: modelConfig.top_p,
+ // max_tokens: Math.max(modelConfig.max_tokens, 1024),
+ // Please do not ask me why not send max_tokens, no reason, this param is just shit, I dont want to explain anymore.
+ };
+
+ console.log("[Request] openai payload: ", requestPayload);
+
+ const shouldStream = !!options.config.stream;
+ const controller = new AbortController();
+ options.onController?.(controller);
+
+ try {
+ const chatPath = this.path(AI302.ChatPath);
+ const chatPayload = {
+ method: "POST",
+ body: JSON.stringify(requestPayload),
+ signal: controller.signal,
+ headers: getHeaders(),
+ };
+
+ // console.log(chatPayload);
+
+ // Use extended timeout for thinking models as they typically require more processing time
+ const requestTimeoutId = setTimeout(
+ () => controller.abort(),
+ getTimeoutMSByModel(options.config.model),
+ );
+
+ if (shouldStream) {
+ const [tools, funcs] = usePluginStore
+ .getState()
+ .getAsTools(
+ useChatStore.getState().currentSession().mask?.plugin || [],
+ );
+ return streamWithThink(
+ chatPath,
+ requestPayload,
+ getHeaders(),
+ tools as any,
+ funcs,
+ controller,
+ // parseSSE
+ (text: string, runTools: ChatMessageTool[]) => {
+ // console.log("parseSSE", text, runTools);
+ const json = JSON.parse(text);
+ const choices = json.choices as Array<{
+ delta: {
+ content: string | null;
+ tool_calls: ChatMessageTool[];
+ reasoning_content: string | null;
+ };
+ }>;
+ const tool_calls = choices[0]?.delta?.tool_calls;
+ if (tool_calls?.length > 0) {
+ const index = tool_calls[0]?.index;
+ const id = tool_calls[0]?.id;
+ const args = tool_calls[0]?.function?.arguments;
+ if (id) {
+ runTools.push({
+ id,
+ type: tool_calls[0]?.type,
+ function: {
+ name: tool_calls[0]?.function?.name as string,
+ arguments: args,
+ },
+ });
+ } else {
+ // @ts-ignore
+ runTools[index]["function"]["arguments"] += args;
+ }
+ }
+ const reasoning = choices[0]?.delta?.reasoning_content;
+ const content = choices[0]?.delta?.content;
+
+ // Skip if both content and reasoning_content are empty or null
+ if (
+ (!reasoning || reasoning.length === 0) &&
+ (!content || content.length === 0)
+ ) {
+ return {
+ isThinking: false,
+ content: "",
+ };
+ }
+
+ if (reasoning && reasoning.length > 0) {
+ return {
+ isThinking: true,
+ content: reasoning,
+ };
+ } else if (content && content.length > 0) {
+ return {
+ isThinking: false,
+ content: content,
+ };
+ }
+
+ return {
+ isThinking: false,
+ content: "",
+ };
+ },
+ // processToolMessage, include tool_calls message and tool call results
+ (
+ requestPayload: RequestPayload,
+ toolCallMessage: any,
+ toolCallResult: any[],
+ ) => {
+ // @ts-ignore
+ requestPayload?.messages?.splice(
+ // @ts-ignore
+ requestPayload?.messages?.length,
+ 0,
+ toolCallMessage,
+ ...toolCallResult,
+ );
+ },
+ options,
+ );
+ } else {
+ const res = await fetch(chatPath, chatPayload);
+ clearTimeout(requestTimeoutId);
+
+ const resJson = await res.json();
+ const message = this.extractMessage(resJson);
+ options.onFinish(message, res);
+ }
+ } catch (e) {
+ console.log("[Request] failed to make a chat request", e);
+ options.onError?.(e as Error);
+ }
+ }
+ async usage() {
+ return {
+ used: 0,
+ total: 0,
+ };
+ }
+
+ async models(): Promise {
+ if (this.disableListModels) {
+ return DEFAULT_MODELS.slice();
+ }
+
+ const res = await fetch(this.path(AI302.ListModelPath), {
+ method: "GET",
+ headers: {
+ ...getHeaders(),
+ },
+ });
+
+ const resJson = (await res.json()) as Ai302ListModelResponse;
+ const chatModels = resJson.data;
+ console.log("[Models]", chatModels);
+
+ if (!chatModels) {
+ return [];
+ }
+
+ let seq = 1000; //同 Constant.ts 中的排序保持一致
+ return chatModels.map((m) => ({
+ name: m.id,
+ available: true,
+ sorted: seq++,
+ provider: {
+ id: "ai302",
+ providerName: "302.AI",
+ providerType: "ai302",
+ sorted: 15,
+ },
+ }));
+ }
+}
diff --git a/app/client/platforms/alibaba.ts b/app/client/platforms/alibaba.ts
index 86229a14705..4875e5c02d9 100644
--- a/app/client/platforms/alibaba.ts
+++ b/app/client/platforms/alibaba.ts
@@ -1,12 +1,16 @@
"use client";
+import { ApiPath, Alibaba, ALIBABA_BASE_URL } from "@/app/constant";
import {
- ApiPath,
- Alibaba,
- ALIBABA_BASE_URL,
- REQUEST_TIMEOUT_MS,
-} from "@/app/constant";
-import { useAccessStore, useAppConfig, useChatStore } from "@/app/store";
-
+ useAccessStore,
+ useAppConfig,
+ useChatStore,
+ ChatMessageTool,
+ usePluginStore,
+} from "@/app/store";
+import {
+ preProcessImageContentForAlibabaDashScope,
+ streamWithThink,
+} from "@/app/utils/chat";
import {
ChatOptions,
getHeaders,
@@ -14,15 +18,15 @@ import {
LLMModel,
SpeechOptions,
MultimodalContent,
+ MultimodalContentForAlibaba,
} from "../api";
-import Locale from "../../locales";
-import {
- EventStreamContentType,
- fetchEventSource,
-} from "@fortaine/fetch-event-source";
-import { prettyObject } from "@/app/utils/format";
import { getClientConfig } from "@/app/config/client";
-import { getMessageTextContent } from "@/app/utils";
+import {
+ getMessageTextContent,
+ getMessageTextContentWithoutThinking,
+ getTimeoutMSByModel,
+ isVisionModel,
+} from "@/app/utils";
import { fetch } from "@/app/utils/stream";
export interface OpenAIListModelResponse {
@@ -90,11 +94,6 @@ export class QwenApi implements LLMApi {
}
async chat(options: ChatOptions) {
- const messages = options.messages.map((v) => ({
- role: v.role,
- content: getMessageTextContent(v),
- }));
-
const modelConfig = {
...useAppConfig.getState().modelConfig,
...useChatStore.getState().currentSession().mask.modelConfig,
@@ -103,6 +102,21 @@ export class QwenApi implements LLMApi {
},
};
+ const visionModel = isVisionModel(options.config.model);
+
+ const messages: ChatOptions["messages"] = [];
+ for (const v of options.messages) {
+ const content = (
+ visionModel
+ ? await preProcessImageContentForAlibabaDashScope(v.content)
+ : v.role === "assistant"
+ ? getMessageTextContentWithoutThinking(v)
+ : getMessageTextContent(v)
+ ) as any;
+
+ messages.push({ role: v.role, content });
+ }
+
const shouldStream = !!options.config.stream;
const requestPayload: RequestPayload = {
model: modelConfig.model,
@@ -122,139 +136,127 @@ export class QwenApi implements LLMApi {
options.onController?.(controller);
try {
- const chatPath = this.path(Alibaba.ChatPath);
+ const headers = {
+ ...getHeaders(),
+ "X-DashScope-SSE": shouldStream ? "enable" : "disable",
+ };
+
+ const chatPath = this.path(Alibaba.ChatPath(modelConfig.model));
const chatPayload = {
method: "POST",
body: JSON.stringify(requestPayload),
signal: controller.signal,
- headers: {
- ...getHeaders(),
- "X-DashScope-SSE": shouldStream ? "enable" : "disable",
- },
+ headers: headers,
};
// make a fetch request
const requestTimeoutId = setTimeout(
() => controller.abort(),
- REQUEST_TIMEOUT_MS,
+ getTimeoutMSByModel(options.config.model),
);
if (shouldStream) {
- let responseText = "";
- let remainText = "";
- let finished = false;
-
- // animate response to make it looks smooth
- function animateResponseText() {
- if (finished || controller.signal.aborted) {
- responseText += remainText;
- console.log("[Response Animation] finished");
- if (responseText?.length === 0) {
- options.onError?.(new Error("empty response from server"));
+ const [tools, funcs] = usePluginStore
+ .getState()
+ .getAsTools(
+ useChatStore.getState().currentSession().mask?.plugin || [],
+ );
+ return streamWithThink(
+ chatPath,
+ requestPayload,
+ headers,
+ tools as any,
+ funcs,
+ controller,
+ // parseSSE
+ (text: string, runTools: ChatMessageTool[]) => {
+ // console.log("parseSSE", text, runTools);
+ const json = JSON.parse(text);
+ const choices = json.output.choices as Array<{
+ message: {
+ content: string | null | MultimodalContentForAlibaba[];
+ tool_calls: ChatMessageTool[];
+ reasoning_content: string | null;
+ };
+ }>;
+
+ if (!choices?.length) return { isThinking: false, content: "" };
+
+ const tool_calls = choices[0]?.message?.tool_calls;
+ if (tool_calls?.length > 0) {
+ const index = tool_calls[0]?.index;
+ const id = tool_calls[0]?.id;
+ const args = tool_calls[0]?.function?.arguments;
+ if (id) {
+ runTools.push({
+ id,
+ type: tool_calls[0]?.type,
+ function: {
+ name: tool_calls[0]?.function?.name as string,
+ arguments: args,
+ },
+ });
+ } else {
+ // @ts-ignore
+ runTools[index]["function"]["arguments"] += args;
+ }
}
- return;
- }
-
- if (remainText.length > 0) {
- const fetchCount = Math.max(1, Math.round(remainText.length / 60));
- const fetchText = remainText.slice(0, fetchCount);
- responseText += fetchText;
- remainText = remainText.slice(fetchCount);
- options.onUpdate?.(responseText, fetchText);
- }
-
- requestAnimationFrame(animateResponseText);
- }
-
- // start animaion
- animateResponseText();
-
- const finish = () => {
- if (!finished) {
- finished = true;
- options.onFinish(responseText + remainText);
- }
- };
- controller.signal.onabort = finish;
-
- fetchEventSource(chatPath, {
- fetch: fetch as any,
- ...chatPayload,
- async onopen(res) {
- clearTimeout(requestTimeoutId);
- const contentType = res.headers.get("content-type");
- console.log(
- "[Alibaba] request response content type: ",
- contentType,
- );
-
- if (contentType?.startsWith("text/plain")) {
- responseText = await res.clone().text();
- return finish();
- }
+ const reasoning = choices[0]?.message?.reasoning_content;
+ const content = choices[0]?.message?.content;
+ // Skip if both content and reasoning_content are empty or null
if (
- !res.ok ||
- !res.headers
- .get("content-type")
- ?.startsWith(EventStreamContentType) ||
- res.status !== 200
+ (!reasoning || reasoning.length === 0) &&
+ (!content || content.length === 0)
) {
- const responseTexts = [responseText];
- let extraInfo = await res.clone().text();
- try {
- const resJson = await res.clone().json();
- extraInfo = prettyObject(resJson);
- } catch {}
-
- if (res.status === 401) {
- responseTexts.push(Locale.Error.Unauthorized);
- }
-
- if (extraInfo) {
- responseTexts.push(extraInfo);
- }
-
- responseText = responseTexts.join("\n\n");
-
- return finish();
- }
- },
- onmessage(msg) {
- if (msg.data === "[DONE]" || finished) {
- return finish();
+ return {
+ isThinking: false,
+ content: "",
+ };
}
- const text = msg.data;
- try {
- const json = JSON.parse(text);
- const choices = json.output.choices as Array<{
- message: { content: string };
- }>;
- const delta = choices[0]?.message?.content;
- if (delta) {
- remainText += delta;
- }
- } catch (e) {
- console.error("[Request] parse error", text, msg);
+
+ if (reasoning && reasoning.length > 0) {
+ return {
+ isThinking: true,
+ content: reasoning,
+ };
+ } else if (content && content.length > 0) {
+ return {
+ isThinking: false,
+ content: Array.isArray(content)
+ ? content.map((item) => item.text).join(",")
+ : content,
+ };
}
+
+ return {
+ isThinking: false,
+ content: "",
+ };
},
- onclose() {
- finish();
- },
- onerror(e) {
- options.onError?.(e);
- throw e;
+ // processToolMessage, include tool_calls message and tool call results
+ (
+ requestPayload: RequestPayload,
+ toolCallMessage: any,
+ toolCallResult: any[],
+ ) => {
+ requestPayload?.input?.messages?.splice(
+ requestPayload?.input?.messages?.length,
+ 0,
+ toolCallMessage,
+ ...toolCallResult,
+ );
},
- openWhenHidden: true,
- });
+ options,
+ );
} else {
const res = await fetch(chatPath, chatPayload);
clearTimeout(requestTimeoutId);
const resJson = await res.json();
const message = this.extractMessage(resJson);
- options.onFinish(message);
+ options.onFinish(message, res);
}
} catch (e) {
console.log("[Request] failed to make a chat request", e);
diff --git a/app/client/platforms/anthropic.ts b/app/client/platforms/anthropic.ts
index 3645cbe6eac..2377470904f 100644
--- a/app/client/platforms/anthropic.ts
+++ b/app/client/platforms/anthropic.ts
@@ -224,7 +224,7 @@ export class ClaudeApi implements LLMApi {
let chunkJson:
| undefined
| {
- type: "content_block_delta" | "content_block_stop";
+ type: "content_block_delta" | "content_block_stop" | "message_delta" | "message_stop";
content_block?: {
type: "tool_use";
id: string;
@@ -234,11 +234,20 @@ export class ClaudeApi implements LLMApi {
type: "text_delta" | "input_json_delta";
text?: string;
partial_json?: string;
+ stop_reason?: string;
};
index: number;
};
chunkJson = JSON.parse(text);
+ // Handle refusal stop reason in message_delta
+ if (chunkJson?.delta?.stop_reason === "refusal") {
+ // Return a message to display to the user
+ const refusalMessage = "\n\n[Assistant refused to respond. Please modify your request and try again.]";
+ options.onError?.(new Error("Content policy violation: " + refusalMessage));
+ return refusalMessage;
+ }
+
if (chunkJson?.content_block?.type == "tool_use") {
index += 1;
const id = chunkJson?.content_block.id;
@@ -317,13 +326,14 @@ export class ClaudeApi implements LLMApi {
};
try {
- controller.signal.onabort = () => options.onFinish("");
+ controller.signal.onabort = () =>
+ options.onFinish("", new Response(null, { status: 400 }));
const res = await fetch(path, payload);
const resJson = await res.json();
const message = this.extractMessage(resJson);
- options.onFinish(message);
+ options.onFinish(message, res);
} catch (e) {
console.error("failed to chat", e);
options.onError?.(e as Error);
diff --git a/app/client/platforms/baidu.ts b/app/client/platforms/baidu.ts
index 2511a696b9b..dc990db4103 100644
--- a/app/client/platforms/baidu.ts
+++ b/app/client/platforms/baidu.ts
@@ -1,10 +1,5 @@
"use client";
-import {
- ApiPath,
- Baidu,
- BAIDU_BASE_URL,
- REQUEST_TIMEOUT_MS,
-} from "@/app/constant";
+import { ApiPath, Baidu, BAIDU_BASE_URL } from "@/app/constant";
import { useAccessStore, useAppConfig, useChatStore } from "@/app/store";
import { getAccessToken } from "@/app/utils/baidu";
@@ -23,7 +18,7 @@ import {
} from "@fortaine/fetch-event-source";
import { prettyObject } from "@/app/utils/format";
import { getClientConfig } from "@/app/config/client";
-import { getMessageTextContent } from "@/app/utils";
+import { getMessageTextContent, getTimeoutMSByModel } from "@/app/utils";
import { fetch } from "@/app/utils/stream";
export interface OpenAIListModelResponse {
@@ -155,13 +150,14 @@ export class ErnieApi implements LLMApi {
// make a fetch request
const requestTimeoutId = setTimeout(
() => controller.abort(),
- REQUEST_TIMEOUT_MS,
+ getTimeoutMSByModel(options.config.model),
);
if (shouldStream) {
let responseText = "";
let remainText = "";
let finished = false;
+ let responseRes: Response;
// animate response to make it looks smooth
function animateResponseText() {
@@ -191,7 +187,7 @@ export class ErnieApi implements LLMApi {
const finish = () => {
if (!finished) {
finished = true;
- options.onFinish(responseText + remainText);
+ options.onFinish(responseText + remainText, responseRes);
}
};
@@ -204,7 +200,7 @@ export class ErnieApi implements LLMApi {
clearTimeout(requestTimeoutId);
const contentType = res.headers.get("content-type");
console.log("[Baidu] request response content type: ", contentType);
-
+ responseRes = res;
if (contentType?.startsWith("text/plain")) {
responseText = await res.clone().text();
return finish();
@@ -267,7 +263,7 @@ export class ErnieApi implements LLMApi {
const resJson = await res.json();
const message = resJson?.result;
- options.onFinish(message);
+ options.onFinish(message, res);
}
} catch (e) {
console.log("[Request] failed to make a chat request", e);
diff --git a/app/client/platforms/bytedance.ts b/app/client/platforms/bytedance.ts
index 000a9e278db..f9524cba28d 100644
--- a/app/client/platforms/bytedance.ts
+++ b/app/client/platforms/bytedance.ts
@@ -1,11 +1,12 @@
"use client";
+import { ApiPath, ByteDance, BYTEDANCE_BASE_URL } from "@/app/constant";
import {
- ApiPath,
- ByteDance,
- BYTEDANCE_BASE_URL,
- REQUEST_TIMEOUT_MS,
-} from "@/app/constant";
-import { useAccessStore, useAppConfig, useChatStore } from "@/app/store";
+ useAccessStore,
+ useAppConfig,
+ useChatStore,
+ ChatMessageTool,
+ usePluginStore,
+} from "@/app/store";
import {
ChatOptions,
@@ -15,14 +16,14 @@ import {
MultimodalContent,
SpeechOptions,
} from "../api";
-import Locale from "../../locales";
-import {
- EventStreamContentType,
- fetchEventSource,
-} from "@fortaine/fetch-event-source";
-import { prettyObject } from "@/app/utils/format";
+
+import { streamWithThink } from "@/app/utils/chat";
import { getClientConfig } from "@/app/config/client";
-import { getMessageTextContent } from "@/app/utils";
+import { preProcessImageContent } from "@/app/utils/chat";
+import {
+ getMessageTextContentWithoutThinking,
+ getTimeoutMSByModel,
+} from "@/app/utils";
import { fetch } from "@/app/utils/stream";
export interface OpenAIListModelResponse {
@@ -34,7 +35,7 @@ export interface OpenAIListModelResponse {
}>;
}
-interface RequestPayload {
+interface RequestPayloadForByteDance {
messages: {
role: "system" | "user" | "assistant";
content: string | MultimodalContent[];
@@ -84,10 +85,14 @@ export class DoubaoApi implements LLMApi {
}
async chat(options: ChatOptions) {
- const messages = options.messages.map((v) => ({
- role: v.role,
- content: getMessageTextContent(v),
- }));
+ const messages: ChatOptions["messages"] = [];
+ for (const v of options.messages) {
+ const content =
+ v.role === "assistant"
+ ? getMessageTextContentWithoutThinking(v)
+ : await preProcessImageContent(v.content);
+ messages.push({ role: v.role, content });
+ }
const modelConfig = {
...useAppConfig.getState().modelConfig,
@@ -98,7 +103,7 @@ export class DoubaoApi implements LLMApi {
};
const shouldStream = !!options.config.stream;
- const requestPayload: RequestPayload = {
+ const requestPayload: RequestPayloadForByteDance = {
messages,
stream: shouldStream,
model: modelConfig.model,
@@ -123,125 +128,108 @@ export class DoubaoApi implements LLMApi {
// make a fetch request
const requestTimeoutId = setTimeout(
() => controller.abort(),
- REQUEST_TIMEOUT_MS,
+ getTimeoutMSByModel(options.config.model),
);
if (shouldStream) {
- let responseText = "";
- let remainText = "";
- let finished = false;
-
- // animate response to make it looks smooth
- function animateResponseText() {
- if (finished || controller.signal.aborted) {
- responseText += remainText;
- console.log("[Response Animation] finished");
- if (responseText?.length === 0) {
- options.onError?.(new Error("empty response from server"));
- }
- return;
- }
-
- if (remainText.length > 0) {
- const fetchCount = Math.max(1, Math.round(remainText.length / 60));
- const fetchText = remainText.slice(0, fetchCount);
- responseText += fetchText;
- remainText = remainText.slice(fetchCount);
- options.onUpdate?.(responseText, fetchText);
- }
-
- requestAnimationFrame(animateResponseText);
- }
-
- // start animaion
- animateResponseText();
-
- const finish = () => {
- if (!finished) {
- finished = true;
- options.onFinish(responseText + remainText);
- }
- };
-
- controller.signal.onabort = finish;
-
- fetchEventSource(chatPath, {
- fetch: fetch as any,
- ...chatPayload,
- async onopen(res) {
- clearTimeout(requestTimeoutId);
- const contentType = res.headers.get("content-type");
- console.log(
- "[ByteDance] request response content type: ",
- contentType,
- );
-
- if (contentType?.startsWith("text/plain")) {
- responseText = await res.clone().text();
- return finish();
+ const [tools, funcs] = usePluginStore
+ .getState()
+ .getAsTools(
+ useChatStore.getState().currentSession().mask?.plugin || [],
+ );
+ return streamWithThink(
+ chatPath,
+ requestPayload,
+ getHeaders(),
+ tools as any,
+ funcs,
+ controller,
+ // parseSSE
+ (text: string, runTools: ChatMessageTool[]) => {
+ // console.log("parseSSE", text, runTools);
+ const json = JSON.parse(text);
+ const choices = json.choices as Array<{
+ delta: {
+ content: string | null;
+ tool_calls: ChatMessageTool[];
+ reasoning_content: string | null;
+ };
+ }>;
+
+ if (!choices?.length) return { isThinking: false, content: "" };
+
+ const tool_calls = choices[0]?.delta?.tool_calls;
+ if (tool_calls?.length > 0) {
+ const index = tool_calls[0]?.index;
+ const id = tool_calls[0]?.id;
+ const args = tool_calls[0]?.function?.arguments;
+ if (id) {
+ runTools.push({
+ id,
+ type: tool_calls[0]?.type,
+ function: {
+ name: tool_calls[0]?.function?.name as string,
+ arguments: args,
+ },
+ });
+ } else {
+ // @ts-ignore
+ runTools[index]["function"]["arguments"] += args;
+ }
}
+ const reasoning = choices[0]?.delta?.reasoning_content;
+ const content = choices[0]?.delta?.content;
+ // Skip if both content and reasoning_content are empty or null
if (
- !res.ok ||
- !res.headers
- .get("content-type")
- ?.startsWith(EventStreamContentType) ||
- res.status !== 200
+ (!reasoning || reasoning.length === 0) &&
+ (!content || content.length === 0)
) {
- const responseTexts = [responseText];
- let extraInfo = await res.clone().text();
- try {
- const resJson = await res.clone().json();
- extraInfo = prettyObject(resJson);
- } catch {}
-
- if (res.status === 401) {
- responseTexts.push(Locale.Error.Unauthorized);
- }
-
- if (extraInfo) {
- responseTexts.push(extraInfo);
- }
-
- responseText = responseTexts.join("\n\n");
-
- return finish();
+ return {
+ isThinking: false,
+ content: "",
+ };
}
- },
- onmessage(msg) {
- if (msg.data === "[DONE]" || finished) {
- return finish();
- }
- const text = msg.data;
- try {
- const json = JSON.parse(text);
- const choices = json.choices as Array<{
- delta: { content: string };
- }>;
- const delta = choices[0]?.delta?.content;
- if (delta) {
- remainText += delta;
- }
- } catch (e) {
- console.error("[Request] parse error", text, msg);
+
+ if (reasoning && reasoning.length > 0) {
+ return {
+ isThinking: true,
+ content: reasoning,
+ };
+ } else if (content && content.length > 0) {
+ return {
+ isThinking: false,
+ content: content,
+ };
}
+
+ return {
+ isThinking: false,
+ content: "",
+ };
},
- onclose() {
- finish();
- },
- onerror(e) {
- options.onError?.(e);
- throw e;
+ // processToolMessage, include tool_calls message and tool call results
+ (
+ requestPayload: RequestPayloadForByteDance,
+ toolCallMessage: any,
+ toolCallResult: any[],
+ ) => {
+ requestPayload?.messages?.splice(
+ requestPayload?.messages?.length,
+ 0,
+ toolCallMessage,
+ ...toolCallResult,
+ );
},
- openWhenHidden: true,
- });
+ options,
+ );
} else {
const res = await fetch(chatPath, chatPayload);
clearTimeout(requestTimeoutId);
const resJson = await res.json();
const message = this.extractMessage(resJson);
- options.onFinish(message);
+ options.onFinish(message, res);
}
} catch (e) {
console.log("[Request] failed to make a chat request", e);
diff --git a/app/client/platforms/deepseek.ts b/app/client/platforms/deepseek.ts
new file mode 100644
index 00000000000..1b38b40cc64
--- /dev/null
+++ b/app/client/platforms/deepseek.ts
@@ -0,0 +1,253 @@
+"use client";
+// azure and openai, using same models. so using same LLMApi.
+import { ApiPath, DEEPSEEK_BASE_URL, DeepSeek } from "@/app/constant";
+import {
+ useAccessStore,
+ useAppConfig,
+ useChatStore,
+ ChatMessageTool,
+ usePluginStore,
+} from "@/app/store";
+import { streamWithThink } from "@/app/utils/chat";
+import {
+ ChatOptions,
+ getHeaders,
+ LLMApi,
+ LLMModel,
+ SpeechOptions,
+} from "../api";
+import { getClientConfig } from "@/app/config/client";
+import {
+ getMessageTextContent,
+ getMessageTextContentWithoutThinking,
+ getTimeoutMSByModel,
+} from "@/app/utils";
+import { RequestPayload } from "./openai";
+import { fetch } from "@/app/utils/stream";
+
+export class DeepSeekApi implements LLMApi {
+ private disableListModels = true;
+
+ path(path: string): string {
+ const accessStore = useAccessStore.getState();
+
+ let baseUrl = "";
+
+ if (accessStore.useCustomConfig) {
+ baseUrl = accessStore.deepseekUrl;
+ }
+
+ if (baseUrl.length === 0) {
+ const isApp = !!getClientConfig()?.isApp;
+ const apiPath = ApiPath.DeepSeek;
+ baseUrl = isApp ? DEEPSEEK_BASE_URL : apiPath;
+ }
+
+ if (baseUrl.endsWith("/")) {
+ baseUrl = baseUrl.slice(0, baseUrl.length - 1);
+ }
+ if (!baseUrl.startsWith("http") && !baseUrl.startsWith(ApiPath.DeepSeek)) {
+ baseUrl = "https://" + baseUrl;
+ }
+
+ console.log("[Proxy Endpoint] ", baseUrl, path);
+
+ return [baseUrl, path].join("/");
+ }
+
+ extractMessage(res: any) {
+ return res.choices?.at(0)?.message?.content ?? "";
+ }
+
+ speech(options: SpeechOptions): Promise {
+ throw new Error("Method not implemented.");
+ }
+
+ async chat(options: ChatOptions) {
+ const messages: ChatOptions["messages"] = [];
+ for (const v of options.messages) {
+ if (v.role === "assistant") {
+ const content = getMessageTextContentWithoutThinking(v);
+ messages.push({ role: v.role, content });
+ } else {
+ const content = getMessageTextContent(v);
+ messages.push({ role: v.role, content });
+ }
+ }
+
+ // 检测并修复消息顺序，确保除system外的第一个消息是user
+ const filteredMessages: ChatOptions["messages"] = [];
+ let hasFoundFirstUser = false;
+
+ for (const msg of messages) {
+ if (msg.role === "system") {
+ // Keep all system messages
+ filteredMessages.push(msg);
+ } else if (msg.role === "user") {
+ // User message directly added
+ filteredMessages.push(msg);
+ hasFoundFirstUser = true;
+ } else if (hasFoundFirstUser) {
+ // After finding the first user message, all subsequent non-system messages are retained.
+ filteredMessages.push(msg);
+ }
+ // If hasFoundFirstUser is false and it is not a system message, it will be skipped.
+ }
+
+ const modelConfig = {
+ ...useAppConfig.getState().modelConfig,
+ ...useChatStore.getState().currentSession().mask.modelConfig,
+ ...{
+ model: options.config.model,
+ providerName: options.config.providerName,
+ },
+ };
+
+ const requestPayload: RequestPayload = {
+ messages: filteredMessages,
+ stream: options.config.stream,
+ model: modelConfig.model,
+ temperature: modelConfig.temperature,
+ presence_penalty: modelConfig.presence_penalty,
+ frequency_penalty: modelConfig.frequency_penalty,
+ top_p: modelConfig.top_p,
+ // max_tokens: Math.max(modelConfig.max_tokens, 1024),
+ // Please do not ask me why not send max_tokens, no reason, this param is just shit, I dont want to explain anymore.
+ };
+
+ console.log("[Request] openai payload: ", requestPayload);
+
+ const shouldStream = !!options.config.stream;
+ const controller = new AbortController();
+ options.onController?.(controller);
+
+ try {
+ const chatPath = this.path(DeepSeek.ChatPath);
+ const chatPayload = {
+ method: "POST",
+ body: JSON.stringify(requestPayload),
+ signal: controller.signal,
+ headers: getHeaders(),
+ };
+
+ // make a fetch request
+ const requestTimeoutId = setTimeout(
+ () => controller.abort(),
+ getTimeoutMSByModel(options.config.model),
+ );
+
+ if (shouldStream) {
+ const [tools, funcs] = usePluginStore
+ .getState()
+ .getAsTools(
+ useChatStore.getState().currentSession().mask?.plugin || [],
+ );
+ return streamWithThink(
+ chatPath,
+ requestPayload,
+ getHeaders(),
+ tools as any,
+ funcs,
+ controller,
+ // parseSSE
+ (text: string, runTools: ChatMessageTool[]) => {
+ // console.log("parseSSE", text, runTools);
+ const json = JSON.parse(text);
+ const choices = json.choices as Array<{
+ delta: {
+ content: string | null;
+ tool_calls: ChatMessageTool[];
+ reasoning_content: string | null;
+ };
+ }>;
+ const tool_calls = choices[0]?.delta?.tool_calls;
+ if (tool_calls?.length > 0) {
+ const index = tool_calls[0]?.index;
+ const id = tool_calls[0]?.id;
+ const args = tool_calls[0]?.function?.arguments;
+ if (id) {
+ runTools.push({
+ id,
+ type: tool_calls[0]?.type,
+ function: {
+ name: tool_calls[0]?.function?.name as string,
+ arguments: args,
+ },
+ });
+ } else {
+ // @ts-ignore
+ runTools[index]["function"]["arguments"] += args;
+ }
+ }
+ const reasoning = choices[0]?.delta?.reasoning_content;
+ const content = choices[0]?.delta?.content;
+
+ // Skip if both content and reasoning_content are empty or null
+ if (
+ (!reasoning || reasoning.length === 0) &&
+ (!content || content.length === 0)
+ ) {
+ return {
+ isThinking: false,
+ content: "",
+ };
+ }
+
+ if (reasoning && reasoning.length > 0) {
+ return {
+ isThinking: true,
+ content: reasoning,
+ };
+ } else if (content && content.length > 0) {
+ return {
+ isThinking: false,
+ content: content,
+ };
+ }
+
+ return {
+ isThinking: false,
+ content: "",
+ };
+ },
+ // processToolMessage, include tool_calls message and tool call results
+ (
+ requestPayload: RequestPayload,
+ toolCallMessage: any,
+ toolCallResult: any[],
+ ) => {
+ // @ts-ignore
+ requestPayload?.messages?.splice(
+ // @ts-ignore
+ requestPayload?.messages?.length,
+ 0,
+ toolCallMessage,
+ ...toolCallResult,
+ );
+ },
+ options,
+ );
+ } else {
+ const res = await fetch(chatPath, chatPayload);
+ clearTimeout(requestTimeoutId);
+
+ const resJson = await res.json();
+ const message = this.extractMessage(resJson);
+ options.onFinish(message, res);
+ }
+ } catch (e) {
+ console.log("[Request] failed to make a chat request", e);
+ options.onError?.(e as Error);
+ }
+ }
+ async usage() {
+ return {
+ used: 0,
+ total: 0,
+ };
+ }
+
+ async models(): Promise {
+ return [];
+ }
+}
diff --git a/app/client/platforms/glm.ts b/app/client/platforms/glm.ts
new file mode 100644
index 00000000000..98b10277de9
--- /dev/null
+++ b/app/client/platforms/glm.ts
@@ -0,0 +1,292 @@
+"use client";
+import { ApiPath, CHATGLM_BASE_URL, ChatGLM } from "@/app/constant";
+import {
+ useAccessStore,
+ useAppConfig,
+ useChatStore,
+ ChatMessageTool,
+ usePluginStore,
+} from "@/app/store";
+import { stream } from "@/app/utils/chat";
+import {
+ ChatOptions,
+ getHeaders,
+ LLMApi,
+ LLMModel,
+ SpeechOptions,
+} from "../api";
+import { getClientConfig } from "@/app/config/client";
+import {
+ getMessageTextContent,
+ isVisionModel,
+ getTimeoutMSByModel,
+} from "@/app/utils";
+import { RequestPayload } from "./openai";
+import { fetch } from "@/app/utils/stream";
+import { preProcessImageContent } from "@/app/utils/chat";
+
+interface BasePayload {
+ model: string;
+}
+
+interface ChatPayload extends BasePayload {
+ messages: ChatOptions["messages"];
+ stream?: boolean;
+ temperature?: number;
+ presence_penalty?: number;
+ frequency_penalty?: number;
+ top_p?: number;
+}
+
+interface ImageGenerationPayload extends BasePayload {
+ prompt: string;
+ size?: string;
+ user_id?: string;
+}
+
+interface VideoGenerationPayload extends BasePayload {
+ prompt: string;
+ duration?: number;
+ resolution?: string;
+ user_id?: string;
+}
+
+type ModelType = "chat" | "image" | "video";
+
+export class ChatGLMApi implements LLMApi {
+ private disableListModels = true;
+
+ private getModelType(model: string): ModelType {
+ if (model.startsWith("cogview-")) return "image";
+ if (model.startsWith("cogvideo-")) return "video";
+ return "chat";
+ }
+
+ private getModelPath(type: ModelType): string {
+ switch (type) {
+ case "image":
+ return ChatGLM.ImagePath;
+ case "video":
+ return ChatGLM.VideoPath;
+ default:
+ return ChatGLM.ChatPath;
+ }
+ }
+
+ private createPayload(
+ messages: ChatOptions["messages"],
+ modelConfig: any,
+ options: ChatOptions,
+ ): BasePayload {
+ const modelType = this.getModelType(modelConfig.model);
+ const lastMessage = messages[messages.length - 1];
+ const prompt =
+ typeof lastMessage.content === "string"
+ ? lastMessage.content
+ : lastMessage.content.map((c) => c.text).join("\n");
+
+ switch (modelType) {
+ case "image":
+ return {
+ model: modelConfig.model,
+ prompt,
+ size: options.config.size,
+ } as ImageGenerationPayload;
+ default:
+ return {
+ messages,
+ stream: options.config.stream,
+ model: modelConfig.model,
+ temperature: modelConfig.temperature,
+ presence_penalty: modelConfig.presence_penalty,
+ frequency_penalty: modelConfig.frequency_penalty,
+ top_p: modelConfig.top_p,
+ } as ChatPayload;
+ }
+ }
+
+ private parseResponse(modelType: ModelType, json: any): string {
+ switch (modelType) {
+ case "image": {
+ const imageUrl = json.data?.[0]?.url;
+ return imageUrl ? `` : "";
+ }
+ case "video": {
+ const videoUrl = json.data?.[0]?.url;
+ return videoUrl ? `` : "";
+ }
+ default:
+ return this.extractMessage(json);
+ }
+ }
+
+ path(path: string): string {
+ const accessStore = useAccessStore.getState();
+ let baseUrl = "";
+
+ if (accessStore.useCustomConfig) {
+ baseUrl = accessStore.chatglmUrl;
+ }
+
+ if (baseUrl.length === 0) {
+ const isApp = !!getClientConfig()?.isApp;
+ const apiPath = ApiPath.ChatGLM;
+ baseUrl = isApp ? CHATGLM_BASE_URL : apiPath;
+ }
+
+ if (baseUrl.endsWith("/")) {
+ baseUrl = baseUrl.slice(0, baseUrl.length - 1);
+ }
+ if (!baseUrl.startsWith("http") && !baseUrl.startsWith(ApiPath.ChatGLM)) {
+ baseUrl = "https://" + baseUrl;
+ }
+
+ console.log("[Proxy Endpoint] ", baseUrl, path);
+ return [baseUrl, path].join("/");
+ }
+
+ extractMessage(res: any) {
+ return res.choices?.at(0)?.message?.content ?? "";
+ }
+
+ speech(options: SpeechOptions): Promise {
+ throw new Error("Method not implemented.");
+ }
+
+ async chat(options: ChatOptions) {
+ const visionModel = isVisionModel(options.config.model);
+ const messages: ChatOptions["messages"] = [];
+ for (const v of options.messages) {
+ const content = visionModel
+ ? await preProcessImageContent(v.content)
+ : getMessageTextContent(v);
+ messages.push({ role: v.role, content });
+ }
+
+ const modelConfig = {
+ ...useAppConfig.getState().modelConfig,
+ ...useChatStore.getState().currentSession().mask.modelConfig,
+ ...{
+ model: options.config.model,
+ providerName: options.config.providerName,
+ },
+ };
+ const modelType = this.getModelType(modelConfig.model);
+ const requestPayload = this.createPayload(messages, modelConfig, options);
+ const path = this.path(this.getModelPath(modelType));
+
+ console.log(`[Request] glm ${modelType} payload: `, requestPayload);
+
+ const controller = new AbortController();
+ options.onController?.(controller);
+
+ try {
+ const chatPayload = {
+ method: "POST",
+ body: JSON.stringify(requestPayload),
+ signal: controller.signal,
+ headers: getHeaders(),
+ };
+
+ const requestTimeoutId = setTimeout(
+ () => controller.abort(),
+ getTimeoutMSByModel(options.config.model),
+ );
+
+ if (modelType === "image" || modelType === "video") {
+ const res = await fetch(path, chatPayload);
+ clearTimeout(requestTimeoutId);
+
+ const resJson = await res.json();
+ console.log(`[Response] glm ${modelType}:`, resJson);
+ const message = this.parseResponse(modelType, resJson);
+ options.onFinish(message, res);
+ return;
+ }
+
+ const shouldStream = !!options.config.stream;
+ if (shouldStream) {
+ const [tools, funcs] = usePluginStore
+ .getState()
+ .getAsTools(
+ useChatStore.getState().currentSession().mask?.plugin || [],
+ );
+ return stream(
+ path,
+ requestPayload,
+ getHeaders(),
+ tools as any,
+ funcs,
+ controller,
+ // parseSSE
+ (text: string, runTools: ChatMessageTool[]) => {
+ const json = JSON.parse(text);
+ const choices = json.choices as Array<{
+ delta: {
+ content: string;
+ tool_calls: ChatMessageTool[];
+ };
+ }>;
+ const tool_calls = choices[0]?.delta?.tool_calls;
+ if (tool_calls?.length > 0) {
+ const index = tool_calls[0]?.index;
+ const id = tool_calls[0]?.id;
+ const args = tool_calls[0]?.function?.arguments;
+ if (id) {
+ runTools.push({
+ id,
+ type: tool_calls[0]?.type,
+ function: {
+ name: tool_calls[0]?.function?.name as string,
+ arguments: args,
+ },
+ });
+ } else {
+ // @ts-ignore
+ runTools[index]["function"]["arguments"] += args;
+ }
+ }
+ return choices[0]?.delta?.content;
+ },
+ // processToolMessage
+ (
+ requestPayload: RequestPayload,
+ toolCallMessage: any,
+ toolCallResult: any[],
+ ) => {
+ // @ts-ignore
+ requestPayload?.messages?.splice(
+ // @ts-ignore
+ requestPayload?.messages?.length,
+ 0,
+ toolCallMessage,
+ ...toolCallResult,
+ );
+ },
+ options,
+ );
+ } else {
+ const res = await fetch(path, chatPayload);
+ clearTimeout(requestTimeoutId);
+
+ const resJson = await res.json();
+ const message = this.extractMessage(resJson);
+ options.onFinish(message, res);
+ }
+ } catch (e) {
+ console.log("[Request] failed to make a chat request", e);
+ options.onError?.(e as Error);
+ }
+ }
+
+ async usage() {
+ return {
+ used: 0,
+ total: 0,
+ };
+ }
+
+ async models(): Promise {
+ return [];
+ }
+}
diff --git a/app/client/platforms/google.ts b/app/client/platforms/google.ts
index 14fecb8f289..654f0e3e4c7 100644
--- a/app/client/platforms/google.ts
+++ b/app/client/platforms/google.ts
@@ -1,4 +1,4 @@
-import { ApiPath, Google, REQUEST_TIMEOUT_MS } from "@/app/constant";
+import { ApiPath, Google } from "@/app/constant";
import {
ChatOptions,
getHeaders,
@@ -22,6 +22,7 @@ import {
getMessageTextContent,
getMessageImages,
isVisionModel,
+ getTimeoutMSByModel,
} from "@/app/utils";
import { preProcessImageContent } from "@/app/utils/chat";
import { nanoid } from "nanoid";
@@ -29,7 +30,7 @@ import { RequestPayload } from "./openai";
import { fetch } from "@/app/utils/stream";
export class GeminiProApi implements LLMApi {
- path(path: string): string {
+ path(path: string, shouldStream = false): string {
const accessStore = useAccessStore.getState();
let baseUrl = "";
@@ -51,15 +52,34 @@ export class GeminiProApi implements LLMApi {
console.log("[Proxy Endpoint] ", baseUrl, path);
let chatPath = [baseUrl, path].join("/");
+ if (shouldStream) {
+ chatPath += chatPath.includes("?") ? "&alt=sse" : "?alt=sse";
+ }
- chatPath += chatPath.includes("?") ? "&alt=sse" : "?alt=sse";
return chatPath;
}
extractMessage(res: any) {
console.log("[Response] gemini-pro response: ", res);
+ const getTextFromParts = (parts: any[]) => {
+ if (!Array.isArray(parts)) return "";
+
+ return parts
+ .map((part) => part?.text || "")
+ .filter((text) => text.trim() !== "")
+ .join("\n\n");
+ };
+
+ let content = "";
+ if (Array.isArray(res)) {
+ res.map((item) => {
+ content += getTextFromParts(item?.candidates?.at(0)?.content?.parts);
+ });
+ }
+
return (
- res?.candidates?.at(0)?.content?.parts.at(0)?.text ||
+ getTextFromParts(res?.candidates?.at(0)?.content?.parts) ||
+ content || //getTextFromParts(res?.at(0)?.candidates?.at(0)?.content?.parts) ||
res?.error?.message ||
""
);
@@ -166,7 +186,10 @@ export class GeminiProApi implements LLMApi {
options.onController?.(controller);
try {
// https://github.com/google-gemini/cookbook/blob/main/quickstarts/rest/Streaming_REST.ipynb
- const chatPath = this.path(Google.ChatPath(modelConfig.model));
+ const chatPath = this.path(
+ Google.ChatPath(modelConfig.model),
+ shouldStream,
+ );
const chatPayload = {
method: "POST",
@@ -175,10 +198,11 @@ export class GeminiProApi implements LLMApi {
headers: getHeaders(),
};
+ const isThinking = options.config.model.includes("-thinking");
// make a fetch request
const requestTimeoutId = setTimeout(
() => controller.abort(),
- REQUEST_TIMEOUT_MS,
+ getTimeoutMSByModel(options.config.model),
);
if (shouldStream) {
@@ -193,7 +217,8 @@ export class GeminiProApi implements LLMApi {
getHeaders(),
// @ts-ignore
tools.length > 0
- ? [{ functionDeclarations: tools.map((tool) => tool.function) }]
+ ? // @ts-ignore
+ [{ functionDeclarations: tools.map((tool) => tool.function) }]
: [],
funcs,
controller,
@@ -216,7 +241,10 @@ export class GeminiProApi implements LLMApi {
},
});
}
- return chunkJson?.candidates?.at(0)?.content.parts.at(0)?.text;
+ return chunkJson?.candidates
+ ?.at(0)
+ ?.content.parts?.map((part: { text: string }) => part.text)
+ .join("\n\n");
},
// processToolMessage, include tool_calls message and tool call results
(
@@ -273,7 +301,7 @@ export class GeminiProApi implements LLMApi {
);
}
const message = apiClient.extractMessage(resJson);
- options.onFinish(message);
+ options.onFinish(message, res);
}
} catch (e) {
console.log("[Request] failed to make a chat request", e);
diff --git a/app/client/platforms/iflytek.ts b/app/client/platforms/iflytek.ts
index 55a39d0ccca..cfc37b3b256 100644
--- a/app/client/platforms/iflytek.ts
+++ b/app/client/platforms/iflytek.ts
@@ -117,6 +117,7 @@ export class SparkApi implements LLMApi {
let responseText = "";
let remainText = "";
let finished = false;
+ let responseRes: Response;
// Animate response text to make it look smooth
function animateResponseText() {
@@ -143,7 +144,7 @@ export class SparkApi implements LLMApi {
const finish = () => {
if (!finished) {
finished = true;
- options.onFinish(responseText + remainText);
+ options.onFinish(responseText + remainText, responseRes);
}
};
@@ -156,7 +157,7 @@ export class SparkApi implements LLMApi {
clearTimeout(requestTimeoutId);
const contentType = res.headers.get("content-type");
console.log("[Spark] request response content type: ", contentType);
-
+ responseRes = res;
if (contentType?.startsWith("text/plain")) {
responseText = await res.clone().text();
return finish();
@@ -231,7 +232,7 @@ export class SparkApi implements LLMApi {
const resJson = await res.json();
const message = this.extractMessage(resJson);
- options.onFinish(message);
+ options.onFinish(message, res);
}
} catch (e) {
console.log("[Request] failed to make a chat request", e);
diff --git a/app/client/platforms/moonshot.ts b/app/client/platforms/moonshot.ts
index 22a34b2e22f..b6812c0d766 100644
--- a/app/client/platforms/moonshot.ts
+++ b/app/client/platforms/moonshot.ts
@@ -180,7 +180,7 @@ export class MoonshotApi implements LLMApi {
const resJson = await res.json();
const message = this.extractMessage(resJson);
- options.onFinish(message);
+ options.onFinish(message, res);
}
} catch (e) {
console.log("[Request] failed to make a chat request", e);
diff --git a/app/client/platforms/openai.ts b/app/client/platforms/openai.ts
index 30f7415c141..cfbff99e978 100644
--- a/app/client/platforms/openai.ts
+++ b/app/client/platforms/openai.ts
@@ -21,10 +21,10 @@ import {
preProcessImageContent,
uploadImage,
base64Image2Blob,
- stream,
+ streamWithThink,
} from "@/app/utils/chat";
import { cloudflareAIGatewayUrl } from "@/app/utils/cloudflare";
-import { DalleSize, DalleQuality, DalleStyle } from "@/app/typing";
+import { ModelSize, DalleQuality, DalleStyle } from "@/app/typing";
import {
ChatOptions,
@@ -41,6 +41,7 @@ import {
getMessageTextContent,
isVisionModel,
isDalle3 as _isDalle3,
+ getTimeoutMSByModel,
} from "@/app/utils";
import { fetch } from "@/app/utils/stream";
@@ -55,7 +56,7 @@ export interface OpenAIListModelResponse {
export interface RequestPayload {
messages: {
- role: "system" | "user" | "assistant";
+ role: "developer" | "system" | "user" | "assistant";
content: string | MultimodalContent[];
}[];
stream?: boolean;
@@ -65,6 +66,7 @@ export interface RequestPayload {
frequency_penalty: number;
top_p: number;
max_tokens?: number;
+ max_completion_tokens?: number;
}
export interface DalleRequestPayload {
@@ -72,7 +74,7 @@ export interface DalleRequestPayload {
prompt: string;
response_format: "url" | "b64_json";
n: number;
- size: DalleSize;
+ size: ModelSize;
quality: DalleQuality;
style: DalleStyle;
}
@@ -194,7 +196,11 @@ export class ChatGPTApi implements LLMApi {
let requestPayload: RequestPayload | DalleRequestPayload;
const isDalle3 = _isDalle3(options.config.model);
- const isO1 = options.config.model.startsWith("o1");
+ const isO1OrO3 =
+ options.config.model.startsWith("o1") ||
+ options.config.model.startsWith("o3") ||
+ options.config.model.startsWith("o4-mini");
+ const isGpt5 = options.config.model.startsWith("gpt-5");
if (isDalle3) {
const prompt = getMessageTextContent(
options.messages.slice(-1)?.pop() as any,
@@ -216,32 +222,52 @@ export class ChatGPTApi implements LLMApi {
const content = visionModel
? await preProcessImageContent(v.content)
: getMessageTextContent(v);
- if (!(isO1 && v.role === "system"))
+ if (!(isO1OrO3 && v.role === "system"))
messages.push({ role: v.role, content });
}
// O1 not support image, tools (plugin in ChatGPTNextWeb) and system, stream, logprobs, temperature, top_p, n, presence_penalty, frequency_penalty yet.
requestPayload = {
messages,
- stream: !isO1 ? options.config.stream : false,
+ stream: options.config.stream,
model: modelConfig.model,
- temperature: !isO1 ? modelConfig.temperature : 1,
- presence_penalty: !isO1 ? modelConfig.presence_penalty : 0,
- frequency_penalty: !isO1 ? modelConfig.frequency_penalty : 0,
- top_p: !isO1 ? modelConfig.top_p : 1,
+ temperature: (!isO1OrO3 && !isGpt5) ? modelConfig.temperature : 1,
+ presence_penalty: !isO1OrO3 ? modelConfig.presence_penalty : 0,
+ frequency_penalty: !isO1OrO3 ? modelConfig.frequency_penalty : 0,
+ top_p: !isO1OrO3 ? modelConfig.top_p : 1,
// max_tokens: Math.max(modelConfig.max_tokens, 1024),
// Please do not ask me why not send max_tokens, no reason, this param is just shit, I dont want to explain anymore.
};
+ if (isGpt5) {
+ // Remove max_tokens if present
+ delete requestPayload.max_tokens;
+ // Add max_completion_tokens (or max_completion_tokens if that's what you meant)
+ requestPayload["max_completion_tokens"] = modelConfig.max_tokens;
+
+ } else if (isO1OrO3) {
+ // by default the o1/o3 models will not attempt to produce output that includes markdown formatting
+ // manually add "Formatting re-enabled" developer message to encourage markdown inclusion in model responses
+ // (https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/reasoning?tabs=python-secure#markdown-output)
+ requestPayload["messages"].unshift({
+ role: "developer",
+ content: "Formatting re-enabled",
+ });
+
+ // o1/o3 uses max_completion_tokens to control the number of tokens (https://platform.openai.com/docs/guides/reasoning#controlling-costs)
+ requestPayload["max_completion_tokens"] = modelConfig.max_tokens;
+ }
+
+
// add max_tokens to vision model
- if (visionModel) {
+ if (visionModel && !isO1OrO3 && ! isGpt5) {
requestPayload["max_tokens"] = Math.max(modelConfig.max_tokens, 4000);
}
}
console.log("[Request] openai payload: ", requestPayload);
- const shouldStream = !isDalle3 && !!options.config.stream && !isO1;
+ const shouldStream = !isDalle3 && !!options.config.stream;
const controller = new AbortController();
options.onController?.(controller);
@@ -285,7 +311,7 @@ export class ChatGPTApi implements LLMApi {
useChatStore.getState().currentSession().mask?.plugin || [],
);
// console.log("getAsTools", tools, funcs);
- stream(
+ streamWithThink(
chatPath,
requestPayload,
getHeaders(),
@@ -300,8 +326,12 @@ export class ChatGPTApi implements LLMApi {
delta: {
content: string;
tool_calls: ChatMessageTool[];
+ reasoning_content: string | null;
};
}>;
+
+ if (!choices?.length) return { isThinking: false, content: "" };
+
const tool_calls = choices[0]?.delta?.tool_calls;
if (tool_calls?.length > 0) {
const id = tool_calls[0]?.id;
@@ -321,7 +351,37 @@ export class ChatGPTApi implements LLMApi {
runTools[index]["function"]["arguments"] += args;
}
}
- return choices[0]?.delta?.content;
+
+ const reasoning = choices[0]?.delta?.reasoning_content;
+ const content = choices[0]?.delta?.content;
+
+ // Skip if both content and reasoning_content are empty or null
+ if (
+ (!reasoning || reasoning.length === 0) &&
+ (!content || content.length === 0)
+ ) {
+ return {
+ isThinking: false,
+ content: "",
+ };
+ }
+
+ if (reasoning && reasoning.length > 0) {
+ return {
+ isThinking: true,
+ content: reasoning,
+ };
+ } else if (content && content.length > 0) {
+ return {
+ isThinking: false,
+ content: content,
+ };
+ }
+
+ return {
+ isThinking: false,
+ content: "",
+ };
},
// processToolMessage, include tool_calls message and tool call results
(
@@ -353,7 +413,7 @@ export class ChatGPTApi implements LLMApi {
// make a fetch request
const requestTimeoutId = setTimeout(
() => controller.abort(),
- isDalle3 || isO1 ? REQUEST_TIMEOUT_MS * 4 : REQUEST_TIMEOUT_MS, // dalle3 using b64_json is slow.
+ getTimeoutMSByModel(options.config.model),
);
const res = await fetch(chatPath, chatPayload);
@@ -361,7 +421,7 @@ export class ChatGPTApi implements LLMApi {
const resJson = await res.json();
const message = await this.extractMessage(resJson);
- options.onFinish(message);
+ options.onFinish(message, res);
}
} catch (e) {
console.log("[Request] failed to make a chat request", e);
diff --git a/app/client/platforms/siliconflow.ts b/app/client/platforms/siliconflow.ts
new file mode 100644
index 00000000000..34f0844c328
--- /dev/null
+++ b/app/client/platforms/siliconflow.ts
@@ -0,0 +1,287 @@
+"use client";
+// azure and openai, using same models. so using same LLMApi.
+import {
+ ApiPath,
+ SILICONFLOW_BASE_URL,
+ SiliconFlow,
+ DEFAULT_MODELS,
+} from "@/app/constant";
+import {
+ useAccessStore,
+ useAppConfig,
+ useChatStore,
+ ChatMessageTool,
+ usePluginStore,
+} from "@/app/store";
+import { preProcessImageContent, streamWithThink } from "@/app/utils/chat";
+import {
+ ChatOptions,
+ getHeaders,
+ LLMApi,
+ LLMModel,
+ SpeechOptions,
+} from "../api";
+import { getClientConfig } from "@/app/config/client";
+import {
+ getMessageTextContent,
+ getMessageTextContentWithoutThinking,
+ isVisionModel,
+ getTimeoutMSByModel,
+} from "@/app/utils";
+import { RequestPayload } from "./openai";
+
+import { fetch } from "@/app/utils/stream";
+export interface SiliconFlowListModelResponse {
+ object: string;
+ data: Array<{
+ id: string;
+ object: string;
+ root: string;
+ }>;
+}
+
+export class SiliconflowApi implements LLMApi {
+ private disableListModels = false;
+
+ path(path: string): string {
+ const accessStore = useAccessStore.getState();
+
+ let baseUrl = "";
+
+ if (accessStore.useCustomConfig) {
+ baseUrl = accessStore.siliconflowUrl;
+ }
+
+ if (baseUrl.length === 0) {
+ const isApp = !!getClientConfig()?.isApp;
+ const apiPath = ApiPath.SiliconFlow;
+ baseUrl = isApp ? SILICONFLOW_BASE_URL : apiPath;
+ }
+
+ if (baseUrl.endsWith("/")) {
+ baseUrl = baseUrl.slice(0, baseUrl.length - 1);
+ }
+ if (
+ !baseUrl.startsWith("http") &&
+ !baseUrl.startsWith(ApiPath.SiliconFlow)
+ ) {
+ baseUrl = "https://" + baseUrl;
+ }
+
+ console.log("[Proxy Endpoint] ", baseUrl, path);
+
+ return [baseUrl, path].join("/");
+ }
+
+ extractMessage(res: any) {
+ return res.choices?.at(0)?.message?.content ?? "";
+ }
+
+ speech(options: SpeechOptions): Promise {
+ throw new Error("Method not implemented.");
+ }
+
+ async chat(options: ChatOptions) {
+ const visionModel = isVisionModel(options.config.model);
+ const messages: ChatOptions["messages"] = [];
+ for (const v of options.messages) {
+ if (v.role === "assistant") {
+ const content = getMessageTextContentWithoutThinking(v);
+ messages.push({ role: v.role, content });
+ } else {
+ const content = visionModel
+ ? await preProcessImageContent(v.content)
+ : getMessageTextContent(v);
+ messages.push({ role: v.role, content });
+ }
+ }
+
+ const modelConfig = {
+ ...useAppConfig.getState().modelConfig,
+ ...useChatStore.getState().currentSession().mask.modelConfig,
+ ...{
+ model: options.config.model,
+ providerName: options.config.providerName,
+ },
+ };
+
+ const requestPayload: RequestPayload = {
+ messages,
+ stream: options.config.stream,
+ model: modelConfig.model,
+ temperature: modelConfig.temperature,
+ presence_penalty: modelConfig.presence_penalty,
+ frequency_penalty: modelConfig.frequency_penalty,
+ top_p: modelConfig.top_p,
+ // max_tokens: Math.max(modelConfig.max_tokens, 1024),
+ // Please do not ask me why not send max_tokens, no reason, this param is just shit, I dont want to explain anymore.
+ };
+
+ console.log("[Request] openai payload: ", requestPayload);
+
+ const shouldStream = !!options.config.stream;
+ const controller = new AbortController();
+ options.onController?.(controller);
+
+ try {
+ const chatPath = this.path(SiliconFlow.ChatPath);
+ const chatPayload = {
+ method: "POST",
+ body: JSON.stringify(requestPayload),
+ signal: controller.signal,
+ headers: getHeaders(),
+ };
+
+ // console.log(chatPayload);
+
+ // Use extended timeout for thinking models as they typically require more processing time
+ const requestTimeoutId = setTimeout(
+ () => controller.abort(),
+ getTimeoutMSByModel(options.config.model),
+ );
+
+ if (shouldStream) {
+ const [tools, funcs] = usePluginStore
+ .getState()
+ .getAsTools(
+ useChatStore.getState().currentSession().mask?.plugin || [],
+ );
+ return streamWithThink(
+ chatPath,
+ requestPayload,
+ getHeaders(),
+ tools as any,
+ funcs,
+ controller,
+ // parseSSE
+ (text: string, runTools: ChatMessageTool[]) => {
+ // console.log("parseSSE", text, runTools);
+ const json = JSON.parse(text);
+ const choices = json.choices as Array<{
+ delta: {
+ content: string | null;
+ tool_calls: ChatMessageTool[];
+ reasoning_content: string | null;
+ };
+ }>;
+ const tool_calls = choices[0]?.delta?.tool_calls;
+ if (tool_calls?.length > 0) {
+ const index = tool_calls[0]?.index;
+ const id = tool_calls[0]?.id;
+ const args = tool_calls[0]?.function?.arguments;
+ if (id) {
+ runTools.push({
+ id,
+ type: tool_calls[0]?.type,
+ function: {
+ name: tool_calls[0]?.function?.name as string,
+ arguments: args,
+ },
+ });
+ } else {
+ // @ts-ignore
+ runTools[index]["function"]["arguments"] += args;
+ }
+ }
+ const reasoning = choices[0]?.delta?.reasoning_content;
+ const content = choices[0]?.delta?.content;
+
+ // Skip if both content and reasoning_content are empty or null
+ if (
+ (!reasoning || reasoning.length === 0) &&
+ (!content || content.length === 0)
+ ) {
+ return {
+ isThinking: false,
+ content: "",
+ };
+ }
+
+ if (reasoning && reasoning.length > 0) {
+ return {
+ isThinking: true,
+ content: reasoning,
+ };
+ } else if (content && content.length > 0) {
+ return {
+ isThinking: false,
+ content: content,
+ };
+ }
+
+ return {
+ isThinking: false,
+ content: "",
+ };
+ },
+ // processToolMessage, include tool_calls message and tool call results
+ (
+ requestPayload: RequestPayload,
+ toolCallMessage: any,
+ toolCallResult: any[],
+ ) => {
+ // @ts-ignore
+ requestPayload?.messages?.splice(
+ // @ts-ignore
+ requestPayload?.messages?.length,
+ 0,
+ toolCallMessage,
+ ...toolCallResult,
+ );
+ },
+ options,
+ );
+ } else {
+ const res = await fetch(chatPath, chatPayload);
+ clearTimeout(requestTimeoutId);
+
+ const resJson = await res.json();
+ const message = this.extractMessage(resJson);
+ options.onFinish(message, res);
+ }
+ } catch (e) {
+ console.log("[Request] failed to make a chat request", e);
+ options.onError?.(e as Error);
+ }
+ }
+ async usage() {
+ return {
+ used: 0,
+ total: 0,
+ };
+ }
+
+ async models(): Promise {
+ if (this.disableListModels) {
+ return DEFAULT_MODELS.slice();
+ }
+
+ const res = await fetch(this.path(SiliconFlow.ListModelPath), {
+ method: "GET",
+ headers: {
+ ...getHeaders(),
+ },
+ });
+
+ const resJson = (await res.json()) as SiliconFlowListModelResponse;
+ const chatModels = resJson.data;
+ console.log("[Models]", chatModels);
+
+ if (!chatModels) {
+ return [];
+ }
+
+ let seq = 1000; //同 Constant.ts 中的排序保持一致
+ return chatModels.map((m) => ({
+ name: m.id,
+ available: true,
+ sorted: seq++,
+ provider: {
+ id: "siliconflow",
+ providerName: "SiliconFlow",
+ providerType: "siliconflow",
+ sorted: 14,
+ },
+ }));
+ }
+}
diff --git a/app/client/platforms/tencent.ts b/app/client/platforms/tencent.ts
index 3610fac0a48..8adeb1b3ea7 100644
--- a/app/client/platforms/tencent.ts
+++ b/app/client/platforms/tencent.ts
@@ -1,5 +1,5 @@
"use client";
-import { ApiPath, TENCENT_BASE_URL, REQUEST_TIMEOUT_MS } from "@/app/constant";
+import { ApiPath, TENCENT_BASE_URL } from "@/app/constant";
import { useAccessStore, useAppConfig, useChatStore } from "@/app/store";
import {
@@ -17,7 +17,11 @@ import {
} from "@fortaine/fetch-event-source";
import { prettyObject } from "@/app/utils/format";
import { getClientConfig } from "@/app/config/client";
-import { getMessageTextContent, isVisionModel } from "@/app/utils";
+import {
+ getMessageTextContent,
+ isVisionModel,
+ getTimeoutMSByModel,
+} from "@/app/utils";
import mapKeys from "lodash-es/mapKeys";
import mapValues from "lodash-es/mapValues";
import isArray from "lodash-es/isArray";
@@ -135,13 +139,14 @@ export class HunyuanApi implements LLMApi {
// make a fetch request
const requestTimeoutId = setTimeout(
() => controller.abort(),
- REQUEST_TIMEOUT_MS,
+ getTimeoutMSByModel(options.config.model),
);
if (shouldStream) {
let responseText = "";
let remainText = "";
let finished = false;
+ let responseRes: Response;
// animate response to make it looks smooth
function animateResponseText() {
@@ -171,7 +176,7 @@ export class HunyuanApi implements LLMApi {
const finish = () => {
if (!finished) {
finished = true;
- options.onFinish(responseText + remainText);
+ options.onFinish(responseText + remainText, responseRes);
}
};
@@ -187,7 +192,7 @@ export class HunyuanApi implements LLMApi {
"[Tencent] request response content type: ",
contentType,
);
-
+ responseRes = res;
if (contentType?.startsWith("text/plain")) {
responseText = await res.clone().text();
return finish();
@@ -253,7 +258,7 @@ export class HunyuanApi implements LLMApi {
const resJson = await res.json();
const message = this.extractMessage(resJson);
- options.onFinish(message);
+ options.onFinish(message, res);
}
} catch (e) {
console.log("[Request] failed to make a chat request", e);
diff --git a/app/client/platforms/xai.ts b/app/client/platforms/xai.ts
index deb74e66c03..830ad4778ac 100644
--- a/app/client/platforms/xai.ts
+++ b/app/client/platforms/xai.ts
@@ -1,6 +1,6 @@
"use client";
// azure and openai, using same models. so using same LLMApi.
-import { ApiPath, XAI_BASE_URL, XAI, REQUEST_TIMEOUT_MS } from "@/app/constant";
+import { ApiPath, XAI_BASE_URL, XAI } from "@/app/constant";
import {
useAccessStore,
useAppConfig,
@@ -17,7 +17,8 @@ import {
SpeechOptions,
} from "../api";
import { getClientConfig } from "@/app/config/client";
-import { getMessageTextContent } from "@/app/utils";
+import { getTimeoutMSByModel } from "@/app/utils";
+import { preProcessImageContent } from "@/app/utils/chat";
import { RequestPayload } from "./openai";
import { fetch } from "@/app/utils/stream";
@@ -62,7 +63,7 @@ export class XAIApi implements LLMApi {
async chat(options: ChatOptions) {
const messages: ChatOptions["messages"] = [];
for (const v of options.messages) {
- const content = getMessageTextContent(v);
+ const content = await preProcessImageContent(v.content);
messages.push({ role: v.role, content });
}
@@ -103,7 +104,7 @@ export class XAIApi implements LLMApi {
// make a fetch request
const requestTimeoutId = setTimeout(
() => controller.abort(),
- REQUEST_TIMEOUT_MS,
+ getTimeoutMSByModel(options.config.model),
);
if (shouldStream) {
@@ -173,7 +174,7 @@ export class XAIApi implements LLMApi {
const resJson = await res.json();
const message = this.extractMessage(resJson);
- options.onFinish(message);
+ options.onFinish(message, res);
}
} catch (e) {
console.log("[Request] failed to make a chat request", e);
diff --git a/app/components/artifacts.tsx b/app/components/artifacts.tsx
index ce187fbcb2c..7546260d92b 100644
--- a/app/components/artifacts.tsx
+++ b/app/components/artifacts.tsx
@@ -29,11 +29,11 @@ type HTMLPreviewProps = {
onLoad?: (title?: string) => void;
};
-export type HTMLPreviewHander = {
+export type HTMLPreviewHandler = {
reload: () => void;
};
-export const HTMLPreview = forwardRef(
+export const HTMLPreview = forwardRef(
function HTMLPreview(props, ref) {
const iframeRef = useRef(null);
const [frameId, setFrameId] = useState(nanoid());
@@ -207,7 +207,7 @@ export function Artifacts() {
const [code, setCode] = useState("");
const [loading, setLoading] = useState(true);
const [fileName, setFileName] = useState("");
- const previewRef = useRef(null);
+ const previewRef = useRef(null);
useEffect(() => {
if (id) {
diff --git a/app/components/auth.tsx b/app/components/auth.tsx
index 539a52eecc9..5375bda3f70 100644
--- a/app/components/auth.tsx
+++ b/app/components/auth.tsx
@@ -18,6 +18,8 @@ import {
trackSettingsPageGuideToCPaymentClick,
trackAuthorizationPageButtonToCPaymentClick,
} from "../utils/auth-settings-events";
+import clsx from "clsx";
+
const storage = safeLocalStorage();
export function AuthPage() {
@@ -54,7 +56,7 @@ export function AuthPage() {
onClick={() => navigate(Path.Home)}
>
-
+
+ 
](https://zeabur.com/templates/ZBUEFA) [
](https://gitpod.io/#https://github.com/Yidadaa/ChatGPT-Next-Web)
+[
](https://opendeploy.dev/github/ChatGPTNextWeb/NextChat)
-[