-
Notifications
You must be signed in to change notification settings - Fork 4.4k
feat(llm): add Doubao (Volcengine Ark) as an LLM provider #12219
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
MackDing
wants to merge
1
commit into
continuedev:main
Choose a base branch
from
MackDing:feat-llm-provider-doubao
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| import { LLMOptions } from "../../index.js"; | ||
|
|
||
| import OpenAI from "./OpenAI.js"; | ||
|
|
||
| /** | ||
| * Doubao (豆包) is ByteDance's large-language-model family, served through | ||
| * Volcengine Ark (火山方舟). | ||
| * | ||
| * API surface: OpenAI-compatible `/chat/completions` at | ||
| * https://ark.cn-beijing.volces.com/api/v3/, Bearer-token auth. Unlike most | ||
| * OpenAI-compatible providers, Doubao requires users to deploy a model as an | ||
| * "endpoint" and use the endpoint ID (e.g. `ep-20240xxx-xxxxx`) as the model | ||
| * identifier — though shared/public endpoints also expose model-name aliases | ||
| * such as `doubao-1-5-pro-32k` and `doubao-seed-1-6`. | ||
| * | ||
| * Docs: https://www.volcengine.com/docs/82379 | ||
| */ | ||
| class Doubao extends OpenAI { | ||
| static providerName = "doubao"; | ||
| static defaultOptions: Partial<LLMOptions> = { | ||
| apiBase: "https://ark.cn-beijing.volces.com/api/v3/", | ||
| // Ark model IDs are date-stamped (e.g. `doubao-seed-1-6-251015`) or are | ||
| // Ark endpoint IDs (`ep-20240xxx-xxxxx`). We intentionally do NOT set a | ||
| // default `model` here: picking a specific dated ID would go stale, and | ||
| // users must in practice verify model availability against their own | ||
| // Ark deployment. Requiring an explicit `model` forces a conscious | ||
| // decision and avoids silent 404s from Ark. | ||
| useLegacyCompletionsEndpoint: false, | ||
| }; | ||
| maxStopWords: number | undefined = 4; | ||
| } | ||
|
|
||
| export default Doubao; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,111 @@ | ||
| --- | ||
| title: "Doubao (豆包)" | ||
| description: "Configure ByteDance Doubao models served via Volcengine Ark in Continue, including the doubao-seed and doubao-1.5-pro model families." | ||
| --- | ||
|
|
||
| [Doubao (豆包)](https://www.volcengine.com/product/ark) is ByteDance's large-language-model family served through Volcengine Ark (火山方舟). The service exposes an OpenAI-compatible `/chat/completions` API and is widely used in the China region for both chat and code-assistance workloads. | ||
|
|
||
| Continue supports Doubao as a first-class provider. Ark addresses models either by a **date-stamped model ID** (e.g. `doubao-seed-1-6-251015`, `doubao-1-5-pro-32k-250115`) or by an **endpoint ID** you provision in the Ark console (`ep-20240xxx-xxxxx`). Always verify the current model ID in the [Ark model list](https://www.volcengine.com/docs/82379/1330310) before configuring Continue — undated aliases are not guaranteed to resolve. | ||
|
|
||
| ## Configuration | ||
|
|
||
| To use Doubao models: | ||
|
|
||
| 1. Create an API key on the [Volcengine Ark console](https://console.volcengine.com/ark/). | ||
| 2. Either deploy the model you want as an endpoint and copy its endpoint ID, or use a public model alias. | ||
| 3. Add the following configuration: | ||
|
|
||
| <Tabs> | ||
| <Tab title="YAML"> | ||
| ```yaml title="config.yaml" | ||
| name: My Config | ||
| version: 0.0.1 | ||
| schema: v1 | ||
|
|
||
| models: | ||
| - name: Doubao Seed 1.6 | ||
| provider: doubao | ||
| model: doubao-seed-1-6-251015 | ||
| apiKey: <YOUR_VOLCENGINE_ARK_API_KEY> | ||
| ``` | ||
| </Tab> | ||
| <Tab title="JSON (Deprecated)"> | ||
| ```json title="config.json" | ||
| { | ||
| "models": [ | ||
| { | ||
| "title": "Doubao Seed 1.6", | ||
| "provider": "doubao", | ||
| "model": "doubao-seed-1-6-251015", | ||
| "apiKey": "<YOUR_VOLCENGINE_ARK_API_KEY>" | ||
| } | ||
| ] | ||
| } | ||
| ``` | ||
| </Tab> | ||
| </Tabs> | ||
|
|
||
| ## Using an endpoint ID | ||
|
|
||
| For custom or newly released models, deploy the model as an Ark endpoint and pass the endpoint ID as the `model`: | ||
|
|
||
| ```yaml title="config.yaml" | ||
| models: | ||
| - name: Doubao (custom endpoint) | ||
| provider: doubao | ||
| model: ep-20240xxx-xxxxx | ||
| apiKey: <YOUR_VOLCENGINE_ARK_API_KEY> | ||
| ``` | ||
|
|
||
| Endpoint IDs bypass model-name routing on the Ark gateway and give you full control over which deployment handles the request, including quota and region. | ||
|
|
||
| ## Configuration Options | ||
|
|
||
| | Option | Description | Default | | ||
| | --------- | ---------------------------------------------------- | ------------------------------------------------- | | ||
| | `apiKey` | Volcengine Ark API key | Required | | ||
| | `apiBase` | Ark API base URL | `https://ark.cn-beijing.volces.com/api/v3/` | | ||
| | `model` | Ark model ID (date-stamped) or endpoint ID | Required | | ||
|
|
||
| ## Example | ||
|
|
||
| Complete configuration with tuned completion options: | ||
|
|
||
| <Tabs> | ||
| <Tab title="YAML"> | ||
| ```yaml title="config.yaml" | ||
| name: My Config | ||
| version: 0.0.1 | ||
| schema: v1 | ||
|
|
||
| models: | ||
| - name: Doubao Seed 1.6 | ||
| provider: doubao | ||
| model: doubao-seed-1-6-251015 | ||
| apiKey: <YOUR_VOLCENGINE_ARK_API_KEY> | ||
| defaultCompletionOptions: | ||
| temperature: 0.7 | ||
| topP: 0.95 | ||
| maxTokens: 2048 | ||
| ``` | ||
| </Tab> | ||
| <Tab title="JSON (Deprecated)"> | ||
| ```json title="config.json" | ||
| { | ||
| "models": [ | ||
| { | ||
| "title": "Doubao Seed 1.6", | ||
| "provider": "doubao", | ||
| "model": "doubao-seed-1-6-251015", | ||
| "apiKey": "<YOUR_VOLCENGINE_ARK_API_KEY>", | ||
| "completionOptions": { | ||
| "temperature": 0.7, | ||
| "topP": 0.95, | ||
| "maxTokens": 2048 | ||
| } | ||
| } | ||
| ] | ||
| } | ||
| ``` | ||
| </Tab> | ||
| </Tabs> | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| import { describe, expect, it } from "vitest"; | ||
| import { constructLlmApi } from "../index.js"; | ||
| import { DoubaoApi } from "./Doubao.js"; | ||
|
|
||
| describe("Doubao (ByteDance / Volcengine Ark) adapter", () => { | ||
| it("registers under the 'doubao' provider and returns a DoubaoApi", () => { | ||
| const api = constructLlmApi({ | ||
| provider: "doubao", | ||
| apiKey: "test-key", | ||
| }); | ||
| expect(api).toBeInstanceOf(DoubaoApi); | ||
| }); | ||
|
|
||
| it("points at the Ark cn-beijing base URL by default", () => { | ||
| const api = new DoubaoApi({ | ||
| provider: "doubao", | ||
| apiKey: "test-key", | ||
| }); | ||
| // apiBase is a public field on the adapter; sanity-check it directly so | ||
| // a future refactor can't silently swap the default to, say, api.openai.com. | ||
| expect(api.apiBase).toBe("https://ark.cn-beijing.volces.com/api/v3/"); | ||
| }); | ||
|
|
||
| it("inherits from OpenAIApi (OpenAI-compatible Ark /chat/completions)", () => { | ||
| const api = new DoubaoApi({ | ||
| provider: "doubao", | ||
| apiKey: "test-key", | ||
| }); | ||
| // Ark is OpenAI-compatible for /chat/completions; the adapter relies on | ||
| // the base class, so it must not accidentally drop that relationship. | ||
| expect(typeof api.chatCompletionStream).toBe("function"); | ||
| expect(typeof api.chatCompletionNonStream).toBe("function"); | ||
| }); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| import { DoubaoConfig } from "../types.js"; | ||
| import { OpenAIApi } from "./OpenAI.js"; | ||
|
|
||
| /** | ||
| * Doubao (豆包) served via Volcengine Ark. | ||
| * | ||
| * Uses the OpenAI-compatible `/chat/completions` endpoint. Unlike most | ||
| * OpenAI-compatible services, Doubao recommends addressing models through | ||
| * a deployed "endpoint ID" (e.g. `ep-20240xxx-xxxxx`), though shared model | ||
| * aliases such as `doubao-1-5-pro-32k` also resolve on the public tenancy. | ||
| * | ||
| * No custom FIM: Ark does not expose a public `beta/completions` or | ||
| * `[fill]`-prompt protocol today. If that changes we can override | ||
| * `fimStream` the way Moonshot and Deepseek do. | ||
| * | ||
| * Reference: https://www.volcengine.com/docs/82379 | ||
| */ | ||
| export class DoubaoApi extends OpenAIApi { | ||
| apiBase: string = "https://ark.cn-beijing.volces.com/api/v3/"; | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P2: Subclass field initialization overwrites Prompt for AI agents |
||
| constructor(config: DoubaoConfig) { | ||
| super({ | ||
| ...config, | ||
| provider: "openai", | ||
| }); | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P2: Configuration instructions are internally contradictory about using model aliases versus date-stamped model IDs/endpoint IDs, which can lead to invalid setup.
Prompt for AI agents