|
1 | 1 | --- |
2 | 2 | title: Model Providers |
3 | | -description: Configure AI model access for your workspace, the foundation that powers all your applications |
| 3 | +description: Add model providers to your workspace so your apps can use LLMs, embedding models, and other AI services |
4 | 4 | --- |
5 | 5 |
|
6 | | -Model providers give your workspace access to AI models. Every application you build needs models to function, and configuring providers at the workspace level means all team members can use them across all projects. |
7 | | - |
8 | | -You configure model providers by adding your own API keys for services like OpenAI, Anthropic, Google, and others. You get full control and direct billing through each provider. |
9 | | - |
10 | | -## Configure Providers |
11 | | - |
12 | | -Only workspace admins and owners can configure model providers. The process is consistent across providers: |
13 | | - |
14 | | -<Steps> |
15 | | - <Step title="Open Integrations > Model Provider"> |
16 | | - Access the model provider configuration from the main navigation. |
17 | | - </Step> |
18 | | - |
19 | | - <Step title="Select your provider"> |
20 | | - Choose from OpenAI, Anthropic, Google, Cohere, or other supported providers. |
21 | | - </Step> |
22 | | - |
23 | | - <Step title="Add credentials"> |
24 | | - Enter your API key and any additional configuration required by the provider. |
25 | | - </Step> |
26 | | - |
27 | | - <Step title="Test and save"> |
28 | | - Dify validates your credentials before making the provider available to your workspace. |
29 | | - </Step> |
30 | | -</Steps> |
31 | | - |
32 | | -## Supported Providers |
33 | | - |
34 | | -**Large Language Models**: |
35 | | -- OpenAI (GPT-4, GPT-3.5-turbo) |
36 | | -- Anthropic (Claude) |
37 | | -- Google (Gemini) |
38 | | -- Cohere |
39 | | -- Local models via Ollama |
40 | | - |
41 | | -**Embedding Models**: |
42 | | -- OpenAI Embeddings |
43 | | -- Cohere Embeddings |
44 | | -- Azure OpenAI |
45 | | -- Local embedding models |
46 | | - |
47 | | -**Specialized Models**: |
48 | | -- Image generation (DALL-E, Stable Diffusion) |
49 | | -- Speech (Whisper, ElevenLabs) |
50 | | -- Moderation APIs |
51 | | - |
52 | | -## Provider Configuration Examples |
53 | | - |
54 | | -<Tabs> |
55 | | - <Tab title="OpenAI"> |
56 | | - **Required**: API Key from OpenAI Platform |
57 | | - |
58 | | - **Optional**: Custom base URL for Azure OpenAI or proxies, Organization ID for organization-scoped usage |
59 | | - |
60 | | - **Available Models**: GPT-4, GPT-3.5-turbo, DALL-E, Whisper, Text embeddings |
61 | | - </Tab> |
62 | | - |
63 | | - <Tab title="Anthropic"> |
64 | | - **Required**: API Key from Anthropic Console |
65 | | - |
66 | | - **Available Models**: Claude 3 (Opus, Sonnet, Haiku), Claude 2.1, Claude Instant |
67 | | - </Tab> |
68 | | - |
69 | | - <Tab title="Local (Ollama)"> |
70 | | - **Required**: Ollama server URL (typically http://localhost:11434) |
71 | | - |
72 | | - **Setup**: Install Ollama, pull models (`ollama pull llama2`), configure Dify connection |
73 | | - |
74 | | - **Benefits**: Complete data privacy, no external API costs, custom model fine-tuning |
75 | | - </Tab> |
76 | | -</Tabs> |
77 | | - |
78 | | -## Manage Model Credentials |
79 | | - |
80 | | -Add multiple credentials for a model provider's predefined and custom models, and easily switch between, delete, or modify these credentials. |
81 | | - |
82 | | -Here are some scenarios where adding multiple credentials is particularly helpful: |
83 | | - |
84 | | -- **Environment Isolation**: Configure separate model credentials for different environments, such as development, testing, and production. For example, use a rate-limited credential in the development environment for debugging, and a paid credential with stable performance and a sufficient quota in the production environment to ensure service quality. |
85 | | - |
86 | | -- **Cost Optimization**: Add and switch between multiple credentials from different accounts or model providers to maximize the use of free or low-cost quotas, thereby reducing application development and operational costs. |
87 | | - |
88 | | -- **Model Testing**: During model fine-tuning or iteration, you may create multiple model versions. By adding credentials for these different versions, you can quickly switch between them to test and evaluate their performance. |
89 | | - |
90 | | -<Tabs> |
91 | | - <Tab title="Predefined Model"> |
92 | | - |
93 | | - After installing a model provider and configuring the first credential, click **Config** in the upper-right corner to perform the following actions: |
94 | | - |
95 | | - - Add a new credential |
96 | | - - Select a credential as the default for all predefined models |
97 | | - - Edit a credential |
98 | | - - Delete a credential |
99 | | - |
100 | | - <Note> |
101 | | - If the default credential is deleted, you must manually specify a new one. |
102 | | - </Note> |
| 6 | +Every Dify app needs at least one model provider. Install a provider, add your API key, and all apps in the workspace can use its models. |
103 | 7 |
|
104 | | - <Frame> |
105 | | -  |
106 | | - </Frame> |
| 8 | +<Warning> |
| 9 | +API keys grant workspace-wide model access and trigger direct billing through each provider's account. Only the workspace owner and admins can manage providers. |
| 10 | +</Warning> |
107 | 11 |
|
108 | | - </Tab> |
109 | | - <Tab title="Custom Model"> |
| 12 | +## Add a Provider |
110 | 13 |
|
111 | | - ### Manage Credentials for a Single Custom Model |
| 14 | +1. In **Integrations** > **Model Provider**, browse the **Install model providers** section or visit the [Marketplace](https://marketplace.dify.ai/) for more options. |
112 | 15 |
|
113 | | - After installing a model provider and adding a custom model, follow these steps: |
| 16 | +2. After installing, click **Setup** on the provider card and enter your API key (and any other required credentials, such as a custom endpoint URL or organization ID). Dify validates the credentials before making the provider available. |
114 | 17 |
|
115 | | - 1. In the model list, click the corresponding **Config**. |
| 18 | +## Add a Custom Model |
116 | 19 |
|
117 | | - 2. In the **Specify model credential** panel, click the default credential to open the credential list, then perform the following actions: |
| 20 | +Most providers bring their own models, so once you authorize one, its models are ready to use. You add a model yourself only when the one you need isn't there, typically in two situations: |
118 | 21 |
|
119 | | - - Add a new credential |
| 22 | +- You run your own inference server, such as Ollama or Xinference, and Dify has no way to know which models you've loaded. |
| 23 | +- A provider lists its models, but the specific one you want—such as a newly released or fine-tuned model—is missing. |
120 | 24 |
|
121 | | - - Select a credential as the default for that custom model |
| 25 | +Click **Add Model** on the provider card, then give the model a name and its credentials. If a provider only serves a fixed set of models, it won't offer this option. |
122 | 26 |
|
123 | | - - Edit a credential |
| 27 | +<Info> |
| 28 | +If you add a custom model whose name and type match one that already exists, Dify attaches the new key to the existing model instead of creating a duplicate. |
| 29 | +</Info> |
124 | 30 |
|
125 | | - - Delete a credential |
| 31 | +## Manage Credentials |
126 | 32 |
|
127 | | - <Warning> |
128 | | - If you delete the only credential for a custom model, the model will also be deleted. |
129 | | - </Warning> |
| 33 | +Add more than one key or endpoint to a provider when you want to keep development and production separate, spread usage across accounts, or send different models to different servers. |
130 | 34 |
|
| 35 | +Where you manage keys depends on the model: |
| 36 | +- For a provider's own models, click **Configure** on the provider card to manage the keys shared across all of them. |
| 37 | + <Frame> |
| 38 | +  |
| 39 | + </Frame> |
| 40 | +- For a custom model you added, click **Configure** on the model to manage its own keys. |
| 41 | + <Columns cols={2}> |
131 | 42 | <Frame> |
132 | | -  |
| 43 | +  |
133 | 44 | </Frame> |
134 | | - |
135 | | - <Info> |
136 | | - When you add a new custom model with a name and type identical to an existing custom model, the system will add the new credential to that existing model rather than creating a duplicate. |
137 | | - </Info> |
138 | | - |
139 | | - ### Manage Credentials for All Custom Models |
140 | | - |
141 | | - Click **Manage Credentials** to view, edit, or delete the credentials for all custom models. |
142 | | - |
143 | 45 | <Frame> |
144 | | -  |
| 46 | +  |
145 | 47 | </Frame> |
| 48 | + </Columns> |
146 | 49 |
|
147 | | - After a custom model is removed, its credentials will remain in the **Manage Credentials** list. When you click **Add Model**, the system will display all removed custom models whose credentials still exist, allowing you to quickly re-add them. |
148 | | - |
149 | | - <Frame> |
150 | | -  |
151 | | - </Frame> |
152 | | - |
153 | | - If you delete all credentials for a removed custom model from the **Manage Credentials** list, that model will no longer appear when you click **Add Model**. |
154 | | - </Tab> |
155 | | -</Tabs> |
156 | | - |
157 | | -<Tip> |
158 | | - On Dify Enterprise, you can automatically load-balance requests across multiple credentials. [Contact sales](https://udify.app/chat/QuwcpW1oBNcfeL55) to learn more. |
159 | | -</Tip> |
| 50 | + <Warning> |
| 51 | + A model you added relies on its own credentials. Deleting its only credential removes the model too. |
| 52 | + </Warning> |
160 | 53 |
|
161 | | -## Access and Permissions |
| 54 | +To view every custom model's keys in one place, click **Manage Credentials** on the provider card. |
162 | 55 |
|
163 | | -Team access follows workspace permissions: |
164 | | -- **Owners/Admins** can configure, modify, and remove providers |
165 | | -- **Editors/Members** can view available providers and use them in applications |
| 56 | +<Frame> |
| 57 | +  |
| 58 | +</Frame> |
166 | 59 |
|
167 | | -<Warning> |
168 | | -API keys grant workspace-wide model access and trigger direct billing through each provider's account. Only give admin privileges to trusted team members. |
169 | | -</Warning> |
| 60 | +Keys stay here even after you remove a model, so you can re-add the model later without re-entering them. |
170 | 61 |
|
171 | | -## Troubleshooting |
| 62 | +<Frame> |
| 63 | +  |
| 64 | +</Frame> |
172 | 65 |
|
173 | | -**Authentication Failed**: Verify API key accuracy, check expiration, ensure sufficient credits, confirm key permissions. |
| 66 | +## Set Default Models |
174 | 67 |
|
175 | | -**Model Not Available**: Check provider configuration includes the model, verify API key tier access, refresh provider settings. |
| 68 | +Click **Default Models** in the toolbar to assign the fallback model for each category. Apps and nodes that don't specify a model use these: |
176 | 69 |
|
177 | | -**Rate Limits**: Upgrade provider account, implement request queuing, consider custom providers for higher limits. |
| 70 | +- **System Reasoning Model**: the default LLM. |
| 71 | +- **Embedding Model**: used for knowledge base indexing and retrieval. |
| 72 | +- **Rerank Model**: re-ranks retrieval results by relevance. |
| 73 | +- **Speech-to-Text Model**: converts audio input to text. |
| 74 | +- **Text-to-Speech Model**: converts text responses to audio. |
0 commit comments