You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on May 22, 2026. It is now read-only.
Expand all lines: _includes/docs/samples/analytics/ai-models.md
+98-15Lines changed: 98 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,26 +19,27 @@ ThingsBoard allows you to configure and connect to different AI providers, manag
19
19
20
20
To add an AI model in ThingsBoard, follow these steps:
21
21
22
-
- Open the "<b>Settings</b>" page in your ThingsBoard instance.
23
-
- Go to the "<b>AI models</b>" tab.
22
+
- Go to the "<b>AI models</b>" tab of the "<b>Settings</b>" page.
24
23
- Click the "<b>Add model</b>" button (located in the top-right corner).
25
24
- This will open a form where you can configure AI model:
26
25
- <b>Name</b> - provide a meaningful name for the AI model.
27
-
-[Provider](#provider-configuration)– select the AI providerand specify its authentication credentials.
28
-
-[Model ID](#model-configuration) – choose which model to use (or deployment name, in the case of Azure OpenAI).
26
+
-[Provider](#provider-configuration)— select the **AI provider**, specify the **base URL** (required only for [OpenAI](#openai)and [Ollama](#ollama)), and enter the provider's **authentication credentials**.
27
+
-[Model ID](#model-configuration) – choose which model to use (or deployment name, in the case of **Azure OpenAI**).
29
28
-[Advanced settings](#advanced-model-settings) – configure optional parameters (such as temperature, top P, max tokens) if supported by the provider.
30
29
- Click "<b>Save</b>" to complete adding the new AI model.
31
30
32
-
Once saved, the model becomes available for use in the [AI request node](/docs/{{docsPrefix}}user-guide/rule-engine-2-0/external-nodes/#ai-request-node){:target="_blank"} of the Rule Engine.
31
+
Once saved, the model becomes available for use in the [AI request node](/docs/{{docsPrefix}}user-guide/rule-engine-2-0/nodes/external/ai-request/){:target="_blank"} of the [Rule Engine](/docs/{{docsPrefix}}user-guide/rule-engine-2-0/overview/){:target="_blank"}.
33
32
34
33
{% include images-gallery.html imageCollection="adding-ai-model" %}
35
34
36
35
## Provider configuration
37
36
38
-
In the "<b>Provider</b>" section you need to select the <b>AI provider</b> you want to use, as well as the authentication method for that provider (e.g., API key, key file, etc.).
37
+
In the <b>Provider</b> section, select the <b>AI provider</b>, specify the <b>base URL</b> (required only for [OpenAI](#openai) and [Ollama](#ollama)), and enter the <b>authentication credentials</b> (e.g., API key, key file, etc.).
39
38
40
39
{% if docsPrefix == "pe/" or docsPrefix == "paas/" or docsPrefix == "paas/eu/" %}
41
-
> We recommend using [Secrets storage](/docs/{{docsPrefix}}user-guide/secrets-storage/){:target="_blank"} to securely store your credentials.
40
+
{% capture difference %}
41
+
We recommend using [Secrets storage](/docs/{{docsPrefix}}user-guide/secrets-storage/){:target="_blank"} to securely store your credentials.{% endcapture %}
42
+
{% include templates/info-banner.md content=difference %}
42
43
{% endif %}
43
44
44
45
<br><b><fontsize="4">Supported AI providers</font></b>
@@ -49,9 +50,44 @@ ThingsBoard currently supports integration with the following AI providers:
49
50
50
51
#### OpenAI
51
52
52
-
- Authentication: <b>API key</b>.
53
+
- <b>Base URL</b>: Specify the address for accessing the OpenAI API.
54
+
- <b>Authentication</b>: API key.
53
55
- You can obtain your API key from the [OpenAI dashboard](https://platform.openai.com/api-keys){:target="_blank"}.
54
56
57
+
##### Using OpenAI-compatible models
58
+
59
+
{% assign sinceVersion = "4.2.1" %}
60
+
{% include templates/since.md %}
61
+
62
+
When working with models compatible with the OpenAI API, an important parameter is the **base URL**, which defines the address used to send requests to the API.
63
+
64
+
<b><fontsize="3">Official Base URL</font></b>
65
+
66
+
The standard OpenAI API endpoint, preconfigured in ThingsBoard. Use this to access models hosted by OpenAI.
67
+
68
+
<b><fontsize="3">Custom Base URL</font></b>
69
+
70
+
An alternative endpoint for providers that implement the OpenAI-compatible API protocol.
71
+
Use this when there is no dedicated integration for your provider and they offer an OpenAI-compatible API (e.g., DeepSeek, Qwen, self-hosted Ollama).
72
+
73
+
{% capture difference %}
74
+
When using a custom base URL, the **API key becomes optional**. This enables working with models that do not require authentication, such as locally hosted models. However, most cloud model providers will still require a valid API key.
75
+
{% endcapture %}
76
+
{% include templates/info-banner.md content=difference %}
[Ollama](#ollama) is also available as a separate integration with additional configuration options.
88
+
{% endcapture %}
89
+
{% include templates/info-banner.md content=ollama_note %}
90
+
55
91
<hr>
56
92
57
93
#### Azure OpenAI
@@ -102,7 +138,10 @@ ThingsBoard currently supports integration with the following AI providers:
102
138
- Secret access key.
103
139
- AWS region (where inference will run).
104
140
105
-
> <b>Note</b>: Authentication with Bedrock API keys is not supported.
141
+
{% capture bedrock_note %}
142
+
Authentication with Bedrock API keys is not supported.
143
+
{% endcapture %}
144
+
{% include templates/info-banner.md content=bedrock_note %}
106
145
107
146
<hr>
108
147
@@ -114,6 +153,41 @@ ThingsBoard currently supports integration with the following AI providers:
114
153
115
154
<hr>
116
155
156
+
#### Ollama
157
+
158
+
{% assign sinceVersion = "4.2.1" %}
159
+
{% include templates/since.md %}
160
+
161
+
[Ollama](https://ollama.com/){:target="_blank"} allows you to easily run open large language models, such as **Llama 3** and **Mistral**, directly on your own machine. This enables local experimentation, offline usage, and greater control over your data.
162
+
163
+
To connect to your Ollama server, you need its **base URL** (e.g., *http://localhost:11434*) and an **authentication method**. The following options are supported:
- Default method for a standard Ollama installation.
169
+
- No authentication is required, and no credentials are sent with requests.
170
+
171
+
- <b>Basic</b>
172
+
- Use when Ollama is secured behind a reverse proxy requiring **HTTP Basic Authentication**.
173
+
- The provided username and password are combined into a `username:password` string, Base64-encoded, and sent in the header:
174
+
```text
175
+
Authorization: Basic <encoded_credentials>
176
+
```
177
+
- <b>Token</b>
178
+
- Use when Ollama is secured behind a reverse proxy requiring **Bearer Token Authentication**.
179
+
- The provided token is sent in the header:
180
+
```text
181
+
Authorization: Bearer <token>
182
+
```
183
+
184
+
{% capture difference %}
185
+
**Security Recommendation:** When using **Basic** or **Token** authentication, always connect via an `HTTPS` URL. Using plain `HTTP` will transmit credentials in clear text, which is insecure.
186
+
{% endcapture %}
187
+
{% include templates/info-banner.md content=difference %}
188
+
189
+
<hr>
190
+
117
191
## Model configuration
118
192
119
193
After you've selected and authenticated your AI provider, you need to specify which particular AI model to use (or deployment name in the case of [Azure OpenAI](#azure-openai)).
@@ -129,13 +203,16 @@ For <b>production usage</b>, we recommend using model snapshots to ensure predic
129
203
Some models support advanced configuration parameters (depending on the provider), such as:
130
204
- <b>Temperature</b> – Adjusts the level of randomness in the model's output. Higher values increase randomness, while lower values decrease it.
131
205
- <b>Top P</b> – Creates a pool of the most probable tokens for the model to choose from. Higher values create a larger and more diverse pool, while lower values create a smaller one.
132
-
- <b>Top K</b> - Restricts the model's choices to a fixed set of the "K" most likely.
133
-
tokens.
206
+
- <b>Top K</b> - Restricts the model's choices to a fixed set of the "K" most likely tokens.
134
207
- <b>Presence penalty</b> - Applies a fixed penalty to the likelihood of a token if it has already appeared in the text.
135
208
- <b>Frequency penalty</b> - Applies a penalty to a token's likelihood that increases based on its frequency in the text.
136
-
- <b>Maximum output tokens</b> – limit response length. Sets the maximum number of tokens that the model can generate in a single response.
209
+
- <b>Maximum output tokens</b> – Sets the maximum number of tokens that the model can generate in a single response.
210
+
- <b>Context length</b> – Defines the size of the context window in tokens. This value sets the total memory limit for the model, including both the user's input and the generated response.
137
211
138
-
> If advanced settings cause errors, try removing their values. In this case, defaults will be applied. This often resolves incompatibility issues for certain models.
212
+
{% capture advanced_settings_tip %}
213
+
If advanced settings cause errors, try removing their values. In this case, defaults will be applied. This often resolves incompatibility issues for certain models.
214
+
{% endcapture %}
215
+
{% include templates/info-banner.md content=advanced_settings_tip %}
139
216
140
217
{% include images-gallery.html imageCollection="advanced-model-settings" %}
141
218
@@ -148,15 +225,21 @@ If the response is successful, you will see a ✅ <b>green checkmark</b>.
148
225
149
226
{% include images-gallery.html imageCollection="check-connectivity-1" %}
150
227
151
-
> <b>Best practice</b>: Always use the connectivity check after configuring a provider to ensure smooth runtime execution.
228
+
{% capture best_practice %}
229
+
Always use the connectivity check after configuring a provider to ensure smooth runtime execution.
230
+
{% endcapture %}
231
+
{% include templates/info-banner.md content=best_practice %}
152
232
153
233
If an error occurs (e.g., invalid API key, non-existing model), an error message with details will be displayed ❌.
154
234
155
235
{% include images-gallery.html imageCollection="check-connectivity-2" %}
156
236
157
237
This feature ensures your configuration is valid and prevents runtime errors when models are used in production.
158
238
159
-
> <b>Note</b>: Even though the test request is trivial (e.g., “What is the capital of X country?”), providers usually charge for it. However, the cost is minimal.
239
+
{% capture test_cost_note %}
240
+
Even though the test request is trivial (e.g., "What is the capital of X country?"), providers usually charge for it. However, the cost is minimal.
241
+
{% endcapture %}
242
+
{% include templates/info-banner.md content=test_cost_note %}
0 commit comments