You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/en/docs/marketplace/genai/reference-guide/genai-commons.md
+13-3Lines changed: 13 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -41,15 +41,15 @@ Although GenAI Commons technically defines additional capabilities typically fou
41
41
42
42
### Token Usage
43
43
44
-
GenAI Commons can help store usage data, allowing admins to understand token usage. Usage data is persisted only if the constant `StoreUsageMetrics` is set to *true* (exception: if [StoreTraces](#traceability) is set to *true*, Usage data is stored as well). In general, this is only supported for chat completions and embedding operations.
44
+
GenAI Commons can help store usage data, allowing admins to understand token usage. Usage data is persisted only if the constant `StoreUsageMetrics` is set to *true* (exception in version 5.3.0 and above: if [StoreTraces](#traceability) is set to *true*, Usage data is stored as well). In general, this is only supported for chat completions and embedding operations.
45
45
46
46
To clean up usage data in a deployed app, you can enable the daily scheduled event `ScE_Usage_Cleanup` in the Mendix Cloud Portal. Use the `Usage_CleanUpAfterDays` constant to control for how long token usage data should be persisted.
47
47
48
48
Lastly, the [Conversational UI module](/appstore/modules/genai/conversational-ui/) provides pages, snippets, and logic to display and export token usage information. For this to work, the module roles `UsageMonitoring` from both Conversational UI as well as GenAI Commons need to be assigned to the applicable project roles.
49
49
50
50
### Traceability {#traceability}
51
51
52
-
By default, the chat completions operations of GenAI Commons store data in your application's database for traceability reasons. This makes it easier to understand the usage of GenAI in your app and why the model behaved in a certain way, for example, by reviewing tool usage. Trace data is only persisted if the constant `StoreTraces` is set to *true*.
52
+
By default, the chat completions operations of GenAI Commons store data in your application's database for traceability reasons. This makes it easier to understand the usage of GenAI in your app and why the model behaved in a certain way, for example, by reviewing tool usage. In version 5.3.0 and above, trace data is only persisted if the constant `StoreTraces` is set to *true*.
53
53
54
54
As traces may contain sensitive and personally identifiable information, you should determine, on a case-by-case basis, whether storing this data is compliant.
55
55
@@ -130,6 +130,8 @@ The data stored in this entity is to be used later on for token consumption moni
130
130
A trace represents the whole LLM interaction from the first user message until the final assistant's response was returned, including tool calls.
131
131
The data stored in this entity is to be used later on for traceability use cases.
132
132
133
+
`Trace` was introduced in version 5.3.0.
134
+
133
135
| Attribute | Description |
134
136
| --- | --- |
135
137
|`TraceId`| The trace ID is set internally to identify a trace. |
@@ -146,6 +148,8 @@ The data stored in this entity is to be used later on for traceability use cases
146
148
147
149
A span is created for each interaction between Mendix and the LLM (such as chat completions, tool calling, etc.). The generalized object is typically not used; instead, its specializations are used.
148
150
151
+
`Span` was introduced in version 5.3.0.
152
+
149
153
| Attribute | Description |
150
154
| --- | --- |
151
155
|`SpanId`| The span ID is set internally to identify a span. |
@@ -159,6 +163,8 @@ A span is created for each interaction between Mendix and the LLM (such as chat
159
163
160
164
A model span is created for each interaction between Mendix and the LLM where content is generated (sent as the assistant's message). Typically, this is a request for text generation. In addition to the [Span's](#span) attributes, it also contains the following:
161
165
166
+
`ModelSpan` was introduced in version 5.3.0.
167
+
162
168
| Attribute | Description |
163
169
| --- | --- |
164
170
|`InputTokens`| Number of tokens in the request. |
@@ -169,6 +175,8 @@ A model span is created for each interaction between Mendix and the LLM where co
169
175
170
176
A tool span is created for each tool call requested by the LLM. The tool call is processed in GenAI Commons, and the result is sent back to the model. In addition to the [Span's](#span) attributes, it also contains the following:
171
177
178
+
`ToolSpan` was introduced in version 5.3.0.
179
+
172
180
| Attribute | Description |
173
181
| --- | --- |
174
182
|`ToolName`| The name of the tool that was called. |
@@ -177,7 +185,9 @@ A tool span is created for each tool call requested by the LLM. The tool call is
177
185
178
186
#### `KnowledgeBaseSpan` {#knowledge-base-span}
179
187
180
-
A knowledge base span is created for each knowledge base retrieval tool call requested by the LLM. The tool call is processed in GenAI Commons, and the result is sent back to the model. It does not contain any additional attributes compared to [ToolSpan](#tool-span).
188
+
A knowledge base span is created for each knowledge base retrieval tool call requested by the LLM. The tool call is processed in GenAI Commons, and the result is sent back to the model. It does not contain any additional attributes compared to [ToolSpan](#tool-span).
189
+
190
+
`KnowledgebaseSpan` was introduced in version 5.3.0.
0 commit comments