You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/en/docs/marketplace/genai/reference-guide/genai-commons.md
+16-16Lines changed: 16 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -41,17 +41,17 @@ Although GenAI Commons technically defines additional capabilities typically fou
41
41
42
42
### Token Usage
43
43
44
-
GenAI Commons can help store usage data which allows admins to understand the token usage. Usage data is persisted only if the constant `StoreUsageMetrics` is set to *true* (exception: if [StoreTraces](#traceability) is set to *true*, Usages data is stored as well). In general, this is only supported for chat completions and embedding operations.
44
+
GenAI Commons can help store usage data, allowing admins to understand token usage. Usage data is persisted only if the constant `StoreUsageMetrics` is set to *true* (exception: if [StoreTraces](#traceability) is set to *true*, Usage data is stored as well). In general, this is only supported for chat completions and embedding operations.
45
45
46
46
To clean up usage data in a deployed app, you can enable the daily scheduled event `ScE_Usage_Cleanup` in the Mendix Cloud Portal. Use the `Usage_CleanUpAfterDays` constant to control for how long token usage data should be persisted.
47
47
48
48
Lastly, the [Conversational UI module](/appstore/modules/genai/conversational-ui/) provides pages, snippets, and logic to display and export token usage information. For this to work, the module roles `UsageMonitoring` from both Conversational UI as well as GenAI Commons need to be assigned to the applicable project roles.
49
49
50
50
### Traceability {#traceability}
51
51
52
-
By default, the chat completions operations of GenAI Commons store data in your application's database for traceability reasons. This makes it easier understand the usage of GenAI in your app and why the model behaved in certain way, for example, by reviewing tool usage. Trace data is only persisted if the constant `StoreTraces` is set to *true*.
52
+
By default, the chat completions operations of GenAI Commons store data in your application's database for traceability reasons. This makes it easier to understand the usage of GenAI in your app and why the model behaved in a certain way, for example, by reviewing tool usage. Trace data is only persisted if the constant `StoreTraces` is set to *true*.
53
53
54
-
As traces may contain sensitive and personally identifiable information, you should decide per use case whether storing this data is compliant.
54
+
As traces may contain sensitive and personally identifiable information, you should determine, on a case-by-case basis, whether storing this data is compliant.
55
55
56
56
To clean up trace data in a deployed app, you can enable the daily scheduled event `ScE_Trace_Cleanup` in the [Mendix Cloud Portal](https://genai.home.mendix.com/). Use the `Trace_CleanUpAfterDays` constant to control the retention period of the trace data.
57
57
@@ -109,7 +109,7 @@ Accepted input modality of the associated deployed model.
109
109
110
110
#### `Usage` {#Usage}
111
111
112
-
This entity represents usage statistics of a call to an LLM. It refers to a complete LLM interaction; in case there are several iterations (e.g. recursive processing of function calls), everything should be aggregated into one Usage record.
112
+
This entity represents usage statistics of a call to an LLM. It refers to a complete LLM interaction; in case there are several iterations (for example, recursive processing of function calls), everything should be aggregated into one Usage record.
113
113
114
114
Following the principles of GenAI Commons, it must be stored based on the response for every successful call to a system of an LLM provider. This is only applicable to text and file operations and embedding operations.
115
115
@@ -132,30 +132,30 @@ The data stored in this entity is to be used later on for traceability use cases
132
132
133
133
| Attribute | Description |
134
134
| --- | --- |
135
-
|`TraceId`| The trace id is set internally to identify a trace. |
135
+
|`TraceId`| The trace ID is set internally to identify a trace. |
136
136
|`StartTime`| The start time of the initial model invocation. |
137
137
|`EndTime`| The end time after the final model invocation is completed. |
138
-
|`DurationMilliseconds`| The duration between start and end of the whole model invocation. |
138
+
|`DurationMilliseconds`| The duration between the start and end of the whole model invocation. |
139
139
|`Input`| The initial input of the model invocation (usually a user prompt). |
140
140
|`Output`| The response of the final message sent by the model (usually an assistant message). |
141
141
|`_AgentVersionId`| The id of the agent version (if applicable) as sent via the request. |
142
-
|`_ConversationId`| The id of the conversation (if applicable) as sent via the request. This usually is created by the model provider. |
142
+
|`_ConversationId`| The id of the conversation (if applicable) as sent via the request. This is usually created by the model provider. |
143
143
144
144
#### `Span` {#span}
145
145
146
-
A span is created for each interaction between Mendix and the LLM (such as chat completions, tool calling, etc.). The generalized object is typically not used instead, its specializations are used.
146
+
A span is created for each interaction between Mendix and the LLM (such as chat completions, tool calling, etc.). The generalized object is typically not used; instead, its specializations are used.
147
147
148
148
| Attribute | Description |
149
149
| --- | --- |
150
-
|`SpanId`| The span id is set internally to identify a span. |
150
+
|`SpanId`| The span ID is set internally to identify a span. |
151
151
|`StartTime`| The start time of the model invocation. |
152
152
|`EndTime`| The end time after the model invocation is completed. |
153
-
|`DurationMilliseconds`| The duration between start and end of the whole model invocation. |
153
+
|`DurationMilliseconds`| The duration between the start and end of the whole model invocation. |
154
154
|`Output`| The output of the span. |
155
155
156
156
#### `ModelSpan` {#model-span}
157
157
158
-
A model span is created for each interaction between Mendix and the LLM where content is generated (sent as the assistant's message). Typically, this is request for text generation. In addition to the [Span's](#span) attributes, it also contains the following:
158
+
A model span is created for each interaction between Mendix and the LLM where content is generated (sent as the assistant's message). Typically, this is a request for text generation. In addition to the [Span's](#span) attributes, it also contains the following:
159
159
160
160
| Attribute | Description |
161
161
| --- | --- |
@@ -165,25 +165,25 @@ A model span is created for each interaction between Mendix and the LLM where co
165
165
166
166
#### `ToolSpan` {#tool-span}
167
167
168
-
A tool span is created for each tool call requested by the LLM. The tool call is processed in GenAI Commons and the result is sent back to the model. In addition to the [Span's](#span) attributes, it also contains the following:
168
+
A tool span is created for each tool call requested by the LLM. The tool call is processed in GenAI Commons, and the result is sent back to the model. In addition to the [Span's](#span) attributes, it also contains the following:
169
169
170
170
| Attribute | Description |
171
171
| --- | --- |
172
172
|`ToolName`| The name of the tool that was called. |
173
173
|`_ToolCallId`| The ID of the tool call used by the model to map an assistant message containing a tool call with the output of the tool call (tool message). |
174
174
|`Input`| The input of the tool call as passed by the LLM. |
175
-
|`IsError`| Indicates if the tool call failed. If so, the span's output will contain the error message that was also logged and sent to the LLM as tool result. |
175
+
|`IsError`| Indicates if the tool call failed. If so, the span's output will contain the error message that was also logged and sent to the LLM as a tool result. |
176
176
177
177
#### `KnowledgeBaseSpan` {#knowledge-base-span}
178
178
179
-
A knowledge base span is created for each knowledge base retrieval tool call requested by the LLM. The tool call is processed in GenAI Commons and the result is sent back to the model. It does not contain any additional attributes compared to [ToolSpan](#tool-span)
179
+
A knowledge base span is created for each knowledge base retrieval tool call requested by the LLM. The tool call is processed in GenAI Commons, and the result is sent back to the model. It does not contain any additional attributes compared to [ToolSpan](#tool-span).
180
180
181
181
| Attribute | Description |
182
182
| --- | --- |
183
183
|`ToolName`| The name of the tool that was called. |
184
-
|`_ToolCallId`| The id of the tool call used by the model to map an assistant message containing a tool call with the output of the tool call (tool message). |
184
+
|`_ToolCallId`| The ID of the tool call used by the model to map an assistant message containing a tool call with the output of the tool call (tool message). |
185
185
|`Input`| The input of the tool call as passed by the LLM. |
186
-
|`IsError`| Indicates if the tool call failed. If so, the span's output will contain the error message that was also logged and sent to the LLM as tool result. |
186
+
|`IsError`| Indicates if the tool call failed. If so, the span's output will contain the error message that was also logged and sent to the LLM as a tool result. |
0 commit comments