You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/en/docs/appstore/use-content/platform-supported-content/modules/genai/genai-commons.md
+32-28Lines changed: 32 additions & 28 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -60,7 +60,7 @@ The domain model in Mendix is a data model that describes the information in you
60
60
61
61
#### `DeployedModel` {#deployed-model}
62
62
63
-
The `DeployedModel` represents a GenAI model that can be invoked by the Mendix app. It contains a display name and a technical name/identifier. It also contains the name of the microflow to be executed for the specified model and other information relevant to connect to a model.
63
+
The `DeployedModel` represents a GenAI model that can be invoked by the Mendix app. It contains a display name and a technical name/identifier. It also contains the name of the microflow to be executed for the specified model and other information relevant to connect to a model. The creation of Deployed Models are handled by the connectors itself (see their specializations) where admins can configure those at runtime.
64
64
65
65
The `DeployedModel` entity replaces the capabilities that were coverd by the `Connection` entity in earlier versions of GenAI Commons.
66
66
@@ -103,7 +103,7 @@ The data stored in this entity is to be used later on for monitoring purposes.
103
103
104
104
#### `Connection` {#connection}
105
105
106
-
The Connection entity used to be an input parameter for Chat completions, Embeddings and Image Generation operations but got replaced by DeployedModel. It is currently only used as a general connection entity for KnowledgeBase interactions.
106
+
The Connection entity used to be an input parameter for Chat completions, Embeddings and Image Generation operations but got replaced by DeployedModel. It is currently only used as a general connection entity for Knowledge Base interactions.
107
107
108
108
#### `Request` {#request}
109
109
@@ -299,13 +299,13 @@ An optional input object for the image generations operations to set optional re
299
299
300
300
Use the exposed microflows and Java Actions to map the required information for GenAI operations from your custom app implementation to the GenAI model and vice versa. Two sets of operations are provided: one for text and files, plus a second one for embeddings and knowledge bases.
301
301
302
-
#### Text and Files: Chat Completions {#chat-completions}
302
+
#### Text and Files: Operations {#text-files-operations}
303
303
304
-
Chat completions operations can be used by passing a [DeployedModel](#deployed-model) object of the desired connector. The action calls the internally assigned microflow of the connector and returns the response. Operations from different connectors can be exchanged very easily without much additional development effort.
304
+
Chat completions and image generation operations can be used by passing a [DeployedModel](#deployed-model) object of the desired connector. The action calls the internally assigned microflow of the connector and returns the response. Operations from different connectors can be exchanged very easily without much additional development effort.
305
305
306
306
We recommend that you adapt to the same interface when developing custom chat completions or image generations operations, such as integration with different AI providers. The generic interfaces are described below. For more detailed information, refer to the documentation of the connector that you want to use, since it may expect specializations of the generic GenAI common entities as an input.
The `Chat Completions (without history)` operation supports scenarios where there is no need to send a list of (historic) messages comprising the conversation so far as part of the request.
311
311
@@ -341,7 +341,7 @@ The `Chat Completions (with history)` operation supports more complex use cases
341
341
| --- | --- | --- |
342
342
|`Response`|[Response](#response)| A `Response` object that contains the assistant's response. |
343
343
344
-
##### Generate Image {#generate-image}
344
+
##### Text and Files: Generate Image {#generate-image}
345
345
346
346
The `Generate Image` operation supports the generation of images based on a `UserPrompt` passed as string. The returned `Response` contains a `FileContent` via `FileCollection` and `Message`. See microflows in the `Handle Response` folder to construct the output.
347
347
@@ -359,7 +359,7 @@ The `Generate Image` operation supports the generation of images based on a `Use
359
359
| --- | --- | --- |
360
360
|`Response`|[Response](#response)| A `Response` object that contains the assistant's response including a `FileContent` which needs to be used in [Get Generated Image (Single)](#image-get-single) or [Get Generated Images (List)]({#image-get-list}).|
361
361
362
-
#### Text and Files: Build request {#text-build-request}
362
+
#### Text and Files: Build request {#text-files-request}
363
363
364
364
The following microflows help you construct the input request structures for the operations for text and files defined in GenAI Commons.
365
365
@@ -508,13 +508,13 @@ Use this microflow to control how the model should determine which function to l
508
508
509
509
This microflow does not have a return value.
510
510
511
-
#### Text and Files: Handle Response {#text-handle-response}
511
+
#### Text and Files: Handle Response {#text-files-response}
512
512
513
513
The following microflows handle the response processing.
514
514
515
515
##### Get Response Text {#chat-get-model-response-text}
516
516
517
-
This microflow can be used to get the content from the latest assistant message over association `Response_Message`. Use this microflow to get the response text from the latest assistant response message. In many cases, this is the main value needed for further logic after the operation or is displayed to the end user.
517
+
This microflow can be used to get the content from the latest assistant message over association `Response_Message`. Use this microflow to get the response text from the latest assistant response message. In many cases, this is the main value needed for further logic after the operation or is displayed to the end user. Note that the content can be directly extracted from the [Response's](#response) attribute `ResponseText`.
518
518
519
519
###### Input Parameters
520
520
@@ -578,7 +578,7 @@ This operation processes a response that was created by an image generations ope
578
578
|---|---|---|
579
579
|`GeneratedImageList`| List of type determined by `ResponseImageEntity`| The list of generated images. |
580
580
581
-
#### Knowledge Bases and Embeddings: Embeddings {#embeddings}
581
+
#### Knowledge Bases and Embeddings: Operations {#knowledge-bases-embeddings-operations}
582
582
583
583
Embeddings operations can be used by passing a [DeployedModel](#deployed-model) object of the desired connector. The action calls the internally assigned microflow of the connector and returns the response. Operations from different connectors can be exchanged very easily without much additional development effort.
584
584
@@ -618,9 +618,9 @@ The `Generate Embeddings (Chunk Collection)` operation allows the invocation of
618
618
| --- | --- | --- |
619
619
|`EmbeddingsResponse`|[EmbeddingsResponse](#embeddingsresponse-entity)| An response object that contains the token usage statistics and the corresponding embedding vector as part of a ChunkCollection. |
620
620
621
-
#### Knowledge Bases and Embeddings Helpers {#knowledge-bases-embeddings}
621
+
#### Knowledge Bases and Embeddings: Build Request {#knowledge-bases-embeddings-request}
622
622
623
-
The following microflows and Java actions help you construct the input structures and handle the response object for the operations for knowledge bases and embeddings as defined in GenAI Commons.
623
+
The following microflows and Java actions help you construct the input structures for the operations for knowledge bases and embeddings as defined in GenAI Commons.
@@ -689,22 +689,6 @@ This microflow creates new [EmbeddingsOptions](#embeddingsoptions-entity).
689
689
|--- |--- |--- |
690
690
|`EmbeddingsOptions`|[EmbeddingsOptions](#embeddingsoptions-entity)| The newly created EmbeddingsOptions object. |
691
691
692
-
##### Embeddings: Get First Vector from Response {#embeddings-get-first-vector}
693
-
694
-
This microflow gets the first embedding vector from the response of an embeddings operation.
695
-
696
-
###### Input Parameters
697
-
698
-
| Name | Type | Mandatory | Description |
699
-
|--- |--- |--- |--- |
700
-
|`EmbeddingsResponse`|[EmbeddingsResponse](#embeddingsresponse-entity)| mandatory | Response object that gets returned by the embeddings operations. |
701
-
702
-
###### Return Value
703
-
704
-
| Name | Type | Description |
705
-
|--- |--- |--- |
706
-
|`Vector`| String | The first vector from the response. |
707
-
708
692
##### Knowledge Base: Initialize MetadataCollection with Metadata {#knowledgebase-initialize-metadatacollection}
709
693
710
694
This microflow creates a new [MetadataCollection](#metadatacollection-entity) and adds a new [Metadata](#metadatacollection-entity). The [MetadataCollection](#metadatacollection-entity) will be returned. To add additional Metadata, use [Add Metadata to MetadataCollection](#knowledgebase-add-metadata).
@@ -738,6 +722,26 @@ This microflow adds a new [Metadata](#metadatacollection-entity) object to a giv
738
722
739
723
This microflow does not have a return value.
740
724
725
+
#### Knowledge Bases and Embeddings: Handle Response {#knowledge-bases-embeddings-response}
726
+
727
+
The following microflows and Java actions help you handle the response object for the operations for embeddings as defined in GenAI Commons.
728
+
729
+
##### Embeddings: Get First Vector from Response {#embeddings-get-first-vector}
730
+
731
+
This microflow gets the first embedding vector from the response of an embeddings operation.
732
+
733
+
###### Input Parameters
734
+
735
+
| Name | Type | Mandatory | Description |
736
+
|--- |--- |--- |--- |
737
+
|`EmbeddingsResponse`|[EmbeddingsResponse](#embeddingsresponse-entity)| mandatory | Response object that gets returned by the embeddings operations. |
738
+
739
+
###### Return Value
740
+
741
+
| Name | Type | Description |
742
+
|--- |--- |--- |
743
+
|`Vector`| String | The first vector from the response. |
0 commit comments