You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/en/docs/appstore/use-content/platform-supported-content/modules/genai/genai-commons.md
+19-23Lines changed: 19 additions & 23 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -305,7 +305,7 @@ Chat completions operations can be used by passing a [DeployedModel](#deployed-m
305
305
306
306
We recommend that you adapt to the same interface when developing custom chat completions or image generations operations, such as integration with different AI providers. The generic interfaces are described below. For more detailed information, refer to the documentation of the connector that you want to use, since it may expect specializations of the generic GenAI common entities as an input.
307
307
308
-
##### Chat Completions (Without History)
308
+
##### Chat Completions (without history)
309
309
310
310
The `Chat Completions (without history)` operation supports scenarios where there is no need to send a list of (historic) messages comprising the conversation so far as part of the request.
311
311
@@ -324,9 +324,9 @@ The `Chat Completions (without history)` operation supports scenarios where ther
324
324
| --- | --- | --- |
325
325
|`Response`|[Response](#response)| A `Response` object that contains the assistant's response.|
326
326
327
-
##### Chat Completions (With History)
327
+
##### Chat Completions (with history)
328
328
329
-
The `Chat Completions (with history)` operation interface supports more complex use cases where a list of (historical) messages (for example, comprising the conversation or context so far) is sent as part of the request to the LLM.
329
+
The `Chat Completions (with history)` operation supports more complex use cases where a list of (historical) messages (for example, comprising the conversation or context so far) is sent as part of the request to the LLM.
330
330
331
331
###### Input Parameters
332
332
@@ -341,29 +341,29 @@ The `Chat Completions (with history)` operation interface supports more complex
341
341
| --- | --- | --- |
342
342
|`Response`|[Response](#response)| A `Response` object that contains the assistant's response. |
343
343
344
-
##### Image Generations
344
+
##### Generate Image
345
345
346
-
The `Image Generations` operation interface supports the generation of images based on a `UserPrompt` passed as string. The returned `Response` contains a `FileContent` via `FileCollection` and `Message`. See microflows in the `Connector Building` folder to construct the output.
346
+
The `Generate Image` operation supports the generation of images based on a `UserPrompt` passed as string. The returned `Response` contains a `FileContent` via `FileCollection` and `Message`. See microflows in the `Handle Response` folder to construct the output.
347
347
348
348
###### Input Parameters
349
349
350
350
| Name | Type | Mandatory | Description |
351
351
| --- | --- | --- |--- |
352
-
|`Connection`|[Connection](#connection)| mandatory |This is an object that contains specifications to interact with an AI provider. |
352
+
|`DeployedModel`|[DeployedModel](#deployed-model)| mandatory |The DeployedModel entity replaces the Connection entity. It contains the name of the microflow to be executed for the specified model and other information relevant to connect to a model. The OutputModality needs to be Image. |
353
353
|`UserPrompt`| String | mandatory | This is the description the image will be based on. |
354
354
|`ImageOptions`|[ImageOptions](#imageoptions-entity)| optional | This can be used to pass optional request attributes. |
355
355
356
356
###### Return Value
357
357
358
358
| Name | Type | Description |
359
359
| --- | --- | --- |
360
-
|`Response`|[Response](#response)| A `Response` object that contains the assistant's response including a `FileContent` which will be used in [Get Generated Image (Single)](#image-get-single). |
360
+
|`Response`|[Response](#response)| A `Response` object that contains the assistant's response including a `FileContent` which needs to be used in [Get Generated Image (Single)](#image-get-single) or [Get Generated Images (List)]({#image-get-list}).|
361
361
362
362
#### Text and Files: Build request {#text-build-request}
363
363
364
364
The following microflows help you construct the input request structures for the operations for text and files defined in GenAI Commons.
365
365
366
-
##### Chat: Create Request {#chat-create-request}
366
+
##### Create Request {#chat-create-request}
367
367
368
368
This microflow can be used to create a request for a chat completion operation. This is the request object that contains the top-level functional input for the language model to generate a response.
369
369
@@ -382,7 +382,7 @@ This microflow can be used to create a request for a chat completion operation.
382
382
|--- |--- |--- |
383
383
|`Request`|[Request](#request)| This is the created request object. |
384
384
385
-
##### Chat: Add Message to Request {#chat-add-message-to-request}
385
+
##### Add Message to Request {#chat-add-message-to-request}
386
386
387
387
This microflow can be used to add a new [Message](#message) to the [Request](#request) object. A message represents the conversation text content and optionally has a collection of files attached that need to be taken into account when generating the response (such as images for vision). Make sure to add messages chronologically so that the most recent message is added last.
388
388
@@ -399,7 +399,7 @@ This microflow can be used to add a new [Message](#message) to the [Request](#re
This microflow can be used to add an optional [StopSequence](#stopsequence) to the request. It can be used after the request has been created. If available for the connector and model of choice, stop sequences let models know when to stop generating text.
405
405
@@ -451,7 +451,7 @@ Use this microflow to add a file to an existing [FileCollection](#filecollection
This microflow creates new [ImageOptions](#imageoptions-entity).
457
457
@@ -512,7 +512,7 @@ This microflow does not have a return value.
512
512
513
513
The following microflows handle the response processing.
514
514
515
-
##### Get Model Response Text {#chat-get-model-response-text}
515
+
##### Get Response Text {#chat-get-model-response-text}
516
516
517
517
This microflow can be used to get the content from the latest assistant message over association `Response_Message`. Use this microflow to get the response text from the latest assistant response message. In many cases, this is the main value needed for further logic after the operation or is displayed to the end user.
518
518
@@ -580,22 +580,18 @@ This operation processes a response that was created by an image generations ope
580
580
581
581
#### Knowledge Bases and Embeddings: Embeddings {#embeddings}
582
582
583
-
To make use of embeddings in a Mendix app, GenAI Commons defines interfaces for embedding operations that connectors can adhere to. We recommend that you adapt to the same interface when developing custom embedding operations, such as integration with different AI providers. The generic interfaces are described below. For more detailed information, refer to the documentation of the connector that you want to use, since it may expect specializations of the generic GenAI common entities as an input.
583
+
Embeddings operations can be used by passing a [DeployedModel](#deployed-model) object of the desired connector. The action calls the internally assigned microflow of the connector and returns the response. Operations from different connectors can be exchanged very easily without much additional development effort.
584
584
585
-
{{% alert color="info" %}}
586
-
These operations are not implemented in this module. The module only describes the interface (microflow input parameters, return value, and expected behavior), and it is up to connectors that adhere to the principles of GenAI Commons to provide an implementation. For an implementation example, see the respective sections in the [OpenAI connector](/appstore/modules/genai/openai/) or the [Amazon Bedrock Connector](/appstore/modules/genai/bedrock/), or take a look at the [showcase app](https://marketplace.mendix.com/link/component/220475) where both connectors are implemented to decide at runtime whether to call the LLM through OpenAI or Amazon Bedrock.
The `Embeddings (String)` operation interface allows the invocation of the embeddings API with a String input and returns an `EmbeddingsResponse` object with token usage statistics, if applicable. The `EmbeddingsResponse_GetFirstVector` microflow from GenAI Commons can be used to retrieve the corresponding embedding vector in a String representation. This operation supports scenarios where the vector embedding of a single string must be generated, e.g. to perform a nearest neighbor search across an existing knowledge base.
587
+
The `Generate Embeddings (String)` operation allows the invocation of the embeddings API with a String input and returns an `EmbeddingsResponse` object with token usage statistics, if applicable. The `EmbeddingsResponse_GetFirstVector` microflow from GenAI Commons can be used to retrieve the corresponding embedding vector in a String representation. This operation supports scenarios where the vector embedding of a single string must be generated, e.g. to perform a nearest neighbor search across an existing knowledge base.
592
588
593
589
###### Input Parameters
594
590
595
591
| Name | Type | Mandatory | Description |
596
592
| --- | --- | ---| --- |
597
593
|`InputText`| String | mandatory | Input text to create the embedding vector for. |
598
-
|`Connection`|[Connection](#connection)| mandatory | Connection object that contains the required endpoint details and API credentials. Depending on the connector module, a specific specialization must be passed. |
594
+
|`DeployedModel`|[DeployedModel](#deployed-model)| mandatory |The DeployedModel entity replaces the Connection entity. It contains the name of the microflow to be executed for the specified model and other information relevant to connect to a model. The OutputModality needs to be Embeddings. |
599
595
|`EmbeddingOptions`|[EmbeddingsOptions](#embeddingsoptions-entity)| optional | Can be used to pass optional request attributes.|
600
596
601
597
###### Return Value
@@ -604,16 +600,16 @@ The `Embeddings (String)` operation interface allows the invocation of the embed
604
600
| --- | --- | --- |
605
601
|`EmbeddingsResponse`|[EmbeddingsResponse](#embeddingsresponse-entity)| A response object that contains the token usage statistics and the corresponding embedding vector as part of a ChunkCollection |
The `Embeddings (ChunkCollection)` operation interface allows the invocation of an embeddings API with a [ChunkCollection](#chunkcollection) and returns an [EmbeddingsResponse](#embeddingsresponse-entity) object with token usage statistics, if applicable. The response object is associated with the original [ChunkCollection](#chunkcollection) used as an input, and the [Chunk](#chunk-entity) (or [KnowledgeBaseChunk](#knowledgebasechunk-entity)) objects will be updated with their corresponding embedding vector retrieved from the Embeddings API within this microflow.
605
+
The `Generate Embeddings (Chunk Collection)` operation allows the invocation of an embeddings API with a [ChunkCollection](#chunkcollection) and returns an [EmbeddingsResponse](#embeddingsresponse-entity) object with token usage statistics, if applicable. The response object is associated with the original [ChunkCollection](#chunkcollection) used as an input, and the [Chunk](#chunk-entity) (or [KnowledgeBaseChunk](#knowledgebasechunk-entity)) objects will be updated with their corresponding embedding vector retrieved from the Embeddings API within this microflow.
610
606
611
607
###### Input Parameters
612
608
613
609
| Name | Type | Mandatory | Description |
614
610
| --- | --- | ---| --- |
615
611
|`ChunkCollection`|[ChunkCollection](#chunkcollection)| mandatory | A ChunkCollection with Chunks for which an embedding vector should be generated. Use operations from GenAI commons to create a ChunkCollection and add Chunks or KnowledgeBaseChunks to it. |
616
-
|`Connection`|[Connection](#connection)| mandatory |A Connection object that contains the required endpoint details and API credentials. Depending on the connector module, a specific specialization must be passed. |
612
+
|`DeployedModel`|[DeployedModel](#deployed-model)| mandatory |The DeployedModel entity replaces the Connection entity. It contains the name of the microflow to be executed for the specified model and other information relevant to connect to a model. The OutputModality needs to be Embeddings. |
617
613
|`EmbeddingOptions`|[EmbeddingsOptions](#embeddingsoptions-entity)| optional | Can be used to pass optional request attributes. |
0 commit comments