Skip to content

Commit 5c85c80

Browse files
committed
SAS-1719: updated openai docs to use Microsoft Foundry as name instead of Azure OpenAI
1 parent 494959d commit 5c85c80

1 file changed

Lines changed: 26 additions & 30 deletions

File tree

  • content/en/docs/marketplace/genai/reference-guide/external-platforms

content/en/docs/marketplace/genai/reference-guide/external-platforms/openai.md

Lines changed: 26 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ aliases:
1111

1212
## Introduction {#introduction}
1313

14-
The [OpenAI Connector](https://marketplace.mendix.com/link/component/220472) allows you to integrate generative AI into your Mendix app. It is compatible with [OpenAI's platform](https://platform.openai.com/) as well as [Azure's OpenAI service](https://oai.azure.com/).
14+
The [OpenAI Connector](https://marketplace.mendix.com/link/component/220472) allows you to integrate generative AI into your Mendix app. It is compatible with [OpenAI's platform](https://platform.openai.com/) and [Microsoft Foundry](https://learn.microsoft.com/en-us/azure/ai-foundry/what-is-ai-foundry), where you can access OpenAI models.
1515

1616
### Features {#features}
1717

@@ -21,7 +21,7 @@ OpenAI provides market-leading LLM capabilities with GPT-4:
2121
* Creativity – Generate, edit, and iterate with end-users on creative and technical writing tasks, such as composing songs, writing screenplays, or learning an end-user’s writing style.
2222
* Longer context – GPT-4 can handle over 25,000 words of text, allowing for use cases like long-form content creation, extended conversations, and document search and analysis.
2323

24-
Mendix provides dual-platform support for both [OpenAI](https://platform.openai.com/) and [Azure OpenAI](https://oai.azure.com/).
24+
Mendix provides support for [OpenAI](https://platform.openai.com/) and [Microsoft Foundry](https://learn.microsoft.com/en-us/azure/ai-foundry/what-is-ai-foundry) (formerly known as Azure OpenAI or Cognitive Services). Microsoft Foundry is Microsoft's unified AI platform that provides access to OpenAI models.
2525

2626
With the current version, Mendix supports the Chat Completions API for [text generation](https://platform.openai.com/docs/guides/text-generation), the Image Generations API for [images](https://platform.openai.com/docs/guides/images), the Embeddings API for [vector embeddings](https://platform.openai.com/docs/guides/embeddings/what-are-embeddings), and indexes via [Azure AI Search](https://learn.microsoft.com/en-us/azure/search/) for knowledge base retrieval.
2727

@@ -33,7 +33,7 @@ By integrating Azure AI Search, the OpenAI Connector enables knowledge base retr
3333

3434
### Prerequisites {#prerequisites}
3535

36-
To use this connector, you need to either sign up for an [OpenAI account](https://platform.openai.com/) or have access to deployments at [Azure OpenAI](https://oai.azure.com/).
36+
To use this connector, you need to either sign up for an [OpenAI account](https://platform.openai.com/) or have access to a [Microsoft Foundry](https://learn.microsoft.com/en-us/azure/ai-foundry/what-is-ai-foundry) project with OpenAI models deployed.
3737

3838
### Dependencies {#dependencies}
3939

@@ -74,33 +74,29 @@ The following inputs are required for the OpenAI configuration:
7474
| Endpoint | This is the API endpoint (for example, `https://api.openai.com/v1`) |
7575
| Token | This is the access token to authorize your API call. <br />To get an API, follow these steps:<ol><li>Create an account and sign in at [OpenAI](https://platform.openai.com/).</li><li> Go to the [API key page](https://platform.openai.com/account/api-keys) to create a new secret key. </li><li>Copy the API key and save this somewhere safe.</li></ol> |
7676

77-
#### Azure OpenAI Configuration {#azure-openai-configuration}
77+
#### Microsoft Foundry Configuration {#azure-openai-configuration}
7878

79-
The following inputs are required for the Azure OpenAI configuration:
79+
The following inputs are required for the Microsoft Foundry configuration:
8080

8181
| Parameter | Value |
8282
| -------------- | ------------------------------------------------------------ |
8383
| Display name | This is the name identifier of a configuration (for example, *MyConfiguration*). |
84-
| API type | Select `AzureOpenAI`. |
85-
| Endpoint | This is the API endpoint (for example, `https://your-resource-name.openai.azure.com/openai/deployments/`).<br />For details on how to obtain `your-resource-name`, see the [Obtaining Azure OpenAI Resource Name](#azure-resource-name) section below. |
84+
| API type | Select `AzureOpenAI` for both Azure OpenAI and Microsoft Foundry deployments. |
85+
| Endpoint | This is the API endpoint (for example, `https://your-resource-name.openai.azure.com/openai/deployments/`).<br />For details on how to obtain `your-resource-name`, see the [Obtaining Resource Name](#azure-resource-name) section below. |
8686
| Azure key type | This is the type of token that is entered in the API key field. For Azure OpenAI, two types of keys are currently supported: Microsoft Entra token and API key. <br />For details on how to generate a Microsoft Entra access token, see [How to Configure Azure OpenAI Service with Managed Identities](https://learn.microsoft.com/en-gb/azure/ai-services/openai/how-to/managed-identity). Alternatively, if your organization allows it, you could use the Azure `api-key` authentication mechanism. For details on how to obtain an API key, see the [Obtaining Azure OpenAI API keys](#azure-api-keys) section below. For more information, see the [Technical Reference](#technical-reference) section. |
8787
| Token / API key | This is the access token to authorize your API call. |
8888

89-
##### Obtaining the Azure OpenAI Resource Name {#azure-resource-name}
89+
##### Obtaining the Resource Name {#azure-resource-name}
9090

91-
1. Go to the [Azure OpenAI portal](https://oai.azure.com/) and sign in.
92-
2. In the upper-right corner, next to your Avatar, click on the scope dropdown.
93-
3. The tab shows your Directory, Subscription, and Azure OpenAI resource.
94-
4. Make sure the right Azure OpenAI resource is selected.
95-
5. Use the copy icon ({{% icon name="copy" %}}) and use it as your resource name in the endpoint URL.
91+
1. Go to the [Microsoft Foundry portal](https://ai.azure.com/) and sign in.
92+
2. Select the right resource in the upper right corner.
93+
3. The home page should show `Resource configuration` where you can find the `Azure OpenAI endpoint`
94+
4. Use the copy icon ({{% icon name="copy" %}}) and use it as your resource name in the endpoint URL.
9695

97-
##### Obtaining the Azure OpenAI API Keys {#azure-api-keys}
96+
##### Obtaining API Keys {#azure-api-keys}
9897

99-
1. Go to the [Azure OpenAI portal](https://oai.azure.com/) and sign in.
100-
2. In the upper-right corner, next to your Avatar, click on the scope dropdown.
101-
3. The tab shows your Directory, Subscription, and Azure OpenAI resource.
102-
4. Make sure the right Azure OpenAI resource is selected.
103-
5. You can now view ({{% icon name="view" %}}) and copy ({{% icon name="copy" %}}) the value of the **key1** or **key2** field as your API key while setting up the configuration. Note that these keys might not be visible for everyone in the Azure OpenAI Portal, depending on your organization's security settings.
98+
1. On the same page where the resource name is located, you can find your API key information.
99+
2. You can now view ({{% icon name="view" %}}) and copy ({{% icon name="copy" %}}) the value of the **key1** or **key2** field as your API key while setting up the configuration. Note that these keys might not be visible for everyone in the portal, depending on your organization's security settings.
104100

105101
##### Adding Azure AI Search Resources {#azure-ai-search}
106102

@@ -119,15 +115,15 @@ Currently, the only supported authorization method for Azure AI Search resources
119115

120116
#### Configuring the OpenAI Deployed Models
121117

122-
A [Deployed Model](/appstore/modules/genai/genai-for-mx/commons/#deployed-model) represents a GenAI model instance that can be used by the app to generate text, embeddings, or images. For every model you want to invoke from your app, you need to create a `OpenAIDeployedModel` record, a specialization of `DeployedModel`. In addition to the model display name and a technical name/identifier, an OpenAI deployed model contains a reference to the additional connection details as configured in the previous step. For OpenAI, a set of common models will be prepopulated automatically upon saving the configuration. If you want to use additional models that are made available by OpenAI you need to configure additional OpenAI deployed models in your Mendix app. For Azure OpenAI no deployed models are created by default. The technical model names depend on the deployment names that were chosen while deploying the models in the [Azure Portal](https://oai.azure.com/resource/deployments). Therefore in this case you always need to configure the deployed models manually in your Mendix app.
118+
A [Deployed Model](/appstore/modules/genai/genai-for-mx/commons/#deployed-model) represents a GenAI model instance that can be used by the app to generate text, embeddings, or images. For every model you want to invoke from your app, you need to create a `OpenAIDeployedModel` record, a specialization of `DeployedModel`. In addition to the model display name and a technical name/identifier, an OpenAI deployed model contains a reference to the additional connection details as configured in the previous step. For OpenAI, a set of common models can be created automatically using the designated button. If you want to use additional models that are made available by OpenAI you need to configure additional OpenAI deployed models in your Mendix app. For Microsoft Foundry the model names can be different. The technical model names depend on the deployment names that were chosen while deploying the models in the [Microsoft Foundry portal](https://ai.azure.com/). Therefore in this case you always need to configure the deployed models manually in your Mendix app.
123119

124120
1. If needed, click the three dots for an OpenAI configuration to open the "Manage Deployed Models" pop-up.
125121
2. For every additional model, add a record. The following fields are required:
126122

127123
| Field | Description |
128124
| -------------- | ------------------------------------------------------------ |
129125
| Display name | This is the reference to the model for app users in case they have to select which one is to be used. |
130-
| Deployment name / Model name | This is the technical reference for the model. For OpenAI this is equal to the [model aliases](https://platform.openai.com/docs/models#current-model-aliases). For Azure OpenAI this is the deployment name from the [Azure Portal](https://oai.azure.com/resource/deployments).
126+
| Deployment name / Model name | This is the technical reference for the model. For OpenAI this is equal to the [model aliases](https://platform.openai.com/docs/models#current-model-aliases). For Microsoft Foundry this is the deployment name from the [Microsoft Foundry portal](https://ai.azure.com/).
131127
| Output modality| Describes what the output of the model is. This connector currently supports Text, Embedding, and Image.
132128
| Input modality| Describes what input modalities are accepted by the model. This connector currently supports Text and Image.
133129
| Azure API version | Azure OpenAI only. This is the API version to use for this operation. It follows the `yyyy-MM-dd` format. For supported versions, see [Azure OpenAI documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference). The supported versions can vary depending on the type of model, so make sure to look for the right section (such as Chat Completions, Image Generation, or Embeddings) on that page. |
@@ -196,7 +192,7 @@ For `Chat Completions with History`, `FileCollection` can optionally be added to
196192
Use the two OpenAI-specific microflow actions from the toolbox [Files: Initialize Collection with OpenAI File](#initialize-filecollection) and [Files: Add OpenAIFile to Collection](#add-file) to construct the input with either `FileDocuments` (for vision, it needs to be of type `Image`) or `URLs`. There are similar file operations exposed by the GenAI commons module that can be used for vision requests with the OpenAIConnector; however, these generic operations do not support the optional OpenAI-specific `Detail` attribute.
197193

198194
{{% alert color="info" %}}
199-
OpenAI and Azure OpenAI for vision do not necessarily all models provide feature parity when it comes to combining functionalities. In other words, Azure OpenAI does not support the use of JSON mode and function calling in combination with image (vision) input for certain models, so make sure to check the [Azure Documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models).
195+
OpenAI and Microsoft Foundry do not necessarily provide feature parity across all models when it comes to combining functionalities. In other words, Microsoft Foundry does not support the use of JSON mode and function calling in combination with image (vision) input for certain models, so make sure to check the [Azure Documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models).
200196

201197
When you use Azure OpenAI, it is recommended to set the optional `MaxTokens` input parameter; otherwise, the return output may be cut off.
202198
{{% /alert %}}
@@ -212,7 +208,7 @@ For [Chat Completions (without history)](/appstore/modules/genai/genai-for-mx/co
212208
You can send up to 100 pages across multiple files, with a maximum combined size of 32 MB per conversation. Currently, processing multiple files with OpenAI is not always guaranteed and can lead to unexpected behavior (for example, only one file being processed).
213209

214210
{{% alert color="info" %}}
215-
Azure OpenAI does not currently support file input.
211+
Microsoft Foundry does not currently support file input.
216212

217213
Note that the model uses the file name when analyzing documents, which may introduce a potential vulnerability to prompt injection. To reduce this risk, consider modifying the string or not passing it at all.
218214
{{% /alert %}}
@@ -323,13 +319,13 @@ To check your JDK version and update it if necessary, follow these steps:
323319
2. You may also need to update Gradle. To do this, go to **Edit** > **Preferences** > **Deployment** > **Gradle directory**. Click **Browse** and select the appropriate Gradle version from the Mendix folder. For Mendix 10.10 and above, use Gradle 8.5. For Mendix 10 versions below 10.10, use Gradle 7.6.3. Then save your settings by clicking **OK**.
324320
3. Rerun the project.
325321

326-
### Chat Completions with Vision and JSON Mode (Azure OpenAI)
322+
### Chat Completions with Vision and JSON Mode (Microsoft Foundry)
327323

328-
Azure OpenAI does not support the use of JSON mode and function calling in combination with image (vision) input and will return a `400 - model error`. Make sure the optional input parameters `ResponseFormat` and `FunctionCollection` are set to `empty` for all chat completion operations if you want to use vision with Azure OpenAI.
324+
Microsoft Foundry does not support the use of JSON mode and function calling in combination with image (vision) input and will return a `400 - model error`. Make sure the optional input parameters `ResponseFormat` and `ToolCollection` are set to `empty` for all chat completion operations if you want to use vision with Microsoft Foundry.
329325

330-
### Chat Completions with Vision Response is Cut Off (Azure OpenAI)
326+
### Chat Completions with Vision Response is Cut Off (Microsoft Foundry)
331327

332-
When you use Azure OpenAI, it is recommended to set the optional `MaxTokens` input parameter; otherwise, the response may be cut off. For more details, see the [Azure OpenAI Documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/gpt-with-vision?tabs=rest%2Csystem-assigned%2Cresource#call-the-chat-completion-apis).
328+
When you use Microsoft Foundry, it is recommended to set the optional `MaxTokens` input parameter; otherwise, the response may be cut off. For more details, see the [Azure OpenAI Documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/gpt-with-vision?tabs=rest%2Csystem-assigned%2Cresource#call-the-chat-completion-apis).
333329

334330
### Attribute or Reference Required Error Message After Upgrade
335331

@@ -342,9 +338,9 @@ If you encounter an error caused by conflicting Java libraries, such as `java.la
342338
## Read More {#read-more}
343339

344340
* [Prompt Engineering – OpenAI Documentation](https://platform.openai.com/docs/guides/prompt-engineering)
345-
* [Introduction to Prompt Engineering – Microsoft Azure Documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/prompt-engineering)
346-
* [Prompt Engineering Techniques – Microsoft Azure Documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/advanced-prompt-engineering?pivots=programming-language-chat-completions)
341+
* [Introduction to Prompt Engineering – Microsoft Foundry Documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/prompt-engineering)
342+
* [Prompt Engineering Techniques – Microsoft Foundry Documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/advanced-prompt-engineering?pivots=programming-language-chat-completions)
347343
* [ChatGPT Prompt Engineering for Developers - DeepLearning.AI](https://www.deeplearning.ai/short-courses/chatgpt-prompt-engineering-for-developers)
348344
* [Function Calling - OpenAI Documentation](https://platform.openai.com/docs/guides/function-calling)
349345
* [Vision - OpenAI Documentation](https://platform.openai.com/docs/guides/vision)
350-
* [Vision - Azure OpenAI Documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/gpt-with-vision)
346+
* [Vision - Microsoft Foundry Documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/gpt-with-vision)

0 commit comments

Comments
 (0)