|
| 1 | +# Azure API Management Setup Guide for Foundry Agents |
| 2 | + |
| 3 | +> **🎯 Step-by-Step Configuration** |
| 4 | +> This guide shows you how to configure Azure API Management (APIM) to make it ready for use by Foundry Agents as a connection. |
| 5 | +
|
| 6 | +## 🏗️ Prerequisites: APIM Instance Setup |
| 7 | + |
| 8 | +Before configuring APIM for Foundry Agents, you need an Azure API Management instance. Choose one of the following options: |
| 9 | + |
| 10 | +### Option 1: 🏢 Use Existing APIM Instance |
| 11 | + |
| 12 | +If you already have an Azure API Management instance (Standard v2 or Premium tier), you can proceed directly to the configuration steps below. |
| 13 | + |
| 14 | +### Option 2: 🔒 Deploy New Private APIM Setup |
| 15 | + |
| 16 | +For a fully secured private network setup, use the Bicep template mentioned in the [Private Network APIM Setup guide](https://github.com/azure-ai-foundry/foundry-samples/tree/main/samples/microsoft/infrastructure-setup/16-private-network-standard-agent-apim-setup-preview). |
| 17 | + |
| 18 | +This template provides: |
| 19 | +- **🔐 Secure Network Configuration**: Private network setup with Agents BYO VNet |
| 20 | +- **🏢 Enterprise-Ready**: Production-ready APIM gateway configuration |
| 21 | +- **🛡️ Network Security**: Fully isolated network access for enterprise scenarios |
| 22 | + |
| 23 | +--- |
| 24 | + |
| 25 | +## 🚀 Configuration Steps |
| 26 | + |
| 27 | +### Step 1: 📥 Import AI APIs into APIM |
| 28 | + |
| 29 | +To use AI models through APIM with Foundry Agents, you need to import the appropriate APIs into your APIM instance. Use the official Microsoft documentation for guidance: |
| 30 | + |
| 31 | +#### 📚 API Import Resources |
| 32 | + |
| 33 | +| Resource | Description | Link | |
| 34 | +|----------|-------------|------| |
| 35 | +| **🔗 Azure AI Foundry API in APIM** | Official guide for integrating Azure AI Foundry APIs with Azure API Management | [Azure AI Foundry API](https://learn.microsoft.com/en-in/azure/api-management/azure-ai-foundry-api) | |
| 36 | +| **🔗 Azure OpenAI API from Specification** | Official guide for importing Azure OpenAI APIs into Azure API Management from specification | [Azure OpenAI API Import](https://learn.microsoft.com/en-in/azure/api-management/azure-openai-api-from-specification) | |
| 37 | + |
| 38 | +#### 🎯 Choose Your Import Method |
| 39 | + |
| 40 | +- **🏢 Azure AI Foundry API**: Use this if you want to import and manage Azure AI Foundry resources through APIM |
| 41 | +- **🤖 Azure OpenAI API**: Use this if you want to import Azure OpenAI services directly from their API specification |
| 42 | + |
| 43 | +### Step 2: 🧪 Test Chat Completions API |
| 44 | + |
| 45 | +Foundry Agents are specifically interested in **chat completions APIs** for AI model interactions. After importing your API: |
| 46 | + |
| 47 | +1. **📍 Navigate to Chat Completions**: In your APIM instance, go to the imported API and locate the **chat completions** operation |
| 48 | +2. **🔧 Use APIM Test Feature**: Use the built-in test functionality in APIM to verify the chat completions endpoint works correctly |
| 49 | +3. **✅ Verify Response**: Ensure the API returns proper chat completion responses before proceeding with connection setup |
| 50 | + |
| 51 | +> **💡 Important**: Agents will primarily use the chat completions endpoint, so it's crucial to verify this specific operation is working through APIM before creating the Foundry connection. |
| 52 | +
|
| 53 | +### Step 3: 🔍 Configure Model Discovery |
| 54 | + |
| 55 | +Once chat completions are working, you need to configure how Foundry Agents will discover available models. You have two options: |
| 56 | + |
| 57 | +#### Option 1: 📋 Static Model List |
| 58 | + |
| 59 | +**✅ Advantages:** |
| 60 | +- **🚀 Better Performance**: Agents don't need to call APIM to fetch model details |
| 61 | +- **🔧 Simpler Setup**: No additional APIM configuration required |
| 62 | +- **💰 Cost Effective**: Reduces API calls to your APIM instance |
| 63 | + |
| 64 | +**📝 Implementation**: Configure the static model list directly in the connection metadata when creating the Foundry connection. No additional APIM setup needed for this approach. |
| 65 | + |
| 66 | +#### Option 2: 🌐 Dynamic Model Discovery via APIM |
| 67 | + |
| 68 | +**📋 When to Use:** |
| 69 | +- Static model configuration is not feasible for your scenario |
| 70 | +- You need dynamic model discovery capabilities |
| 71 | +- Models change frequently and need real-time discovery |
| 72 | + |
| 73 | +**🔧 Implementation**: Configure list deployments and get deployment APIs in APIM to enable dynamic model discovery. |
| 74 | + |
| 75 | +##### 📝 Dynamic Discovery Setup Instructions |
| 76 | + |
| 77 | +If you choose dynamic discovery, you need to manually add **2 operations** to your API in APIM: |
| 78 | + |
| 79 | +1. **📋 List Deployments Operation** - Returns all available models/deployments |
| 80 | +2. **🎯 Get Deployment Operation** - Returns details for a specific model/deployment |
| 81 | + |
| 82 | +##### 🛠️ Adding Get Deployment Operation |
| 83 | + |
| 84 | +1. **📍 Navigate to Your API**: In APIM, go to your imported API (e.g., `agent-aoai`) |
| 85 | +2. **➕ Add Operation**: Click **"Add operation"** button |
| 86 | +3. **📋 Configure Operation Details**: |
| 87 | + - **Display name**: `Get Deployment By Name` |
| 88 | + - **Name**: `get-deployment-by-name` |
| 89 | + - **URL**: `GET /deployments/{deploymentName}` |
| 90 | + - **Description**: (Optional) Add description for the operation |
| 91 | + - **Tags**: (Optional) Add relevant tags like `xyz` |
| 92 | + |
| 93 | +4. **💾 Save**: Click **"Save"** to create the operation |
| 94 | + |
| 95 | +##### 🔧 Configure Get Deployment Policy |
| 96 | + |
| 97 | +After creating the operation, you need to configure a policy to route the request to the Azure Management endpoint: |
| 98 | + |
| 99 | +1. **🎯 Select the Operation**: Click on the **"Get Deployment"** operation you just created |
| 100 | +2. **📝 Edit Policy**: Click on **"Policies"** to edit the policy for this specific operation |
| 101 | +3. **⚠️ Ensure Operation-Level Policy**: Make sure the policy is applied to **this operation only**, not at the API level |
| 102 | +4. **📋 Add Policy XML**: Replace the policy content with the following XML: |
| 103 | + |
| 104 | +```xml |
| 105 | +<!-- |
| 106 | + - Policies are applied in the order they appear. |
| 107 | + - Position <base/> inside a section to inherit policies from the outer scope. |
| 108 | + - Comments within policies are not preserved. |
| 109 | +--> |
| 110 | +<!-- Add policies as children to the <inbound>, <outbound>, <backend>, and <on-error> elements --> |
| 111 | +<policies> |
| 112 | + <!-- Throttle, authorize, validate, cache, or transform the requests --> |
| 113 | + <inbound> |
| 114 | + <authentication-managed-identity resource="https://management.azure.com/" /> |
| 115 | + <rewrite-uri template="/deployments/{deploymentName}?api-version=2023-05-01" copy-unmatched-params="false" /> |
| 116 | + <!--Azure Resource Manager--> |
| 117 | + <set-backend-service base-url="https://management.azure.com/subscriptions/YOUR-SUBSCRIPTION-ID/resourceGroups/YOUR-RESOURCE-GROUP/providers/Microsoft.CognitiveServices/accounts/YOUR-COGNITIVE-SERVICE-ACCOUNT" /> |
| 118 | + </inbound> |
| 119 | + <!-- Control if and how the requests are forwarded to services --> |
| 120 | + <backend> |
| 121 | + <base /> |
| 122 | + </backend> |
| 123 | + <!-- Customize the responses --> |
| 124 | + <outbound> |
| 125 | + <base /> |
| 126 | + </outbound> |
| 127 | + <!-- Handle exceptions and customize error responses --> |
| 128 | + <on-error> |
| 129 | + <base /> |
| 130 | + </on-error> |
| 131 | +</policies> |
| 132 | +``` |
| 133 | + |
| 134 | +> **🔧 Important**: Update the `set-backend-service` base-url with your actual Azure resource details: |
| 135 | +> - Replace `YOUR-SUBSCRIPTION-ID` with your Azure subscription ID |
| 136 | +> - Replace `YOUR-RESOURCE-GROUP` with your resource group name |
| 137 | +> - Replace `YOUR-COGNITIVE-SERVICE-ACCOUNT` with your Cognitive Services account name |
| 138 | +
|
| 139 | +5. **💾 Save Policy**: Save the policy configuration |
| 140 | + |
| 141 | +This policy will route the get deployment request to the Azure Management endpoint to retrieve deployment details. |
| 142 | + |
| 143 | +##### 🛠️ Adding List Deployments Operation |
| 144 | + |
| 145 | +Now create the second operation for listing all deployments: |
| 146 | + |
| 147 | +1. **📍 Navigate to Your API**: Go back to your API operations list |
| 148 | +2. **➕ Add Operation**: Click **"Add operation"** button again |
| 149 | +3. **📋 Configure Operation Details**: |
| 150 | + - **Display name**: `List Deployments` |
| 151 | + - **Name**: `list-deployments` |
| 152 | + - **URL**: `GET /deployments` |
| 153 | + - **Description**: (Optional) Add description for the operation |
| 154 | + - **Tags**: (Optional) Add relevant tags |
| 155 | + |
| 156 | +4. **💾 Save**: Click **"Save"** to create the operation |
| 157 | + |
| 158 | +##### 🔧 Configure List Deployments Policy |
| 159 | + |
| 160 | +Configure the policy for the list deployments operation: |
| 161 | + |
| 162 | +1. **🎯 Select the Operation**: Click on the **"List Deployments"** operation you just created |
| 163 | +2. **📝 Edit Policy**: Click on **"Policies"** to edit the policy for this specific operation |
| 164 | +3. **⚠️ Ensure Operation-Level Policy**: Make sure the policy is applied to **this operation only** |
| 165 | +4. **📋 Add Policy XML**: Replace the policy content with the following XML: |
| 166 | + |
| 167 | +```xml |
| 168 | +<!-- |
| 169 | + - Policies are applied in the order they appear. |
| 170 | + - Position <base/> inside a section to inherit policies from the outer scope. |
| 171 | + - Comments within policies are not preserved. |
| 172 | +--> |
| 173 | +<!-- Add policies as children to the <inbound>, <outbound>, <backend>, and <on-error> elements --> |
| 174 | +<policies> |
| 175 | + <!-- Throttle, authorize, validate, cache, or transform the requests --> |
| 176 | + <inbound> |
| 177 | + <authentication-managed-identity resource="https://management.azure.com/" /> |
| 178 | + <rewrite-uri template="/deployments?api-version=2023-05-01" copy-unmatched-params="false" /> |
| 179 | + <!--Azure Resource Manager--> |
| 180 | + <set-backend-service base-url="https://management.azure.com/subscriptions/YOUR-SUBSCRIPTION-ID/resourceGroups/YOUR-RESOURCE-GROUP/providers/Microsoft.CognitiveServices/accounts/YOUR-COGNITIVE-SERVICE-ACCOUNT" /> |
| 181 | + </inbound> |
| 182 | + <!-- Control if and how the requests are forwarded to services --> |
| 183 | + <backend> |
| 184 | + <base /> |
| 185 | + </backend> |
| 186 | + <!-- Customize the responses --> |
| 187 | + <outbound> |
| 188 | + <base /> |
| 189 | + </outbound> |
| 190 | + <!-- Handle exceptions and customize error responses --> |
| 191 | + <on-error> |
| 192 | + <base /> |
| 193 | + </on-error> |
| 194 | +</policies> |
| 195 | +``` |
| 196 | + |
| 197 | +> **🔧 Important**: Update the `set-backend-service` base-url with your actual Azure resource details (same as the get deployment operation). |
| 198 | +
|
| 199 | +5. **💾 Save Policy**: Save the policy configuration |
| 200 | + |
| 201 | +This policy will route the list deployments request to the Azure Management endpoint to retrieve all available deployments. |
| 202 | + |
| 203 | +### Step 4: 📋 Gather Connection Details |
| 204 | + |
| 205 | +Once your APIM operations are configured, you need to collect the following details to create your Foundry connection: |
| 206 | + |
| 207 | +#### 🎯 1. Target URL |
| 208 | + |
| 209 | +1. **📍 Navigate to Chat Completions**: Go to your chat completions operation in APIM |
| 210 | +2. **🧪 Open Test Tab**: Click on the **"Test"** tab for the chat completions operation |
| 211 | +3. **🔍 Check Request URL**: Look at the endpoint URL that **you are hitting** during the test |
| 212 | +4. **✂️ Extract Base URL**: Take everything **before** `/chat/completions` or `/deployments/{deploymentId}/chat/completions` |
| 213 | + |
| 214 | +**Examples:** |
| 215 | +- If endpoint is: `https://my-apim.azure-api.net/foundrymodels/chat/completions` |
| 216 | +- Target URL would be: `https://my-apim.azure-api.net/foundry/models` |
| 217 | + |
| 218 | +#### 🔧 2. Inference API Version |
| 219 | + |
| 220 | +1. **📋 Check API Version Parameter**: In the chat completions test, look for an **api-version** parameter |
| 221 | +2. **📝 Note the Value**: If an API version is required when hitting chat completions, record that value |
| 222 | +3. **📄 Common Values**: Typically values like `2024-02-01`, `2023-12-01-preview`, etc. |
| 223 | + |
| 224 | +#### 🛤️ 3. Deployment in Path |
| 225 | + |
| 226 | +Determine if your chat completions URL includes the deployment name in the path: |
| 227 | + |
| 228 | +- **✅ Set to "true"**: If your URL is like `/deployments/{deploymentName}/chat/completions` |
| 229 | +- **❌ Set to "false"**: If your URL is like `/chat/completions` (deployment passed as parameter) |
| 230 | + |
| 231 | +**Examples:** |
| 232 | +- `"true"`: `/deployments/gpt-4/chat/completions` |
| 233 | +- `"false"`: `/chat/completions?deployment=gpt-4` |
| 234 | + |
| 235 | +> **📝 Note**: These values will be used when creating your APIM connection in Foundry using the Bicep templates. |
0 commit comments