From a4898d354e2c1ea84aae300bb864df6dcd789d94 Mon Sep 17 00:00:00 2001 From: alexsin368 Date: Mon, 27 Apr 2026 18:12:42 -0700 Subject: [PATCH 1/5] simplify env setup Signed-off-by: alexsin368 --- sample_solutions/MultiAgentQnA/.env.example | 32 ++++ sample_solutions/MultiAgentQnA/README.md | 179 ++++-------------- .../MultiAgentQnA/api/.env.example | 40 ---- .../MultiAgentQnA/docker-compose.yml | 2 +- 4 files changed, 65 insertions(+), 188 deletions(-) create mode 100644 sample_solutions/MultiAgentQnA/.env.example delete mode 100644 sample_solutions/MultiAgentQnA/api/.env.example diff --git a/sample_solutions/MultiAgentQnA/.env.example b/sample_solutions/MultiAgentQnA/.env.example new file mode 100644 index 00000000..d14569bb --- /dev/null +++ b/sample_solutions/MultiAgentQnA/.env.example @@ -0,0 +1,32 @@ +# Inference API Configuration +# Note: model names must match one of the deployed models with Enterprise Inference. +# INFERENCE_API_ENDPOINT: URL to your inference service (without /v1 suffix). Uncomment ONE of the below. +# - For GenAI Gateway: +#INFERENCE_API_ENDPOINT=https://api.example.com +# - For APISIX Gateway (default), requires the model name in the URL path: +INFERENCE_API_ENDPOINT=https://api.example.com/Llama-3.1-8B-Instruct + +# INFERENCE_API_TOKEN: Authentication token/API key for the inference service +# - For GenAI Gateway: Your GenAI Gateway API key +# - For APISIX Gateway: Your APISIX authentication token +INFERENCE_API_TOKEN=your-pre-generated-token-here + +# Model Configuration +# IMPORTANT: Use the full model names as they appear in your inference service +# Check available models: curl https://api.example.com/v1/models -H "Authorization: Bearer your-token" +INFERENCE_MODEL_NAME=meta-llama/Llama-3.1-8B-Instruct +EMBEDDING_MODEL_NAME=BAAI/bge-base-en-v1.5 + +# Embedding Model API (APISIX Gateway only) +# Set only when using APISIX Gateway (default). Comment out for GenAI Gateway +EMBEDDING_API_ENDPOINT=https://api.example.com/bge-base-en-v1.5 + +# Local URL Endpoint (only needed for non-public domains) +# If using a local domain like api.example.com mapped to localhost: +# Set this to: api.example.com (domain without https://) +# If using a public domain, set any placeholder value like: not-needed +LOCAL_URL_ENDPOINT=not-needed + +# SSL Verification Settings +# Set to false only for dev with self-signed certs +VERIFY_SSL=true diff --git a/sample_solutions/MultiAgentQnA/README.md b/sample_solutions/MultiAgentQnA/README.md index 5908827b..2012e5fd 100644 --- a/sample_solutions/MultiAgentQnA/README.md +++ b/sample_solutions/MultiAgentQnA/README.md @@ -113,40 +113,34 @@ Before you begin, ensure you have the following installed: This application supports multiple inference deployment patterns: +| API Configuration | Validated | +|---|:---:| +| GenAI Gateway | ✅ | +| Keycloak/APISIX | ✅ | + **GenAI Gateway**: Provide your GenAI Gateway URL and API key - - URL format: https://api.example.com - To generate the GenAI Gateway API key, use the [generate-vault-secrets.sh](https://github.com/opea-project/Enterprise-Inference/blob/main/core/scripts/generate-vault-secrets.sh) script - The API key is the litellm_master_key value from the generated vault.yml file **APISIX Gateway**: Provide your APISIX Gateway URL and authentication token - - URL format: https://api.example.com/bge-base-en-v1.5 (for embedding) and https://api.example.com/Llama-3.1-8B-Instruct (for inference) - - Note: APISIX requires the model name in the URL path - To generate the APISIX authentication token, use the [generate-token.sh](https://github.com/opea-project/Enterprise-Inference/blob/main/core/scripts/generate-token.sh) script - The token is generated using Keycloak client credentials -Configuration requirements: -- INFERENCE_API_ENDPOINT: URL to your inference service (example: https://api.example.com) -- INFERENCE_API_TOKEN: Authentication token/API key for your chosen service -- EMBEDDING_MODEL_NAME: Embedding model name (default: BAAI/bge-base-en-v1.5) -- INFERENCE_MODEL_NAME: Inference model name (default: meta-llama/Llama-3.1-8B-Instruct) -- EMBEDDING_API_ENDPOINT: (Optional, for APISIX only) URL with embedding model in path -- INFERENCE_MODEL_ENDPOINT: (Optional, for APISIX only) URL with inference model in path - -### Local Development Configuration - -**For Local Testing Only** +### Deploy Required Model(s) -If you're testing with a local inference endpoint using a custom domain (e.g., `api.example.com` mapped to localhost in your hosts file): +The following models have been validated on different hardware platforms. -1. Edit `api/.env` and set: - ```bash - LOCAL_URL_ENDPOINT=inference.example.com - ``` - (Use the domain name from your INFERENCE_API_ENDPOINT without `https://`) +At least one of the following models must be deployed. -2. This allows Docker containers to resolve your local domain correctly. +| Model | Xeon | Gaudi | +|---|:---:|:---:| +| **meta-llama/Llama-3.1-8B-Instruct** | ❌ | ✅ Validated on Dell XE7740 | +| **Qwen/Qwen3-4B-Instruct-2507** | ✅ Validated on Dell XE7740 | ❌ | -**Note:** For public domains or cloud-hosted endpoints, leave the default value `not-needed`. +In addition, an embedding model must be deployed. +| Model | Xeon | Gaudi | +|---|:---:|:---:| +| **BAAI/bge-base-en-v1.5** | ❌ | ✅ Validated on Dell XE7740 | ### Verify Docker Installation @@ -173,126 +167,29 @@ cd Enterprise-Inference/sample_solutions/MultiAgentQnA ### Set up the Environment -This application requires **two `.env` files** for proper configuration: - -1. **Root `.env` file** (for Docker Compose variables) -2. **`api/.env` file** (for backend application configuration) - -#### Step 1: Create Root `.env` File - -```bash -# From the MultiAgentQnA directory -cat > .env << EOF -# Docker Compose Configuration -LOCAL_URL_ENDPOINT=not-needed -EOF -``` - -**Note:** If using a local domain (e.g., `api.example.com` mapped to localhost), replace `not-needed` with your domain name (without `https://`). - -#### Step 2: Create `api/.env` File - -This application requires an `api/.env` file for proper configuration. Create it with the commands below: - -```bash -# Create the api/.env file -cat > api/.env << EOF -# Inference API Configuration -# INFERENCE_API_ENDPOINT: URL to your inference service (without /v1 suffix) -# -# **GenAI Gateway**: Provide your GenAI Gateway URL and API key -# - URL format: https://genai-gateway.example.com -# - To generate the GenAI Gateway API key, use the [generate-vault-secrets.sh] script -# - The API key is the litellm_master_key value from the generated vault.yml file -# -# **APISIX Gateway**: Provide your APISIX Gateway URL and authentication token -# - For APISIX, include the model name in the INFERENCE_API_ENDPOINT path -# - Example: https://apisix-gateway.example.com/Llama-3.1-8B-Instruct -# - Set EMBEDDING_API_ENDPOINT separately for the embedding model -# - Example: https://apisix-gateway.example.com/bge-base-en-v1.5 -# - To generate the APISIX authentication token, use the [generate-token.sh] script -# - The token is generated using Keycloak client credentials -# -# INFERENCE_API_TOKEN: Authentication token/API key for the inference service -INFERENCE_API_ENDPOINT=https://your-api.example.com/ -INFERENCE_API_TOKEN=your-pre-generated-token-here - -# Model Configuration -EMBEDDING_MODEL_NAME=BAAI/bge-base-en-v1.5 -INFERENCE_MODEL_NAME=meta-llama/Llama-3.1-8B-Instruct - -# APISIX Gateway Endpoints -# Uncomment and set these when using APISIX Gateway: -# IMPORTANT: Use exact APISIX route paths: -# Example routes: /bge-base-en-v1.5/* and /Llama-3.1-8B-Instruct/* -# INFERENCE_API_ENDPOINT=https://api.example.com/Llama-3.1-8B-Instruct -# EMBEDDING_API_ENDPOINT=https://api.example.com/bge-base-en-v1.5 - -# Local URL Endpoint (only needed for non-public domains) -# If using a local domain like api.example.com mapped to localhost: -# Set this to: api.example.com (domain without https://) -# If using a public domain, set any placeholder value like: not-needed -LOCAL_URL_ENDPOINT=not-needed - -# SSL Verification Settings -# Set to false only for dev with self-signed certs -VERIFY_SSL=true -EOF -``` - -Or manually create `api/.env` with: +This application requires an `.env` file in the root directory for proper configuration. Create it using [.env.example](./.env.example) with the commands below: ```bash -# Inference API Configuration -# INFERENCE_API_ENDPOINT: URL to your inference service (without /v1 suffix) -# -# **GenAI Gateway**: Provide your GenAI Gateway URL and API key -# - URL format: https://genai-gateway.example.com -# - To generate the GenAI Gateway API key, use the [generate-vault-secrets.sh] script -# - The API key is the litellm_master_key value from the generated vault.yml file -# -# **APISIX Gateway**: Provide your APISIX Gateway URL and authentication token -# - For APISIX, include the model name in the INFERENCE_API_ENDPOINT path -# - Example: https://apisix-gateway.example.com/Llama-3.1-8B-Instruct -# - Set EMBEDDING_API_ENDPOINT separately for the embedding model -# - Example: https://apisix-gateway.example.com/bge-base-en-v1.5 -# - To generate the APISIX authentication token, use the [generate-token.sh] script -# - The token is generated using Keycloak client credentials -# -# INFERENCE_API_TOKEN: Authentication token/API key for the inference service -INFERENCE_API_ENDPOINT=https://your-api.example.com/ -INFERENCE_API_TOKEN=your-pre-generated-token-here - -# Model Configuration -EMBEDDING_MODEL_NAME=BAAI/bge-base-en-v1.5 -INFERENCE_MODEL_NAME=meta-llama/Llama-3.1-8B-Instruct - -# APISIX Gateway Endpoints -# Uncomment and set these when using APISIX Gateway: -# IMPORTANT: Use exact APISIX route paths: -# Example routes: /bge-base-en-v1.5/* and /Llama-3.1-8B-Instruct/* -# INFERENCE_API_ENDPOINT=https://your-api-endpoint.com/Llama-3.1-8B-Instruct -# EMBEDDING_API_ENDPOINT=https://your-api-endpoint.com/bge-base-en-v1.5 - -# Local URL Endpoint (only needed for non-public domains) -# If using a local domain like api.example.com mapped to localhost: -# Set this to: api.example.com (domain without https://) -# If using a public domain, set any placeholder value like: not-needed -LOCAL_URL_ENDPOINT=not-needed - -# SSL Verification Settings -# Set to false only for dev with self-signed certs -VERIFY_SSL=true +cp .env.example .env ``` +Then modify it as needed, with special consideration to certain environment variables mentioned below. Read through the .env file for full instructions. **Important Configuration Notes:** -- **INFERENCE_API_ENDPOINT**: Your actual inference service URL (replace `https://your-api-endpoint.com/deployment`) - - For APISIX/Keycloak deployments, the model name must be included in the endpoint URL (e.g., `https://apisix-gateway.example.com/Llama-3.1-8B-Instruct`) +- **INFERENCE_API_ENDPOINT**: Your actual inference service URL (replace `https://api.example.com`) + - For APISIX/Keycloak deployments, the model name must be included in the endpoint URL (e.g., `https://api.example.com/Llama-3.1-8B-Instruct`) - **INFERENCE_API_TOKEN**: Your actual pre-generated authentication token -- **LOCAL_URL_ENDPOINT**: Only needed if using local domain mapping (see [Local Development Configuration](#local-development-configuration)) - -**Note**: The docker-compose.yml file automatically loads environment variables from both `.env` (root) and `./api/.env` (backend) files. +- **EMBEDDING_MODEL_NAME**, **INFERENCE_MODEL_NAME**: Use the exact model name from your inference service + - To check available models: `curl https://api.example.com/v1/models -H "Authorization: Bearer your-token"` +- **EMBEDDING_API_ENDPOINT** (APISIX only): Your actual embedding service URL +- **LOCAL_URL_ENDPOINT**: Only needed if using local domain mapping (i.e. `api.example.com` mapped to localhost) for Docker containers to resolve correctly. + - Use the domain name from INFERENCE_API_ENDPOINT without `https://` + - For public domains or cloud-hosted endpoints, leave the default value `not-needed` +- **VERIFY_SSL**: Controls SSL certificate verification (default: `true`) + - Set to `false` only for development environments with self-signed certificates + - Keep as `true` for production environments + +**Note**: The docker-compose.yaml file automatically loads environment variables from `.env` for the backend service. ### Running the Application @@ -378,15 +275,3 @@ docker compose down For comprehensive troubleshooting guidance, common issues, and solutions, refer to: [TROUBLESHOOTING.md](TROUBLESHOOTING.md) - ---- - -## Additional Info - -The following models have been validated with MultiAgentQnA: - -| Model | Hardware | -|-------|----------| -| **meta-llama/Llama-3.1-8B-Instruct** | Gaudi | -| **Qwen/Qwen3-4B-Instruct-2507** | Xeon | -| **BAAI/bge-base-en-v1.5** (embeddings) | Gaudi | diff --git a/sample_solutions/MultiAgentQnA/api/.env.example b/sample_solutions/MultiAgentQnA/api/.env.example deleted file mode 100644 index 5c72e049..00000000 --- a/sample_solutions/MultiAgentQnA/api/.env.example +++ /dev/null @@ -1,40 +0,0 @@ -# Inference API Configuration -# INFERENCE_API_ENDPOINT: URL to your inference service (without /v1 suffix) -# -# **GenAI Gateway**: Provide your GenAI Gateway URL and API key -# - URL format: https://genai-gateway.example.com -# - To generate the GenAI Gateway API key, use the [generate-vault-secrets.sh](https://github.com/opea-project/Enterprise-Inference/blob/main/core/scripts/generate-vault-secrets.sh) script -# - The API key is the litellm_master_key value from the generated vault.yml file -# -# **APISIX Gateway**: Provide your APISIX Gateway URL and authentication token -# - For APISIX, include the model name in the INFERENCE_API_ENDPOINT path -# - Example: https://apisix-gateway.example.com/Llama-3.1-8B-Instruct -# - Set EMBEDDING_API_ENDPOINT separately for the embedding model -# - Example: https://apisix-gateway.example.com/bge-base-en-v1.5 -# - To generate the APISIX authentication token, use the [generate-token.sh](https://github.com/opea-project/Enterprise-Inference/blob/main/core/scripts/generate-token.sh) script -# - The token is generated using Keycloak client credentials -# -# INFERENCE_API_TOKEN: Authentication token/API key for the inference service -INFERENCE_API_ENDPOINT=https://api.example.com -INFERENCE_API_TOKEN=your-pre-generated-token-here - -# Model Configuration -EMBEDDING_MODEL_NAME=BAAI/bge-base-en-v1.5 -INFERENCE_MODEL_NAME=meta-llama/Llama-3.1-8B-Instruct - -# APISIX Gateway Endpoints -# Uncomment and set these when using APISIX Gateway: -# IMPORTANT: Use exact APISIX route paths: -# Example routes: /bge-base-en-v1.5/* and /Llama-3.1-8B-Instruct/* -# INFERENCE_API_ENDPOINT=https://api.example.com/Llama-3.1-8B-Instruct -# EMBEDDING_API_ENDPOINT=https://api.example.com/bge-base-en-v1.5 - -# Local URL Endpoint (only needed for non-public domains) -# If using a local domain like api.example.com mapped to localhost: -# Set this to: api.example.com (domain without https://) -# If using a public domain, set any placeholder value like: not-needed -LOCAL_URL_ENDPOINT=not-needed - -# SSL Verification Settings -# Set to false only for dev with self-signed certs -VERIFY_SSL=true diff --git a/sample_solutions/MultiAgentQnA/docker-compose.yml b/sample_solutions/MultiAgentQnA/docker-compose.yml index 7a6d0455..cf8e762d 100644 --- a/sample_solutions/MultiAgentQnA/docker-compose.yml +++ b/sample_solutions/MultiAgentQnA/docker-compose.yml @@ -8,7 +8,7 @@ services: ports: - "5001:5001" env_file: - - ./api/.env + - .env volumes: - ./api:/app networks: From da176c4e2d425ee4336d3fe66fea66a648f2f4dd Mon Sep 17 00:00:00 2001 From: alexsin368 Date: Tue, 28 Apr 2026 14:44:15 -0700 Subject: [PATCH 2/5] change VECTOR_STORE_DIR to writable /tmp folder Signed-off-by: alexsin368 --- sample_solutions/MultiAgentQnA/api/services/rag_service.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sample_solutions/MultiAgentQnA/api/services/rag_service.py b/sample_solutions/MultiAgentQnA/api/services/rag_service.py index 3dec786a..aca015db 100644 --- a/sample_solutions/MultiAgentQnA/api/services/rag_service.py +++ b/sample_solutions/MultiAgentQnA/api/services/rag_service.py @@ -16,7 +16,7 @@ logger = logging.getLogger(__name__) # Constants -VECTOR_STORE_DIR = "./rag_index" +VECTOR_STORE_DIR = "/tmp/rag_index" CHUNK_SIZE = 1000 # Characters per chunk CHUNK_OVERLAP = 200 # Overlap between chunks From 1f4c024af56c445aececece2c736d3d927e69bd3 Mon Sep 17 00:00:00 2001 From: alexsin368 Date: Tue, 28 Apr 2026 17:06:13 -0700 Subject: [PATCH 3/5] update tables and notes Signed-off-by: alexsin368 --- sample_solutions/MultiAgentQnA/.env.example | 3 +- sample_solutions/MultiAgentQnA/README.md | 44 ++++++++++----------- 2 files changed, 21 insertions(+), 26 deletions(-) diff --git a/sample_solutions/MultiAgentQnA/.env.example b/sample_solutions/MultiAgentQnA/.env.example index d14569bb..54a78338 100644 --- a/sample_solutions/MultiAgentQnA/.env.example +++ b/sample_solutions/MultiAgentQnA/.env.example @@ -12,8 +12,7 @@ INFERENCE_API_ENDPOINT=https://api.example.com/Llama-3.1-8B-Instruct INFERENCE_API_TOKEN=your-pre-generated-token-here # Model Configuration -# IMPORTANT: Use the full model names as they appear in your inference service -# Check available models: curl https://api.example.com/v1/models -H "Authorization: Bearer your-token" +# IMPORTANT: Use the full model name i.e. model card INFERENCE_MODEL_NAME=meta-llama/Llama-3.1-8B-Instruct EMBEDDING_MODEL_NAME=BAAI/bge-base-en-v1.5 diff --git a/sample_solutions/MultiAgentQnA/README.md b/sample_solutions/MultiAgentQnA/README.md index 2012e5fd..a7f8317f 100644 --- a/sample_solutions/MultiAgentQnA/README.md +++ b/sample_solutions/MultiAgentQnA/README.md @@ -105,43 +105,39 @@ The application consists of: Before you begin, ensure you have the following installed: - **Docker and Docker Compose** -- **Enterprise inference endpoint access** (token-based authentication) +- **Enterprise Inference endpoint access** (token-based authentication, see below for models and configs) + +#### Deploy Required Models + +See the table below for supported models, hardware, and gateway configuration. + +| Model | Xeon w/APISIX/Keycloak | Xeon w/GenAI Gateway | Gaudi w/APISIX/Keycloak | Gaudi w/GenAI Gateway | +|---|:---:|:---:|:---:|:---:| +| **meta-llama/Llama-3.1-8B-Instruct** | ❌ | ❌ | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 | +| **Qwen/Qwen3-4B-Instruct-2507** | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 | ❌ | ❌ | + +In addition, an embedding model must be deployed. +| Model | Xeon w/APISIX/Keycloak | Xeon w/GenAI Gateway | Gaudi w/APISIX/Keycloak | Gaudi w/GenAI Gateway | +|---|:---:|:---:|:---:|:---:| +| **BAAI/bge-base-en-v1.5** | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 | ### Required API Configuration -**For Inference Service:** +**For Inference Service (RAG Chatbot):** This application supports multiple inference deployment patterns: -| API Configuration | Validated | -|---|:---:| -| GenAI Gateway | ✅ | -| Keycloak/APISIX | ✅ | - **GenAI Gateway**: Provide your GenAI Gateway URL and API key + - URL format: https://api.example.com - To generate the GenAI Gateway API key, use the [generate-vault-secrets.sh](https://github.com/opea-project/Enterprise-Inference/blob/main/core/scripts/generate-vault-secrets.sh) script - The API key is the litellm_master_key value from the generated vault.yml file **APISIX Gateway**: Provide your APISIX Gateway URL and authentication token + - URL format: https://api.example.com/Llama-3.1-8B-Instruct + - Note: APISIX requires the model name in the URL path - To generate the APISIX authentication token, use the [generate-token.sh](https://github.com/opea-project/Enterprise-Inference/blob/main/core/scripts/generate-token.sh) script - The token is generated using Keycloak client credentials -### Deploy Required Model(s) - -The following models have been validated on different hardware platforms. - -At least one of the following models must be deployed. - -| Model | Xeon | Gaudi | -|---|:---:|:---:| -| **meta-llama/Llama-3.1-8B-Instruct** | ❌ | ✅ Validated on Dell XE7740 | -| **Qwen/Qwen3-4B-Instruct-2507** | ✅ Validated on Dell XE7740 | ❌ | - -In addition, an embedding model must be deployed. -| Model | Xeon | Gaudi | -|---|:---:|:---:| -| **BAAI/bge-base-en-v1.5** | ❌ | ✅ Validated on Dell XE7740 | - ### Verify Docker Installation ```bash @@ -181,7 +177,7 @@ Then modify it as needed, with special consideration to certain environment vari - **INFERENCE_API_TOKEN**: Your actual pre-generated authentication token - **EMBEDDING_MODEL_NAME**, **INFERENCE_MODEL_NAME**: Use the exact model name from your inference service - To check available models: `curl https://api.example.com/v1/models -H "Authorization: Bearer your-token"` -- **EMBEDDING_API_ENDPOINT** (APISIX only): Your actual embedding service URL +- **EMBEDDING_API_ENDPOINT** (APISIX only): Your actual embedding service URL. **Comment out** for GenAI Gateway - **LOCAL_URL_ENDPOINT**: Only needed if using local domain mapping (i.e. `api.example.com` mapped to localhost) for Docker containers to resolve correctly. - Use the domain name from INFERENCE_API_ENDPOINT without `https://` - For public domains or cloud-hosted endpoints, leave the default value `not-needed` From 477bb970acfdfdf8e2f3abf08ae2607c2c3bbcd2 Mon Sep 17 00:00:00 2001 From: alexsin368 Date: Tue, 28 Apr 2026 17:20:19 -0700 Subject: [PATCH 4/5] remove typo Signed-off-by: alexsin368 --- sample_solutions/MultiAgentQnA/README.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/sample_solutions/MultiAgentQnA/README.md b/sample_solutions/MultiAgentQnA/README.md index a7f8317f..0de14c09 100644 --- a/sample_solutions/MultiAgentQnA/README.md +++ b/sample_solutions/MultiAgentQnA/README.md @@ -123,8 +123,6 @@ In addition, an embedding model must be deployed. ### Required API Configuration -**For Inference Service (RAG Chatbot):** - This application supports multiple inference deployment patterns: **GenAI Gateway**: Provide your GenAI Gateway URL and API key From 379e549b33a66dcee146ce418aa3e71f14878300 Mon Sep 17 00:00:00 2001 From: alexsin368 Date: Tue, 28 Apr 2026 17:22:46 -0700 Subject: [PATCH 5/5] fix typo Signed-off-by: alexsin368 --- sample_solutions/MultiAgentQnA/README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sample_solutions/MultiAgentQnA/README.md b/sample_solutions/MultiAgentQnA/README.md index 0de14c09..a98ec3aa 100644 --- a/sample_solutions/MultiAgentQnA/README.md +++ b/sample_solutions/MultiAgentQnA/README.md @@ -123,6 +123,8 @@ In addition, an embedding model must be deployed. ### Required API Configuration +**For LLM Inference and Embedding Services:** + This application supports multiple inference deployment patterns: **GenAI Gateway**: Provide your GenAI Gateway URL and API key