You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
|`BAAI/bge-base-en-v1.5`| Embedding generation for dense retrieval | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 |
124
+
|`BAAI/bge-reranker-base`| Reranking retrieved results for improved relevance | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 |
125
+
|`Qwen/Qwen3-4B-Instruct-2507`| LLM for question answering and generation | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 |
130
126
131
127
### Verify Docker Installation
132
128
```bash
@@ -140,9 +136,6 @@ docker compose version
140
136
docker ps
141
137
```
142
138
143
-
### Credentials & Authentication
144
-
The system uses GenAI Gateway for authentication with API key-based access.
145
-
146
139
## Deployment & Configuration
147
140
148
141
### Clone the Repository
@@ -152,36 +145,39 @@ cd Enterprise-Inference/sample_solutions/HybridSearch
152
145
```
153
146
154
147
### Set up the Environment
155
-
Create a `.env` file from the example and configure your credentials.
148
+
This application requires an `.env` file in the root directory for proper configuration. Create it using [.env.example](./.env.example) with the commands below:
156
149
157
150
```bash
158
151
cp .env.example .env
159
152
```
153
+
Then modify it as needed, with special consideration to certain environment variables mentioned below. Read through the .env file for full instructions. Additional notes are given below.
160
154
161
-
**Note:** The `LOCAL_URL_ENDPOINT` variable enables Docker's `extra_hosts` configuration for local domain resolution. Set it to `not-needed` if you don't require custom domain mapping.
155
+
### Local Development Configuration (Local Testing Only)
156
+
If using a local domain like api.example.com mapped to localhost, set `LOCAL_URL_ENDPOINT` to the domain without `https://`. Otherwise, set to: `not-needed`.
162
157
163
-
### Configure Authentication
158
+
### Configure Inference Gateway
164
159
Configure GenAI Gateway authentication in your `.env` file:
165
160
166
161
```bash
167
162
# GenAI Gateway Configuration
168
163
GENAI_GATEWAY_URL=https://api.example.com
169
164
GENAI_API_KEY=your-api-key-here
170
-
171
-
# SSL Verification (set to false only for dev with self-signed certs)
172
-
VERIFY_SSL=true
173
165
```
174
166
175
167
**Generating API tokens by gateway type:**
176
168
177
-
-**GenAI Gateway**: Provide your GenAI Gateway URL and API key
169
+
This application supports multiple inference deployment patterns:
170
+
171
+
**GenAI Gateway**: Provide your GenAI Gateway URL and API key
172
+
- URL format: https://api.example.com
178
173
- To generate the GenAI Gateway API key, use the [generate-vault-secrets.sh](https://github.com/opea-project/Enterprise-Inference/blob/main/core/scripts/generate-vault-secrets.sh) script
179
-
- The API key is the `litellm_master_key` value from the generated `vault.yml` file
174
+
- The API key is the litellm_master_key value from the generated vault.yml file
180
175
181
-
-**Keycloak / APISIX Gateway**: Provide your APISIX Gateway URL and authentication token
176
+
**APISIX Gateway**: Provide your APISIX Gateway URL and authentication token
- Note: APISIX requires the model name in the URL path
182
179
- To generate the APISIX authentication token, use the [generate-token.sh](https://github.com/opea-project/Enterprise-Inference/blob/main/core/scripts/generate-token.sh) script
183
-
- The token is generated using Keycloak client credentials (expires in 15 minutes by default; a Keycloak admin can configure longer-lived tokens in the Keycloak console)
184
-
- For Keycloak, each model has its own APISIX route path. Run `kubectl get apisixroutes` to find the route names for your deployed models (e.g., `bge-base-en-v1.5`, `bge-reranker-base`, `Qwen3-4B-Instruct-2507`)
180
+
- The token is generated using Keycloak client credentials
185
181
186
182
### Configure Models
187
183
Model endpoint names differ by deployment type. Use the table below to determine the correct values:
0 commit comments