Skip to content

Commit ba5141d

Browse files
authored
HybridSearch: simplify instructions and env setup (#107)
* update instructions Signed-off-by: alexsin368 <alex.sin@intel.com> * update table Signed-off-by: alexsin368 <alex.sin@intel.com> --------- Signed-off-by: alexsin368 <alex.sin@intel.com>
1 parent d793701 commit ba5141d

2 files changed

Lines changed: 33 additions & 33 deletions

File tree

sample_solutions/HybridSearch/.env.example

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,8 @@ DEPLOYMENT_PHASE=production
22
SYSTEM_MODE=document
33

44
# Local URL Endpoint (only needed for non-public domains)
5-
# If using a local domain like api.example.com mapped to localhost:
6-
# Set this to: api.example.com (domain without https://)
7-
# If using a public domain, set any placeholder value like: not-needed
5+
# If using a local domain like api.example.com mapped to localhost, set to the domain without https://
6+
# Otherwise, set to: not-needed
87
LOCAL_URL_ENDPOINT=not-needed
98

109
# Service Ports
@@ -52,9 +51,9 @@ INFERENCE_BACKEND=vllm
5251
# Set these to the full APISIX route URL for each model.
5352
# The route name matches the APISIX route (kubectl get apisixroutes).
5453
# Xeon default route names use the -vllmcpu suffix.
55-
# EMBEDDING_API_ENDPOINT=https://api.example.com/bge-base-en-v1.5-vllmcpu
56-
# RERANKER_API_ENDPOINT=https://api.example.com/bge-reranker-base-vllmcpu
57-
# LLM_API_ENDPOINT=https://api.example.com/Qwen3-4B-Instruct-2507-vllmcpu
54+
#EMBEDDING_API_ENDPOINT=https://api.example.com/bge-base-en-v1.5-vllmcpu
55+
#RERANKER_API_ENDPOINT=https://api.example.com/bge-reranker-base-vllmcpu
56+
#LLM_API_ENDPOINT=https://api.example.com/Qwen3-4B-Instruct-2507-vllmcpu
5857

5958
# Retrieval Configuration
6059
USE_RERANKING=true

sample_solutions/HybridSearch/README.md

Lines changed: 28 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -111,22 +111,18 @@ UI (8501) → Gateway (8000) → [Embedding (8001), Retrieval (8002), LLM (8003)
111111
### System Requirements
112112
Before you begin, ensure you have the following installed:
113113

114-
- Docker and Docker Compose
115-
- Python 3.10+ (optional, for local development)
114+
- **Docker and Docker Compose**
115+
- **Enterprise Inference endpoint access** (token-based authentication, see below for models and configs)
116116

117-
### Required Models
118-
The following models must be deployed on Enterprise Inference before running this application:
117+
#### Deploy Required Models
119118

120-
| Model | Purpose |
121-
|-------|---------|
122-
| `BAAI/bge-base-en-v1.5` | Embedding generation for dense retrieval |
123-
| `BAAI/bge-reranker-base` | Reranking retrieved results for improved relevance |
124-
| `Qwen/Qwen3-4B-Instruct-2507` | LLM for question answering and generation |
119+
See the table below for supported models, hardware, and gateway configuration.
125120

126-
Verify your models are available:
127-
```bash
128-
curl https://your-gateway-url/v1/models -H "Authorization: Bearer your-token"
129-
```
121+
| Model | Purpose | Xeon w/APISIX/Keycloak | Xeon w/GenAI Gateway | Gaudi w/APISIX/Keycloak | Gaudi w/GenAI Gateway |
122+
|---|---|:---:|:---:|:---:|:---:|
123+
| `BAAI/bge-base-en-v1.5` | Embedding generation for dense retrieval | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 |
124+
| `BAAI/bge-reranker-base` | Reranking retrieved results for improved relevance | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 |
125+
| `Qwen/Qwen3-4B-Instruct-2507` | LLM for question answering and generation | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 |
130126

131127
### Verify Docker Installation
132128
```bash
@@ -140,9 +136,6 @@ docker compose version
140136
docker ps
141137
```
142138

143-
### Credentials & Authentication
144-
The system uses GenAI Gateway for authentication with API key-based access.
145-
146139
## Deployment & Configuration
147140

148141
### Clone the Repository
@@ -152,36 +145,39 @@ cd Enterprise-Inference/sample_solutions/HybridSearch
152145
```
153146

154147
### Set up the Environment
155-
Create a `.env` file from the example and configure your credentials.
148+
This application requires an `.env` file in the root directory for proper configuration. Create it using [.env.example](./.env.example) with the commands below:
156149

157150
```bash
158151
cp .env.example .env
159152
```
153+
Then modify it as needed, with special consideration to certain environment variables mentioned below. Read through the .env file for full instructions. Additional notes are given below.
160154

161-
**Note:** The `LOCAL_URL_ENDPOINT` variable enables Docker's `extra_hosts` configuration for local domain resolution. Set it to `not-needed` if you don't require custom domain mapping.
155+
### Local Development Configuration (Local Testing Only)
156+
If using a local domain like api.example.com mapped to localhost, set `LOCAL_URL_ENDPOINT` to the domain without `https://`. Otherwise, set to: `not-needed`.
162157

163-
### Configure Authentication
158+
### Configure Inference Gateway
164159
Configure GenAI Gateway authentication in your `.env` file:
165160

166161
```bash
167162
# GenAI Gateway Configuration
168163
GENAI_GATEWAY_URL=https://api.example.com
169164
GENAI_API_KEY=your-api-key-here
170-
171-
# SSL Verification (set to false only for dev with self-signed certs)
172-
VERIFY_SSL=true
173165
```
174166

175167
**Generating API tokens by gateway type:**
176168

177-
- **GenAI Gateway**: Provide your GenAI Gateway URL and API key
169+
This application supports multiple inference deployment patterns:
170+
171+
**GenAI Gateway**: Provide your GenAI Gateway URL and API key
172+
- URL format: https://api.example.com
178173
- To generate the GenAI Gateway API key, use the [generate-vault-secrets.sh](https://github.com/opea-project/Enterprise-Inference/blob/main/core/scripts/generate-vault-secrets.sh) script
179-
- The API key is the `litellm_master_key` value from the generated `vault.yml` file
174+
- The API key is the litellm_master_key value from the generated vault.yml file
180175

181-
- **Keycloak / APISIX Gateway**: Provide your APISIX Gateway URL and authentication token
176+
**APISIX Gateway**: Provide your APISIX Gateway URL and authentication token
177+
- URL format: https://api.example.com/Llama-3.1-8B-Instruct
178+
- Note: APISIX requires the model name in the URL path
182179
- To generate the APISIX authentication token, use the [generate-token.sh](https://github.com/opea-project/Enterprise-Inference/blob/main/core/scripts/generate-token.sh) script
183-
- The token is generated using Keycloak client credentials (expires in 15 minutes by default; a Keycloak admin can configure longer-lived tokens in the Keycloak console)
184-
- For Keycloak, each model has its own APISIX route path. Run `kubectl get apisixroutes` to find the route names for your deployed models (e.g., `bge-base-en-v1.5`, `bge-reranker-base`, `Qwen3-4B-Instruct-2507`)
180+
- The token is generated using Keycloak client credentials
185181

186182
### Configure Models
187183
Model endpoint names differ by deployment type. Use the table below to determine the correct values:
@@ -213,6 +209,11 @@ RERANKER_API_ENDPOINT=https://api.example.com/bge-reranker-base-vllmcpu
213209
LLM_API_ENDPOINT=https://api.example.com/Qwen3-4B-Instruct-2507-vllmcpu
214210
```
215211

212+
### SSL Verification (set to false only for dev with self-signed certs)
213+
```bash
214+
VERIFY_SSL=false
215+
```
216+
216217
### Running the Application
217218
Start all services together with Docker Compose:
218219

0 commit comments

Comments
 (0)