Skip to content

Commit 6a9f7da

Browse files
authored
DocSummarization: simplify instructions and env setup (#102)
* simplify env setup Signed-off-by: alexsin368 <alex.sin@intel.com> * update table, fix model endpoint Signed-off-by: alexsin368 <alex.sin@intel.com> --------- Signed-off-by: alexsin368 <alex.sin@intel.com>
1 parent d0fa8a7 commit 6a9f7da

4 files changed

Lines changed: 63 additions & 181 deletions

File tree

Lines changed: 36 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,38 @@
1-
# Docker Compose Configuration
1+
# Inference API Configuration
2+
# Note: model names must match one of the deployed models with Enterprise Inference.
3+
# INFERENCE_API_ENDPOINT: URL to your inference service (without /v1 suffix). Uncomment ONE of the below.
4+
# - For GenAI Gateway:
5+
#INFERENCE_API_ENDPOINT=https://api.example.com
6+
# - For APISIX Gateway (default), requires the model name in the URL path:
7+
INFERENCE_API_ENDPOINT=https://api.example.com/Llama-3.1-8B-Instruct
8+
9+
# INFERENCE_API_TOKEN: Authentication token/API key for the inference service
10+
# - For GenAI Gateway: Your GenAI Gateway API key
11+
# - For APISIX Gateway: Your APISIX authentication token
12+
INFERENCE_API_TOKEN=your-pre-generated-token-here
13+
14+
# Model Configuration
15+
# IMPORTANT: Use the full model name i.e. model card
16+
INFERENCE_MODEL_NAME=meta-llama/Llama-3.1-8B-Instruct
17+
18+
# LLM Configuration
19+
LLM_TEMPERATURE=0.7
20+
LLM_MAX_TOKENS=2000
21+
222
# Local URL Endpoint (only needed for non-public domains)
3-
# If using a local domain like api.example.com mapped to localhost:
4-
# Set this to: api.example.com (domain without https://)
5-
# If using a public domain, set any placeholder value like: not-needed
23+
# If using a local domain like api.example.com mapped to localhost, set to the domain without https://
24+
# Otherwise, set to: not-needed
625
LOCAL_URL_ENDPOINT=not-needed
26+
27+
# Service Configuration
28+
SERVICE_PORT=8000
29+
LOG_LEVEL=INFO
30+
31+
# File Upload Limits
32+
MAX_FILE_SIZE=524288000
33+
MAX_PDF_SIZE=52428800
34+
MAX_PDF_PAGES=100
35+
36+
# SSL Verification Settings
37+
# Set to false only for dev with self-signed certs
38+
VERIFY_SSL=true

sample_solutions/DocSummarization/README.md

Lines changed: 26 additions & 108 deletions
Original file line numberDiff line numberDiff line change
@@ -101,38 +101,34 @@ The application consists of:
101101
Before you begin, ensure you have the following installed:
102102

103103
- **Docker and Docker Compose**
104-
- **Enterprise inference endpoint access** (token-based authentication)
104+
- **Enterprise Inference endpoint access** (token-based authentication, see below for models and configs)
105+
106+
#### Deploy Required Models
107+
108+
See the table below for supported models, hardware, and gateway configuration.
109+
110+
| Model | Xeon w/APISIX/Keycloak | Xeon w/GenAI Gateway | Gaudi w/APISIX/Keycloak | Gaudi w/GenAI Gateway |
111+
|---|:---:|:---:|:---:|:---:|
112+
| **meta-llama/Llama-3.1-8B-Instruct** ||| ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 |
113+
| **Qwen/Qwen3-4B-Instruct-2507** | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 |||
105114

106115
### Required API Configuration
107116

108117
**For Inference Service:**
109118

110119
This application supports multiple inference deployment patterns:
111120

112-
- **GenAI Gateway**: Provide your GenAI Gateway URL and API key
121+
**GenAI Gateway**: Provide your GenAI Gateway URL and API key
122+
- URL format: https://api.example.com
113123
- To generate the GenAI Gateway API key, use the [generate-vault-secrets.sh](https://github.com/opea-project/Enterprise-Inference/blob/main/core/scripts/generate-vault-secrets.sh) script
114-
- The API key is the `litellm_master_key` value from the generated `vault.yml` file
115-
116-
- **APISIX Gateway**: Provide your APISIX Gateway URL and authentication token
124+
- The API key is the litellm_master_key value from the generated vault.yml file
125+
126+
**APISIX Gateway**: Provide your APISIX Gateway URL and authentication token
127+
- URL format: https://api.example.com/Llama-3.1-8B-Instruct
128+
- Note: APISIX requires the model name in the URL path
117129
- To generate the APISIX authentication token, use the [generate-token.sh](https://github.com/opea-project/Enterprise-Inference/blob/main/core/scripts/generate-token.sh) script
118130
- The token is generated using Keycloak client credentials
119131

120-
### Local Development Configuration
121-
122-
**For Local Testing Only (Optional)**
123-
124-
If you're testing with a local inference endpoint using a custom domain (e.g., `api.example.com` mapped to localhost in your hosts file):
125-
126-
1. Edit `backend/.env` and set:
127-
```bash
128-
LOCAL_URL_ENDPOINT=api.example.com
129-
```
130-
(Use the domain name from your INFERENCE_API_ENDPOINT without `https://`)
131-
132-
2. This allows Docker containers to resolve your local domain correctly.
133-
134-
**Note:** For public domains or cloud-hosted endpoints, leave the default value `not-needed`.
135-
136132
### Verify Docker Installation
137133

138134
```bash
@@ -159,95 +155,28 @@ cd Enterprise-Inference/sample_solutions/DocSummarization
159155

160156
### Set up the Environment
161157

162-
This application requires **two `.env` files** for proper configuration:
163-
164-
1. **Root `.env` file** (for Docker Compose variables)
165-
2. **`backend/.env` file** (for backend application configuration)
166-
167-
#### Step 1: Create Root `.env` File
168-
169-
You can either copy from the example file:
158+
This application requires an `.env` file in the root directory for proper configuration. Create it using [.env.example](./.env.example) with the commands below:
170159

171160
```bash
172161
cp .env.example .env
173162
```
174-
175-
Then edit `.env` if needed, **OR** create it directly:
176-
177-
```bash
178-
# From the DocSummarization directory
179-
cat > .env << EOF
180-
# Docker Compose Configuration
181-
LOCAL_URL_ENDPOINT=not-needed
182-
EOF
183-
```
184-
185-
**Note:** If using a local domain (e.g., `api.example.com` mapped to localhost), replace `not-needed` with your domain name (without `https://`).
186-
187-
#### Step 2: Create `backend/.env` File
188-
189-
You can either copy from the example file:
190-
191-
```bash
192-
cp backend/.env.example backend/.env
193-
```
194-
195-
Then edit `backend/.env` with your actual credentials, **OR** create it directly:
196-
197-
```bash
198-
cat > backend/.env << EOF
199-
# Inference API Configuration
200-
# INFERENCE_API_ENDPOINT: URL to your inference service (without /v1 suffix)
201-
# - For GenAI Gateway: https://genai-gateway.example.com
202-
# - For APISIX Gateway: https://apisix-gateway.example.com/Llama-3.1-8B-Instruct
203-
# Note: APISIX Gateway requires the model name in the URL path
204-
#
205-
# INFERENCE_API_TOKEN: Authentication token/API key for the inference service
206-
# - For GenAI Gateway: Your GenAI Gateway API key
207-
# - For APISIX Gateway: Your APISIX authentication token
208-
INFERENCE_API_ENDPOINT=https://your-actual-api-endpoint.com
209-
INFERENCE_API_TOKEN=your-actual-token-here
210-
211-
# Model Configuration
212-
# IMPORTANT: Use the full model names as they appear in your inference service
213-
# Check available models: curl https://your-api-endpoint.com/v1/models -H "Authorization: Bearer your-token"
214-
INFERENCE_MODEL_NAME=meta-llama/Llama-3.1-8B-Instruct
215-
216-
# LLM Configuration
217-
LLM_TEMPERATURE=0.7
218-
LLM_MAX_TOKENS=2000
219-
220-
# Local URL Endpoint (for Docker)
221-
LOCAL_URL_ENDPOINT=not-needed
222-
223-
# Service Configuration
224-
SERVICE_PORT=8000
225-
LOG_LEVEL=INFO
226-
227-
# File Upload Limits
228-
MAX_FILE_SIZE=524288000
229-
MAX_PDF_SIZE=52428800
230-
MAX_PDF_PAGES=100
231-
232-
# SSL Verification Settings
233-
# Set to false only for dev with self-signed certs
234-
VERIFY_SSL=true
235-
EOF
236-
```
163+
Then modify it as needed, with special consideration to certain environment variables mentioned below. Read through the .env file for full instructions.
237164

238165
**Important Configuration Notes:**
239166

240-
- **INFERENCE_API_ENDPOINT**: Your actual inference service URL (replace `https://your-actual-api-endpoint.com`)
241-
- For APISIX/Keycloak deployments, the model name must be included in the endpoint URL (e.g., `https://apisix-gateway.example.com/Llama-3.1-8B-Instruct`)
167+
- **INFERENCE_API_ENDPOINT**: Your actual inference service URL (replace `https://api.example.com`)
168+
- For APISIX/Keycloak deployments, the model name must be included in the endpoint URL (e.g., `https://api.example.com/Llama-3.1-8B-Instruct`)
242169
- **INFERENCE_API_TOKEN**: Your actual pre-generated authentication token
243170
- **INFERENCE_MODEL_NAME**: Use the exact model name from your inference service
244-
- To check available models: `curl https://your-api-endpoint.com/v1/models -H "Authorization: Bearer your-token"`
245-
- **LOCAL_URL_ENDPOINT**: Only needed if using local domain mapping
171+
- To check available models: `curl https://api.example.com/v1/models -H "Authorization: Bearer your-token"`
172+
- **LOCAL_URL_ENDPOINT**: Only needed if using local domain mapping (i.e. `api.example.com` mapped to localhost) for Docker containers to resolve correctly.
173+
- Use the domain name from INFERENCE_API_ENDPOINT without `https://`
174+
- For public domains or cloud-hosted endpoints, leave the default value `not-needed`
246175
- **VERIFY_SSL**: Controls SSL certificate verification (default: `true`)
247176
- Set to `false` only for development environments with self-signed certificates
248177
- Keep as `true` for production environments
249178

250-
**Note**: The docker-compose.yml file automatically loads environment variables from both `.env` (root) and `./backend/.env` (backend) files.
179+
**Note**: The docker-compose.yaml file automatically loads environment variables from `.env` for the backend service.
251180

252181
### Running the Application
253182

@@ -329,14 +258,3 @@ docker compose down
329258
For comprehensive troubleshooting guidance, common issues, and solutions, refer to:
330259

331260
[TROUBLESHOOTING.md](./TROUBLESHOOTING.md)
332-
333-
---
334-
335-
## Additional Info
336-
337-
The following models have been validated with DocSummarization:
338-
339-
| Model | Hardware |
340-
|-------|----------|
341-
| **meta-llama/Llama-3.1-8B-Instruct** | Gaudi |
342-
| **Qwen/Qwen3-4B-Instruct-2507** | Xeon |

sample_solutions/DocSummarization/backend/.env.example

Lines changed: 0 additions & 39 deletions
This file was deleted.

sample_solutions/DocSummarization/docker-compose.yml

Lines changed: 1 addition & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ services:
88
ports:
99
- "8000:8000"
1010
env_file:
11-
- ./backend/.env
11+
- .env
1212
volumes:
1313
- ./backend:/app
1414
extra_hosts:
@@ -34,32 +34,3 @@ services:
3434
networks:
3535
app_network:
3636
driver: bridge
37-
38-
# =============================================================================
39-
# Usage Instructions:
40-
# =============================================================================
41-
#
42-
# 1. Create .env file in root directory (for Docker Compose variables):
43-
# cat > .env << EOF
44-
# LOCAL_URL_ENDPOINT=not-needed
45-
# EOF
46-
#
47-
# 2. Copy backend/.env.example to backend/.env and configure your inference API credentials:
48-
# cp backend/.env.example backend/.env
49-
# # Edit backend/.env with your INFERENCE_API_ENDPOINT and INFERENCE_API_TOKEN
50-
#
51-
# 3. Start the application:
52-
# docker compose up -d --build
53-
#
54-
# 4. Access the application:
55-
# - Frontend: http://localhost:5173
56-
# - Backend API: http://localhost:8000
57-
# - API Docs: http://localhost:8000/docs
58-
#
59-
# 5. View logs:
60-
# docker compose logs -f
61-
#
62-
# 6. Stop the application:
63-
# docker compose down
64-
#
65-
# =============================================================================

0 commit comments

Comments
 (0)