You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
|**Qwen/Qwen3-4B-Instruct-2507**| ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 | ✅ Validated on Dell XE7740 |
248
250
249
-
All nine AI agents (Code Explorer, API Reference, Call Graph, Error Analysis, Environment Config, Dependency Analyzer, Planner, Mermaid Generator, and QA Validator) use this model for efficient documentation generation.
251
+
>**Note** When deploying, the model must have the `--enable-auto-tool-choice` and `--tool-call-parser "hermes"` arguments for the AI agents to use the model endpoint. On Xeon deployments, it is included, but on Gaudi it is not (as of Enterprise Inference release v1.4.0). This needs to be added into the core/helm-charts/vllm folder's `gaudi3_values.yaml` or `gaudi-values.yaml` as a new model config before deploying the model. If the model config is already there, disregard this note.
250
252
251
-
**Note:** This model must be available through your GenAI Gateway or APISIX Gateway deployment before running the application.
253
+
All nine AI agents (Code Explorer, API Reference, Call Graph, Error Analysis, Environment Config, Dependency Analyzer, Planner, Mermaid Generator, and QA Validator) use this model for efficient documentation generation.
252
254
253
255
### Required API Configuration
254
256
@@ -257,36 +259,15 @@ All nine AI agents (Code Explorer, API Reference, Call Graph, Error Analysis, En
257
259
This application supports multiple inference deployment patterns:
258
260
259
261
**GenAI Gateway**: Provide your GenAI Gateway URL and API key
260
-
-**URL format**: `https://api.example.com`
261
-
- To generate the GenAI Gateway API key, use the [generate-vault-secrets.sh](https://github.com/opea-project/Enterprise-Inference/blob/main/core/scripts/generate-vault-secrets.sh) script
262
-
- The API key is the `litellm_master_key` value from the generated `vault.yml` file
262
+
-URL format: https://api.example.com
263
+
- To generate the GenAI Gateway API key, use the [generate-vault-secrets.sh](https://github.com/opea-project/Enterprise-Inference/blob/main/core/scripts/generate-vault-secrets.sh) script
264
+
- The API key is the litellm_master_key value from the generated vault.yml file
263
265
264
266
**APISIX Gateway**: Provide your APISIX Gateway URL and authentication token
-**Note**: APISIX requires the model name in the URL path (without company/family prefixes)
267
-
- To generate the APISIX authentication token, use the [generate-token.sh](https://github.com/opea-project/Enterprise-Inference/blob/main/core/scripts/generate-token.sh) script
268
-
- The token is generated using Keycloak client credentials
269
-
270
-
**Configuration requirements:**
271
-
-**INFERENCE_API_ENDPOINT**: URL to your inference service (example: `https://api.example.com`)
272
-
-**INFERENCE_API_TOKEN**: Authentication token/API key for your chosen service
273
-
274
-
275
-
### Local Development Configuration
276
-
277
-
**For Local Testing Only (Optional)**
278
-
279
-
If you're testing with a local inference endpoint using a custom domain (e.g., `api.example.com` mapped to localhost in your hosts file):
280
-
281
-
1. Edit `api/.env` and set:
282
-
```bash
283
-
LOCAL_URL_ENDPOINT=api.example.com
284
-
```
285
-
(Use the domain name from your INFERENCE_API_ENDPOINT without `https://`)
286
-
287
-
2. This allows Docker containers to resolve your local domain correctly.
288
-
289
-
**Note:** For public domains or cloud-hosted endpoints, leave the default value `not-needed`.
- Note: APISIX requires the model name in the URL path
269
+
- To generate the APISIX authentication token, use the [generate-token.sh](https://github.com/opea-project/Enterprise-Inference/blob/main/core/scripts/generate-token.sh) script
270
+
- The token is generated using Keycloak client credentials
290
271
291
272
### GitHub Personal Access Token (Optional)
292
273
@@ -323,152 +304,27 @@ cd Enterprise-Inference/sample_solutions/Docugen-Microagents
323
304
324
305
### Set up the Environment
325
306
326
-
This application requires **two `.env` files** for proper configuration:
# Local URL Endpoint (only needed for non-public domains)
339
-
# If using a local domain like api.example.com mapped to localhost, set to the domain without https://
340
-
# Otherwise, set to: not-needed
341
-
LOCAL_URL_ENDPOINT=not-needed
342
-
343
-
BACKEND_PORT=8000
344
-
FRONTEND_PORT=3000
345
-
346
-
EOF
347
-
```
348
-
349
-
OR
350
-
351
-
Copy from the example file and edit with your credentials as required.
307
+
This application requires an `.env` file in the root directory for proper configuration. Create it using [.env.example](./.env.example) with the commands below:
352
308
353
309
```bash
354
310
cp .env.example .env
355
311
```
356
-
357
-
**Note:** If using a local domain (e.g., `api.example.com` mapped to localhost), replace `not-needed` with your domain name (without `https://`).
358
-
359
-
#### Step 2: Create `api/.env` File
360
-
361
-
Copy from the example file and edit with your actual credentials:
362
-
363
-
```bash
364
-
cp api/.env.example api/.env
365
-
```
366
-
367
-
Then edit `api/.env` to set your `INFERENCE_API_ENDPOINT` and `INFERENCE_API_TOKEN`.
368
-
369
-
Or manually create `api/.env` with:
370
-
371
-
```bash
372
-
# ==========================================
373
-
# Inference API Configuration
374
-
# ==========================================
375
-
# INFERENCE_API_ENDPOINT: URL to your inference service (without /v1 suffix)
376
-
#
377
-
# **GenAI Gateway**: Provide your GenAI Gateway URL and API key
378
-
# - URL format: https://genai-gateway.example.com
379
-
# - To generate the GenAI Gateway API key, use the [generate-vault-secrets.sh] script
380
-
# - The API key is the litellm_master_key value from the generated vault.yml file
381
-
#
382
-
# **APISIX Gateway**: Provide your APISIX Gateway URL and authentication token
383
-
# - For APISIX, include the model name in the INFERENCE_API_ENDPOINT path
# SSL Verification: Set to false only for development with self-signed certificates
457
-
VERIFY_SSL=true
458
-
```
459
-
460
-
**Note:** All nine AI agents (Code Explorer, API Reference, Call Graph, Error Analysis, Env Config, Dependency Analyzer, Planner, Mermaid Generator, QA Validator) plus PR Agent use Qwen/Qwen3-4B-Instruct-2507 optimized for Intel Xeon processors
312
+
Then modify it as needed, with special consideration to certain environment variables mentioned below. Read through the .env file for full instructions.
461
313
462
314
**Important Configuration Notes:**
463
315
464
316
-**INFERENCE_API_ENDPOINT**: Your actual inference service URL (replace `https://api.example.com`)
465
-
- For APISIX/Keycloak deployments, the model name must be included in the endpoint URL (e.g., `https://api.example.com/Qwen3-4B-Instruct`)
317
+
- For APISIX/Keycloak deployments, the model name must be included in the endpoint URL (e.g., `https://api.example.com/Qwen3-4B-Instruct-2507`)
466
318
-**INFERENCE_API_TOKEN**: Your actual pre-generated authentication token
467
-
-**Model Names**: Use the exact model names from your inference service
468
-
-**LOCAL_URL_ENDPOINT**: Only needed if using local domain mapping
469
-
-**SSL Verification**: set to false only for development with self-signed certificates
470
-
471
-
**Note**: The docker-compose.yml file automatically loads environment variables from both `.env` (root) and `./api/.env` (backend) files.
319
+
-**LOCAL_URL_ENDPOINT**: Only needed if using local domain mapping (i.e. `api.example.com` mapped to localhost) for Docker containers to resolve correctly.
320
+
- Use the domain name from INFERENCE_API_ENDPOINT without `https://`
321
+
- For public domains or cloud-hosted endpoints, leave the default value `not-needed`
322
+
-**Model Names**: Use the exact model name i.e. model card from Hugging Face
0 commit comments