Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 9 additions & 10 deletions CodeGen/docker_compose/intel/cpu/xeon/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,13 +83,13 @@ Different Docker Compose files are available to select the LLM serving backend.

- **Compose File:** `compose.yaml`
- **Description:** Uses vLLM optimized for Intel CPUs as the LLM serving engine. This is the default deployment option used in the Quick Start.
- **Services Deployed:** `codegen-vllm-server`, `codegen-llm-server`, `codegen-tei-embedding-server`, `codegen-retriever-server`, `redis-vector-db`, `codegen-dataprep-server`, `codegen-backend-server`, `codegen-gradio-ui-server`.
- **Services Deployed:** `codegen-vllm-server`, `codegen-llm-server`, `codegen-tei-embedding-server`, `codegen-retriever-server`, `redis-vector-db`, `codegen-dataprep-server`, `codegen-backend-server`, `codegen-ui-server`.

#### TGI-based Deployment (`compose_tgi.yaml`)

- **Compose File:** `compose_tgi.yaml`
- **Description:** Uses Hugging Face Text Generation Inference (TGI) optimized for Intel CPUs as the LLM serving engine.
- **Services Deployed:** `codegen-tgi-server`, `codegen-llm-server`, `codegen-tei-embedding-server`, `codegen-retriever-server`, `redis-vector-db`, `codegen-dataprep-server`, `codegen-backend-server`, `codegen-gradio-ui-server`.
- **Services Deployed:** `codegen-tgi-server`, `codegen-llm-server`, `codegen-tei-embedding-server`, `codegen-retriever-server`, `redis-vector-db`, `codegen-dataprep-server`, `codegen-backend-server`, `codegen-ui-server`.
- **To Run:**

```bash
Expand All @@ -101,7 +101,7 @@ Different Docker Compose files are available to select the LLM serving backend.

- **Compose File:** `compose_remote.yaml`
- **Description:** Uses remote endpoints to access the served LLM's. This is the default configurations except for the LLM serving engine.
- **Services Deployed:** `codegen-tei-embedding-server`, `codegen-retriever-server`, `redis-vector-db`, `codegen-dataprep-server`, `codegen-backend-server`, `codegen-gradio-ui-server`.
- **Services Deployed:** `codegen-tei-embedding-server`, `codegen-retriever-server`, `redis-vector-db`, `codegen-dataprep-server`, `codegen-backend-server`, `codegen-ui-server`.
- **To Run:**

When models are deployed on a remote server, a base URL and an API key are required to access them. To set up a remote server and acquire the base URL and API key, refer to [Intel® AI for Enterprise Inference](https://www.intel.com/content/www/us/en/developer/topic-technology/artificial-intelligence/enterprise-inference.html) offerings.
Expand Down Expand Up @@ -158,7 +158,7 @@ Different Docker Compose files (`compose.yaml`, `compose_tgi.yaml`) control whic
If you need to modify the microservices:

1. Clone the [OPEA GenAIComps](https://github.com/opea-project/GenAIComps) repository.
2. Follow build instructions in the respective component directories (e.g., `comps/llms/text-generation`, `comps/codegen`, `comps/ui/gradio`, etc.). Use the provided Dockerfiles (e.g., `CodeGen/Dockerfile`, `CodeGen/ui/docker/Dockerfile.gradio`).
2. Follow build instructions in the respective component directories (e.g., `comps/llms/text-generation`, `comps/codegen`, etc.). Use the provided Dockerfiles (e.g., `CodeGen/Dockerfile`, `CodeGen/ui/docker/Dockerfile`).
3. Tag your custom images appropriately (e.g., `my-custom-codegen:latest`).
4. Update the `image:` fields in the compose files (`compose.yaml` or `compose_tgi.yaml`) to use your custom image tags.

Expand Down Expand Up @@ -206,18 +206,17 @@ Use `curl` commands to test the main service endpoints. Ensure `HOST_IP` is corr

Multiple UI options can be configured via the compose files.

### Gradio UI (Default)
### Svelte UI (Default)

Access the default Gradio UI by navigating to:
Comment thread
ZePan110 marked this conversation as resolved.
Outdated
`http://{HOST_IP}:5173`
_(Port `5173` is the default host mapping for `codegen-gradio-ui-server`)_
_(Port `5173` is the default host mapping for `codegen-ui-server`)_

![Gradio UI - Code Generation](../../../../assets/img/codegen_gradio_ui_main.png)
![Gradio UI - Resource Management](../../../../assets/img/codegen_gradio_ui_dataprep.png)
![Svelte UI - Code Generation](../../../../assets/img/codegen_gradio_ui_main.png)
Comment thread
ZePan110 marked this conversation as resolved.
Outdated

### Svelte UI (Optional)
### Gradio UI (Optional)

1. Modify the compose file (either `compose.yaml` or `compose_tgi.yaml`): Comment out the `codegen-gradio-ui-server` service and uncomment/add the `codegen-xeon-ui-server` (Svelte) service definition, ensuring the port mapping is correct (e.g., `"- 5173:5173"`).
1. Modify the compose file (either `compose.yaml` or `compose_tgi.yaml`): Comment out the `codegen-xeon-ui-server` (Svelte) service and uncomment/add the `codegen-gradio-ui-server` service definition, ensuring the port mapping is correct (e.g., `"- 5173:5173"`).
2. Restart Docker Compose: `docker compose up -d` or `docker compose -f compose_tgi.yaml up -d`
3. Access: `http://{HOST_IP}:5173` (or the host port you mapped).

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ services:
ipc: host
restart: always
codegen-xeon-ui-server:
image: ${REGISTRY:-opea}/codegen-gradio-ui:${TAG:-latest}
image: ${REGISTRY:-opea}/codegen-ui:${TAG:-latest}
container_name: codegen-xeon-ui-server
depends_on:
- codegen-xeon-backend-server
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ services:
ipc: host
restart: always
codegen-xeon-ui-server:
image: ${REGISTRY:-opea}/codegen-gradio-ui:${TAG:-latest}-openeuler
image: ${REGISTRY:-opea}/codegen-ui:${TAG:-latest}-openeuler
container_name: codegen-xeon-ui-server
depends_on:
- codegen-xeon-backend-server
Expand Down
14 changes: 7 additions & 7 deletions CodeGen/docker_compose/intel/hpu/gaudi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ There are two separate Docker Compose files to select the LLM serving backend ac
- **Compose File:** `compose.yaml`
- **Description:** Uses vLLM optimized for Intel Gaudi HPUs as the LLM serving engine. This is the default deployment used in the Quick Start.
- **Gaudi Service:** `codegen-vllm-gaudi-server`
- **Other Services:** `codegen-llm-server`, `codegen-tei-embedding-server` (CPU), `codegen-retriever-server` (CPU), `redis-vector-db` (CPU), `codegen-dataprep-server` (CPU), `codegen-backend-server` (CPU), `codegen-gradio-ui-server` (CPU).
- **Other Services:** `codegen-llm-server`, `codegen-tei-embedding-server` (CPU), `codegen-retriever-server` (CPU), `redis-vector-db` (CPU), `codegen-dataprep-server` (CPU), `codegen-backend-server` (CPU), `codegen-ui-server` (CPU).

### TGI-based Deployment (`compose_tgi.yaml`)

Expand Down Expand Up @@ -194,23 +194,23 @@ Use `curl` commands targeting the main service endpoints. Ensure `HOST_IP` is co

UI options are similar to the Xeon deployment.

### Gradio UI (Default)
### Svelte UI (Default)

Access the default Gradio UI:
Access the default Svelte UI:
`http://{HOST_IP}:5173`
_(Port `5173` is the default host mapping)_

![Gradio UI](../../../../assets/img/codegen_gradio_ui_main.png)
![Svelte UI](../../../../assets/img/codeGen_ui_init.jpg)

### Svelte UI (Optional)
### Gradio UI (Optional)

1. Modify the compose file (either `compose.yaml` or `compose_tgi.yaml`): Swap Gradio service for Svelte (`codegen-gaudi-ui-server`), check port map (e.g., `5173:5173`).
1. Modify the compose file (either `compose.yaml` or `compose_tgi.yaml`): Swap Svelte service for Gradio (`codegen-gradio-ui-server`), check port map (e.g., `5173:5173`).
2. Restart: `docker compose up -d` or `docker compose -f compose_tgi.yaml up -d`
3. Access: `http://{HOST_IP}:5173`

### React UI (Optional)

1. Modify the compose file (either `compose.yaml` or `compose_tgi.yaml`): Swap Gradio service for React (`codegen-gaudi-react-ui-server`), check port map (e.g., `5174:80`).
1. Modify the compose file (either `compose.yaml` or `compose_tgi.yaml`): Swap Svelte service for React (`codegen-gaudi-react-ui-server`), check port map (e.g., `5174:80`).
2. Restart: `docker compose up -d` or `docker compose -f compose_tgi.yaml up -d`
3. Access: `http://{HOST_IP}:5174`

Expand Down
2 changes: 1 addition & 1 deletion CodeGen/docker_compose/intel/hpu/gaudi/compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ services:
ipc: host
restart: always
codegen-gaudi-ui-server:
image: ${REGISTRY:-opea}/codegen-gradio-ui:${TAG:-latest}
image: ${REGISTRY:-opea}/codegen-ui:${TAG:-latest}
container_name: codegen-gaudi-ui-server
depends_on:
- codegen-gaudi-backend-server
Expand Down
2 changes: 1 addition & 1 deletion CodeGen/docker_compose/intel/hpu/gaudi/compose_tgi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ services:
ipc: host
restart: always
codegen-gaudi-ui-server:
image: ${REGISTRY:-opea}/codegen-gradio-ui:${TAG:-latest}
image: ${REGISTRY:-opea}/codegen-ui:${TAG:-latest}
container_name: codegen-gaudi-ui-server
depends_on:
- codegen-gaudi-backend-server
Expand Down
2 changes: 1 addition & 1 deletion CodeGen/tests/test_compose_on_epyc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ function build_docker_images() {
popd && sleep 1s

echo "Build all the images with --no-cache, check docker_image_build.log for details..."
service_list="codegen codegen-gradio-ui llm-textgen dataprep retriever embedding"
service_list="codegen codegen-ui llm-textgen dataprep retriever embedding"

docker compose -f build.yaml build ${service_list} --no-cache >${LOG_PATH}/docker_image_build.log

Expand Down
2 changes: 1 addition & 1 deletion CodeGen/tests/test_compose_on_gaudi.sh
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ function build_docker_images() {
popd && sleep 1s

echo "Build all the images with --no-cache, check docker_image_build.log for details..."
service_list="codegen codegen-gradio-ui llm-textgen dataprep retriever embedding"
service_list="codegen codegen-ui llm-textgen dataprep retriever embedding"
docker compose -f build.yaml build ${service_list} --no-cache > ${LOG_PATH}/docker_image_build.log

docker images && sleep 1s
Expand Down
2 changes: 1 addition & 1 deletion CodeGen/tests/test_compose_tgi_on_epyc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ function build_docker_images() {
popd && sleep 1s

echo "Build all the images with --no-cache, check docker_image_build.log for details..."
service_list="codegen codegen-gradio-ui llm-textgen dataprep retriever embedding"
service_list="codegen codegen-ui llm-textgen dataprep retriever embedding"

docker compose -f build.yaml build ${service_list} --no-cache >${LOG_PATH}/docker_image_build.log

Expand Down
Loading