You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: AvatarChatbot/README.md
+15-33Lines changed: 15 additions & 33 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,8 +1,13 @@
1
1
# AvatarChatbot Application
2
2
3
-
The AvatarChatbot service can be effortlessly deployed on either Intel Gaudi2 or Intel XEON Scalable Processors.
3
+
The AvatarChatbot example demonstrates the implementation of AI avatar chatbot using OPEA component-level microservices.
4
4
5
-
## AI Avatar Workflow
5
+
## Table of contents
6
+
7
+
1.[Architecture](#architecture)
8
+
2.[Deployment Options](#deployment-options)
9
+
10
+
## Architecture
6
11
7
12
The AI Avatar example is implemented using both megaservices and the component-level microservices defined in [GenAIComps](https://github.com/opea-project/GenAIComps). The flow chart below shows the information flow between different megaservices and microservices for this example.
8
13
@@ -69,37 +74,14 @@ flowchart LR
69
74
UI <-.->|6'| WAV2LIP
70
75
```
71
76
72
-
## Deploy AvatarChatbot Service
73
-
74
-
The AvatarChatbot service can be deployed on either Intel Gaudi2 AI Accelerator or Intel Xeon Scalable Processor.
75
-
76
-
### Deploy AvatarChatbot on Gaudi
77
-
78
-
Refer to the [Gaudi Guide](./docker_compose/intel/hpu/gaudi/README.md) for instructions on deploying AvatarChatbot on Gaudi, and on setting up an UI for the application.
79
-
80
-
### Deploy AvatarChatbot on Xeon
81
-
82
-
Refer to the [Xeon Guide](./docker_compose/intel/cpu/xeon/README.md) for instructions on deploying AvatarChatbot on Xeon.
83
-
84
-
## Supported Models
85
-
86
-
### ASR
87
-
88
-
The default model is [openai/whisper-small](https://huggingface.co/openai/whisper-small). It also supports all models in the Whisper family, such as `openai/whisper-large-v3`, `openai/whisper-medium`, `openai/whisper-base`, `openai/whisper-tiny`, etc.
89
-
90
-
To replace the model, please edit the `compose.yaml` and add the `command` line to pass the name of the model you want to use:
91
-
92
-
```yaml
93
-
services:
94
-
whisper-service:
95
-
...
96
-
command: --model_name_or_path openai/whisper-tiny
97
-
```
98
-
99
-
### TTS
77
+
This AvatarChatbot use case performs AI avatar chatbot across multiple platforms. Currently, we provide the example for [Intel Gaudi2](https://www.intel.com/content/www/us/en/products/details/processors/ai-accelerators/gaudi-overview.html) and [Intel Xeon Scalable Processors](https://www.intel.com/content/www/us/en/products/details/processors/xeon.html), and we invite contributions from other hardware vendors to expand OPEA ecosystem.
100
78
101
-
The default model is [microsoft/SpeechT5](https://huggingface.co/microsoft/speecht5_tts). We currently do not support replacing the model. More models under the commercial license will be added in the future.
79
+
## Deployment Options
102
80
103
-
### Animation
81
+
The table below lists the available deployment options and their implementation details for different hardware platforms.
104
82
105
-
The default model is [Rudrabha/Wav2Lip](https://github.com/Rudrabha/Wav2Lip) and [TencentARC/GFPGAN](https://github.com/TencentARC/GFPGAN). We currently do not support replacing the model. More models under the commercial license such as [OpenTalker/SadTalker](https://github.com/OpenTalker/SadTalker) will be added in the future.
# Example AvatarChatbot Deployment on Intel® Xeon® Platform
2
2
3
-
This document outlines the deployment process for a AvatarChatbot application utilizing the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline on Intel Xeon server.
3
+
This document outlines the deployment process for a AvatarChatbot application utilizing the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline on Intel Xeon server. This example includes the following sections:
4
4
5
-
## 🚀 Build Docker images
5
+
-[AvatarChatbot Quick Start Deployment](#avatarchatbot-quick-start-deployment): Demonstrates how to quickly deploy a AvatarChatbot service/pipeline on Intel Xeon server.
6
+
-[AvatarChatbot Docker Compose Files](#avatarchatbot-docker-compose-files): Describes some example deployments and their docker compose files.
7
+
-[AvatarChatbot Service Configuration](#avatarchatbot-service-configuration): Describes the service and possible configuration changes.
This section describes how to quickly deploy and test the AvatarChatbot service manually on Intel Xeon server. The basic steps are:
21
12
22
-
Intel Xeon optimized image hosted in huggingface repo will be used for TGI service: ghcr.io/huggingface/text-generation-inference:2.4.0-intel-cpu (https://github.com/huggingface/text-generation-inference)
13
+
1.[Access the Code](#access-the-code)
14
+
2.[Generate a HuggingFace Access Token](#generate-a-huggingface-access-token)
15
+
3.[Configure the Deployment Environment](#configure-the-deployment-environment)
16
+
4.[Deploy the Service Using Docker Compose](#deploy-the-service-using-docker-compose)
17
+
5.[Check the Deployment Status](#check-the-deployment-status)
18
+
6.[Test the Pipeline](#test-the-pipeline)
19
+
7.[Cleanup the Deployment](#cleanup-the-deployment)
To construct the Mega Service, we utilize the [GenAIComps](https://github.com/opea-project/GenAIComps.git) microservice pipeline within the `audioqna.py` Python script. Build the MegaService Docker image using the command below:
cd GenAIExamples/AvatarChatbot/docker_compose/intel/cpu/xeon/
46
28
```
47
29
48
-
Then run the command `docker images`, you will have following images ready:
49
-
50
-
1.`opea/whisper:latest`
51
-
2.`opea/speecht5:latest`
52
-
3.`opea/wav2lip:latest`
53
-
4.`opea/animation:latest`
54
-
5.`opea/avatarchatbot:latest`
30
+
Checkout a released version, such as v1.3:
55
31
56
-
## 🚀 Set the environment variables
32
+
```
33
+
git checkout v1.3
34
+
```
57
35
58
-
Before starting the services with `docker compose`, you have to recheck the following environment variables.
36
+
### Generate a HuggingFace Access Token
59
37
60
-
```bash
61
-
export HF_TOKEN=<your_hf_token>
62
-
export host_ip=$(hostname -I | awk '{print $1}')
38
+
Some HuggingFace resources, such as some models, are only accessible if you have an access token. If you do not already have a HuggingFace access token, you can create one by first creating an account by following the steps provided at [HuggingFace](https://huggingface.co/) and then generating a [user access token](https://huggingface.co/docs/transformers.js/en/guides/private#step-1-generating-a-user-access-token).
63
39
64
-
export LLM_MODEL_ID=Intel/neural-chat-7b-v3-3
40
+
### Configure the Deployment Environment
65
41
66
-
export WAV2LIP_ENDPOINT=http://$host_ip:7860
42
+
To set up environment variables for deploying AvatarChatbot service, source the _set_env.sh_ script in this directory:
67
43
68
-
export MEGA_SERVICE_HOST_IP=${host_ip}
69
-
export WHISPER_SERVER_HOST_IP=${host_ip}
70
-
export WHISPER_SERVER_PORT=7066
71
-
export SPEECHT5_SERVER_HOST_IP=${host_ip}
72
-
export SPEECHT5_SERVER_PORT=7055
73
-
export LLM_SERVER_HOST_IP=${host_ip}
74
-
export LLM_SERVER_PORT=3006
75
-
export ANIMATION_SERVICE_HOST_IP=${host_ip}
76
-
export ANIMATION_SERVICE_PORT=3008
77
-
78
-
export MEGA_SERVICE_PORT=8888
44
+
```
45
+
source set_env.sh
79
46
```
80
47
81
-
- Xeon CPU
48
+
The set_env.sh script will prompt for required and optional environment variables used to configure the AvatarChatbot service. If a value is not entered, the script will use a default value for the same. It will also generate a env file defining the desired configuration. Consult the section on [AvatarChatbot Service configuration](#avatarchatbot-service-configuration) for information on how service specific configuration parameters affect deployments.
In the current version v1.3, you need to set the avatar figure image/video and the DL model choice in the environment variables before starting AvatarChatbot backend service and running the UI. Please just customize the audio question in the UI.
168
-
\*\* We will enable change of avatar figure between runs in v2.0
The table provides a comprehensive overview of the AvatarChatbot service utilized across various deployments as illustrated in the example Docker Compose files. Each row in the table represents a distinct service, detailing its possible images used to enable it and a concise description of its function within the deployment architecture.
149
+
150
+
| Service Name | Possible Image Names | Optional | Description |
| tgi-service | ghcr.io/huggingface/text-generation-inference:2.4.0-intel-cpu | No | Specific to the TGI deployment, focuses on text generation inference using Xeon hardware. |
153
+
| whisper-service | opea/whisper:latest| No | Provides automatic speech recognition (ASR), converting spoken audio input into text. |
154
+
| speecht5-service | opea/speecht5:latest| No | Performs text-to-speech (TTS) synthesis, generating natural-sounding speech from text. |
155
+
| wav2lip-service | opea/wav2lip:latest| No | Generates realistic lip-sync animations by aligning speech audio with a video of a face. |
156
+
| animation | opea/animation:latest| No | Handles avatar animation, rendering facial expressions and movements for the chatbot avatar. |
157
+
| avatarchatbot-xeon-backend-server | opea/avatarchatbot:latest| No | Orchestrates the overall AvatarChatbot pipeline, managing requests and integrating all services. |
![pre-commit-ci[bot]](https://avatars.githubusercontent.com/in/68672?v=4&size=40){kind=avatar}
0 commit comments