|
2 | 2 | sidebar_position: 3 |
3 | 3 | --- |
4 | 4 |
|
| 5 | +import Tabs from '@theme/Tabs'; |
| 6 | +import TabItem from '@theme/TabItem'; |
| 7 | + |
5 | 8 | # Docker Deployment |
6 | 9 |
|
7 | | -:::danger |
8 | | -Due to significant refactoring, the Docker image has not yet been updated to version `v1.0.0`, but it will be updated soon. |
| 10 | +## Download configuration files |
| 11 | + |
| 12 | +### Git clone (for users familiar with git) |
| 13 | +First, [install git following the Quick Start](/docs/quick-start.md#安装-git), then download the Docker user configuration: |
| 14 | + |
| 15 | +```bash |
| 16 | +git clone https://github.com/Open-LLM-VTuber/OLV-Docker-Config |
| 17 | +``` |
| 18 | + |
| 19 | +### Download ZIP (for users without coding experience) |
| 20 | +:::warning |
| 21 | +This download method cannot be updated later, but it will work right now. |
| 22 | +::: |
| 23 | +First, [click here to go to the user configuration repository](https://github.com/Open-LLM-Vtuber/OLV-Docker-Config) |
| 24 | + |
| 25 | +Then, on the repository page, click `Code` -> `Download ZIP` to download the Docker user configuration, as shown below. |
| 26 | + |
| 27 | + |
| 28 | + |
| 29 | +## Edit configuration files |
| 30 | +If you want to change any settings, edit the `conf.yaml` file. |
| 31 | + |
| 32 | +:::info |
| 33 | +The default `conf.yaml` is in English. If you want to replace it with Chinese, copy `conf.ZH.default.yaml` from the `config_templates` folder, go back one level, and replace the original `conf.yaml`. |
| 34 | +::: |
| 35 | + |
| 36 | +If you're a beginner, it's recommended to use the default configuration file for one-click startup. |
| 37 | + |
| 38 | +If you want to change settings like character profile, model, ASR, TTS, etc., modify the corresponding fields in `conf.yaml`. |
| 39 | + |
| 40 | +For information about each field in the config file, refer to the comments inside the configuration file. |
| 41 | + |
| 42 | +## Install Docker |
| 43 | + |
| 44 | +:::tip About installation methods |
| 45 | +Two installation methods are supported currently: |
| 46 | +- If you're a beginner, we recommend [using Docker Desktop](#install-docker-using-docker-desktop). |
| 47 | +- If you're familiar with Docker, you can [use terminal commands](#install-docker-using-terminal-commands). |
| 48 | +::: |
| 49 | + |
| 50 | +:::caution About GPU support |
| 51 | +Currently **only CPU and Nvidia GPU** are supported. Nvidia GPUs require proper Nvidia driver configuration. If you haven't set it up yet, please [**click here to view the setup guide in Quick Start**](/docs/quick-start.md#nvidia-gpu-支持). |
| 52 | + |
| 53 | +If you don't plan to use a GPU and will instead **use API calls** (this is **the best experience for most users**) or use CPU only, you can skip this step. |
9 | 54 | ::: |
10 | 55 |
|
11 | | -:::warning |
12 | | -⚠️ Please note: This feature is currently in an experimental stage but works normally in most cases. |
| 56 | +:::note About our Docker team |
| 57 | +We created an [openllmvtuber team on Docker Hub](https://hub.docker.com/orgs/openllmvtuber/members). Docker images are currently maintained by [@Harry_Y](https://github.com/Harry-Yu-Shuhang). |
| 58 | +::: |
| 59 | + |
| 60 | +## Install Docker using Docker Desktop |
| 61 | + |
| 62 | +:::warning |
| 63 | +Docker Desktop may hang during restart; try to avoid restarting when possible. If it becomes unresponsive for a long time, kill the Docker Desktop background process in Task Manager (Windows) or Activity Monitor (macOS) and then start it again. |
| 64 | +::: |
| 65 | + |
| 66 | +### Download Docker Desktop |
| 67 | + |
| 68 | +Visit the [Docker Desktop download page](https://www.docker.com/products/docker-desktop). |
| 69 | + |
| 70 | +Click "Download Docker Desktop" |
| 71 | + |
| 72 | + |
| 73 | + |
| 74 | +Choose the version that matches your computer. |
| 75 | + |
| 76 | +:::tip How to choose the right version for your computer |
| 77 | +- **Mac Apple Silicon** → Newer Macs (M1/M2/M3 chips) |
| 78 | +- **Mac Intel Chip** → Older Macs (Intel processors) |
| 79 | +- **Windows AMD64** → Most Windows PCs (Intel/AMD 64-bit) |
| 80 | +- **Windows ARM64** → Rare, for ARM-based Windows (e.g., Surface Pro X) |
13 | 81 | ::: |
14 | 82 |
|
15 | | -You can choose to build the Docker image yourself or pull the pre-built image directly from Docker Hub: [](https://hub.docker.com/r/t41372/open-llm-vtuber) |
| 83 | +If you are not sure, check your system settings as follows: |
| 84 | + |
| 85 | +<Tabs groupId="operating-systems"> |
| 86 | + <TabItem value="windows" label="Windows"> |
| 87 | + |
| 88 | +1. Right-click "This PC" → Properties |
| 89 | +2. Under "System type" check: |
| 90 | + - "x64-based processor" → Choose **Windows AMD64** |
| 91 | + - "ARM-based processor" → Choose **Windows ARM64** |
| 92 | + |
| 93 | + </TabItem> |
| 94 | + <TabItem value="macos" label="macOS"> |
| 95 | + |
| 96 | +1. Click the Apple icon at the top-left → "About This Mac" |
| 97 | +2. Check the "Chip" (or "Processor") field: |
| 98 | + - Shows **Apple M1/M2/M3** → Choose **Mac Apple Silicon** |
| 99 | + - Shows **Intel** → Choose **Mac Intel Chip** |
| 100 | + |
| 101 | + </TabItem> |
| 102 | +</Tabs> |
| 103 | + |
| 104 | +### Start Docker Desktop |
16 | 105 |
|
17 | | -## Usage Notes |
| 106 | +Double-click the downloaded installer and follow the prompts to install. |
18 | 107 |
|
19 | | -- The Docker image is large (about 13GB), and some optional models require additional downloads when used, occupying more storage space |
20 | | -- Only supports devices with NVIDIA GPUs |
21 | | -- Requires installation and configuration of [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) to support GPU passthrough |
22 | | -- Current issue: Some models need to be re-downloaded after the container stops (this will be fixed in future versions) |
23 | | -- Building the image on ARM architecture devices is not recommended due to potential build failures with the gRPC dependency ([related issue](https://github.com/grpc/grpc/issues/34998)) |
24 | | -- If running on a remote server, HTTPS must be configured. This is because the frontend webpage's microphone/camera/screen recording features can only be used in a secure context (localhost or HTTPS environment) |
| 108 | +After installation, click the Docker Desktop icon to start Docker. |
25 | 109 |
|
26 | | -## Pre-installed Model Information |
| 110 | +On first start, Docker Desktop will ask you to sign in to a Docker Hub account. If you don't have one, register first. |
27 | 111 |
|
28 | | -The default image comes with most Automatic Speech Recognition (ASR) and Text-to-Speech (TTS) models pre-installed. However, considering size factors (about 8GB, which would increase the total size to 25GB) and performance, the default build does not include the following models: |
29 | | -- Bark TTS |
30 | | -- Original OpenAI Whisper (Note: This refers to Whisper, not WhisperCPP) |
| 112 | +### Pull the Docker image |
| 113 | + |
| 114 | +Click the top menu "Search" icon (or use the shortcut `Ctrl + K` on Windows or `Cmd + K` on macOS). |
| 115 | + |
| 116 | + |
| 117 | + |
| 118 | +Enter the following in the search box. When you find it, click the "Pull" button on the right. |
| 119 | + |
| 120 | +``` |
| 121 | +openllmvtuber/open-llm-vtuber:latest |
| 122 | +``` |
| 123 | + |
| 124 | + |
| 125 | + |
| 126 | +### Configure LLM |
| 127 | +:::info About how to use LLM |
| 128 | +If you use a local model (for example the default `Ollama`), this step is required. |
| 129 | + |
| 130 | +If you choose API calls, you can skip this step. |
| 131 | +::: |
| 132 | + |
| 133 | +:::info |
| 134 | +If you are a beginner with no coding background |
| 135 | +We recommend following these steps to use the default `Ollama` model and avoid extra configuration. |
| 136 | + |
| 137 | +<details> |
| 138 | + 1. Download and install the Ollama client from the [Ollama official site](https://ollama.com/). |
| 139 | + |
| 140 | + 2. Verify Ollama installation: |
| 141 | + ```bash |
| 142 | + ollama --version |
| 143 | + ``` |
| 144 | + |
| 145 | + 3. Download and run a model (the default config uses `qwen2.5:latest` as an example): |
| 146 | + ```bash |
| 147 | + ollama run qwen2.5:latest |
| 148 | + ``` |
| 149 | + |
| 150 | + 4. List installed models: |
| 151 | + ```bash |
| 152 | + ollama list |
| 153 | + ``` |
| 154 | +</details> |
| 155 | +::: |
| 156 | + |
| 157 | +For more details, refer to the LLM configuration section in [Quick Start](/docs/quick-start.md#3-配置-llm). |
| 158 | + |
| 159 | +### Run the Docker container |
| 160 | + |
| 161 | +Click "Images", find **openllmvtuber/open-llm-vtuber**, and click the run button on the right. |
| 162 | + |
| 163 | + |
| 164 | +Open "Optional settings" and configure as shown below: |
| 165 | + |
| 166 | + |
| 167 | +### Open the web interface in your browser |
| 168 | +When you see lines like the ones below, the service has started successfully. The first pull may be slow because it installs necessary tools—please be patient. |
| 169 | + |
| 170 | + |
| 171 | +Click the `12393:12393` link in the top-left as shown below, or open `http://localhost:12393` in your browser. |
| 172 | + |
| 173 | + |
| 174 | +**Congratulations — it's working!** Chat with your virtual companion! |
| 175 | + |
| 176 | +## Install Docker using terminal commands |
| 177 | +1. Install Docker Desktop. |
| 178 | +<Tabs groupId="operating-systems"> |
| 179 | + <TabItem value="windows" label="Windows"> |
31 | 180 |
|
32 | | -To include these models in the image, add the following parameters when building: |
33 | 181 | ```bash |
34 | | ---build-arg INSTALL_ORIGINAL_WHISPER=true --build-arg INSTALL_BARK=true |
| 182 | +winget install -e --id Docker.DockerDesktop |
35 | 183 | ``` |
36 | 184 |
|
37 | | -## Deployment Steps |
| 185 | + </TabItem> |
| 186 | + <TabItem value="macos" label="macOS"> |
| 187 | + |
| 188 | +```bash |
| 189 | +brew install --cask docker |
| 190 | +``` |
38 | 191 |
|
39 | | -1. Before building the image, check and adjust the `conf.yaml` configuration file (the current configuration will be built into the image) |
| 192 | + </TabItem> |
| 193 | +</Tabs> |
40 | 194 |
|
41 | | -2. Build the Docker image: |
| 195 | +2. Pull the latest image. |
| 196 | + |
| 197 | +```bash |
| 198 | +docker pull openllmvtuber/open-llm-vtuber:latest |
| 199 | +``` |
| 200 | + |
| 201 | +3. Start the container. You can modify `docker-compose.yml` if needed; the default can also be started with one command. |
42 | 202 | ```bash |
43 | | -docker build -t open-llm-vtuber . |
| 203 | +cd OLV-Docker-Config\Path |
44 | 204 | ``` |
45 | | - > Note: The build process may take a considerable amount of time |
46 | 205 |
|
47 | | -3. Prepare the `conf.yaml` configuration file |
48 | | - You can obtain it from the project repository or download it directly via this [link](https://raw.githubusercontent.com/t41372/Open-LLM-VTuber/main/conf.yaml) |
| 206 | +```bash |
| 207 | +docker-compose up -d |
| 208 | +``` |
49 | 209 |
|
50 | | -4. Run the container: |
| 210 | +4. View container logs. |
51 | 211 | ```bash |
52 | | -docker run -it --net=host --rm -v $(pwd)/conf.yaml:/app/conf.yaml -p 12393:12393 open-llm-vtuber |
| 212 | +docker logs -f open-llm-vtuber-latest |
53 | 213 | ``` |
54 | | - Note: Replace `$(pwd)/conf.yaml` with the actual path to your configuration file |
55 | 214 |
|
56 | | -5. Access `localhost:12393` for testing |
| 215 | +5. Open the web interface in your browser. |
| 216 | + |
| 217 | +After the logs show `Uvicorn running on http://0.0.0.0:12393`, open `http://localhost:12393` in your browser to access the web interface. |
| 218 | + |
| 219 | +That's it — you have successfully deployed Open LLM VTuber! |
0 commit comments