11---
2- title : " Containers"
3- description : " Run open models locally, offline, and on edge devices"
4- icon : " docker"
2+ title : ' Containers'
3+ description : ' Run open models locally, offline, and on edge devices'
4+ icon : ' docker'
55---
66
77Every open-source model on Bytez is available as a Docker image. Pull it, run it, and make requests to ` localhost ` .
@@ -18,18 +18,33 @@ Every open-source model on Bytez is available as a Docker image. Pull it, run it
1818 <Tip >
1919 Find model IDs at [ bytez.com/models] ( https://bytez.com/models ) or via the [ List Models API] ( /http-reference/list/models ) .
2020 </Tip >
21+
2122 </Accordion >
2223
2324 <Accordion title = " Start a Container" icon = " play" >
2425 ``` bash
25- docker run -it \
26+ docker run -d \
2627 -e KEY=YOUR_BYTEZ_KEY \
2728 -e PORT=8000 \
2829 -p 8000:8000 \
2930 bytez/qwen_qwen3-4b
3031 ```
3132
32- Get your API key at [ bytez.com/settings] ( https://bytez.com/settings ) .
33+ This runs the container in the background. Get your API key at [ bytez.com/api/key] ( https://bytez.com/api/key ) .
34+
35+ ** View logs**
36+
37+ ``` bash
38+ # Follow logs (live)
39+ docker logs -f < container_id>
40+
41+ # View recent logs
42+ docker logs < container_id>
43+ ```
44+
45+ <Tip >
46+ To run attached and watch logs directly, replace ` -d ` with ` -it ` . Press ` Ctrl+C ` to stop.
47+ </Tip >
3348
3449 ** Environment Variables**
3550
@@ -46,26 +61,38 @@ Every open-source model on Bytez is available as a Docker image. Pull it, run it
4661 | ` --gpus all ` | Enable GPU acceleration (requires NVIDIA drivers + CUDA) |
4762 | ` -v /local/path:/server/model ` | Mount a local directory for weight caching |
4863 | ` -p HOST:CONTAINER ` | Map container port to host port |
64+
4965 </Accordion >
5066
5167 <Accordion title = " Common Configurations" icon = " gear" >
5268 ** Run on GPU**
5369
5470 ``` bash
55- docker run -it \
71+ docker run -d \
5672 --gpus all \
5773 -e KEY=YOUR_BYTEZ_KEY \
5874 -e PORT=8000 \
5975 -p 8000:8000 \
6076 bytez/qwen_qwen3-4b
6177 ```
6278
79+ ** Run on CPU**
80+
81+ ``` bash
82+ docker run -d \
83+ -e DEVICE=cpu \
84+ -e KEY=YOUR_BYTEZ_KEY \
85+ -e PORT=8000 \
86+ -p 8000:8000 \
87+ bytez/qwen_qwen3-4b
88+ ```
89+
6390 ** Cache Weights Locally**
6491
6592 Avoid re-downloading weights on every run by mounting a local directory:
6693
6794 ``` bash
68- docker run -it \
95+ docker run -d \
6996 --gpus all \
7097 -v /path/to/cache:/server/model \
7198 -e HF_HOME=/server/model \
@@ -79,16 +106,6 @@ Every open-source model on Bytez is available as a Docker image. Pull it, run it
79106 If you're going to create the same model container multiple times, then for large models (70B+), caching is highly recommended. Downloads can take hours otherwise.
80107 </Warning >
81108
82- ** Force CPU-Only**
83-
84- ``` bash
85- docker run -it \
86- -e DEVICE=cpu \
87- -e KEY=YOUR_BYTEZ_KEY \
88- -e PORT=8000 \
89- -p 8000:8000 \
90- bytez/qwen_qwen3-4b
91- ```
92109 </Accordion >
93110
94111 <Accordion title = " Run Inference" icon = " bolt" >
@@ -144,6 +161,7 @@ Every open-source model on Bytez is available as a Docker image. Pull it, run it
144161 <Card title = " Full HTTP Reference" icon = " book" href = " /http-reference/overview" >
145162 See complete request/response params and examples for all 30+ task types.
146163 </Card >
164+
147165 </Accordion >
148166
149167 <Accordion title = " Run Offline (Air-Gapped)" icon = " wifi-slash" >
@@ -152,15 +170,19 @@ Every open-source model on Bytez is available as a Docker image. Pull it, run it
152170 ** Step 1: Run the container once to download weights**
153171
154172 ``` bash
155- docker run \
173+ docker run -d \
156174 -e KEY=YOUR_BYTEZ_KEY \
157175 -e PORT=8000 \
158176 -p 8000:8000 \
159177 --name my-model \
160178 bytez/qwen_qwen3-4b
161179 ```
162180
163- Wait for the model to fully load (you'll see logs indicating it's ready). Then press ` Ctrl+C ` to stop the container.
181+ Wait for the model to fully load (check with ` docker logs -f my-model ` ). Once ready, stop it:
182+
183+ ``` bash
184+ docker stop my-model
185+ ```
164186
165187 ** Step 2: Save as a new image**
166188
@@ -171,7 +193,7 @@ Every open-source model on Bytez is available as a Docker image. Pull it, run it
171193 ** Step 3: Run offline (no internet needed)**
172194
173195 ``` bash
174- docker run \
196+ docker run -d \
175197 -e KEY=YOUR_BYTEZ_KEY \
176198 -e PORT=8000 \
177199 -p 8000:8000 \
@@ -191,6 +213,7 @@ Every open-source model on Bytez is available as a Docker image. Pull it, run it
191213 # Load on another machine
192214 docker load -i my-model-offline.tar
193215 ```
216+
194217 </Accordion >
195218
196219 <Accordion title = " Troubleshooting" icon = " wrench" >
@@ -209,6 +232,7 @@ Every open-source model on Bytez is available as a Docker image. Pull it, run it
209232 ** Model only works with specific DEVICE setting**
210233
211234 Some models only support ` auto ` , ` cuda ` , or ` cpu ` . If one doesn't work, try another.
235+
212236 </Accordion >
213237</AccordionGroup >
214238
@@ -220,4 +244,4 @@ Every open-source model on Bytez is available as a Docker image. Pull it, run it
220244 <Card title = " Discord" icon = " discord" href = " https://discord.com/invite/Z723PfCFWf" >
221245 Get live support from the community
222246 </Card >
223- </CardGroup >
247+ </CardGroup >
0 commit comments