Skip to content

Commit da7401d

Browse files
committed
Improve docker docs
1 parent edd4446 commit da7401d

1 file changed

Lines changed: 45 additions & 21 deletions

File tree

docs/model-api/docs/containers.mdx

Lines changed: 45 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
2-
title: "Containers"
3-
description: "Run open models locally, offline, and on edge devices"
4-
icon: "docker"
2+
title: 'Containers'
3+
description: 'Run open models locally, offline, and on edge devices'
4+
icon: 'docker'
55
---
66

77
Every open-source model on Bytez is available as a Docker image. Pull it, run it, and make requests to `localhost`.
@@ -18,18 +18,33 @@ Every open-source model on Bytez is available as a Docker image. Pull it, run it
1818
<Tip>
1919
Find model IDs at [bytez.com/models](https://bytez.com/models) or via the [List Models API](/http-reference/list/models).
2020
</Tip>
21+
2122
</Accordion>
2223

2324
<Accordion title="Start a Container" icon="play">
2425
```bash
25-
docker run -it \
26+
docker run -d \
2627
-e KEY=YOUR_BYTEZ_KEY \
2728
-e PORT=8000 \
2829
-p 8000:8000 \
2930
bytez/qwen_qwen3-4b
3031
```
3132

32-
Get your API key at [bytez.com/settings](https://bytez.com/settings).
33+
This runs the container in the background. Get your API key at [bytez.com/api/key](https://bytez.com/api/key).
34+
35+
**View logs**
36+
37+
```bash
38+
# Follow logs (live)
39+
docker logs -f <container_id>
40+
41+
# View recent logs
42+
docker logs <container_id>
43+
```
44+
45+
<Tip>
46+
To run attached and watch logs directly, replace `-d` with `-it`. Press `Ctrl+C` to stop.
47+
</Tip>
3348

3449
**Environment Variables**
3550

@@ -46,26 +61,38 @@ Every open-source model on Bytez is available as a Docker image. Pull it, run it
4661
| `--gpus all` | Enable GPU acceleration (requires NVIDIA drivers + CUDA) |
4762
| `-v /local/path:/server/model` | Mount a local directory for weight caching |
4863
| `-p HOST:CONTAINER` | Map container port to host port |
64+
4965
</Accordion>
5066

5167
<Accordion title="Common Configurations" icon="gear">
5268
**Run on GPU**
5369

5470
```bash
55-
docker run -it \
71+
docker run -d \
5672
--gpus all \
5773
-e KEY=YOUR_BYTEZ_KEY \
5874
-e PORT=8000 \
5975
-p 8000:8000 \
6076
bytez/qwen_qwen3-4b
6177
```
6278

79+
**Run on CPU**
80+
81+
```bash
82+
docker run -d \
83+
-e DEVICE=cpu \
84+
-e KEY=YOUR_BYTEZ_KEY \
85+
-e PORT=8000 \
86+
-p 8000:8000 \
87+
bytez/qwen_qwen3-4b
88+
```
89+
6390
**Cache Weights Locally**
6491

6592
Avoid re-downloading weights on every run by mounting a local directory:
6693

6794
```bash
68-
docker run -it \
95+
docker run -d \
6996
--gpus all \
7097
-v /path/to/cache:/server/model \
7198
-e HF_HOME=/server/model \
@@ -79,16 +106,6 @@ Every open-source model on Bytez is available as a Docker image. Pull it, run it
79106
If you're going to create the same model container multiple times, then for large models (70B+), caching is highly recommended. Downloads can take hours otherwise.
80107
</Warning>
81108

82-
**Force CPU-Only**
83-
84-
```bash
85-
docker run -it \
86-
-e DEVICE=cpu \
87-
-e KEY=YOUR_BYTEZ_KEY \
88-
-e PORT=8000 \
89-
-p 8000:8000 \
90-
bytez/qwen_qwen3-4b
91-
```
92109
</Accordion>
93110

94111
<Accordion title="Run Inference" icon="bolt">
@@ -144,6 +161,7 @@ Every open-source model on Bytez is available as a Docker image. Pull it, run it
144161
<Card title="Full HTTP Reference" icon="book" href="/http-reference/overview">
145162
See complete request/response params and examples for all 30+ task types.
146163
</Card>
164+
147165
</Accordion>
148166

149167
<Accordion title="Run Offline (Air-Gapped)" icon="wifi-slash">
@@ -152,15 +170,19 @@ Every open-source model on Bytez is available as a Docker image. Pull it, run it
152170
**Step 1: Run the container once to download weights**
153171

154172
```bash
155-
docker run \
173+
docker run -d \
156174
-e KEY=YOUR_BYTEZ_KEY \
157175
-e PORT=8000 \
158176
-p 8000:8000 \
159177
--name my-model \
160178
bytez/qwen_qwen3-4b
161179
```
162180

163-
Wait for the model to fully load (you'll see logs indicating it's ready). Then press `Ctrl+C` to stop the container.
181+
Wait for the model to fully load (check with `docker logs -f my-model`). Once ready, stop it:
182+
183+
```bash
184+
docker stop my-model
185+
```
164186

165187
**Step 2: Save as a new image**
166188

@@ -171,7 +193,7 @@ Every open-source model on Bytez is available as a Docker image. Pull it, run it
171193
**Step 3: Run offline (no internet needed)**
172194

173195
```bash
174-
docker run \
196+
docker run -d \
175197
-e KEY=YOUR_BYTEZ_KEY \
176198
-e PORT=8000 \
177199
-p 8000:8000 \
@@ -191,6 +213,7 @@ Every open-source model on Bytez is available as a Docker image. Pull it, run it
191213
# Load on another machine
192214
docker load -i my-model-offline.tar
193215
```
216+
194217
</Accordion>
195218

196219
<Accordion title="Troubleshooting" icon="wrench">
@@ -209,6 +232,7 @@ Every open-source model on Bytez is available as a Docker image. Pull it, run it
209232
**Model only works with specific DEVICE setting**
210233

211234
Some models only support `auto`, `cuda`, or `cpu`. If one doesn't work, try another.
235+
212236
</Accordion>
213237
</AccordionGroup>
214238

@@ -220,4 +244,4 @@ Every open-source model on Bytez is available as a Docker image. Pull it, run it
220244
<Card title="Discord" icon="discord" href="https://discord.com/invite/Z723PfCFWf">
221245
Get live support from the community
222246
</Card>
223-
</CardGroup>
247+
</CardGroup>

0 commit comments

Comments
 (0)