Skip to content

Commit 2cca702

Browse files
TimPietruskyMeptl
TimPietrusky
and
Meptl
authored
feat: network-volume; execution time config; skip default images; access ComfyUI via web (runpod-workers#35)
* feat: provide option to run the handler locally as API * ci: run the workflow on our extended instance * feat: the local API should run on 0.0.0.0 * feat: make the image smaller * ci: use semantic-version to create releases automatically * chore: we don't want to break anyone with a minor release * docs: added section for local API testing * ci: use custom runner * fix: added .releaserc, otherwise semantic-release will complain about a missing "package.json" * feat: support network volumes, skip default models (runpod-workers#16) * Support network volumes * README tweaks * docs: added comment on what is happening * feat: don't overwrite the default paths, but add "runpod_worker_comfy" to have additional paths * docs: updated "bring your own models" --------- Co-authored-by: Tim Pietrusky <timpietrusky@gmail.com> * feat: provide access to ComfyUI via web * fix: use the full path to the output image * feat: added env vars COMFY_POLLING_INTERVAL_MS and COMFY_POLLING_MAX_RETRIES * test: added "subfolder" * docs: update CUDA guide for Ubuntu * feat(network-volume): added "custom_nodes" * chore: removed lora from civitai * docs: removed lora from civitai * docs: simplified multiple sections; added an image to describe how to find the endpointID * feat: remove "custom_nodes" * feat: move models before python installation * feat: added "runpod-volume"; use "dev" instead of "latest" * docs: removed "custom nodes"; fixed links to GitHub Actions * docs: fixed typo in GitHub * docs: fixed hosts for local testing * docs: updated link to docs from RunPod to create a network volume * feat: expose the port of ComfyUI * docs: fixed example link to download sdxl-turbo; removed nodes from network volume * docs: moved example response into their own block * chore: added example workflows for webp and sdxl-turbo --------- Co-authored-by: Meptl <git-client@meptl.com>
1 parent 7d8195a commit 2cca702

10 files changed

Lines changed: 422 additions & 87 deletions

File tree

‎Dockerfile‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,12 @@ RUN git clone https://github.com/comfyanonymous/ComfyUI.git /comfyui
2424
# Change working directory to ComfyUI
2525
WORKDIR /comfyui
2626

27+
ARG SKIP_DEFAULT_MODELS
28+
# Download checkpoints/vae/LoRA to include in image.
29+
RUN if [ -z "$SKIP_DEFAULT_MODELS" ]; then wget -O models/checkpoints/sd_xl_base_1.0.safetensors https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors; fi
30+
RUN if [ -z "$SKIP_DEFAULT_MODELS" ]; then wget -O models/vae/sdxl_vae.safetensors https://huggingface.co/stabilityai/sdxl-vae/resolve/main/sdxl_vae.safetensors; fi
31+
RUN if [ -z "$SKIP_DEFAULT_MODELS" ]; then wget -O models/vae/sdxl-vae-fp16-fix.safetensors https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/resolve/main/sdxl_vae.safetensors; fi
32+
2733
# Install ComfyUI dependencies
2834
RUN pip3 install --no-cache-dir torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 \
2935
&& pip3 install --no-cache-dir xformers==0.0.21 \
@@ -32,14 +38,8 @@ RUN pip3 install --no-cache-dir torch torchvision torchaudio --index-url https:/
3238
# Install runpod
3339
RUN pip3 install runpod requests
3440

35-
# Download checkpoints/vae/LoRA to include in image
36-
RUN wget -O models/checkpoints/sd_xl_base_1.0.safetensors https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors
37-
RUN wget -O models/vae/sdxl_vae.safetensors https://huggingface.co/stabilityai/sdxl-vae/resolve/main/sdxl_vae.safetensors
38-
RUN wget -O models/vae/sdxl-vae-fp16-fix.safetensors https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/resolve/main/sdxl_vae.safetensors
39-
40-
# Example for adding specific models into image
41-
# ADD models/checkpoints/sd_xl_base_1.0.safetensors models/checkpoints/
42-
# ADD models/vae/sdxl_vae.safetensors models/vae/
41+
# Support for the network volume
42+
ADD src/extra_model_paths.yaml ./
4343

4444
# Go back to the root
4545
WORKDIR /

‎README.md‎

Lines changed: 185 additions & 68 deletions
Large diffs are not rendered by default.

‎assets/my-endpoint-with-endpointID.png‎

22.1 KB
Loading

‎docker-compose.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,15 @@ version: "3.8"
22

33
services:
44
comfyui:
5-
image: timpietruskyblibla/runpod-worker-comfy:latest
5+
image: timpietruskyblibla/runpod-worker-comfy:dev
66
container_name: comfyui-worker
77
environment:
88
- NVIDIA_VISIBLE_DEVICES=all
99
- SERVE_API_LOCALLY=true
1010
ports:
1111
- "8000:8000"
12+
- "8188:8188"
1213
runtime: nvidia
1314
volumes:
1415
- ./data/comfyui/output:/comfyui/output
16+
- ./data/runpod-volume:/runpod-volume

‎src/extra_model_paths.yaml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
runpod_worker_comfy:
2+
base_path: /runpod-volume
3+
checkpoints: models/checkpoints/
4+
clip: models/clip/
5+
clip_vision: models/clip_vision/
6+
configs: models/configs/
7+
controlnet: models/controlnet/
8+
embeddings: models/embeddings/
9+
loras: models/loras/
10+
upscale_models: models/upscale_models/
11+
vae: models/vae/

‎src/rp_handler.py‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -14,9 +14,9 @@
1414
# Maximum number of API check attempts
1515
COMFY_API_AVAILABLE_MAX_RETRIES = 500
1616
# Time to wait between poll attempts in milliseconds
17-
COMFY_POLLING_INTERVAL_MS = 250
17+
COMFY_POLLING_INTERVAL_MS = os.environ.get("COMFY_POLLING_INTERVAL_MS", 250)
1818
# Maximum number of poll attempts
19-
COMFY_POLLING_MAX_RETRIES = 500
19+
COMFY_POLLING_MAX_RETRIES = os.environ.get("COMFY_POLLING_MAX_RETRIES", 500)
2020
# Host where ComfyUI is running
2121
COMFY_HOST = "127.0.0.1:8188"
2222
# Enforce a clean state after each job is done
@@ -237,7 +237,7 @@ def process_output_images(outputs, job_id):
237237
for node_id, node_output in outputs.items():
238238
if "images" in node_output:
239239
for image in node_output["images"]:
240-
output_images = f"{image['subfolder']}/{image['filename']}"
240+
output_images = os.path.join(image["subfolder"], image["filename"])
241241

242242
print(f"runpod-worker-comfy - image generation is done")
243243

‎src/start.sh‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,17 @@
44
TCMALLOC="$(ldconfig -p | grep -Po "libtcmalloc.so.\d" | head -n 1)"
55
export LD_PRELOAD="${TCMALLOC}"
66

7-
echo "runpod-worker-comfy: Starting ComfyUI"
8-
python3 /comfyui/main.py --disable-auto-launch --disable-metadata &
9-
10-
echo "runpod-worker-comfy: Starting RunPod Handler"
11-
127
# Serve the API and don't shutdown the container
138
if [ "$SERVE_API_LOCALLY" == "true" ]; then
9+
echo "runpod-worker-comfy: Starting ComfyUI"
10+
python3 /comfyui/main.py --disable-auto-launch --disable-metadata --listen &
11+
12+
echo "runpod-worker-comfy: Starting RunPod Handler"
1413
python3 -u /rp_handler.py --rp_serve_api --rp_api_host=0.0.0.0
1514
else
15+
echo "runpod-worker-comfy: Starting ComfyUI"
16+
python3 /comfyui/main.py --disable-auto-launch --disable-metadata &
17+
18+
echo "runpod-worker-comfy: Starting RunPod Handler"
1619
python3 -u /rp_handler.py
1720
fi

‎test_resources/workflows/workflow_sdxl_turbo.json‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
{
2+
"input": {
3+
"workflow": {
4+
"3": {
5+
"inputs": {
6+
"seed": 457699577674669,
7+
"steps": 3,
8+
"cfg": 1.5,
9+
"sampler_name": "euler_ancestral",
10+
"scheduler": "normal",
11+
"denoise": 1,
12+
"model": ["4", 0],
13+
"positive": ["6", 0],
14+
"negative": ["7", 0],
15+
"latent_image": ["5", 0]
16+
},
17+
"class_type": "KSampler",
18+
"_meta": {
19+
"title": "KSampler"
20+
}
21+
},
22+
"4": {
23+
"inputs": {
24+
"ckpt_name": "sd_xl_turbo_1.0_fp16.safetensors"
25+
},
26+
"class_type": "CheckpointLoaderSimple",
27+
"_meta": {
28+
"title": "Load Checkpoint"
29+
}
30+
},
31+
"5": {
32+
"inputs": {
33+
"width": 1024,
34+
"height": 1024,
35+
"batch_size": 1
36+
},
37+
"class_type": "EmptyLatentImage",
38+
"_meta": {
39+
"title": "Empty Latent Image"
40+
}
41+
},
42+
"6": {
43+
"inputs": {
44+
"text": "ancient rome, 4k photo",
45+
"clip": ["4", 1]
46+
},
47+
"class_type": "CLIPTextEncode",
48+
"_meta": {
49+
"title": "CLIP Text Encode (Prompt)"
50+
}
51+
},
52+
"7": {
53+
"inputs": {
54+
"text": "text, watermark, blurry, ugly, deformed",
55+
"clip": ["4", 1]
56+
},
57+
"class_type": "CLIPTextEncode",
58+
"_meta": {
59+
"title": "CLIP Text Encode (Prompt)"
60+
}
61+
},
62+
"8": {
63+
"inputs": {
64+
"samples": ["3", 0],
65+
"vae": ["4", 2]
66+
},
67+
"class_type": "VAEDecode",
68+
"_meta": {
69+
"title": "VAE Decode"
70+
}
71+
},
72+
"9": {
73+
"inputs": {
74+
"filename_prefix": "images/rome",
75+
"images": ["8", 0]
76+
},
77+
"class_type": "SaveImage",
78+
"_meta": {
79+
"title": "Save Image"
80+
}
81+
}
82+
}
83+
}
84+
}

‎test_resources/workflows/workflow_webp.json‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
{
2+
"input": {
3+
"workflow": {
4+
"3": {
5+
"inputs": {
6+
"seed": 416138702284529,
7+
"steps": 20,
8+
"cfg": 8,
9+
"sampler_name": "euler",
10+
"scheduler": "normal",
11+
"denoise": 1,
12+
"model": ["4", 0],
13+
"positive": ["6", 0],
14+
"negative": ["7", 0],
15+
"latent_image": ["5", 0]
16+
},
17+
"class_type": "KSampler",
18+
"_meta": {
19+
"title": "KSampler"
20+
}
21+
},
22+
"4": {
23+
"inputs": {
24+
"ckpt_name": "v1-5-pruned-emaonly.safetensors"
25+
},
26+
"class_type": "CheckpointLoaderSimple",
27+
"_meta": {
28+
"title": "Load Checkpoint"
29+
}
30+
},
31+
"5": {
32+
"inputs": {
33+
"width": 512,
34+
"height": 512,
35+
"batch_size": 1
36+
},
37+
"class_type": "EmptyLatentImage",
38+
"_meta": {
39+
"title": "Empty Latent Image"
40+
}
41+
},
42+
"6": {
43+
"inputs": {
44+
"text": "beautiful scenery nature glass bottle landscape, purple galaxy bottle,",
45+
"clip": ["4", 1]
46+
},
47+
"class_type": "CLIPTextEncode",
48+
"_meta": {
49+
"title": "CLIP Text Encode (Prompt)"
50+
}
51+
},
52+
"7": {
53+
"inputs": {
54+
"text": "text, watermark",
55+
"clip": ["4", 1]
56+
},
57+
"class_type": "CLIPTextEncode",
58+
"_meta": {
59+
"title": "CLIP Text Encode (Prompt)"
60+
}
61+
},
62+
"8": {
63+
"inputs": {
64+
"samples": ["3", 0],
65+
"vae": ["4", 2]
66+
},
67+
"class_type": "VAEDecode",
68+
"_meta": {
69+
"title": "VAE Decode"
70+
}
71+
},
72+
"10": {
73+
"inputs": {
74+
"output_path": "output_path",
75+
"filename_prefix": "filename_prefix",
76+
"filename_delimiter": "___",
77+
"filename_number_padding": 4,
78+
"filename_number_start": "false",
79+
"extension": "webp",
80+
"quality": 10,
81+
"lossless_webp": "false",
82+
"overwrite_mode": "false",
83+
"show_history": "false",
84+
"show_history_by_prefix": "false",
85+
"embed_workflow": "false",
86+
"show_previews": "false",
87+
"images": ["8", 0]
88+
},
89+
"class_type": "Image Save",
90+
"_meta": {
91+
"title": "Image Save"
92+
}
93+
},
94+
"11": {
95+
"inputs": {
96+
"images": ["8", 0]
97+
},
98+
"class_type": "PreviewImage",
99+
"_meta": {
100+
"title": "Preview Image"
101+
}
102+
},
103+
"12": {
104+
"inputs": {
105+
"images": ["8", 0]
106+
},
107+
"class_type": "PreviewImage",
108+
"_meta": {
109+
"title": "Preview Image"
110+
}
111+
}
112+
}
113+
}
114+
}

‎tests/test_rp_handler.py‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -131,7 +131,9 @@ def test_bucket_endpoint_not_configured(self, mock_upload_image, mock_exists):
131131
mock_exists.return_value = True
132132
mock_upload_image.return_value = "simulated_uploaded/image.png"
133133

134-
outputs = {"node_id": {"images": [{"filename": "ComfyUI_00001_.png", "subfolder": ""}]}}
134+
outputs = {
135+
"node_id": {"images": [{"filename": "ComfyUI_00001_.png", "subfolder": ""}]}
136+
}
135137
job_id = "123"
136138

137139
result = rp_handler.process_output_images(outputs, job_id)
@@ -188,7 +190,9 @@ def test_bucket_image_upload_fails_env_vars_wrong_or_missing(
188190
# When AWS credentials are wrong or missing, upload_image should return 'simulated_uploaded/...'
189191
mock_upload_image.return_value = "simulated_uploaded/image.png"
190192

191-
outputs = {"node_id": {"images": [{"filename": "ComfyUI_00001_.png", "subfolder": "test"}]}}
193+
outputs = {
194+
"node_id": {"images": [{"filename": "ComfyUI_00001_.png", "subfolder": ""}]}
195+
}
192196
job_id = "123"
193197

194198
result = rp_handler.process_output_images(outputs, job_id)

0 commit comments

Comments
 (0)