v0.3.33-NetworkVolume

davefojtik · davefojtik · commit e7c90d9d7511 · 2024-04-09T00:15:00.000+02:00
Update Release compatible with Fooocus-API v0.3.33 Changelog: - Added preview streaming based on #11 - Added inpainting methods based on #13 - Updated and refractored code - Changed used model to Juggernaut V8 See docs/request_examples.js to learn how to use the new params
diff --git a/Dockerfile_NetworkEndpoint b/Dockerfile_NetworkEndpoint
@@ -1,8 +1,8 @@
 # Base image
 FROM runpod/pytorch:2.1.0-py3.10-cuda11.8.0-devel-ubuntu22.04
 
-ENV PIP_PREFER_BINARY=1 \
-    LD_PRELOAD=libtcmalloc.so \
+ENV DEBIAN_FRONTEND=noninteractive \
+    PIP_PREFER_BINARY=1 \
     PYTHONUNBUFFERED=1
 SHELL ["/bin/bash", "-o", "pipefail", "-c"]
 
diff --git a/Dockerfile_NetworkSetup b/Dockerfile_NetworkSetup
@@ -1,4 +1,4 @@
-FROM alpine:latest
+FROM alpine:3.19.1
 RUN apk add --no-cache git curl
 
 COPY builder/clone.sh /clone.sh
diff --git a/README.md b/README.md
@@ -1,4 +1,4 @@
-![Static Badge](https://img.shields.io/badge/API_version-0.3.30-blue) ![Static Badge](https://img.shields.io/badge/API_coverage-100%25-vividgreen) ![Static Badge](https://img.shields.io/badge/API_tests-passed-vividgreen) ![Static Badge](https://img.shields.io/badge/Known_bugs-0-vividgreen) ![Static Badge](https://img.shields.io/badge/Fooocus_version-2.1.862-lightgrey)
+![Static Badge](https://img.shields.io/badge/API_version-0.3.33-blue) ![Static Badge](https://img.shields.io/badge/API_coverage-100%25-vividgreen) ![Static Badge](https://img.shields.io/badge/API_tests-passed-vividgreen) ![Static Badge](https://img.shields.io/badge/Fooocus_version-2.3.0-lightgrey)
 
 # RunPod-Fooocus-API
 
@@ -8,15 +8,15 @@ This repository consists of two branches:
 ![image](https://github.com/davefojtik/RunPod-Fooocus-API/assets/66263283/88d74dd7-2dcd-44a8-af01-f1ce29bfb713)
 
 
-The **NetworkVolume** expects you to install and prepare your own `Fooocus-API v0.3.30` instance on the RunPod network volume, or to use our `3wad/runpod-fooocus-api:0.3.30-networksetup` image. This is ideal if you want to change models, loras or other contents on the fly. The downside of this solution is slower starts, especially when the endpoint is not used frequently. See [network-guide](https://github.com/davefojtik/RunPod-Fooocus-API/blob/NetworkVolume/docs/network-guide.md) for step by step instructions.
+The **NetworkVolume** expects you to install and prepare your own `Fooocus-API v0.3.33` instance on the RunPod network volume, or to use our `3wad/runpod-fooocus-api:0.3.33-networksetup` image. This is ideal if you want to change models, loras or other contents on the fly. The downside of this solution is slower starts, especially when the endpoint is not used frequently. See [network-guide](https://github.com/davefojtik/RunPod-Fooocus-API/blob/NetworkVolume/docs/network-guide.md) for step-by-step instructions.
 
-The **Standalone** branch is ready-to-use docker image with all the files and models already baked and installed into it. You can still customize it to use your own contents, but they can't be changed without rebuilding and redeploying the image. This is ideal if you want the fastest, cheapest possible endpoint for long-term usage without needs for frequent changes of models or loras. See [standalone-guide](https://github.com/davefojtik/RunPod-Fooocus-API/blob/Standalone/docs/standalone-guide.md) or simply use `3WaD/RunPod-Fooocus-API:v0.3.30-standalone` as the image for a quick "default" deploy on your RunPod serverless endpoint.
+The **Standalone** branch is a ready-to-use docker image with all the files and models already baked and installed into it. You can still customize it to use your own content, but it can't be changed without rebuilding and redeploying the image. This is ideal if you want the fastest, cheapest possible endpoint for long-term usage without the need for frequent changes of models or loras. See [standalone-guide](https://github.com/davefojtik/RunPod-Fooocus-API/blob/Standalone/docs/standalone-guide.md) or simply use `3WaD/RunPod-Fooocus-API:v0.3.33-standalone` as the image for a quick deploy with the default Juggernaut V8 on your RunPod serverless endpoint.
 
 ## How to send requests
 [request_examples.js](https://github.com/davefojtik/RunPod-Fooocus-API/blob/NetworkVolume/docs/request_examples.js) contain example payloads for all endpoints on your serverless worker, regardless of the branch. But don't hesitate to ask if you need more help.
 
 ## Contributors Welcomed
-Feel free to do pull requests, fixes, improvements and suggestions to the code. I can spend only limited time on this as it's a side project for our community discord bot. So any cooperation will help manage this repo better.
+Feel free to make pull requests, fixes, improvements and suggestions to the code. I can spend only limited time on this as it's only a side tool project for an AI Agent. So any cooperation will help manage this repo better.
 
 ## Updates
-The version of compatible Fooocus-API is always stated on the top of this readme. We're not always on the latest version automatically, as there can be breaking changes. The updates are being made only after troughrough tests on our community discord bot.
+The version of compatible Fooocus-API is always stated at the top of this readme. We're not always on the latest version automatically, as there can be breaking changes. The updates are being made only after thorough tests on our community discord bot, and only if we see that the new version performs better or solves some problems.
diff --git a/builder/requirements.txt b/builder/requirements.txt
@@ -3,7 +3,7 @@
 
 # Custom dependencies
 requests-toolbelt==1.0.0
-runpod==1.6.0
+runpod==1.6.2
 
 # Pulled from https://github.com/konieshadow/Fooocus-API/blob/main/requirements.txt
 torchsde==0.2.5
@@ -26,4 +26,9 @@ fastapi==0.103.1
 pydantic==2.4.2
 pydantic_core==2.10.1
 python-multipart==0.0.6
-uvicorn[standard]==0.23.2
+uvicorn[standard]==0.23.2
+colorlog
+requests
+numpy
+sqlalchemy
+packaging
diff --git a/docs/network-guide.md b/docs/network-guide.md
@@ -1,12 +1,12 @@
 ## How to prepare Network Volume
 - [**Create RunPod network volume:**](https://www.runpod.io/console/user/storage)
   20GB is just enough for the generic Foocus with Juggernaut and **all** controlnet models (you can save some space by not downloading those you don't plan to use - by modifying the [script](https://github.com/davefojtik/RunPod-Fooocus-API/NetworkVolume/src/networksetup.sh) and making your own setup image). You can increase its size any time if you need additional models, loras etc. But unfortunately, it cannot be reduced back without creating a new one.
-- [**Create a custom CPU Pod Template:**](https://www.runpod.io/console/user/templates) use the `3wad/runpod-fooocus-api:v0.3.30-networksetup` image,  20GB disk size and mount path `/workspace`.
+- [**Create a custom CPU Pod Template:**](https://www.runpod.io/console/user/templates) use the `3wad/runpod-fooocus-api:v0.3.33-networksetup` image,  20GB disk size and mount path `/workspace`. *Note: 20GB is max for CPU pods, if you need more space, you'll have to use GPU pod even for the installation*
 - [**Run a CPU pod:**](https://www.runpod.io/console/pods) with **network volume** and runpod-fooocus-api template you've just created. Any CPU pod will do, the installation is just download-intensive. After a while, you should see a "Setup complete!" message in Pod logs. After that, you can move to the serverless setup steps. 
 ---
-- Now you can use our premade image: `3wad/runpod-fooocus-api:0.3.30-networkendpoint` and skip the next step OR create your custom docker image from this repo that will run on the actual serverless API. Feel free to adjust the code to your needs.
+- Now you can use our premade image: `3wad/runpod-fooocus-api:0.3.33-networkendpoint` and skip the next step OR create your custom docker image from this repo that will run on the actual serverless API. Feel free to adjust the code to your needs.
 - *If you built your own image, upload it to the Docker Hub.*
 - [**Create a custom Serverless Pod Template:**](https://www.runpod.io/console/user/templates) using the Docker Hub image you've just uploaded (or our premade one).
-- [**Create a new Serverless API Endpoint:**](https://www.runpod.io/console/serverless) Make sure to choose your (or our) Docker Hub image and not the `3wad/runpod-fooocus-api:v0.3.30-networksetup` from the step 2. In Advanced settings choose your created network volume.
+- [**Create a new Serverless API Endpoint:**](https://www.runpod.io/console/serverless) Make sure to choose your (or our) Docker Hub image and not the `3wad/runpod-fooocus-api:v0.3.33-networksetup` from the step 2. In Advanced settings choose your created network volume.
 - Other settings are your choice, but I personally found that using 4090/L4 GPUs + Flashboot is the most cost-effective one.
-- That's it! See the [request_examples]() for how to make requests to this endpoint from your app.
+- That's it! See the [request_examples]() for how to make requests to this endpoint from your app.
diff --git a/docs/request_examples.js b/docs/request_examples.js
@@ -1,14 +1,20 @@
-// ########################
 // EXAMPLE REQUEST PAYLOADS
 // ########################
 // The requests can be done in several possible ways:
 // - Send the payload (with require_base64:true on img endpoints) to https://api.runpod.ai/v2/$your-endpont-id/runsync, in this case your app will simply send a request and wait for the final response. (This works fine if the requests don't take too long and you don't have enough of them to be queued)
 // - Send the payload (with require_base64:true on img endpoints) to https://api.runpod.ai/v2/$your-endpont-id/run, in this async case your app will send a request, immediately receive obj with runpod job id that you can then use to receive the final result on https://api.runpod.ai/v2/$your-endpoint-id/status/$id
 //   To get notified when the job is finished, you can either use the Fooocus-API webhook feature, or you can just periodically fetch the job's status endpoint
+// - Send the payload (with require_base64:true, async_process:true and preview_url:"https://your.app/endpoint") to https://api.runpod.ai/v2/$your-endpont-id/run, in this case your app's listening endpoint will receive POST requests with previews periodically and also the final img once it's finished. 
 // - You can also customize the handler.py and add your own logic at the end, for example to save the image files into some external storage.
 
+// Contents:
+// 1. Minimal required
+// 2. Full payload
+// 3. Custom params (async preview stream, ..)
+
+
 // ----------------------------------------------------
-// Part one: Minimal required
+// 1: Minimal required
 // ----------------------------------------------------
 
 // **************************
@@ -298,6 +304,7 @@ models_refresh =
         "api_name":"models-refresh",
     }
 }
+// WARNING! This endpoint is deprecated! Features are merged into all-models endpoint. This endpoint will be removed in next releases!
 // returns: {"delayTime": 0, "executionTime": 0, "id":"runpod-job-id", 'output':{"model_filenames": ["string"],"lora_filenames": ["string"]}, "status":"COMPLETED"}
 
 // ----------------
@@ -312,10 +319,8 @@ styles =
 // returns: {"delayTime": 0, "executionTime": 0, "id":"runpod-job-id", 'output':["string"], "status":"COMPLETED"}
 
 
-
-
 // ----------------------------------------------------
-// Part two: Full payload
+// 2: Full payload
 // ----------------------------------------------------
 // The basic idea of params on v1/v2 (multipart/form-data vs. application/json) endpoints is: 
 // on v2 params can be standard json format, on v1 they have to be sent stringified.
@@ -348,4 +353,17 @@ v2 =
             "adaptive_cfg":7
         }
     }
-}
+}
+
+// ----------------------------------------------------
+// 3: Custom params
+// ----------------------------------------------------
+// To overcome some differences which arise from running Fooocus on RunPod serverless
+// we have to add several custom parameters to our payloads that are not part of the original Fooocus-API
+
+"api_name" // "string" - Chooses what Fooocus-API endpoint we're actually calling
+"preview_url" // "string" - If you use Fooocus-API "async_process:true" and want to get a stream of preview images, you should add your app's url endpoint where POST requests with previews will arrive
+    "preview_interval" // "number" - Optional param when using preview_url, sets how often in seconds the preview is checked and sent to your app. If not set defaults to 1
+    "preview_headers" // "object" - Optional param when using preview_url, sets custom headers to send with the preview request (for tokens, auth etc.)
+"inpaint_preset" // "string" - Custom fix for missing "Inpaint Method" selection in Fooocus-API. Can be one of: 'Improve Detail', 'Modify Content' or 'Inpaint or Outpaint'
+"clear_output" // "boolean" - Chooses if you want to keep the image files on network volume/worker local storage or not. Can be true or false. Default is true (not saving image files)
diff --git a/src/config.txt b/src/config.txt
@@ -1,54 +1,121 @@
-
-"path_checkpoints": "\\workspace\\repositories\\Fooocus\\models\\checkpoints",
-"path_loras": "\\workspace\\repositories\\Fooocus\\models\\loras",
-"path_embeddings": "\\workspace\\repositories\\Fooocus\\models\\embeddings",
-"path_vae_approx": "\\workspace\\repositories\\Fooocus\\models\\vae_approx",
-"path_upscale_models": "\\workspace\\repositories\\Fooocus\\models\\upscale_models",
-"path_inpaint": "\\workspace\\repositories\\Fooocus\\models\\inpaint",
-"path_controlnet": "\\workspace\\repositories\\Fooocus\\models\\controlnet",
-"path_clip_vision": "\\workspace\\repositories\\Fooocus\\models\\clip_vision",
-"path_fooocus_expansion": "\\workspace\\repositories\\Fooocus\\models\\prompt_expansion\\fooocus_expansion",
-"path_outputs": "\\workspace\\repositories\\Fooocus\\outputs",
-"default_model": "juggernautXL_version6Rundiffusion.safetensors",
-"default_refiner": "None",
-"default_refiner_switch": 0.5,
-"default_loras": [
-    [
-        "sd_xl_offset_example-lora_1.0.safetensors",
-        0.1
+{
+    "path_checkpoints": [
+        "/workspace/repositories/Fooocus/models/checkpoints"
     ],
-    [
-        "None",
-        1.0
+    "path_loras": [
+        "/workspace/repositories/Fooocus/models/loras"
     ],
-    [
-        "None",
-        1.0
+    "path_embeddings": "/workspace/repositories/Fooocus/models/embeddings",
+    "path_vae_approx": "/workspace/repositories/Fooocus/models/vae_approx",
+    "path_upscale_models": "/workspace/repositories/Fooocus/models/upscale_models",
+    "path_inpaint": "/workspace/repositories/Fooocus/models/inpaint",
+    "path_controlnet": "/workspace/repositories/Fooocus/models/controlnet",
+    "path_clip_vision": "/workspace/repositories/Fooocus/models/clip_vision",
+    "path_fooocus_expansion": "/workspace/repositories/Fooocus/models/prompt_expansion/fooocus_expansion",
+    "path_wildcards": "/workspace/repositories/Fooocus/wildcards",
+    "path_outputs": "/workspace/repositories/Fooocus/outputs",
+    "temp_path": "/tmp/fooocus",
+    "temp_path_cleanup_on_launch": true,
+    "default_model": "juggernautXL_v8Rundiffusion.safetensors",
+    "previous_default_models": [
+        "juggernautXL_version8Rundiffusion.safetensors",
+        "juggernautXL_version7Rundiffusion.safetensors",
+        "juggernautXL_v7Rundiffusion.safetensors",
+        "juggernautXL_version6Rundiffusion.safetensors",
+        "juggernautXL_v6Rundiffusion.safetensors"
     ],
-    [
-        "None",
-        1.0
+    "default_refiner": "None",
+    "default_refiner_switch": 0.5,
+    "default_loras_min_weight": -2,
+    "default_loras_max_weight": 2,
+    "default_loras": [
+        [
+            true,
+            "sd_xl_offset_example-lora_1.0.safetensors",
+            0.1
+        ],
+        [
+            true,
+            "None",
+            1.0
+        ],
+        [
+            true,
+            "None",
+            1.0
+        ],
+        [
+            true,
+            "None",
+            1.0
+        ],
+        [
+            true,
+            "None",
+            1.0
+        ]
     ],
-    [
-        "None",
-        1.0
-    ]
-],
-"default_cfg_scale": 4.0,
-"default_sample_sharpness": 2.0,
-"default_sampler": "dpmpp_2m_sde_gpu",
-"default_scheduler": "karras",
-"default_styles": [
-    "Fooocus V2",
-    "Fooocus Enhance",
-    "Fooocus Sharp"
-],
-"default_prompt_negative": "",
-"default_prompt": "",
-"default_performance": "Speed",
-"default_advanced_checkbox": false,
-"default_max_image_number": 32,
-"default_image_number": 1,
-"checkpoint_downloads": {},
-"lora_downloads": {},
-"embeddings_downloads": {}
+    "default_max_lora_number": 5,
+    "default_cfg_scale": 4.0,
+    "default_sample_sharpness": 2.0,
+    "default_sampler": "dpmpp_2m_sde_gpu",
+    "default_scheduler": "karras",
+    "default_styles": [
+        "Fooocus V2",
+        "Fooocus Enhance",
+        "Fooocus Sharp"
+    ],
+    "default_prompt_negative": "",
+    "default_prompt": "",
+    "default_performance": "Speed",
+    "default_advanced_checkbox": false,
+    "default_max_image_number": 32,
+    "default_output_format": "png",
+    "default_image_number": 2,
+    "checkpoint_downloads": {},
+    "lora_downloads": {},
+    "embeddings_downloads": {},
+    "available_aspect_ratios": [
+        "704*1408",
+        "704*1344",
+        "768*1344",
+        "768*1280",
+        "832*1216",
+        "832*1152",
+        "896*1152",
+        "896*1088",
+        "960*1088",
+        "960*1024",
+        "1024*1024",
+        "1024*960",
+        "1088*960",
+        "1088*896",
+        "1152*896",
+        "1152*832",
+        "1216*832",
+        "1280*768",
+        "1344*768",
+        "1344*704",
+        "1408*704",
+        "1472*704",
+        "1536*640",
+        "1600*640",
+        "1664*576",
+        "1728*576"
+    ],
+    "default_aspect_ratio": "1152*896",
+    "default_inpaint_engine_version": "v2.6",
+    "default_cfg_tsnr": 7.0,
+    "default_overwrite_step": -1,
+    "default_overwrite_switch": -1,
+    "example_inpaint_prompts": [
+        "highly detailed face",
+        "detailed girl face",
+        "detailed man face",
+        "detailed hand",
+        "beautiful eyes"
+    ],
+    "default_save_metadata_to_images": false,
+    "default_metadata_scheme": "fooocus",
+    "metadata_created_by": ""
+}
diff --git a/src/default.json b/src/default.json
@@ -1,25 +1,30 @@
 {
-    "default_model": "juggernautXL_version6Rundiffusion.safetensors",
+    "default_model": "juggernautXL_v8Rundiffusion.safetensors",
     "default_refiner": "None",
     "default_refiner_switch": 0.5,
     "default_loras": [
         [
+            true,
             "sd_xl_offset_example-lora_1.0.safetensors",
             0.1
         ],
         [
+            true,
             "None",
             1.0
         ],
         [
+            true,
             "None",
             1.0
         ],
         [
+            true,
             "None",
             1.0
         ],
         [
+            true,
             "None",
             1.0
         ]
@@ -34,9 +39,17 @@
     "default_styles": [
         "Fooocus V2",
         "Fooocus Enhance",
-        "Fooocus Sharp"],
+        "Fooocus Sharp"
+    ],
     "default_aspect_ratio": "1152*896",
     "checkpoint_downloads": {},
     "embeddings_downloads": {},
-    "lora_downloads": {}
+    "lora_downloads": {},
+    "previous_default_models": [
+        "juggernautXL_version8Rundiffusion.safetensors",
+        "juggernautXL_version7Rundiffusion.safetensors",
+        "juggernautXL_v7Rundiffusion.safetensors",
+        "juggernautXL_version6Rundiffusion.safetensors",
+        "juggernautXL_v6Rundiffusion.safetensors"
+    ]
 }
diff --git a/src/handler.py b/src/handler.py
diff --git a/src/networksetup.sh b/src/networksetup.sh
diff --git a/src/start.sh b/src/start.sh

Original file line number	Diff line number	Diff line change
`@@ -1,4 +1,4 @@`
`1`		`-FROM alpine:latest`
	`1`	`+FROM alpine:3.19.1`
`2`	`2`	`RUN apk add --no-cache git curl`
`3`	`3`
`4`	`4`	`COPY builder/clone.sh /clone.sh`