feat(stablediffusion-ggml): distributed RPC across ggml workers

Enable the ggml RPC backend (-DSD_RPC=ON) so image generation can be
sharded across remote rpc-server workers. The ggml rpc-server is
backend-agnostic, so this reuses the exact same worker pool as the
llama.cpp backend - one set of `local-ai worker llama-cpp-rpc` /
`p2p-llama-cpp-rpc` workers accelerates both text and image generation.

RPC servers are selected by precedence:
- the explicit `rpc_servers` option, else
- the LLAMACPP_GRPC_SERVERS env var, which LocalAI's p2p worker mode
  populates automatically with discovered workers (the backend inherits
  it from the parent process env), so distributed image generation needs
  no per-model configuration.

Documented manual and p2p setup in the image-generation guide.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]

Author: mudler

backend/go/stablediffusion-ggml/Makefile

@@ -12,6 +12,12 @@ STABLEDIFFUSION_GGML_VERSION?=5a34bc7f6e0621dd2f899daa64476eac667d7ed3
 
 CMAKE_ARGS+=-DGGML_MAX_NAME=128
 
+# Enable the ggml RPC backend so generation can be sharded across remote
+# rpc-server workers (the same backend-agnostic ggml rpc-server used by the
+# llama.cpp backend). Servers are selected via the `rpc_servers` option or the
+# LLAMACPP_GRPC_SERVERS env var (populated automatically in p2p worker mode).
+CMAKE_ARGS+=-DSD_RPC=ON
+
 ifeq ($(NATIVE),false)
 	CMAKE_ARGS+=-DGGML_NATIVE=OFF
 endif

backend/go/stablediffusion-ggml/cpp/gosd.cpp

@@ -638,7 +638,18 @@ int load_model(const char *model, char *model_path, char* options[], int threads
     if (keep_control_net_on_cpu) prepend_spec(backend_spec, "controlnet=cpu");
     if (!backend_spec.empty()) ctx_params.backend = backend_spec.c_str();
     if (!params_backend_spec.empty()) ctx_params.params_backend = params_backend_spec.c_str();
-    if (strlen(rpc_servers_arg) > 0) ctx_params.rpc_servers = rpc_servers_arg;
+    // RPC servers: prefer the explicit option, otherwise fall back to the
+    // LLAMACPP_GRPC_SERVERS env var. LocalAI's p2p worker mode populates that
+    // var with discovered ggml rpc-server workers (shared with the llama.cpp
+    // backend), so distributed image generation works with no extra config.
+    if (strlen(rpc_servers_arg) > 0) {
+        ctx_params.rpc_servers = rpc_servers_arg;
+    } else {
+        const char* env_rpc_servers = std::getenv("LLAMACPP_GRPC_SERVERS");
+        if (env_rpc_servers != NULL && strlen(env_rpc_servers) > 0) {
+            ctx_params.rpc_servers = env_rpc_servers;
+        }
+    }
     // max_vram: GiB budget or per-backend spec for graph-cut segmented param
     // offload ("0" = disabled, "-1" = auto). stream_layers only has effect when
     // max_vram is set.

docs/content/features/image-generation.md

@@ -107,6 +107,27 @@ options:
 `vae_decode_only` is still accepted for backwards compatibility but is now a no-op: upstream removed the flag and the model decides automatically.
 {{% /alert %}}
 
+#### Distributed inference (RPC workers)
+
+The `stablediffusion-ggml` backend can offload computation to remote `ggml` RPC workers, sharding a model that does not fit on a single machine. It reuses the **same backend-agnostic `rpc-server` workers as the llama.cpp backend**, so one worker pool can serve both.
+
+**Manual:** point the model at running workers with the `rpc_servers` option:
+
+```yaml
+options:
+- "rpc_servers:192.168.1.10:50052,192.168.1.11:50052"
+```
+
+Start a worker on each remote machine the same way you would for llama.cpp:
+
+```bash
+local-ai worker llama-cpp-rpc --llama-cpp-args="--host 0.0.0.0 --port 50052"
+```
+
+**Automatic (peer-to-peer):** when LocalAI runs in [p2p worker mode]({{%relref "features/distributed_inferencing" %}}), discovered workers are published in the `LLAMACPP_GRPC_SERVERS` environment variable. The image-generation backend reads that variable automatically (when `rpc_servers` is not set), so the same `local-ai worker p2p-llama-cpp-rpc` workers used for text generation also accelerate image generation - no per-model configuration needed.
+
+By default the RPC devices join the pool and participate in placement; combine with the `backend` / `params_backend` options above to pin specific components to them (e.g. `backend:diffusion=rpc0`).
+
 
 ### Diffusers