You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
LocalAI supports distributing inference workloads across multiple machines. There are two approaches, each suited to different use cases:
9
+
10
+
## Distributed Mode (PostgreSQL + NATS)
11
+
12
+
Production-grade horizontal scaling with centralized management. Frontends are stateless LocalAI instances behind a load balancer; workers self-register and receive backends dynamically via NATS. State lives in PostgreSQL.
13
+
14
+
**Best for:** production deployments, Kubernetes, managed infrastructure.
Peer-to-peer networking via libp2p. Share a token to form a cluster with automatic discovery — no central server required. Supports federated load balancing and worker-mode weight sharding.
21
+
22
+
**Best for:** ad-hoc clusters, community sharing, quick experimentation.
0 commit comments