Skip to content

Commit 11dc54b

Browse files
committed
fix(docs): commit distribution.md
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
1 parent 7e0b73d commit 11dc54b

1 file changed

Lines changed: 34 additions & 0 deletions

File tree

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
+++
2+
disableToc = false
3+
title = "Distribution"
4+
weight = 13
5+
url = "/features/distribution/"
6+
+++
7+
8+
LocalAI supports distributing inference workloads across multiple machines. There are two approaches, each suited to different use cases:
9+
10+
## Distributed Mode (PostgreSQL + NATS)
11+
12+
Production-grade horizontal scaling with centralized management. Frontends are stateless LocalAI instances behind a load balancer; workers self-register and receive backends dynamically via NATS. State lives in PostgreSQL.
13+
14+
**Best for:** production deployments, Kubernetes, managed infrastructure.
15+
16+
[Read more]({{% relref "features/distributed-mode" %}})
17+
18+
## P2P / Federated Inference
19+
20+
Peer-to-peer networking via libp2p. Share a token to form a cluster with automatic discovery — no central server required. Supports federated load balancing and worker-mode weight sharding.
21+
22+
**Best for:** ad-hoc clusters, community sharing, quick experimentation.
23+
24+
[Read more]({{% relref "features/distributed_inferencing" %}})
25+
26+
## Quick Comparison
27+
28+
| | P2P / Federation | Distributed Mode |
29+
|---|---|---|
30+
| **Discovery** | Automatic via libp2p token | Self-registration to frontend URL |
31+
| **State storage** | In-memory / ledger | PostgreSQL |
32+
| **Coordination** | Gossip protocol | NATS messaging |
33+
| **Node management** | Automatic | REST API + WebUI |
34+
| **Setup complexity** | Minimal (share a token) | Requires PostgreSQL + NATS |

0 commit comments

Comments
 (0)