Skip to content

Commit c9468fd

Browse files
committed
docs(distributed): note restart-safe reconciler behavior
Adds a bullet to the Replica Reconciler section explaining that per-model load metadata is persisted across frontend restarts via the new model_load_infos PostgreSQL table, so a rolling upgrade no longer needs a fresh inference request per model before the reconciler can replace dead replicas. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-7[1m]
1 parent 4221121 commit c9468fd

1 file changed

Lines changed: 1 addition & 0 deletions

File tree

docs/content/features/distributed-mode.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -490,6 +490,7 @@ The **Replica Reconciler** runs as a background process on the frontend:
490490
- **Scale down**: Removes idle replicas after 5 minutes of inactivity
491491
- **Maintain minimum**: Ensures `min_replicas` are always loaded (recovers from node failures)
492492
- **Eviction protection**: Models with auto-scaling enabled are never evicted below `min_replicas`
493+
- **Restart-safe**: Per-model load metadata (backend type + `ModelOptions`) is persisted in the `model_load_infos` PostgreSQL table on the first successful dispatch, so a frontend restart or rolling upgrade does not require a fresh inference request to repopulate state before the reconciler can scale up replacement replicas.
493494

494495
All fields are optional and composable:
495496
- Node selector only: pin model to matching nodes, single replica

0 commit comments

Comments
 (0)