diff --git a/daprdocs/content/en/operations/hosting/kubernetes/kubernetes-production.md b/daprdocs/content/en/operations/hosting/kubernetes/kubernetes-production.md index 1b737f0665e..709f5d63102 100644 --- a/daprdocs/content/en/operations/hosting/kubernetes/kubernetes-production.md +++ b/daprdocs/content/en/operations/hosting/kubernetes/kubernetes-production.md @@ -122,6 +122,30 @@ To scale scheduler and placement to three instances independently of the `global --wait ``` +### Spreading Placement and Scheduler replicas + +With HA enabled, the default Helm values add pod anti-affinity so Scheduler and Placement service replicas are attempted to be scheduled across different failure domains if cluster resourcing allows. This is configured using the following Helm chart [options](https://github.com/dapr/dapr/blob/master/charts/dapr/README.md) + +- `global.ha.topologyKey`: Kubernetes label used meaning “different domain”. Set by default to `topology.kubernetes.io/zone`. +- `global.ha.podAntiAffinityPolicy`: Enables soft spreading with `preferredDuringSchedulingIgnoredDuringExecution` (the default) or hard spreading with `requiredDuringSchedulingIgnoredDuringExecution`. + +When running in production, consider the following recommendations. + +1. **Availability zones**: The default topology key spreads pods across zones. This limits the chance that a single zone outage removes multiple Scheduler or Placement pods at the same time. +2. **Single zone, many nodes**: Zone labels are often identical across the cluster; set `topologyKey` to `kubernetes.io/hostname` to enable spreading by node. +3. **Soft spreading**: The default behaviour is recommended here allowing pod scheduling to succeed when the cluster is short on zones or nodes. +4. **Hard spreading**: Only use when you must never place two replicas on the same topology value. This requires as many distinct values for `topologyKey` as replicas, or pods will stay `Pending`. + +For example, too configure hard spreading by hostname in a single-zone cluster, use the following Helm values: + +```yaml +global: + ha: + enabled: true + topologyKey: kubernetes.io/hostname + podAntiAffinityPolicy: requiredDuringSchedulingIgnoredDuringExecution +``` + ## Setting cluster critical priority class name for control plane services In some scenarios, nodes may have memory and/or cpu pressure and the Dapr control plane pods might get selected