Skip to content

chore: add an explicit nodeTaintsPolicy#206

Merged
savme merged 2 commits into
mainfrom
chore/nso-node-taint-honor
Jun 19, 2026
Merged

chore: add an explicit nodeTaintsPolicy#206
savme merged 2 commits into
mainfrom
chore/nso-node-taint-honor

Conversation

@savme

@savme savme commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Problem

The extension server Deployment uses a topologySpreadConstraints rule with
maxSkew: 1 across kubernetes.io/hostname. During a rolling update, the
strategy is maxSurge: 1 / maxUnavailable: 0, which temporarily requires one
more pod than the desired replica count (2 → 3).

On edge clusters that have a mix of general worker nodes and tainted
special-purpose nodes, the surge pod gets stuck Pending. Kubernetes counts
all nodes with the topology key when computing skew, including tainted nodes
the pod can never land on. With 2 workers (1 pod each) and 3 tainted nodes (0
pods each), the current skew is already 1. Adding the surge pod to either
worker would raise the skew to 2, exceeding maxSkew: 1.

The stuck pod causes the Deployment to enter ProgressDeadlineExceeded, which
in turn keeps the Flux nso-extension-server kustomization in a failing health
check loop. Flux retries every hour, and each retry triggers an Envoy Gateway
xDS re-translation. If the extension server is slow or returns a partial
response during that push, Envoy briefly serves a config with no TPP routes
applied — producing a ~1-minute spike of envoy_http_no_route errors on the
HTTPS listener.

Fix

Add nodeTaintsPolicy: Honor to the topology spread constraint. This tells
Kubernetes to exclude nodes with untolerated taints from the skew calculation.
With only the 2 schedulable workers counted, the surge pod sees a skew of 0 (1
vs 1) and can schedule freely (1+1=2 vs 1, skew=1 ≤ maxSkew).

Requirements

nodeTaintsPolicy is GA since Kubernetes 1.33. Verify edge cluster version before deploying.

@savme
savme merged commit 8b8cda1 into main Jun 19, 2026
11 checks passed
@savme
savme deleted the chore/nso-node-taint-honor branch June 19, 2026 14:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants