EN_K8s_Operations

Kubernetes Operations

16. How Does Kubernetes Graceful Shutdown Work?

One-line answer: On Pod termination, the kube-apiserver sends SIGTERM and waits terminationGracePeriodSeconds (default 30s) for the preStop hook and connection draining; if it times out, it force-kills with SIGKILL. Meanwhile the Pod is removed from endpoints, giving zero-downtime shutdown.

In Kubernetes, configuring Graceful Shutdown and handling SIGTERM and SIGKILL signals play an important role in managing resources and preventing data loss when containers are terminated. Each concept is explained as follows:

terminationGracePeriodSeconds

When a Pod is terminated, Kubernetes sends a SIGTERM signal to the Pod and waits before forcefully terminating the Pod with SIGKILL.

terminationGracePeriodSeconds is an option to set the Grace Period, which gives the container that received SIGTERM time to terminate gracefully.
The default value is 30 seconds, and if the container is not terminated within this time, Kubernetes forcefully sends a SIGKILL to terminate the container.
This setting allows services to safely perform cleanup operations (e.g., closing connections, saving state) without interrupting requests.

spec:
  terminationGracePeriodSeconds: 60 # Give the Pod a grace period of 60 seconds before receiving a SIGKILL.

preStop Hook

preStop is a Kubernetes Lifecycle Hook that specifies a command or script to be executed when a Pod is terminated. When a Pod receives a SIGTERM signal, this preStop hook is executed first, and after the configured actions are completed, the container starts its termination process.

You can use this hook to perform tasks such as notifying external services or cleaning up logs before termination.
The preStop hook runs before the SIGTERM signal is processed, allowing some preparation before the container starts its Graceful Shutdown.

lifecycle:
  preStop:
    exec:
      command: ["/bin/sh", "-c", "sleep 10"] # Wait 10 seconds before terminating the container.

By using the preStop hook to delay for 10 seconds, the container waits before it begins handling SIGTERM. After this time, it can still use the remainder of terminationGracePeriodSeconds to finish cleanup tasks.

SIGTERM (Terminate Signal)

SIGTERM is a signal that sends a graceful shutdown request to a container or process.
Applications that support Graceful Shutdown need to capture and handle this signal, which can trigger tasks such as closing open connections or completing the current work.
In Kubernetes, SIGTERM is sent first when a Pod is terminated. Upon receiving this signal, the application can proceed with cleanup operations for a graceful termination.

SIGKILL (Kill Signal)

SIGKILL is a force kill signal that immediately terminates a process or container.
This signal stops the process without allowing for cleanup operations, which can result in data loss or broken connections.
In Kubernetes, if a container is not terminated within terminationGracePeriodSeconds, SIGKILL is sent to force the container to terminate.

Graceful Shutdown Flow Summary

When a Pod termination request occurs, Kubernetes first sends a SIGTERM signal to the corresponding Pod.
If a preStop hook is defined, it is executed, preparing the application for the SIGTERM signal.
The application processes the SIGTERM signal within terminationGracePeriodSeconds, performing any necessary cleanup tasks.
If the container is not terminated within terminationGracePeriodSeconds, Kubernetes sends a SIGKILL to force the container to terminate.

graph LR
    A[Pod termination request] --> B[SIGTERM sent to Pod]
    B --> C{preStop hook defined?}
    C -- Yes --> D[Execute preStop hook]
    D --> E[SIGTERM signal handled by application]
    C -- No --> E
    E --> F[Graceful shutdown within terminationGracePeriodSeconds]
    F -->|Success| G[Pod terminates cleanly]
    F -->|Timeout| H[SIGKILL sent to force termination]

17. What Are the Three imagePullPolicy Values?

One-line answer: Always pulls from the registry every time, IfNotPresent prefers the local cache, and Never uses only local images. The default is Always when the tag is :latest or omitted, and IfNotPresent for any other fixed tag.

In Kubernetes, imagePullPolicy is a setting that controls how and when container images are pulled from the container registry. This is configured for each container in the Pod specification and determines whether Kubernetes should pull the image from the registry or use a locally cached version. There are three main imagePullPolicy values:

Always
IfNotPresent
Never

Always

Kubernetes always pulls the image from the registry whenever a Pod is created, even if the image already exists on the node.
This is useful if you frequently update images in your container registry without changing the image tag (e.g., using the 'latest' tag or a fixed tag for continuous deployment).

IfNotPresent

Kubernetes pulls the image from the registry only if the image is not yet on the node. If the image exists locally, it uses the cached version.
This is the default policy if a specific image tag (other than 'latest') is used. It helps reduce the time and bandwidth required to pull the image if the same image is already available on the node.

Never

Kubernetes does not pull the image from the registry and expects the image to already exist on the node.
This is useful in environments where you manually preload container images on nodes or want to avoid pulling images from external sources.

Preferences

Use the latest tag: When you use the latest tag, Kubernetes defaults to setting the image pull policy to Always.
Use other tags: If you specify a tag that is not the latest tag, Kubernetes defaults to IfNotPresent.

18. What Are the Kubernetes Deployment Strategies (RollingUpdate vs Recreate)?

One-line answer: RollingUpdate (default) replaces Pods incrementally via maxUnavailable/maxSurge for zero-downtime deploys, while Recreate terminates all existing Pods before starting new ones — brief downtime but no version mixing.

In Kubernetes, deployment strategies define how to update a Pod (or set of Pods) when deploying a new application version or container image. There are two main deployment strategies:

RollingUpdate (default strategy)
Recreate

RollingUpdate (default)

RollingUpdate is the default and most widely used strategy in Kubernetes. It gradually updates Pods within a deployment.
During a rolling update, Kubernetes creates new versions of Pods while simultaneously terminating older Pods in small batches. Downtime is minimal as some Pods continue to run during the process.
This strategy is ideal if you need to maintain high availability.

Key configuration options:

maxUnavailable: Specifies the maximum number (or percentage) of Pods that can be unavailable during the update. For example, maxUnavailable: 25% means that up to 25% of Pods may go down during an update.
maxSurge: Specifies the maximum number (or percentage) of additional Pods that can be temporarily created beyond the desired number of replicas. For example, maxSurge: 1 means that one additional Pod can be created temporarily during an update.
Advantage: Updates and deploys new Pods incrementally, so there is no service downtime.
Use case: Ideal for production environments that require continuous availability during updates.

spec:
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
      maxSurge: 1

Recreate

In the Recreate strategy, Kubernetes deletes all existing Pods before creating new ones.
This strategy ensures that old applications are fully stopped when a new version is deployed.
Advantages: Simple; terminates all existing instances before cleanly deploying the new version.
Disadvantages: All Pods are terminated until new Pods are created, resulting in downtime.
Use case: Suitable for applications where short downtime is acceptable or where the previous version must be fully stopped before deployment.

spec:
  strategy:
    type: Recreate

Infrastructure Operations Q&A (Q22-Q30)

Q22. How do you automate etcd Compaction and Defragmentation?

etcd stores all change history via revision-based MVCC, so its disk grows. Compaction deletes old revisions, and Defragmentation reclaims the actual disk space.

Run periodically via CronJob:

① Check DB size with etcdctl endpoint status →
② Delete revisions older than 3 hours with etcdctl compact →
③ Defrag all members with etcdctl defrag →
④ Raise etcd_quota_backend_bytes to 8GB. Enable automatic compaction with the --auto-compaction-retention=1h option.

Monitor etcd_mvcc_db_total_size_in_bytes with Prometheus.

Q22-1. What is the Kubernetes Control Plane HA setup and etcd quorum strategy?

etcd needs a quorum (majority) via the Raft consensus algorithm. 3 nodes (tolerates 1 failure) and 5 nodes (tolerates 2 failures) are common; even numbers are discouraged as they tolerate fewer failures than odd.
Run multiple kube-apiservers and distribute across a LoadBalancer.
Configure kube-scheduler and kube-controller-manager in Active-Standby via leader election.
For multi-AZ placement, put etcd across an odd number of AZs and choose AZs with low network latency.
Back up etcd periodically with etcdctl snapshot save and store to S3.

Q23. What is the practical configuration strategy for a Pod Disruption Budget (PDB)?

A PDB guarantees a minimum number of available Pods during voluntary disruptions.
minAvailable vs maxUnavailable: minAvailable=2 always keeps at least 2, maxUnavailable=1 allows at most 1 to be disrupted.
Set critical services to minAvailable=replicas-1 to allow only minimal disruption. During a Drain, if the PDB would be violated, Pods are not evicted and the Drain is blocked.
In that case, force-delete with the --disable-eviction option or temporarily modify the PDB. During a cluster upgrade, drain nodes sequentially considering the PDB.

Q24. What are the QoS classes of Resource Requests vs Limits and how do you resolve OOMKilled?

QoS classes:

① Guaranteed (Requests=Limits) - highest protection,
② Burstable (Requests<Limits) - medium protection,
③ BestEffort (unset) - terminated first. On memory pressure, the OOM Killer terminates the lowest QoS class first.

OOMKilled resolution:

① Set Requests based on actual usage (reference VPA recommendations) →
② Set Limits with headroom to allow spikes →
③ Set namespace defaults with LimitRange →
④ Cap total resources with ResourceQuota.
Set the JVM with -XX:MaxRAMPercentage so it recognizes the container memory.

Q25. What is the difference between Cluster Autoscaler and Karpenter, and how do you choose?

Cluster Autoscaler scales by Node Group and is slow, checking every 10 seconds after detecting Pending Pods.

It uses only specific instance types, and downscale occurs after 10 minutes.

Karpenter looks at Pod requirements (CPU, memory, GPU) and immediately provisions the optimal instance.

It provides Spot/On-Demand mixing, automatic selection of diverse instance types, resource optimization via Consolidation, and fast scaling (seconds).
CA is recommended for small scale, Karpenter for large/complex workloads.

Q29. How do you use Admission Webhooks (Mutating/Validating) in practice, and how do you implement them?

Admission Webhooks perform validation/transformation on resource creation/modification.
A Mutating Webhook modifies resources (sidecar injection, label addition), and a Validating Webhook validates them (naming rules, security policies).

Practical uses:

① Automatic sidecar injection by Istio/Linkerd →
② Policy validation by OPA Gatekeeper →
③ Secret injection by Vault →
④ Custom validation (per-team namespace rules). Implementation: a webhook server (Go/Python) + TLS certificate + MutatingWebhookConfiguration/ValidatingWebhookConfiguration resources. failurePolicy: Ignore (allow on failure) vs Fail (deny on failure).

Q30. What are the Kubernetes API server's Rate Limiting and Priority & Fairness?

API Priority & Fairness (APF, K8s 1.20+) classifies client requests and handles them fairly.
A FlowSchema classifies requests, and a PriorityLevelConfiguration defines the concurrency and queue length.
It assigns priorities such as system-leader-election (high), workload-high (medium), workload-low (low).
Rate Limiting is controlled by --max-requests-inflight (default 400) and --max-mutating-requests-inflight (default 200).
In large clusters, List requests can overload the API server, so increase the Informer resync period and use pagination.

Q31. (Real case) On ALB + Karpenter, how did you fix nodes never being reclaimed after `rollout restart`?

One-line answer: It was not a single bug but the interaction of three settings (NodePool policy, deploy strategy, resource request); I guarded both ends of ALB register/deregister to stay zero-downtime while reclaiming the leftover node.

① Symptom

A single kubectl rollout restart added one node that was never reclaimed after the rollout finished.

② Root cause (interacting settings)

NodePool consolidationPolicy: WhenEmpty → a node with even one Pod left is never reclaimed.
maxSurge: 1 + oversized memory request (2Gi vs. measured idle ~101Mi, ~20x) → the surge Pod wakes a new node, and after the rollout Pods straddle the old and new nodes → neither is empty → it persists.

③ Debunk "sessions get dropped" first

The app is stateless HTTP (Fastify) with sessions in Redis → session STATE survives a Pod kill.
The real cause is not session loss but 502s from ALB still routing to a dying Pod.

④ Fix — guard both ends of ALB register/deregister (interview key: "each mechanism guards a different window")

Mechanism	Window guarded	Role
preStop `sleep 35` + `grace 60`	shutdown (Pod leaving)	Keep Pod alive until ALB deregister completes → preserve in-flight
`deregistration_delay=30` (ingress)	shutdown	Cut draining from default 300s → 30s (requires `target-type: ip`)
Readiness Gate (namespace label)	startup (Pod entering)	Hold new Pod NotReady until its ALB target is healthy → block premature cutover
RollingUpdate strategy	replace pace	Small `maxSurge:1` (+1 node) / Large `maxSurge:0 + maxUnavailable:20~25%` (zero node growth). maxSurge as absolute value (never percent)
PDB `minAvailable:1` (replica≥2)	availability	Keep Ready Pods above 0 during consolidation/evict

⑤ Timing invariants that must hold

preStop(35) > ALB dereg(30) → if the Pod dies before dereg ends, 502 again.
grace(60) > preStop(35) → so preStop isn't truncated.
grace − preStop(25) ≥ app shutdown time → headroom to clean up after SIGTERM.

⑥ Root prevention

Switch NodePool to WhenEmptyOrUnderutilized + consolidateAfter: 5m (actively reclaim spare nodes; 5m avoids churn).
Right-size memory request 2Gi → 512Mi → better bin-packing removes the straddle itself.

⑦ Verification (for follow-up questions)

Health loop (curl every 0.2s) confirms 0 × 502 during rollout/restart.
Confirm the readiness gate holds new Pods NotReady until healthy.
Track node count returning to baseline after consolidateAfter.
Classify per-domain responses with curl → most 404s and SPA catch-all 200s are intended behavior; isolate only the real failures.

Caveat: preStop is best-effort — guaranteed only for voluntary disruptions (rolling/drain/consolidation), not hard crashes / SIGKILL paths. Watch for distroless images lacking /bin/sh.

Q32. (Real case) How did you measure and set the three timing values (`preStop` / `grace` / ALB `dereg`) for Graceful Shutdown, and how did you verify them?

One-line answer: The three numbers come from measurement, not intuition — ⓐ ALB deregistration_delay is the lower bound for preStop; ⓑ/ⓒ the app's SIGTERM behavior and drain (p99) are the lower bound for grace. After setting them you verify each window independently, and the final pass criterion is "0 × 502 during rollout". (Q31 is why you guard each window; Q32 is how many seconds — measured and verified.)

① Termination order and the back-calculation formula

t=0    Pod Terminating → removed from Service Endpoints + preStop sleep starts
       (Pod stays alive and keeps serving existing requests)
t=P    preStop ends → SIGTERM delivered to container PID 1
t=G    grace expires → SIGKILL force-kill

preStop sleep   ≥ ALB deregistration_delay (+ slack)   ← drive new-request inflow to 0
grace           > preStop + app drain (p99) + slack     ← finish in-flight requests

Prerequisite: ingress must already have target-type: ip + deregistration_delay.

② Measure to set the values (before deploy)

ⓐ ALB deregistration_delay (decision value) — "time for new requests to a draining target to stop." Default 300s, but can be shortened without HTTP keepalive → decided 30s here (= lower bound for preStop). Annotation alb.ingress.kubernetes.io/target-group-attributes: deregistration_delay.timeout_seconds=30.
ⓑ Is the app graceful on SIGTERM? — make a slow response in-flight on a standalone container, send SIGTERM, see whether it finishes 200 or is cut off.
```
curl -s -o /dev/null -w '%{http_code} %{time_total}s\n' localhost:8080/<slow-api> &
sleep 1; kill -15 1; wait        # 200=graceful / 000·Empty reply=cut off
```
Note: the container's default sh (dash/busybox) can't take kill -SIGTERM. Always use kill -15 / kill -TERM.
ⓒ How many seconds is the app drain? (only when graceful, p99-based) — give grace generous headroom, then measure "SIGTERM received → last request completed" under load. Back-calculate preStop from p99, not worst-case.
```
hey -z 90s -c 20 https://game.example.com/health &
kubectl -n game rollout restart deployment/game
kubectl -n game logs -l app.kubernetes.io/name=game --prefix | grep -i 'SIGTERM\|shutdown\|closed'
```

Derivation:

preStop sleep = ALB dereg + slack          = 30 + 5  = 35
grace         > preStop + app drain + slack = 35 + drain + buffer

③ Per-runtime SIGTERM behavior (preStop alone can still leave 502s)

Runtime	On SIGTERM	Default drain	What to do
ASP.NET Core (.NET 5+)	auto graceful drain	~30s	mostly OK as-is (3.x or lower: 5s)
Spring Boot	exits immediately by default	~30s when enabled	needs `server.shutdown=graceful`
Node.js / Go	exits immediately	none	preStop is the only line of defense (app handler recommended)
Python (gunicorn/uvicorn)	graceful	~30s	generally OK
nginx / PHP-FPM	fast exit → in-flight cut off	—	`STOPSIGNAL SIGQUIT` switch required

In particular nginx reads SIGTERM as a fast shutdown and drops in-flight requests → use STOPSIGNAL SIGQUIT or preStop nginx -s quit. (The official nginx image has defaulted STOPSIGNAL to SIGQUIT since 2020-11.)

④ Verify the values (after deploy) — inspect each window independently. A single total-delete time cannot separate preStop from grace.

Target	Method	Pass criterion
preStop 35s	after `delete --wait=false`, on the terminating Pod run `ps -ef \| grep '[s]leep 35'`	present 0~35s, gone afterward
grace 60s	`time kubectl -n game delete pod $POD`	total ≈ 60s (SIGKILL ceiling)
app SIGTERM	`kubectl exec $POD -- sh -c 'kill -15 1; sleep 5; kill -0 1 && echo IGNORED \|\| echo EXITED'` + RESTARTS	IGNORED / EXITED verdict
ALB dereg 30s	`aws elbv2 describe-target-health --target-group-arn <TG>`	podIP `draining` → removed within ~30s
end-to-end (zero-502)	`hey -z 90s -c 20 https://game.example.com/health` during `kubectl rollout restart`	5xx = 0 ← final pass

The total of time kubectl delete only shows the grace ceiling (SIGKILL). To isolate preStop(35) you must watch the sleep 35 process living inside the terminating Pod.

POD=$(kubectl -n game get pod -l app.kubernetes.io/name=game -o name | head -1 | cut -d/ -f2)
kubectl -n game delete pod $POD --wait=false
kubectl -n game exec $POD -- ps -ef | grep '[s]leep 35'   # visible 0~35s / gone after

⑤ Measured result (example, 2026-06-24) — deployed values grace 60 / preStop sleep 35 / ALB dereg 30.

Measurement	Command	Result	Meaning
grace ceiling	`time kubectl delete pod`	62s	grace 60 + overhead = SIGKILL ceiling
app SIGTERM	`kill -15 1` + RESTARTS	IGNORED, RESTARTS=0	app ignores SIGTERM (no handler)

t=0    Terminating + preStop sleep 35 starts
t=35   preStop ends → SIGTERM delivered  ← app ignores it
t=60   grace expires → SIGKILL
─────  total ≈ 62s  (not a preStop failure — SIGTERM ignored, so it waits out the grace ceiling)

⑥ Conclusion

preStop 35 > ALB dereg 30 → ✅ 502 prevention (Pod survives until traffic drains) works via this inequality; new requests = 0.
grace 60 > preStop 35 → ✅ valid but currently oversized. Since the app ignores SIGTERM, t=35~60 (25s) is dead time waiting for SIGKILL (harmless to 502, but delete always takes 60s).
For that 60s to mean something, add a SIGTERM handler to the app (e.g. Fastify enableShutdownHooks + graceful close) → drain ⓒ appears and grace is justified as 35 + drain + slack < 60. If not adding one, cut grace to 40s for faster teardown (no impact on correctness).

One-line summary: measurement (ⓑ/ⓒ) gives you the app drain that fixes the grace number; verification (④/⑤) confirms it runs that way and that 502 = 0. The example above has "app drain = 0 (SIGTERM ignored)", so preStop=35 alone blocks 502s and grace=60 remains as headroom for a future handler.

Q33. (Real case) A blue-green Argo Rollout threw ELB 503s on every deploy while the app was healthy — what caused it and how did you fix it?

One-line answer: The 503s came from the ALB itself (ELB 5xx > 0 while Target 5xx = 0), because blue-green's all-at-once active-target-group swap momentarily left the group with 0 healthy targets. Switching to a stepless canary (maxSurge:1 / maxUnavailable:0, no trafficRouting) keeps ready ≥ desired, so the active TG is never 0-healthy — the 503 race is gone. (Q31/Q32 make an individual pod swap safe; Q33 makes the sequence of swaps never empty the target group.)

① The signal that localizes the fault (ALB vs app)

Signal	During a deploy	Meaning
`HTTPCode_ELB_5XX_Count`	22 in one minute	The ALB generated the 5xx
`HTTPCode_Target_5XX_Count`	0	No backend pod ever returned a 5xx
`RequestCount`	~200/min, flat	Not overload — normal traffic

ELB 5xx > 0 while Target 5xx = 0 means the ALB had no healthy target to route to at that instant — never an app bug. It was also non-deterministic (one deploy 22×503, the next 0×) → the fingerprint of a race, not a static misconfiguration.

② Why blue-green races to 0 healthy targets

Blue-green keeps the old ReplicaSet (blue) on the active TG while the new one (green) comes up in a preview TG. On promotion it flips the active Service selector blue → green all-at-once. But green must now register + pass health checks in the active TG all over again. If blue drains out before green finishes registering there, the active TG is momentarily empty → the ALB emits 503 directly.

[blue ×2 active] → promote (selector flip) → ⚠️ blue drained, green not yet
                                              registered-here → 0 healthy → 503
                                            → [green ×2 active] → 503s stop

prePromotionAnalysis does not help — it gates whether to promote, not the atomicity of the swap.

③ The decision — can the service tolerate mixed traffic?

Canary replaces the all-at-once swap with a pod-by-pod one, so old + new serve real traffic at the same time for a few seconds. That is only safe if a request hitting either version is equivalent:

Property	This service	Mixed-traffic safe?
Protocol	Stateless HTTP (Fastify)	Yes — no long-lived connection
WebSocket / gRPC stream	None	Yes — no connection affinity
Auth	JWT (stateless)	Yes — any pod validates any token
Session / state	Externalized to Redis + DB	Yes — no in-pod state to pin to

Standing requirement (blue-green already demanded it too): every deploy must be backward-compatible across versions (no removed field, no incompatible payload, no destructive migration). A genuinely breaking change is instead shipped as a deliberate maintenance-window cutover.

④ The fix — a stepless canary

strategy:
  canary:
    maxSurge: 1        # add one new pod ABOVE desired first
    maxUnavailable: 0  # never let ready drop below desired

maxUnavailable: 0 + a single active TG = the group is never 0-healthy. maxSurge: 1 makes it add-before-remove, one pod at a time. No steps, no trafficRouting.

[old ×2] ready=2 → [old ×2][new ×1] ready=2 → [old ×1][new ×1] ready=2
        → [old ×1][new ×2] ready=2 → [new ×2] ready=2   (never below 2)

Backward-compat when this is a shared chart: make rollout.strategy a switch defaulting to blueGreen (every other service re-renders unchanged), and gate the blue-green preview scaffolding (preview Service + preview Ingress/TG) on strategy == blueGreen so canary stops rendering — and prunes — it.

⑤ Why no weight-based traffic splitting

	Stepless canary (used)	Weighted canary + trafficRouting
trafficRouting	None	Required (ALB / NGINX / SMI)
steps / setWeight	None	5% → 25% → 50% → …
Traffic split	Not controlled — incidental old:new ratio, always rolls to 100%	Controlled % — holds at each weight to observe / gate / roll back
Goal	Gap-free replacement (kill the 503 race)	Progressive, analysis-gated delivery
Target groups	1 (active)	2 (stable + canary)

setWeight only works with trafficRouting; without a traffic router there is no weight to set. Weight is for progressive delivery (send 5%, watch, advance) — a different goal than "eliminate the 0-healthy race". Adding it would be extra machinery for a problem we didn't have.

⑥ Verification — forced a rolling restart through the canary strategy and watched the metrics:

Signal	Result
Pods	Replaced one at a time; old pods drained gracefully (preStop)
Ready / desired	Dipped to 1 briefly, never 0
`HTTPCode_ELB_5XX_Count`	0 across the whole window
`HTTPCode_Target_5XX_Count`	0

Nuance: a restart recreates pods in the same ReplicaSet (delete-then-recreate), so ready can dip to 1 — but the draining pod keeps serving (preStop > dereg, Q32), so still no 503. A new-image deploy surges a new ReplicaSet pod with maxUnavailable:0 strictly honored, so ready never drops below desired at all.

⑦ Conclusion

ELB 5xx up + Target 5xx flat = the load balancer had no healthy target, not an app failure.
Blue-green's all-at-once swap opens a 0-healthy window; it is timing-dependent and immune to prePromotionAnalysis.
A stepless canary (maxSurge:1 / maxUnavailable:0) keeps the single active TG at ≥ desired ready → gap-free, no 503, minimal config.
Weight-based canary is a different tool for progressive, analysis-gated delivery — reach for it when you need to bleed a rollout in slowly, not when you just need it gap-free.

EN_K8s_Operations

Kubernetes Operations

16. How Does Kubernetes Graceful Shutdown Work?

terminationGracePeriodSeconds

preStop Hook

SIGTERM (Terminate Signal)

SIGKILL (Kill Signal)

Graceful Shutdown Flow Summary

17. What Are the Three imagePullPolicy Values?

Always

IfNotPresent

Never

Preferences

18. What Are the Kubernetes Deployment Strategies (RollingUpdate vs Recreate)?

RollingUpdate (default)

Recreate

Infrastructure Operations Q&A (Q22-Q30)

Q22. How do you automate etcd Compaction and Defragmentation?

Run periodically via CronJob:

Q22-1. What is the Kubernetes Control Plane HA setup and etcd quorum strategy?

Q23. What is the practical configuration strategy for a Pod Disruption Budget (PDB)?

Q24. What are the QoS classes of Resource Requests vs Limits and how do you resolve OOMKilled?

QoS classes:

OOMKilled resolution:

Q25. What is the difference between Cluster Autoscaler and Karpenter, and how do you choose?

Q29. How do you use Admission Webhooks (Mutating/Validating) in practice, and how do you implement them?

Practical uses:

Q30. What are the Kubernetes API server's Rate Limiting and Priority & Fairness?

Q31. (Real case) On ALB + Karpenter, how did you fix nodes never being reclaimed after rollout restart?

Q32. (Real case) How did you measure and set the three timing values (preStop / grace / ALB dereg) for Graceful Shutdown, and how did you verify them?

Q33. (Real case) A blue-green Argo Rollout threw ELB 503s on every deploy while the app was healthy — what caused it and how did you fix it?

Reference

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Q31. (Real case) On ALB + Karpenter, how did you fix nodes never being reclaimed after `rollout restart`?

Q32. (Real case) How did you measure and set the three timing values (`preStop` / `grace` / ALB `dereg`) for Graceful Shutdown, and how did you verify them?