@@ -27,15 +27,27 @@ Kubernetes upgrades are non-disruptive from Talos, but Kubelet upgrades may caus
2727
2828To trigger a Kubernetes upgrade, issue a command specifying the version of Kubernetes to upgrade to, such as:
2929
30- ` talosctl --nodes <controlplane node> upgrade-k8s --to ${k8s_release} `
30+ <CodeBlock lang = " sh" >
31+ {
32+ ` talosctl --nodes <control-plane-ip> upgrade-k8s --to <new-kubernetes-version> `
33+ }
34+ </CodeBlock >
35+
3136
3237Note that the ` --nodes ` parameter specifies the control plane node to send the API call to, but all members of the cluster will be upgraded.
3338
3439To check what will be upgraded you can run ` talosctl upgrade-k8s ` with the ` --dry-run ` flag:
3540
3641<CodeBlock lang = " sh" >
3742 { `
38- $ talosctl --nodes <controlplane node> upgrade-k8s --to ${k8s_release } --dry-run
43+ talosctl --nodes <control-plane-ip> upgrade-k8s --to ${k8s_release } --dry-run
44+ ` }
45+ </CodeBlock >
46+
47+ You should get an output similar to this:
48+
49+ <CodeBlock lang = " sh" >
50+ { `
3951WARNING: found resources which are going to be deprecated/migrated in the version ${k8s_release }
4052RESOURCE COUNT
4153validatingwebhookconfigurations.v1beta1.admissionregistration.k8s.io 4
@@ -71,30 +83,11 @@ updating manifests
7183 > apply manifest ClusterRoleBinding system-bootstrap-approve-node-client-csr
7284 > apply skipped in dry run
7385<snip>
74- ` }
75- </CodeBlock >
76-
77- To upgrade Kubernetes from ` ${k8s_prev_release} ` to v` ${k8s_release} ` run:
7886
79- <CodeBlock lang = " sh" >
80- { `
81- $ talosctl --nodes <controlplane node> upgrade-k8s --to ${k8s_release }
82- automatically detected the lowest Kubernetes version ${k8s_prev_release }
83- checking for resource APIs to be deprecated in version ${k8s_release }
84- discovered controlplane nodes ["172.20.0.2" "172.20.0.3" "172.20.0.4"]
85- discovered worker nodes ["172.20.0.5" "172.20.0.6"]
86- updating "kube-apiserver" to version "${k8s_release }"
87- > "172.20.0.2": starting update
88- > update kube-apiserver: ${k8s_prev_release } -> ${k8s_release }
89- > "172.20.0.2": machine configuration patched
90- > "172.20.0.2": waiting for API server state pod update
91- < "172.20.0.2": successfully updated
92- > "172.20.0.3": starting update
93- > update kube-apiserver: ${k8s_prev_release } -> ${k8s_release }
94- <snip>
9587 ` }
9688</CodeBlock >
9789
90+
9891This command runs in several phases:
9992
100931 . Images for new Kubernetes components are pre-pulled to the nodes to minimize downtime and test for image availability.
@@ -111,7 +104,7 @@ This command runs in several phases:
111104
112105If the command fails for any reason, it can be safely restarted to continue the upgrade process from the moment of the failure.
113106
114- > Note: When using custom/overridden Kubernetes component images, use flags ` --*-image ` to override the default image names.
107+ < Note > When using custom/overridden Kubernetes component images, use flags ` --*-image ` to override the default image names.</ Note >
115108
116109### Kubernetes manifest synchronization
117110
@@ -122,7 +115,7 @@ If any services were deleted or disabled, the resources associated with them wil
122115For example if [ kube-proxy] ( https://docs.siderolabs.com/talos/latest/reference/configuration/v1alpha1/config#proxy ) was disabled in the machine configuration the resources associated with it would be deleted at this stage.
123116Pruning can be disabled by passing the ` --manifests-no-prune ` flag.
124117
125- > Note: Pruning is supported from Talos and talosctl v1.13 onwards.
118+ < Note > Pruning is supported from Talos and talosctl v1.13 onwards.</ Note >
126119
127120From Talos v1.13 onwards, all Kubernetes manifests are applied via [ Kubernetes Server-Side Apply] ( https://kubernetes.io/docs/reference/using-api/server-side-apply/ ) .
128121Talos forces ownership of all fields it applies, even if they have a different field manager.
@@ -131,7 +124,7 @@ If you wish to manage a resource previously applied by Talos, you need to take t
1311241 . Remove the inline manifest or extra manifest entry from the machine configuration.
132125After this step you can run ` talosctl upgrade-k8s --dry-run --to <in-cluster-k8s-version> ` .
133126Resources affected will be marked for pruning in the output.
134- 2 . Remove the resource entries from the talos tracking inventory configmap data block:
127+ 2 . Remove the resource entries from the talos tracking inventory configmap data block by running :
135128
136129``` bash
137130kubectl edit cm --namespace kube-system talos-bootstrap-manifests-inventory
@@ -164,16 +157,15 @@ In order to edit the control plane, you need a working `kubectl` config.
164157If you don't already have one, you can get one by running:
165158
166159``` bash
167- talosctl --nodes < controlplane node > kubeconfig
160+ talosctl --nodes < control-plane-ip > kubeconfig
168161```
169162
170163### API server
171164
172165Patch machine configuration using ` talosctl patch ` command:
173166
174167``` bash
175- $ talosctl -n < CONTROL_PLANE_IP_1> patch mc --mode=no-reboot -p ' [{"op": "replace", "path": "/cluster/apiServer/image", "value": "registry.k8s.io/kube-apiserver:v${k8s_release}"}]'
176- patched mc at the node 172.20.0.2
168+ talosctl -n < control-plane-ip> patch mc --mode=no-reboot -p ' [{"op": "replace", "path": "/cluster/apiServer/image", "value": "registry.k8s.io/kube-apiserver:v${k8s_release}"}]'
177169```
178170
179171The JSON patch might need to be adjusted if current machine configuration is missing ` .cluster.apiServer.image ` key.
@@ -183,7 +175,12 @@ Also the machine configuration can be edited manually with `talosctl -n <IP> ed
183175Capture the new version of ` kube-apiserver ` config with:
184176
185177``` bash
186- $ talosctl -n < CONTROL_PLANE_IP_1> get apiserverconfig -o yaml
178+ talosctl -n < control-plane-ip> get apiserverconfig -o yaml
179+ ```
180+
181+ You should get an output similar to this:
182+
183+ ``` bash
187184node: 172.20.0.2
188185metadata:
189186 namespace: controlplane
@@ -217,16 +214,14 @@ In this example, the new version is `5`.
217214Wait for the new pod definition to propagate to the API server state (replace ` talos-default-controlplane-1 ` with the node name):
218215
219216``` bash
220- $ kubectl get pod -n kube-system -l k8s-app=kube-apiserver --field-selector spec.nodeName=talos-default-controlplane-1 -o jsonpath=' {.items[0].metadata.annotations.talos\.dev/config\-version}'
217+ kubectl get pod -n kube-system -l k8s-app=kube-apiserver --field-selector spec.nodeName=talos-default-controlplane-1 -o jsonpath=' {.items[0].metadata.annotations.talos\.dev/config\-version}'
2212185
222219```
223220
224221Check that the pod is running:
225222
226223``` bash
227- $ kubectl get pod -n kube-system -l k8s-app=kube-apiserver --field-selector spec.nodeName=talos-default-controlplane-1
228- NAME READY STATUS RESTARTS AGE
229- kube-apiserver-talos-default-controlplane-1 1/1 Running 0 16m
224+ kubectl get pod -n kube-system -l k8s-app=kube-apiserver --field-selector spec.nodeName=talos-default-controlplane-1
230225```
231226
232227Repeat this process for every control plane node, verifying that state got propagated successfully between each node update.
@@ -236,16 +231,20 @@ Repeat this process for every control plane node, verifying that state got propa
236231Patch machine configuration using ` talosctl patch ` command:
237232
238233``` bash
239- $ talosctl -n < CONTROL_PLANE_IP_1> patch mc --mode=no-reboot -p ' [{"op": "replace", "path": "/cluster/controllerManager/image", "value": "registry.k8s.io/kube-controller-manager:v${k8s_release}"}]'
240- patched mc at the node 172.20.0.2
234+ talosctl -n < control-plane-ip> patch mc --mode=no-reboot -p ' [{"op": "replace", "path": "/cluster/controllerManager/image", "value": "registry.k8s.io/kube-controller-manager:v${k8s_release}"}]'
241235```
242236
243237The JSON patch might need be adjusted if current machine configuration is missing ` .cluster.controllerManager.image ` key.
244238
245239Capture new version of ` kube-controller-manager ` config with:
246240
247241``` bash
248- $ talosctl -n < CONTROL_PLANE_IP_1> get controllermanagerconfig -o yaml
242+ talosctl -n < control-plane-ip> get controllermanagerconfig -o yaml
243+ ```
244+
245+ You should see an output similar to this:
246+
247+ ``` yaml
249248node : 172.20.0.2
250249metadata :
251250 namespace : controlplane
@@ -270,22 +269,19 @@ spec:
270269 cpu : " "
271270 memory : " "
272271 limits : {}
273- ```
272+ ` ` `
274273
275274In this example, new version is ` 3`.
276275Wait for the new pod definition to propagate to the API server state (replace `talos-default-controlplane-1` with the node name) :
277276
278277` ` ` bash
279- $ kubectl get pod -n kube-system -l k8s-app=kube-controller-manager --field-selector spec.nodeName=talos-default-controlplane-1 -o jsonpath=' {.items[0].metadata.annotations.talos\.dev/config\-version}'
280- 3
278+ kubectl get pod -n kube-system -l k8s-app=kube-controller-manager --field-selector spec.nodeName=talos-default-controlplane-1 -o jsonpath='{.items[0].metadata.annotations.talos\. dev/config\- version}'
281279` ` `
282280
283281Check that the pod is running :
284282
285283` ` ` bash
286- $ kubectl get pod -n kube-system -l k8s-app=kube-controller-manager --field-selector spec.nodeName=talos-default-controlplane-1
287- NAME READY STATUS RESTARTS AGE
288- kube-controller-manager-talos-default-controlplane-1 1/1 Running 0 35m
284+ kubectl get pod -n kube-system -l k8s-app=kube-controller-manager --field-selector spec.nodeName=talos-default-controlplane-1
289285` ` `
290286
291287Repeat this process for every control plane node, verifying that state propagated successfully between each node update.
@@ -295,16 +291,20 @@ Repeat this process for every control plane node, verifying that state propagate
295291Patch machine configuration using `talosctl patch` command :
296292
297293` ` ` bash
298- $ talosctl -n < CONTROL_PLANE_IP_1> patch mc --mode=no-reboot -p ' [{"op": "replace", "path": "/cluster/scheduler/image", "value": "registry.k8s.io/kube-scheduler:v${k8s_release}"}]'
299- patched mc at the node 172.20.0.2
294+ talosctl -n <control-plane-ip> patch mc --mode=no-reboot -p '[{"op": "replace", "path": "/cluster/scheduler/image", "value": "registry.k8s.io/kube-scheduler:v${k8s_release}"}]'
300295` ` `
301296
302297JSON patch might need be adjusted if current machine configuration is missing `.cluster.scheduler.image` key.
303298
304299Capture new version of `kube-scheduler` config with :
305300
306301` ` ` bash
307- $ talosctl -n < CONTROL_PLANE_IP_1> get schedulerconfig -o yaml
302+ talosctl -n <control-plane-ip> get schedulerconfig -o yaml
303+ ` ` `
304+
305+ You should see an output similar to this :
306+
307+ ` ` ` yaml
308308node: 172.20.0.2
309309metadata:
310310 namespace: controlplane
@@ -328,21 +328,18 @@ spec:
328328 limits: {}
329329 config: {}
330330` ` `
331-
332331In this example, new version is `3`.
332+
333333Wait for the new pod definition to propagate to the API server state (replace `talos-default-controlplane-1` with the node name) :
334334
335335` ` ` bash
336- $ kubectl get pod -n kube-system -l k8s-app=kube-scheduler --field-selector spec.nodeName=talos-default-controlplane-1 -o jsonpath=' {.items[0].metadata.annotations.talos\.dev/config\-version}'
337- 3
336+ kubectl get pod -n kube-system -l k8s-app=kube-scheduler --field-selector spec.nodeName=talos-default-controlplane-1 -o jsonpath='{.items[0].metadata.annotations.talos\. dev/config\- version}'
338337` ` `
339338
340339Check that the pod is running :
341340
342341` ` ` bash
343- $ kubectl get pod -n kube-system -l k8s-app=kube-scheduler --field-selector spec.nodeName=talos-default-controlplane-1
344- NAME READY STATUS RESTARTS AGE
345- kube-scheduler-talos-default-controlplane-1 1/1 Running 0 39m
342+ kubectl get pod -n kube-system -l k8s-app=kube-scheduler --field-selector spec.nodeName=talos-default-controlplane-1
346343` ` `
347344
348345Repeat this process for every control plane node, verifying that state got propagated successfully between each node update.
@@ -397,7 +394,7 @@ kubectl edit daemonsets -n kube-system kube-proxy
397394Bootstrap manifests can be retrieved in a format which works for `kubectl` with the following command :
398395
399396` ` ` bash
400- talosctl -n <controlplane IP > get manifests -o yaml | yq eval-all '.spec | .[] | splitDoc' - > manifests.yaml
397+ talosctl -n <control-plane-ip > get manifests -o yaml | yq eval-all '.spec | .[] | splitDoc' - > manifests.yaml
401398` ` `
402399
403400Diff the manifests with the cluster :
@@ -412,21 +409,18 @@ Apply the manifests:
412409kubectl apply -f manifests.yaml
413410` ` `
414411
415- > Note: if some bootstrap resources were removed, they have to be removed from the cluster manually.
412+ <Note>If some bootstrap resources were removed, they have to be removed from the cluster manually.</Note>
416413
417414# ## kubelet
418415
419416For every node, patch machine configuration with new kubelet version, wait for the kubelet to restart with new version :
420417
421418` ` ` bash
422- $ talosctl -n <IP> patch mc --mode=no-reboot -p '[{"op": "replace", "path": "/machine/kubelet/image", "value": "ghcr.io/siderolabs/kubelet:v${k8s_release}"}]'
423- patched mc at the node 172.20.0.2
419+ talosctl -n <control-plane-ip> patch mc --mode=no-reboot -p '[{"op": "replace", "path": "/machine/kubelet/image", "value": "ghcr.io/siderolabs/kubelet:v${k8s_release}"}]'
424420` ` `
425421
426422Once `kubelet` restarts with the new configuration, confirm upgrade with `kubectl get nodes <name>` :
427423
428424` ` ` bash
429- $ kubectl get nodes talos-default-controlplane-1
430- NAME STATUS ROLES AGE VERSION
431- talos-default-controlplane-1 Ready control-plane 123m v${k8s_release}
425+ kubectl get nodes talos-default-controlplane-1
432426` ` `
0 commit comments