11---
2- title : Node failure test
3- linktitle : Node failure
2+ title : Restoring etcd quorum (lost quorum)
3+ linktitle : Restoring etcd quorum (lost quorum)
44description : Let's switch off some control plane nodes.
55tags : ['etcd','control-plane','v4.17']
66---
77
88# Control plane node failure test
99
10- * Just some stupid tests with OpenShift 4.17
11- * ** With [ OCPSTRAT-539] ( https://issues.redhat.com/browse/OCPSTRAT-539 ) there will be a improvement of the process. Hopefully, land in 4.18!**
10+ Let's test how to restore a control-plane from a lost quorum. Losing the majority of control plane nodes leads to a quorum loss
11+
12+ Tested with OpenShift Cluster Version 4.17.0.
13+
14+ ** With [ OCPSTRAT-539] ( https://issues.redhat.com/browse/OCPSTRAT-539 ) there will be a improvement of the process. Hopefully, land in 4.18!**
1215
1316Useful etcd commands
1417
@@ -62,7 +65,7 @@ Test Workload
6265 +---------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
6366 ```
6467
65- ## One control plane node stopeed
68+ ## One control plane node stopped
6669
6770Date CET: ` 2025-01-08 15:06:37 +0100 `
6871
@@ -133,7 +136,7 @@ Test Workload
133136 stormshift-ocp1 #
134137 ```
135138
136- ## Two control plane nodes stopeed
139+ ## Two control plane nodes stopped
137140
138141Date CET: ` 2025-01-08 15:16:06 +0100 `
139142
@@ -156,7 +159,7 @@ Test Workload
156159| ---| ---| ---| ---| ---|
157160| ` 2025-01-08 13:51:23 +0000 ` | 🟢 🔴| 🟢 🟢* | 🟢 🟢| 🟢 🟢|
158161
159- * OpenShift Web console is " available" but without API useless and not really available .
162+ * OpenShift Web console is available. But the React application cannot connect to Kubernetes/OpenShift API any more .
160163* Control plane is read-only == offline / not available
161164* Workload is still running as expected
162165
@@ -308,7 +311,7 @@ Important points:
308311 [root@ocp1-cp-1 ~]#
309312 ```
310313
311- Accordint to monitoring, API was online at `2025-01-08 15:44:01`
314+ According to monitoring, API was online at `2025-01-08 15:44:01`
312315
313316??? quote "oc get nodes (stormshift-ocp1)"
314317
@@ -413,7 +416,7 @@ Contiue with steps in docs (kubelet restart, ovn-kubernetes, csr,...)
413416* ✅ DNS (A / PTR) done
414417* ✅ DHCP done
415418* ✅ RHCOS Live ISO Uploaded (rhcos-417.94.202410090854-0-live.x86_64.iso)
416- * ✅ control plane igntion exported and available at ` http://10.32.96.31/stormshift-ocp1-cp.ign `
419+ * ✅ control plane ignition file exported and available at ` http://10.32.96.31/stormshift-ocp1-cp.ign `
417420
418421### Add cp-4
419422
@@ -677,7 +680,7 @@ Patch object in another terminal
677680 stormshift-ocp1 #
678681 ```
679682
680- Let's delete the old control-plane artifacts
683+ Delete the old control-plane artifacts.
681684
682685#### Delete two old control plane artifacts
683686
0 commit comments