|
5 | 5 |
|
6 | 6 | The Hetzner Cloud [cloud-controller-manager](https://kubernetes.io/docs/concepts/architecture/cloud-controller/) integrates your Kubernetes cluster with the Hetzner Cloud & Robot APIs. |
7 | 7 |
|
8 | | -## Features |
9 | | - |
10 | | -- **Node**: |
11 | | - - Updates your `Node` objects with information about the server from the Cloud & Robot API. |
12 | | - - Instance Type, Location, Datacenter, Server ID, IPs. |
13 | | -- **Node Lifecycle**: |
14 | | - - Cleans up stale `Node` objects when the server is deleted in the API. |
15 | | -- **Routes** (if enabled): |
16 | | - - Routes traffic to the pods through Hetzner Cloud Networks. Removes one layer of indirection in CNIs that support this. |
17 | | -- **Load Balancer**: |
18 | | - - Watches Services with `type: LoadBalancer` and creates Hetzner Cloud Load Balancers for them, adds Kubernetes Nodes as targets for the Load Balancer. |
19 | | - |
20 | | -Read more about cloud controllers in the [Kubernetes documentation](https://kubernetes.io/docs/tasks/administer-cluster/running-cloud-controller/). |
21 | | - |
22 | | -### Node Metadata Example |
23 | | - |
24 | | -```yaml |
25 | | -apiVersion: v1 |
26 | | -kind: Node |
27 | | -metadata: |
28 | | - labels: |
29 | | - node.kubernetes.io/instance-type: cx22 |
30 | | - topology.kubernetes.io/region: fsn1 |
31 | | - topology.kubernetes.io/zone: fsn1-dc8 |
32 | | - instance.hetzner.cloud/provided-by: cloud |
33 | | - name: node |
34 | | -spec: |
35 | | - podCIDR: 10.244.0.0/24 |
36 | | - providerID: hcloud://123456 # <-- Hetzner Cloud Server ID |
37 | | -status: |
38 | | - addresses: |
39 | | - - address: node |
40 | | - type: Hostname |
41 | | - - address: 1.2.3.4 # <-- Hetzner Cloud Server public ipv4 |
42 | | - type: ExternalIP |
43 | | -``` |
44 | | -
|
45 | | -## Deployment |
46 | | -
|
47 | | -This deployment example uses `kubeadm` to bootstrap an Kubernetes |
48 | | -cluster, with [flannel](https://github.com/coreos/flannel) as overlay |
49 | | -network agent. Feel free to adapt the steps to your preferred method of |
50 | | -installing Kubernetes. |
51 | | - |
52 | | -These deployment instructions are designed to guide with the |
53 | | -installation of the `hcloud-cloud-controller-manager` and are by no |
54 | | -means an in depth tutorial of setting up Kubernetes clusters. |
55 | | -**Previous knowledge about the involved components is required.** |
56 | | - |
57 | | -Please refer to the [kubeadm cluster creation |
58 | | -guide](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/), |
59 | | -which these instructions are meant to augment and the [kubeadm |
60 | | -documentation](https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/). |
61 | | - |
62 | | -1. The cloud controller manager adds the labels when a node is added to |
63 | | - the cluster. For current Kubernetes versions, this means we |
64 | | - have to add the `--cloud-provider=external` flag to the `kubelet`. How you |
65 | | - do this depends on your Kubernetes distribution. With `kubeadm` you can |
66 | | - either set it in the kubeadm config |
67 | | - ([`nodeRegistration.kubeletExtraArgs`][kubeadm-config]) or through a systemd |
68 | | - drop-in unit `/etc/systemd/system/kubelet.service.d/20-hcloud.conf`: |
69 | | - |
70 | | - ```ini |
71 | | - [Service] |
72 | | - Environment="KUBELET_EXTRA_ARGS=--cloud-provider=external" |
73 | | - ``` |
74 | | - |
75 | | - Note: the `--cloud-provider` flag is deprecated since K8S 1.19. You |
76 | | - will see a log message regarding this. For now (v1.32) it is still required. |
77 | | - |
78 | | -2. Now the control plane can be initialized: |
79 | | - |
80 | | - ```sh |
81 | | - sudo kubeadm init --pod-network-cidr=10.244.0.0/16 |
82 | | - ``` |
83 | | - |
84 | | -3. Configure kubectl to connect to the kube-apiserver: |
85 | | - |
86 | | - ```sh |
87 | | - mkdir -p $HOME/.kube |
88 | | - sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config |
89 | | - sudo chown $(id -u):$(id -g) $HOME/.kube/config |
90 | | - ``` |
91 | | - |
92 | | -4. Deploy the flannel CNI plugin: |
93 | | - |
94 | | - ```sh |
95 | | - kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml |
96 | | - ``` |
97 | | - |
98 | | -5. Patch the flannel deployment to tolerate the `uninitialized` taint: |
99 | | - |
100 | | - ```sh |
101 | | - kubectl -n kube-system patch ds kube-flannel-ds --type json -p '[{"op":"add","path":"/spec/template/spec/tolerations/-","value":{"key":"node.cloudprovider.kubernetes.io/uninitialized","value":"true","effect":"NoSchedule"}}]' |
102 | | - ``` |
| 8 | +## Docs |
103 | 9 |
|
104 | | -6. Create a secret containing your Hetzner Cloud API token. |
105 | | - |
106 | | - ```sh |
107 | | - kubectl -n kube-system create secret generic hcloud --from-literal=token=<hcloud API token> |
108 | | - ``` |
109 | | - |
110 | | -7. Deploy `hcloud-cloud-controller-manager` |
111 | | - |
112 | | - **Using Helm (recommended):** |
113 | | - |
114 | | - ``` |
115 | | - helm repo add hcloud https://charts.hetzner.cloud |
116 | | - helm repo update hcloud |
117 | | - helm install hccm hcloud/hcloud-cloud-controller-manager -n kube-system |
118 | | - ``` |
119 | | -
|
120 | | - See the [Helm chart README](./chart/README.md) for more info. |
121 | | -
|
122 | | - **Legacy installation method**: |
123 | | -
|
124 | | - ```sh |
125 | | - kubectl apply -f https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/latest/download/ccm.yaml |
126 | | - ``` |
127 | | - |
128 | | -[kubeadm-config]: https://kubernetes.io/docs/reference/config-api/kubeadm-config.v1beta4/#kubeadm-k8s-io-v1beta4-NodeRegistrationOptions |
129 | | - |
130 | | -## Networks support |
131 | | - |
132 | | -When you use the Cloud Controller Manager with networks support, the CCM is in favor of allocating the IPs (& setup the |
133 | | -routing) (Docs: https://kubernetes.io/docs/concepts/architecture/cloud-controller/#route-controller). The CNI plugin you |
134 | | -use needs to support this k8s native functionality (Cilium does it, I don't know about Calico & WeaveNet), so basically |
135 | | -you use the Hetzner Cloud Networks as the underlying networking stack. |
136 | | - |
137 | | -When you use the CCM without Networks support it just disables the RouteController part, all other parts work completely |
138 | | -the same. Then just the CNI is in charge of making all the networking stack things. Using the CCM with Networks support |
139 | | -has the benefit that your node is connected to a private network so the node doesn't need to encrypt the connections and |
140 | | -you have a bit less operational overhead as you don't need to manage the Network. |
141 | | - |
142 | | -If you want to use the Hetzner Cloud `Networks` Feature, head over to |
143 | | -the [Deployment with Networks support |
144 | | -documentation](./docs/deploy_with_networks.md). |
145 | | - |
146 | | -If you manage the network yourself it might still be required to let the CCM know about private networks. For example, |
147 | | -even with a self-managed network, it's still possible to enable private network attachment of CCM-provisioned Load |
148 | | -Balancers by setting the `load-balancer.hetzner.cloud/use-private-ip` annotation to `true` on the Kubernetes Service. |
149 | | -This functionality requires setting the following environment variables in the CCM deployment: |
150 | | - |
151 | | -``` |
152 | | - env: |
153 | | - - name: HCLOUD_NETWORK |
154 | | - valueFrom: |
155 | | - secretKeyRef: |
156 | | - name: hcloud |
157 | | - key: network |
158 | | - - name: HCLOUD_NETWORK_ROUTES_ENABLED |
159 | | - value: "false" |
160 | | -``` |
| 10 | +- :rocket: See the [quick start guide](docs/guides/quickstart.md) to get you started. |
| 11 | +- :book: See the [configuration reference](docs/reference/README.md) for the available configuration. |
161 | 12 |
|
162 | | -You also need to add the network name/ID to the |
163 | | -secret: `kubectl -n kube-system create secret generic hcloud --from-literal=token=<hcloud API token> --from-literal=network=<hcloud Network_ID_or_Name>` |
164 | | -. |
165 | | - |
166 | | -## Kube-proxy mode IPVS and HCloud LoadBalancer |
167 | | - |
168 | | -If `kube-proxy` is run in IPVS mode, the `Service` manifest needs to have the |
169 | | -annotation `load-balancer.hetzner.cloud/hostname` where the FQDN resolves to the HCloud LoadBalancer IP. |
170 | | - |
171 | | -See https://github.com/hetznercloud/hcloud-cloud-controller-manager/issues/212 |
172 | | - |
173 | | -## Versioning policy |
174 | | - |
175 | | -We aim to support the latest three versions of Kubernetes. When a Kubernetes |
176 | | -version is marked as _End Of Life_, we will stop support for it and remove the |
177 | | -version from our CI tests. This does not necessarily mean that the |
178 | | -Cloud Controller Manager does not still work with this version. We will |
179 | | -not fix bugs related only to an unsupported version. |
180 | | - |
181 | | -Current Kubernetes Releases: https://kubernetes.io/releases/ |
182 | | - |
183 | | -### With Networks support |
184 | | - |
185 | | -| Kubernetes | Cloud Controller Manager | Deployment File | |
186 | | -| ---------- | -----------------------: | ----------------------------------------------------------------------------------------------------------: | |
187 | | -| 1.33 | latest | https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/latest/download/ccm-networks.yaml | |
188 | | -| 1.32 | latest | https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/latest/download/ccm-networks.yaml | |
189 | | -| 1.31 | latest | https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/latest/download/ccm-networks.yaml | |
190 | | -| 1.30 | latest | https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/latest/download/ccm-networks.yaml | |
191 | | -| 1.29 | v1.25.1 | https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/download/v1.25.1/ccm-networks.yaml | |
192 | | -| 1.28 | v1.20.0 | https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/download/v1.20.0/ccm-networks.yaml | |
193 | | -| 1.27 | v1.20.0 | https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/download/v1.20.0/ccm-networks.yaml | |
194 | | -| 1.26 | v1.19.0 | https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/download/v1.19.0/ccm-networks.yaml | |
195 | | -| 1.25 | v1.19.0 | https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/download/v1.19.0/ccm-networks.yaml | |
196 | | -| 1.24 | v1.17.2 | https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/download/v1.17.2/ccm-networks.yaml | |
197 | | -| 1.23 | v1.13.2 | https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/download/v1.13.2/ccm-networks.yaml | |
198 | | - |
199 | | -### Without Networks support |
200 | | - |
201 | | -| Kubernetes | Cloud Controller Manager | Deployment File | |
202 | | -| ---------- | -----------------------: | -------------------------------------------------------------------------------------------------: | |
203 | | -| 1.33 | latest | https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/latest/download/ccm.yaml | |
204 | | -| 1.32 | latest | https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/latest/download/ccm.yaml | |
205 | | -| 1.31 | latest | https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/latest/download/ccm.yaml | |
206 | | -| 1.30 | latest | https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/latest/download/ccm.yaml | |
207 | | -| 1.29 | v1.25.1 | https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/download/v1.25.1/ccm.yaml | |
208 | | -| 1.28 | v1.20.0 | https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/download/v1.20.0/ccm.yaml | |
209 | | -| 1.27 | v1.20.0 | https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/download/v1.20.0/ccm.yaml | |
210 | | -| 1.26 | v1.19.0 | https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/download/v1.19.0/ccm.yaml | |
211 | | -| 1.25 | v1.19.0 | https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/download/v1.19.0/ccm.yaml | |
212 | | -| 1.24 | v1.17.2 | https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/download/v1.17.2/ccm.yaml | |
213 | | -| 1.23 | v1.13.2 | https://github.com/hetznercloud/hcloud-cloud-controller-manager/releases/download/v1.13.2/ccm.yaml | |
| 13 | +For more information, see the [documentation](docs/). |
214 | 14 |
|
215 | 15 | ## Development |
216 | 16 |
|
|
0 commit comments