Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,8 @@ else
helm repo update syself
KUBECONFIG=$(WORKER_CLUSTER_KUBECONFIG) helm upgrade --install ccm syself/ccm-hetzner --version 2.0.6 \
--namespace kube-system \
--set privateNetwork.enabled=$(PRIVATE_NETWORK)
--set privateNetwork.enabled=$(PRIVATE_NETWORK) \
--set-json 'extraEnvVars=[{"name":"HCLOUD_USE_HROBOT_PROVIDER_ID_FOR_BAREMETAL","value":"true"}]'
@echo
@echo 'run "kubectl --kubeconfig=$(WORKER_CLUSTER_KUBECONFIG) ..." to work with the new target cluster'
@echo
Expand Down
47 changes: 11 additions & 36 deletions docs/caph/02-topics/05-baremetal/03-creating-workload-cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,17 +13,17 @@ Secrets as of now are hardcoded given we are using a flavor which is essentially

{% /callout %}

First you need to decide if you want to use the Syself CCM or the upstream HCloud CCM.
For bare metal, use the Syself CCM until an upstream release includes [PR fix robot name lookup
after stale cache miss](https://github.com/hetznercloud/hcloud-cloud-controller-manager/pull/1204).

In the long run we (Syself) want to switch from our fork to the upstream CCM.

The CCM is the "Cloud Controller" which runs in the workload-cluster. The most important tasks of the CCM are:

- Set ProviderID on Nodes. This is important, so that CAPI in the mgt-cluster knows which CAPI
machine (in mgt-cluster) is which Node (in wl-cluster).
- Creates LoadBalancers

If you are unsure, use the HCloud CCM. In the long run we (Syself) want to switch from our fork to
the upstream CCM.

The CCM calls the Hetzner APIs. To authenticate, it reads the credentials from a secret. This secret
has to be in the workload cluster, when the CCM runs in the workload cluster. CAPH creates the
secret and syncs the credentials specified in the management cluster to the workload cluster. In our
Expand All @@ -36,7 +36,7 @@ Important: CAPH and the CCM must both use the same ProviderID format for bare me
- old: `hcloud://bm-NNNN`
- new: `hrobot://NNNN`

The Syself CCM uses the old format by default. The HCloud CCM always uses the new format.
The upstream HCloud CCM uses the new format.

If you use the new format, set the annotation `capi.syself.com/use-hrobot-provider-id-for-baremetal`
to `"true"` on the `HetznerCluster`. Our default templates have this annotation set.
Expand Down Expand Up @@ -108,47 +108,22 @@ This requires a secret containing access credentials to both Hetzner Robot and H

{% /callout %}

If you want to use the HCloud CCM:
For bare metal, use the Syself CCM until an upstream release includes [PR fix robot name lookup
after stale cache miss](https://github.com/hetznercloud/hcloud-cloud-controller-manager/pull/1204).

```shell
helm repo add hcloud https://charts.hetzner.cloud
helm repo update hcloud
helm repo add syself https://charts.syself.com
helm repo update syself

helm upgrade --install ccm hcloud/hcloud-cloud-controller-manager \
helm upgrade --install ccm syself/ccm-hetzner --version 2.0.6 \
--namespace kube-system \
--set env.HCLOUD_TOKEN.valueFrom.secretKeyRef.name=hetzner \
--set env.HCLOUD_TOKEN.valueFrom.secretKeyRef.key=hcloud \
--set env.ROBOT_USER.valueFrom.secretKeyRef.name=hetzner \
--set env.ROBOT_USER.valueFrom.secretKeyRef.key=robot-user \
--set env.ROBOT_PASSWORD.valueFrom.secretKeyRef.name=hetzner \
--set env.ROBOT_PASSWORD.valueFrom.secretKeyRef.key=robot-password \
--set-json 'additionalTolerations=[{"key":"node.cluster.x-k8s.io/uninitialized","operator":"Exists","effect":"NoSchedule"},{"key":"node.cilium.io/agent-not-ready","operator":"Exists","effect":"NoSchedule"}]' \
--set robot.enabled=true \
--set-json 'extraEnvVars=[{"name":"HCLOUD_USE_HROBOT_PROVIDER_ID_FOR_BAREMETAL","value":"true"}]' \
--kubeconfig workload-kubeconfig
```

Be sure that the HetznerCluster has the annotation
`capi.syself.com/use-hrobot-provider-id-for-baremetal: "true"`.

If you use a different `hetznerSecretRef.name` or different keys, then adjust the chart values
above accordingly.

---

If you want to use the Syself CCM (not recommended for new clusters):

```shell
helm repo add syself https://charts.syself.com
helm repo update syself

$ helm upgrade --install ccm syself/ccm-hetzner --version 2.0.6 \
--namespace kube-system \
--kubeconfig workload-kubeconfig
```

Be sure that the HetznerCluster does not have the annotation
`capi.syself.com/use-hrobot-provider-id-for-baremetal`.

### Installing CNI

For CNI, let's deploy cilium in the workload cluster that will facilitate the networking in the cluster.
Expand Down
1 change: 1 addition & 0 deletions hack/verify-generated-files.sh
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ if ! git diff --quiet || ! git diff --cached --quiet || [[ -n "$(git ls-files --
fi

(
export PATH="$(git rev-parse --show-toplevel)/hack/tools/bin:$PATH"
make kubectl
cd test/e2e
HCLOUD_TOKEN=dummy_hcloud_token
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,6 @@ spec:
hetznerSecretRef:
name: hcloud
key:
hcloudToken: token
hcloudToken: hcloud
hetznerRobotUser: robot-user
hetznerRobotPassword: robot-password
26 changes: 10 additions & 16 deletions test/e2e/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ e2e-cilium-templates:

e2e-ccm-templates:
helm repo add hetzner-cloud https://charts.hetzner.cloud
helm repo add syself https://charts.syself.com

helm template ccm hetzner-cloud/hcloud-cloud-controller-manager --version 1.30.1 \
--namespace kube-system \
Expand All @@ -89,27 +90,20 @@ e2e-ccm-templates:
--set networking.network.valueFrom.secretKeyRef.name=hcloud \
--set networking.network.valueFrom.secretKeyRef.key=network > $(REPO_ROOT)/test/e2e/data/ccm/hcloud-ccm-network.yaml

helm template ccm hetzner-cloud/hcloud-cloud-controller-manager --version 1.30.1 \
helm template ccm syself/ccm-hetzner --version 2.0.6 \
--namespace kube-system \
--set image.repository=ghcr.io/syself/hcloud-cloud-controller-manager \
--set-string env.HCLOUD_DEBUG.value=true \
--set env.HCLOUD_TOKEN.valueFrom.secretKeyRef.name=hcloud \
--set env.HCLOUD_TOKEN.valueFrom.secretKeyRef.key=token \
--set env.ROBOT_USER.valueFrom.secretKeyRef.name=hcloud \
--set env.ROBOT_USER.valueFrom.secretKeyRef.key=robot-user \
--set env.ROBOT_PASSWORD.valueFrom.secretKeyRef.name=hcloud \
--set env.ROBOT_PASSWORD.valueFrom.secretKeyRef.key=robot-password \
--set-json 'additionalTolerations=[{"key":"node.cluster.x-k8s.io/uninitialized","operator":"Exists","effect":"NoSchedule"},{"key":"node.cilium.io/agent-not-ready","operator":"Exists","effect":"NoSchedule"}]' \
--set args.route-reconciliation-period=null \
--set args.feature-gates=null \
--set robot.enabled=true \
--set nameOverride=ccm-ccm-hetzner \
--set networking.enabled=false > $(REPO_ROOT)/test/e2e/data/ccm/hcloud-ccm-hetzner.yaml
--set env.debug=true \
--set pdb.enabled=false \
--set privateNetwork.enabled=false \
--set secret.name=hcloud \
--set-json 'extraEnvVars=[{"name":"HCLOUD_USE_HROBOT_PROVIDER_ID_FOR_BAREMETAL","value":"true"}]' \
--set-json 'tolerations=[{"key":"node.cluster.x-k8s.io/uninitialized","operator":"Exists","effect":"NoSchedule"},{"key":"node.cilium.io/agent-not-ready","operator":"Exists","effect":"NoSchedule"}]' \
> $(REPO_ROOT)/test/e2e/data/ccm/syself-ccm-hetzner.yaml

for file in \
$(REPO_ROOT)/test/e2e/data/ccm/hcloud-ccm.yaml \
$(REPO_ROOT)/test/e2e/data/ccm/hcloud-ccm-network.yaml \
$(REPO_ROOT)/test/e2e/data/ccm/hcloud-ccm-hetzner.yaml; do \
$(REPO_ROOT)/test/e2e/data/ccm/syself-ccm-hetzner.yaml; do \
tmp_file="$${file}.tmp"; \
{ \
# Keep the generated-file marker split so boilerplate verification does not classify this Makefile as generated. \
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/config/hetzner.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ variables:
CILIUM: "./data/cni/cilium/values.yaml"
CCM: "./data/ccm/hcloud-ccm.yaml"
CCM_NETWORK: "./data/ccm/hcloud-ccm-network.yaml"
CCM_HETZNER: "./data/ccm/hcloud-ccm-hetzner.yaml"
CCM_HETZNER: "./data/ccm/syself-ccm-hetzner.yaml"

# Conformance testing variables
CONFORMANCE_WORKER_MACHINE_COUNT: 3
Expand Down
183 changes: 0 additions & 183 deletions test/e2e/data/ccm/hcloud-ccm-hetzner.yaml

This file was deleted.

Loading
Loading