diff --git a/public/kubernetes-guides/advanced-guides/upgrading-kubernetes.mdx b/public/kubernetes-guides/advanced-guides/upgrading-kubernetes.mdx index e4073d2d..a01948d4 100644 --- a/public/kubernetes-guides/advanced-guides/upgrading-kubernetes.mdx +++ b/public/kubernetes-guides/advanced-guides/upgrading-kubernetes.mdx @@ -27,7 +27,12 @@ Kubernetes upgrades are non-disruptive from Talos, but Kubelet upgrades may caus To trigger a Kubernetes upgrade, issue a command specifying the version of Kubernetes to upgrade to, such as: -`talosctl --nodes upgrade-k8s --to ${k8s_release}` + +{ +`talosctl --nodes upgrade-k8s --to ` +} + + Note that the `--nodes` parameter specifies the control plane node to send the API call to, but all members of the cluster will be upgraded. @@ -35,7 +40,14 @@ To check what will be upgraded you can run `talosctl upgrade-k8s` with the `--dr {` -$ talosctl --nodes upgrade-k8s --to ${k8s_release} --dry-run +talosctl --nodes upgrade-k8s --to ${k8s_release} --dry-run + `} + + +You should get an output similar to this: + + + {` WARNING: found resources which are going to be deprecated/migrated in the version ${k8s_release} RESOURCE COUNT validatingwebhookconfigurations.v1beta1.admissionregistration.k8s.io 4 @@ -71,30 +83,11 @@ updating manifests > apply manifest ClusterRoleBinding system-bootstrap-approve-node-client-csr > apply skipped in dry run - `} - - -To upgrade Kubernetes from `${k8s_prev_release}` to v`${k8s_release}` run: - - {` -$ talosctl --nodes upgrade-k8s --to ${k8s_release} -automatically detected the lowest Kubernetes version ${k8s_prev_release} -checking for resource APIs to be deprecated in version ${k8s_release} -discovered controlplane nodes ["172.20.0.2" "172.20.0.3" "172.20.0.4"] -discovered worker nodes ["172.20.0.5" "172.20.0.6"] -updating "kube-apiserver" to version "${k8s_release}" - > "172.20.0.2": starting update - > update kube-apiserver: ${k8s_prev_release} -> ${k8s_release} - > "172.20.0.2": machine configuration patched - > "172.20.0.2": waiting for API server state pod update - < "172.20.0.2": successfully updated - > "172.20.0.3": starting update - > update kube-apiserver: ${k8s_prev_release} -> ${k8s_release} - `} + This command runs in several phases: 1. Images for new Kubernetes components are pre-pulled to the nodes to minimize downtime and test for image availability. @@ -111,7 +104,7 @@ This command runs in several phases: If the command fails for any reason, it can be safely restarted to continue the upgrade process from the moment of the failure. -> Note: When using custom/overridden Kubernetes component images, use flags `--*-image` to override the default image names. + When using custom/overridden Kubernetes component images, use flags `--*-image` to override the default image names. ### Kubernetes manifest synchronization @@ -122,7 +115,7 @@ If any services were deleted or disabled, the resources associated with them wil For example if [kube-proxy](https://docs.siderolabs.com/talos/latest/reference/configuration/v1alpha1/config#proxy) was disabled in the machine configuration the resources associated with it would be deleted at this stage. Pruning can be disabled by passing the `--manifests-no-prune` flag. -> Note: Pruning is supported from Talos and talosctl v1.13 onwards. +Pruning is supported from Talos and talosctl v1.13 onwards. From Talos v1.13 onwards, all Kubernetes manifests are applied via [Kubernetes Server-Side Apply](https://kubernetes.io/docs/reference/using-api/server-side-apply/). Talos forces ownership of all fields it applies, even if they have a different field manager. @@ -131,7 +124,7 @@ If you wish to manage a resource previously applied by Talos, you need to take t 1. Remove the inline manifest or extra manifest entry from the machine configuration. After this step you can run `talosctl upgrade-k8s --dry-run --to `. Resources affected will be marked for pruning in the output. -2. Remove the resource entries from the talos tracking inventory configmap data block: +2. Remove the resource entries from the talos tracking inventory configmap data block by running: ```bash kubectl edit cm --namespace kube-system talos-bootstrap-manifests-inventory @@ -164,7 +157,7 @@ In order to edit the control plane, you need a working `kubectl` config. If you don't already have one, you can get one by running: ```bash -talosctl --nodes kubeconfig +talosctl --nodes kubeconfig ``` ### API server @@ -172,8 +165,7 @@ talosctl --nodes kubeconfig Patch machine configuration using `talosctl patch` command: ```bash -$ talosctl -n patch mc --mode=no-reboot -p '[{"op": "replace", "path": "/cluster/apiServer/image", "value": "registry.k8s.io/kube-apiserver:v${k8s_release}"}]' -patched mc at the node 172.20.0.2 +talosctl -n patch mc --mode=no-reboot -p '[{"op": "replace", "path": "/cluster/apiServer/image", "value": "registry.k8s.io/kube-apiserver:v${k8s_release}"}]' ``` The JSON patch might need to be adjusted if current machine configuration is missing `.cluster.apiServer.image` key. @@ -183,7 +175,12 @@ Also the machine configuration can be edited manually with `talosctl -n ed Capture the new version of `kube-apiserver` config with: ```bash -$ talosctl -n get apiserverconfig -o yaml +talosctl -n get apiserverconfig -o yaml +``` + +You should get an output similar to this: + +```bash node: 172.20.0.2 metadata: namespace: controlplane @@ -217,16 +214,14 @@ In this example, the new version is `5`. Wait for the new pod definition to propagate to the API server state (replace `talos-default-controlplane-1` with the node name): ```bash -$ kubectl get pod -n kube-system -l k8s-app=kube-apiserver --field-selector spec.nodeName=talos-default-controlplane-1 -o jsonpath='{.items[0].metadata.annotations.talos\.dev/config\-version}' +kubectl get pod -n kube-system -l k8s-app=kube-apiserver --field-selector spec.nodeName=talos-default-controlplane-1 -o jsonpath='{.items[0].metadata.annotations.talos\.dev/config\-version}' 5 ``` Check that the pod is running: ```bash -$ kubectl get pod -n kube-system -l k8s-app=kube-apiserver --field-selector spec.nodeName=talos-default-controlplane-1 -NAME READY STATUS RESTARTS AGE -kube-apiserver-talos-default-controlplane-1 1/1 Running 0 16m +kubectl get pod -n kube-system -l k8s-app=kube-apiserver --field-selector spec.nodeName=talos-default-controlplane-1 ``` Repeat this process for every control plane node, verifying that state got propagated successfully between each node update. @@ -236,8 +231,7 @@ Repeat this process for every control plane node, verifying that state got propa Patch machine configuration using `talosctl patch` command: ```bash -$ talosctl -n patch mc --mode=no-reboot -p '[{"op": "replace", "path": "/cluster/controllerManager/image", "value": "registry.k8s.io/kube-controller-manager:v${k8s_release}"}]' -patched mc at the node 172.20.0.2 +talosctl -n patch mc --mode=no-reboot -p '[{"op": "replace", "path": "/cluster/controllerManager/image", "value": "registry.k8s.io/kube-controller-manager:v${k8s_release}"}]' ``` The JSON patch might need be adjusted if current machine configuration is missing `.cluster.controllerManager.image` key. @@ -245,7 +239,12 @@ The JSON patch might need be adjusted if current machine configuration is missin Capture new version of `kube-controller-manager` config with: ```bash -$ talosctl -n get controllermanagerconfig -o yaml +talosctl -n get controllermanagerconfig -o yaml +``` + +You should see an output similar to this: + +```yaml node: 172.20.0.2 metadata: namespace: controlplane @@ -270,22 +269,19 @@ spec: cpu: "" memory: "" limits: {} -``` +``` In this example, new version is `3`. Wait for the new pod definition to propagate to the API server state (replace `talos-default-controlplane-1` with the node name): ```bash -$ kubectl get pod -n kube-system -l k8s-app=kube-controller-manager --field-selector spec.nodeName=talos-default-controlplane-1 -o jsonpath='{.items[0].metadata.annotations.talos\.dev/config\-version}' -3 +kubectl get pod -n kube-system -l k8s-app=kube-controller-manager --field-selector spec.nodeName=talos-default-controlplane-1 -o jsonpath='{.items[0].metadata.annotations.talos\.dev/config\-version}' ``` Check that the pod is running: ```bash -$ kubectl get pod -n kube-system -l k8s-app=kube-controller-manager --field-selector spec.nodeName=talos-default-controlplane-1 -NAME READY STATUS RESTARTS AGE -kube-controller-manager-talos-default-controlplane-1 1/1 Running 0 35m +kubectl get pod -n kube-system -l k8s-app=kube-controller-manager --field-selector spec.nodeName=talos-default-controlplane-1 ``` Repeat this process for every control plane node, verifying that state propagated successfully between each node update. @@ -295,8 +291,7 @@ Repeat this process for every control plane node, verifying that state propagate Patch machine configuration using `talosctl patch` command: ```bash -$ talosctl -n patch mc --mode=no-reboot -p '[{"op": "replace", "path": "/cluster/scheduler/image", "value": "registry.k8s.io/kube-scheduler:v${k8s_release}"}]' -patched mc at the node 172.20.0.2 +talosctl -n patch mc --mode=no-reboot -p '[{"op": "replace", "path": "/cluster/scheduler/image", "value": "registry.k8s.io/kube-scheduler:v${k8s_release}"}]' ``` JSON patch might need be adjusted if current machine configuration is missing `.cluster.scheduler.image` key. @@ -304,7 +299,12 @@ JSON patch might need be adjusted if current machine configuration is missing `. Capture new version of `kube-scheduler` config with: ```bash -$ talosctl -n get schedulerconfig -o yaml +talosctl -n get schedulerconfig -o yaml +``` + +You should see an output similar to this: + +```yaml node: 172.20.0.2 metadata: namespace: controlplane @@ -328,21 +328,18 @@ spec: limits: {} config: {} ``` - In this example, new version is `3`. + Wait for the new pod definition to propagate to the API server state (replace `talos-default-controlplane-1` with the node name): ```bash -$ kubectl get pod -n kube-system -l k8s-app=kube-scheduler --field-selector spec.nodeName=talos-default-controlplane-1 -o jsonpath='{.items[0].metadata.annotations.talos\.dev/config\-version}' -3 +kubectl get pod -n kube-system -l k8s-app=kube-scheduler --field-selector spec.nodeName=talos-default-controlplane-1 -o jsonpath='{.items[0].metadata.annotations.talos\.dev/config\-version}' ``` Check that the pod is running: ```bash -$ kubectl get pod -n kube-system -l k8s-app=kube-scheduler --field-selector spec.nodeName=talos-default-controlplane-1 -NAME READY STATUS RESTARTS AGE -kube-scheduler-talos-default-controlplane-1 1/1 Running 0 39m +kubectl get pod -n kube-system -l k8s-app=kube-scheduler --field-selector spec.nodeName=talos-default-controlplane-1 ``` Repeat this process for every control plane node, verifying that state got propagated successfully between each node update. @@ -397,7 +394,7 @@ kubectl edit daemonsets -n kube-system kube-proxy Bootstrap manifests can be retrieved in a format which works for `kubectl` with the following command: ```bash -talosctl -n get manifests -o yaml | yq eval-all '.spec | .[] | splitDoc' - > manifests.yaml +talosctl -n get manifests -o yaml | yq eval-all '.spec | .[] | splitDoc' - > manifests.yaml ``` Diff the manifests with the cluster: @@ -412,21 +409,18 @@ Apply the manifests: kubectl apply -f manifests.yaml ``` -> Note: if some bootstrap resources were removed, they have to be removed from the cluster manually. +If some bootstrap resources were removed, they have to be removed from the cluster manually. ### kubelet For every node, patch machine configuration with new kubelet version, wait for the kubelet to restart with new version: ```bash -$ talosctl -n patch mc --mode=no-reboot -p '[{"op": "replace", "path": "/machine/kubelet/image", "value": "ghcr.io/siderolabs/kubelet:v${k8s_release}"}]' -patched mc at the node 172.20.0.2 +talosctl -n patch mc --mode=no-reboot -p '[{"op": "replace", "path": "/machine/kubelet/image", "value": "ghcr.io/siderolabs/kubelet:v${k8s_release}"}]' ``` Once `kubelet` restarts with the new configuration, confirm upgrade with `kubectl get nodes `: ```bash -$ kubectl get nodes talos-default-controlplane-1 -NAME STATUS ROLES AGE VERSION -talos-default-controlplane-1 Ready control-plane 123m v${k8s_release} +kubectl get nodes talos-default-controlplane-1 ``` diff --git a/public/kubernetes-guides/cni/deploying-cilium.mdx b/public/kubernetes-guides/cni/deploying-cilium.mdx index c950bcee..2bab0cb3 100644 --- a/public/kubernetes-guides/cni/deploying-cilium.mdx +++ b/public/kubernetes-guides/cni/deploying-cilium.mdx @@ -1,28 +1,42 @@ --- title: "Deploy Cilium CNI" -description: "In this guide you will learn how to set up Cilium CNI on Talos." +description: "Learn how to install Cilium CNI on Talos Linux using one of several supported methods." aliases: - ../../guides/deploying-cilium --- import { k8s_release } from '/snippets/custom-variables.mdx'; -> Cilium can be installed either via the `cilium` cli or using `helm`. +Cilium CNI can be installed on Talos Linux using several methods, ranging from a fully declarative GitOps-friendly approach to a quick one-command install for development clusters. -This documentation will outline installing Cilium CNI v1.18.0 on Talos in six different ways. -Adhering to Talos principles we'll deploy Cilium with IPAM mode set to Kubernetes, and using the `cgroupv2` and `bpffs` mount that talos already provides. -As Talos does not allow loading kernel modules by Kubernetes workloads, `SYS_MODULE` capability needs to be dropped from the Cilium default set of values, this override can be seen in the helm/cilium cli install commands. -Each method can either install Cilium using kube proxy (default) or without: [Kubernetes Without kube-proxy](https://docs.cilium.io/en/stable/network/kubernetes/kubeproxy-free/). +The right method depends on whether you're using Omni, what your production requirements are, and how much you want the installation to persist across reboots. -In this guide we assume that [KubePrism](../advanced-guides/kubeprism) is enabled and configured to use the port 7445. +Each method supports both the default kube-proxy mode and [kube-proxy-free](https://docs.cilium.io/en/stable/network/kubernetes/kubeproxy-free/) operation, covered by tabs in each section. -## Machine configuration preparation +| Method | Recommended for | Notes | +|---|---|---| +| [Omni manifest sync](#method-1-omni-manifest-sync) | Omni users | Recommended path for Omni clusters | +| [Helm](#method-2-helm) | All other Talos clusters | Four variants — inline, apply, hosted, CLI; choose based on your workflow | +| [Cilium CLI](#method-3-cilium-cli) | Development / testing | Quickest path, least declarative | -When generating the machine config for a node set the CNI to none. -For example using a config patch: +## Prerequisites + +Talos Linux has specific requirements that apply regardless of which installation method you choose: + +- IPAM mode must be set to `kubernetes` +- Talos already provides `cgroupv2` and `bpffs` mounts — do not let Cilium attempt to mount these +- Talos does not allow Kubernetes workloads to load kernel modules, so the `SYS_MODULE` capability must be dropped from Cilium's default capability set +- This guide assumes [KubePrism](../advanced-guides/kubeprism) is enabled and configured on port `7445` + +### Machine configuration prerequisities + +In addition to the above prerequisities, the Cilium CLI and Helm methods all require that you first configure your machine with CNI set to `none`. Set this before bootstrapping. + +To do that: +**1. Create a patch file** that sets the CNI to `none`: ```bash cat < patch.yaml @@ -33,6 +47,8 @@ cluster: EOF ``` +**2. Generate the machine config** with the patch applied: + ```bash talosctl gen config \ my-cluster https://mycluster.local:6443 \ @@ -41,8 +57,7 @@ talosctl gen config \ - -If you want to deploy Cilium without kube-proxy, you also need to disable kube proxy: +**1. Create a patch file** that sets the CNI to `none` and disables kube-proxy: ```bash cat < patch.yaml @@ -55,22 +70,28 @@ cluster: EOF ``` +**2. Generate the machine config** with the patch applied: + ```bash talosctl gen config \ my-cluster https://mycluster.local:6443 \ --config-patch @patch.yaml ``` - -### Installation using Omni + After applying the machine config with CNI set to `none` and bootstrapping, Talos will appear to hang at phase 18/19 with the message `retrying error: node not ready`. This is expected, nodes are only marked ready once a CNI is running. You have approximately 10 minutes to apply Cilium before the node reboots to retry. -If you are using [Omni](https://www.siderolabs.com/platform/saas-for-kubernetes/), you can deploy Cilium using the [manifest sync](../../omni/cluster-management/sync-kubernetes-manifests) feature in a cluster template. +## Method 1: Omni manifest sync -**Step 1.** Create a `values.yaml` file with the Cilium Helm values: +If you are using [Omni](https://www.siderolabs.com/platform/saas-for-kubernetes/), the recommended approach is to deploy Cilium using the [manifest sync](../../omni/cluster-management/sync-kubernetes-manifests) feature in a cluster template. Omni will wait until the Kubernetes API is available and the cluster is healthy before applying the Cilium manifests. -```yaml +**Step 1.** Create a `values.yaml` file with the Cilium Helm values by running: + + + +```bash +cat < values.yaml ipam: mode: kubernetes kubeProxyReplacement: false @@ -96,15 +117,43 @@ cgroup: autoMount: enabled: false hostRoot: /sys/fs/cgroup +EOF ``` - -If deploying without kube-proxy, also add: - -```yaml + + +```bash +cat < values.yaml +ipam: + mode: kubernetes kubeProxyReplacement: true k8sServiceHost: localhost k8sServicePort: 7445 +securityContext: + capabilities: + ciliumAgent: + - CHOWN + - KILL + - NET_ADMIN + - NET_RAW + - IPC_LOCK + - SYS_ADMIN + - SYS_RESOURCE + - DAC_OVERRIDE + - FOWNER + - SETGID + - SETUID + cleanCiliumState: + - NET_ADMIN + - SYS_ADMIN + - SYS_RESOURCE +cgroup: + autoMount: + enabled: false + hostRoot: /sys/fs/cgroup +EOF ``` + + **Step 2.** Render the Cilium manifests using Helm: @@ -114,30 +163,27 @@ helm repo update helm template \ cilium \ cilium/cilium \ - --version 1.19.3 \ + --version 1.18.0 \ --namespace kube-system \ --values values.yaml > cilium.yaml ``` **Step 3.** Reference the rendered manifest in your Omni cluster template using the `file` field: + + {`kind: Cluster\nname: my-cluster\nkubernetes:\n version: ${k8s_release}\n manifests:\n - name: cilium\n file: cilium.yaml\n mode: one-time\npatches:\n - name: disable-default-cni\n inline:\n cluster:\n network:\n cni:\n name: none\n...\n# Include machines for template`} + + -If deploying without kube-proxy, also disable it in the patches: - -```yaml -patches: - - name: disable-default-cni-and-proxy - inline: - cluster: - network: - cni: - name: none - proxy: - disabled: true -``` +When deploying without kube-proxy, set `proxy.disabled: true` in the `patches` block: + + {`kind: Cluster\nname: my-cluster\nkubernetes:\n version: ${k8s_release}\n manifests:\n - name: cilium\n file: cilium.yaml\n mode: one-time\npatches:\n - name: disable-default-cni-and-proxy\n inline:\n cluster:\n network:\n cni:\n name: none\n proxy:\n disabled: true\n...\n# Include machines for template`} + + + **Step 4.** Apply the cluster template: @@ -145,19 +191,34 @@ patches: omnictl cluster template sync --file cluster-template.yaml ``` -Omni will wait until the Kubernetes API is available and the cluster is healthy before applying the Cilium manifests. See [Sync Kubernetes Manifests](../../omni/cluster-management/sync-kubernetes-manifests) for more details on manifest sync modes and status monitoring. +See [Sync Kubernetes Manifests](../../omni/cluster-management/sync-kubernetes-manifests) for more on manifest sync modes and status monitoring. -### Installation using Cilium CLI +## Method 2: Helm -> Note: It is recommended to template the cilium manifest using helm and use it as part of Talos machine config, but if you want to install Cilium using the Cilium CLI, you can follow the steps below. +Helm is the recommended approach for Talos clusters not managed by Omni. There are four variants depending on how you want to manage and apply the Cilium manifest. Choose the one that fits your workflow: -Install the [Cilium CLI](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-default/#install-the-cilium-cli) following the steps here. +- **CLI install**: Installs Cilium directly with `helm install` without generating a manifest file. Simplest Helm path but least declarative. +- **Inline manifest** : Embeds the manifest directly in the machine configuration; applied automatically on bootstrap and reapplied on control plane reboots. Recommended for production. +- **Manifest apply**: Generates the manifest with `helm template` and applies it manually with `kubectl`. Useful for one-off installs or when you want to inspect the manifest first. +- **Hosted manifest**: Generates the manifest, hosts it at a URL, and patches the machine config to fetch it automatically during bootstrap. Requires secure internal hosting. + + + + +Install Cilium directly using `helm install` during the bootstrap window. ```bash -cilium install \ +helm repo add cilium https://helm.cilium.io/ +helm repo update + +helm install \ + cilium \ + cilium/cilium \ + --version 1.18.0 \ + --namespace kube-system \ --set ipam.mode=kubernetes \ --set kubeProxyReplacement=false \ --set securityContext.capabilities.ciliumAgent="{CHOWN,KILL,NET_ADMIN,NET_RAW,IPC_LOCK,SYS_ADMIN,SYS_RESOURCE,DAC_OVERRIDE,FOWNER,SETGID,SETUID}" \ @@ -170,7 +231,14 @@ cilium install \ ```bash -cilium install \ +helm repo add cilium https://helm.cilium.io/ +helm repo update + +helm install \ + cilium \ + cilium/cilium \ + --version 1.18.0 \ + --namespace kube-system \ --set ipam.mode=kubernetes \ --set kubeProxyReplacement=true \ --set securityContext.capabilities.ciliumAgent="{CHOWN,KILL,NET_ADMIN,NET_RAW,IPC_LOCK,SYS_ADMIN,SYS_RESOURCE,DAC_OVERRIDE,FOWNER,SETGID,SETUID}" \ @@ -184,10 +252,17 @@ cilium install \ -Install [Gateway API CRDs](https://docs.cilium.io/en/stable/network/servicemesh/gateway-api/gateway-api/#prerequisites), then install Cilium: +Install [Gateway API CRDs](https://docs.cilium.io/en/stable/network/servicemesh/gateway-api/gateway-api/#prerequisites) first, then: ```bash -cilium install \ +helm repo add cilium https://helm.cilium.io/ +helm repo update + +helm install \ + cilium \ + cilium/cilium \ + --version 1.18.0 \ + --namespace kube-system \ --set ipam.mode=kubernetes \ --set kubeProxyReplacement=true \ --set securityContext.capabilities.ciliumAgent="{CHOWN,KILL,NET_ADMIN,NET_RAW,IPC_LOCK,SYS_ADMIN,SYS_RESOURCE,DAC_OVERRIDE,FOWNER,SETGID,SETUID}" \ @@ -201,34 +276,31 @@ cilium install \ --set gatewayAPI.enableAppProtocol=true ``` -> Note: If you plan to use gRPC and GRPCRoutes with TLS, you must enable ALPN by setting `gatewayAPI.enableAlpn=true`. -> Since gRPC relies on HTTP/2, ALPN is required to negotiate HTTP/2 support between the client and server. - -### Installation using Helm - -Refer to [Installing with Helm](https://docs.cilium.io/en/stable/installation/k8s-install-helm/) for more information. +After Cilium is installed successfully, the bootstrap process will continue and complete. -First we'll need to add the helm repo for Cilium. + + -```bash -helm repo add cilium https://helm.cilium.io/ -helm repo update -``` +The Cilium manifest is embedded directly in the machine configuration and applied automatically during bootstrap. - - +> **Before you start:** +> - Only add the Cilium inline manifest to control plane node machine configurations, and make sure all control plane nodes have an identical configuration. +> - Changing the namespace when templating with Helm does not generate a manifest that creates that namespace. If you change the namespace, manually prepend the namespace YAML at the top of the inline manifest. +> - Talos only creates missing resources from inline manifests — it never deletes or updates them. To update the manifest, edit all control plane machine configurations and run `talosctl upgrade-k8s`. -After applying the machine config and bootstrapping Talos will appear to hang on phase 18/19 with the message: retrying error: node not ready. -This happens because nodes in Kubernetes are only marked as ready once the CNI is up. -As there is no CNI defined, the boot process is pending and will reboot the node to retry after 10 minutes, this is expected behavior. +**Step 1.** Generate the Cilium manifest: -During this window you can install Cilium manually by running the following: + + ```bash -helm install \ +helm repo add cilium https://helm.cilium.io/ +helm repo update + +helm template \ cilium \ cilium/cilium \ --version 1.18.0 \ @@ -238,13 +310,17 @@ helm install \ --set securityContext.capabilities.ciliumAgent="{CHOWN,KILL,NET_ADMIN,NET_RAW,IPC_LOCK,SYS_ADMIN,SYS_RESOURCE,DAC_OVERRIDE,FOWNER,SETGID,SETUID}" \ --set securityContext.capabilities.cleanCiliumState="{NET_ADMIN,SYS_ADMIN,SYS_RESOURCE}" \ --set cgroup.autoMount.enabled=false \ - --set cgroup.hostRoot=/sys/fs/cgroup + --set cgroup.hostRoot=/sys/fs/cgroup > cilium.yaml ``` -Or if you want to deploy Cilium without kube-proxy, also set some extra parameters: + + ```bash -helm install \ +helm repo add cilium https://helm.cilium.io/ +helm repo update + +helm template \ cilium \ cilium/cilium \ --version 1.18.0 \ @@ -256,26 +332,42 @@ helm install \ --set cgroup.autoMount.enabled=false \ --set cgroup.hostRoot=/sys/fs/cgroup \ --set k8sServiceHost=localhost \ - --set k8sServicePort=7445 + --set k8sServicePort=7445 > cilium.yaml ``` -And with GatewayAPI support: + + -```bash -... - --set=gatewayAPI.enabled=true \ - --set=gatewayAPI.enableAlpn=true \ - --set=gatewayAPI.enableAppProtocol=true -``` +**Step 2.** Embed the generated manifest in your machine configuration: -After Cilium is installed the boot process should continue and complete successfully. +```yaml +cluster: + inlineManifests: + - name: cilium + contents: | + --- + # Source: cilium/templates/cilium-agent/serviceaccount.yaml + apiVersion: v1 + kind: ServiceAccount + metadata: + name: "cilium" + namespace: kube-system + --- + # Paste the full contents of cilium.yaml here +``` - + -Instead of directly installing Cilium you can instead first generate the manifest and then apply it: +Generate the manifest with `helm template` and apply it manually with `kubectl` during the bootstrap window. + + + ```bash +helm repo add cilium https://helm.cilium.io/ +helm repo update + helm template \ cilium \ cilium/cilium \ @@ -291,9 +383,13 @@ helm template \ kubectl apply -f cilium.yaml ``` -Without kube-proxy: + + ```bash +helm repo add cilium https://helm.cilium.io/ +helm repo update + helm template \ cilium \ cilium/cilium \ @@ -312,11 +408,18 @@ kubectl apply -f cilium.yaml ``` - + + + + + +Generate the manifest, host it at a secure internal URL, and patch the machine config to fetch it automatically during bootstrap. -After generating `cilium.yaml` using `helm template`, instead of applying this manifest directly during the Talos boot window (before the reboot timeout). -You can also host this file somewhere and patch the machine config to apply this manifest automatically during bootstrap. -To do this patch your machine configuration to include this config instead of the above: +> **Warning:** The Helm-generated Cilium manifest contains sensitive key material. Do not host it somewhere publicly accessible. + +**Step 1.** Generate `cilium.yaml` using `helm template` as shown in the Manifest apply tab above. + +**Step 2.** Host the file at a secure internal URL, then patch your machine configuration: ```bash cat < patch.yaml @@ -334,69 +437,83 @@ talosctl gen config \ my-cluster https://mycluster.local:6443 \ --config-patch @patch.yaml ``` + + -However, beware of the fact that the helm generated Cilium manifest contains sensitive key material. -As such you should definitely not host this somewhere publicly accessible. +## Method 3: Cilium CLI - - +Install the [Cilium CLI](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-default/#install-the-cilium-cli) following the upstream instructions, then run: -A more secure option would be to include the `helm template` output manifest inside the machine configuration. -Using the `patch.yaml` created in the [machine configuration preparation](#machine-configuration-preparation) step, patch this into your machine configuration: + + -``` yaml -cluster: - inlineManifests: - - name: cilium - contents: | - -- - # Source: cilium/templates/cilium-agent/serviceaccount.yaml - apiVersion: v1 - kind: ServiceAccount - metadata: - name: "cilium" - namespace: kube-system - --- - # Source: cilium/templates/cilium-operator/serviceaccount.yaml - apiVersion: v1 - kind: ServiceAccount - -> Your cilium.yaml file will be pretty long.... +```bash +cilium install \ + --set ipam.mode=kubernetes \ + --set kubeProxyReplacement=false \ + --set securityContext.capabilities.ciliumAgent="{CHOWN,KILL,NET_ADMIN,NET_RAW,IPC_LOCK,SYS_ADMIN,SYS_RESOURCE,DAC_OVERRIDE,FOWNER,SETGID,SETUID}" \ + --set securityContext.capabilities.cleanCiliumState="{NET_ADMIN,SYS_ADMIN,SYS_RESOURCE}" \ + --set cgroup.autoMount.enabled=false \ + --set cgroup.hostRoot=/sys/fs/cgroup +``` + + + + +```bash +cilium install \ + --set ipam.mode=kubernetes \ + --set kubeProxyReplacement=true \ + --set securityContext.capabilities.ciliumAgent="{CHOWN,KILL,NET_ADMIN,NET_RAW,IPC_LOCK,SYS_ADMIN,SYS_RESOURCE,DAC_OVERRIDE,FOWNER,SETGID,SETUID}" \ + --set securityContext.capabilities.cleanCiliumState="{NET_ADMIN,SYS_ADMIN,SYS_RESOURCE}" \ + --set cgroup.autoMount.enabled=false \ + --set cgroup.hostRoot=/sys/fs/cgroup \ + --set k8sServiceHost=localhost \ + --set k8sServicePort=7445 ``` -This will install the Cilium manifests at just the right time during bootstrap. + + + +Install [Gateway API CRDs](https://docs.cilium.io/en/stable/network/servicemesh/gateway-api/gateway-api/#prerequisites) first, then: -Beware though: +> **Note:** If you plan to use gRPC and GRPCRoutes with TLS, you must enable ALPN (`gatewayAPI.enableAlpn=true`). gRPC relies on HTTP/2, and ALPN is required to negotiate HTTP/2 support between client and server. -- Changing the namespace when templating with Helm does not generate a manifest containing the yaml to create that namespace. -As the inline manifest is processed from top to bottom make sure to manually put the namespace yaml at the start of the inline manifest. -- Only add the Cilium inline manifest to the control plane nodes machine configuration. -- Make sure all control plane nodes have an identical configuration. -- If you delete any of the generated resources they will be restored whenever a control plane node reboots. -- As a safety measure, Talos only creates missing resources from inline manifests, it never deletes or updates anything. -- If you need to update a manifest make sure to first edit all control plane machine configurations and then run `talosctl upgrade-k8s` as it will take care of updating inline manifests. +```bash +cilium install \ + --set ipam.mode=kubernetes \ + --set kubeProxyReplacement=true \ + --set securityContext.capabilities.ciliumAgent="{CHOWN,KILL,NET_ADMIN,NET_RAW,IPC_LOCK,SYS_ADMIN,SYS_RESOURCE,DAC_OVERRIDE,FOWNER,SETGID,SETUID}" \ + --set securityContext.capabilities.cleanCiliumState="{NET_ADMIN,SYS_ADMIN,SYS_RESOURCE}" \ + --set cgroup.autoMount.enabled=false \ + --set cgroup.hostRoot=/sys/fs/cgroup \ + --set k8sServiceHost=localhost \ + --set k8sServicePort=7445 \ + --set gatewayAPI.enabled=true \ + --set gatewayAPI.enableAlpn=true \ + --set gatewayAPI.enableAppProtocol=true +``` ## Known issues -- There are some gotchas when using Talos and Cilium on the Google cloud platform when using internal load balancers. -For more details: [GCP ILB support / support scope local routes to be configured](https://github.com/siderolabs/talos/issues/4109) +The following are known issues when running Cilium on Talos Linux. Each entry includes a workaround where one exists. + +- **GCP internal load balancers**: There are known issues when using Talos and Cilium on Google Cloud Platform with internal load balancers. See [GCP ILB support / support scope local routes to be configured](https://github.com/siderolabs/talos/issues/4109) for details. -- When using Talos `forwardKubeDNSToHost=true` option (which is enabled by default) in combination with cilium `bpf.masquerade=true`. -There is a known issue that causes `CoreDNS` to not work correctly. -As a workaround, configuring `forwardKubeDNSToHost=false` resolves the issue. -For more details see [the discussion here](https://github.com/siderolabs/talos/pull/9200) +- **CoreDNS not working with `forwardKubeDNSToHost` and `bpf.masquerade`**: When using Talos `forwardKubeDNSToHost=true` (enabled by default) together with Cilium `bpf.masquerade=true`, CoreDNS may not work correctly. Setting `forwardKubeDNSToHost=false` resolves the issue. See [the discussion here](https://github.com/siderolabs/talos/pull/9200) for more context. -## Other things to know +- **`cilium connectivity test` fails with PodSecurity errors**: After installing Cilium, `cilium connectivity test` may hang or fail with an error like: -- After installing Cilium, `cilium connectivity test` might hang and/or fail with errors similar to + ```text + Error creating: pods "client-69748f45d8-9b9jg" is forbidden: violates PodSecurity "baseline:latest": + non-default capabilities (container "client" must not include "NET_RAW" in securityContext.capabilities.add) + ``` - ```Error creating: pods "client-69748f45d8-9b9jg" is forbidden: violates PodSecurity "baseline:latest": non-default capabilities (container "client" must not include "NET_RAW" in securityContext.capabilities.add)``` + This is expected. Work around it by adding the `pod-security.kubernetes.io/enforce=privileged` [label on the namespace level](../security/pod-security). - This is expected, you can workaround it by adding the `pod-security.kubernetes.io/enforce=privileged` [label on the namespace level](../security/pod-security). +## Additional resources -- Talos has full kernel module support for eBPF, See: - - [Cilium System Requirements](https://docs.cilium.io/en/stable/operations/system_requirements/) - - [Talos Kernel Config AMD64](https://github.com/siderolabs/pkgs/blob/main/kernel/build/config-amd64) - - [Talos Kernel Config ARM64](https://github.com/siderolabs/pkgs/blob/main/kernel/build/config-arm64) +Talos has full kernel module support for eBPF. See [Cilium System Requirements](https://docs.cilium.io/en/stable/operations/system_requirements/), [Talos Kernel Config AMD64](https://github.com/siderolabs/pkgs/blob/main/kernel/build/config-amd64), and [Talos Kernel Config ARM64](https://github.com/siderolabs/pkgs/blob/main/kernel/build/config-arm64). diff --git a/public/kubernetes-guides/csi/local-storage.mdx b/public/kubernetes-guides/csi/local-storage.mdx index 34a0ccfa..5316a960 100644 --- a/public/kubernetes-guides/csi/local-storage.mdx +++ b/public/kubernetes-guides/csi/local-storage.mdx @@ -16,8 +16,7 @@ Once the user volume is created, it is automatically mounted under `/var/mnt/u-< For example, create a configuration patch for a user volume named `local-storage`: -```yaml -# local-storage.yaml +```yaml local-storage.yaml apiVersion: v1alpha1 kind: UserVolumeConfig name: local-storage @@ -31,18 +30,14 @@ provisioning: Apply the patch to the machine configuration: ```bash -talosctl --nodes patch mc --patch @local-storage.yaml +talosctl --nodes patch mc --patch @local-storage.yaml ``` If there is enough space available on a non-system disk (see `diskSelector`), the user volume will be created and mounted under `/var/mnt/u-local-storage` path on the node. ```bash -$ talosctl -n get volumestatus u-local-storage -NODE NAMESPACE TYPE ID VERSION TYPE PHASE LOCATION SIZE -172.20.0.5 runtime VolumeStatus u-local-storage 3 partition ready /dev/nvme0n2p1 2.0 GB -$ talosctl -n get mountstatus u-local-storage -NODE NAMESPACE TYPE ID VERSION SOURCE TARGET FILESYSTEM VOLUME -172.20.0.5 runtime MountStatus u-local-storage 2 /dev/nvme0n2p1 /var/mnt/local-storage xfs u-local-storage +talosctl -n get volumestatus u-local-storage +talosctl -n get mountstatus u-local-storage ``` Now you can use the `/var/mnt/local-storage` path in your Kubernetes manifests to refer to the local storage: @@ -73,7 +68,7 @@ spec: First, we will create a separate [user volume](../../talos/v1.10/configure-your-talos-cluster/storage-and-disk-management/disk-management#user-volumes) for the Local Path Provisioner to use. Apply the following machine configuration patch: -> Note: make sure you have [enough space](../../talos/v1.10/configure-your-talos-cluster/storage-and-disk-management/disk-management#disk-layout) available to provision the user volume. +Ensure that you have [enough space](../../talos/v1.10/configure-your-talos-cluster/storage-and-disk-management/disk-management#disk-layout) available to provision the user volume. ```yaml apiVersion: v1alpha1 @@ -90,8 +85,7 @@ Make sure to update Local Path Provisioner configuration to use the user volume For example, Local Path Provisioner can be installed using [kustomize](https://kustomize.io/) with the following configuration: -```yaml -# kustomization.yaml +```yaml kustomization.yaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: diff --git a/public/kubernetes-guides/csi/storage.mdx b/public/kubernetes-guides/csi/storage.mdx index 9be6c208..3e633d6b 100644 --- a/public/kubernetes-guides/csi/storage.mdx +++ b/public/kubernetes-guides/csi/storage.mdx @@ -18,24 +18,26 @@ It is easy and automatic. ## Storage clusters -> **Sidero Labs** recommends having separate disks (separate from the Talos install disk) dedicated for storage. +Sidero Labs recommends having separate disks (separate from the Talos install disk) dedicated for storage. Redundancy, scaling capabilities, reliability, speed, maintenance load, and ease of use are all factors you must consider when managing your own storage. Running a storage cluster can be a very good choice when managing your own storage. The following projects are known to work with Talos Linux and provide good options, depending on your situation. -**Mayastor**: Ultra-low latency and high-performance workloads. +- **Mayastor**: Ultra-low latency and high-performance workloads. -**Longhorn**: Simple, reliable, easy-to-use Kubernetes storage with easy replication and snapshots. +- **Longhorn**: Simple, reliable, easy-to-use Kubernetes storage with easy replication and snapshots. -**Rook/Ceph**: Enterprise-scale, distributed, multi-tenant storage (block, file, and object storage) +- **Rook/Ceph**: Enterprise-scale, distributed, multi-tenant storage (block, file, and object storage) Also, if you need _both_ mount-once _and_ mount-many capabilities, Ceph is your answer. -> Please note that _most_ people should not use mount-many semantics. -> NFS is pervasive because it is old and easy, _not_ because it is a good idea. -> There are all manner of locking, performance, change control, and reliability concerns inherent in _any_ mount-many situation, so we **strongly** recommend you avoid this method. + +Please note that _most_ people should not use mount-many semantics. +NFS is pervasive because it is old and easy, _not_ because it is a good idea. +There are all manner of locking, performance, change control, and reliability concerns inherent in _any_ mount-many situation, so we **strongly** recommend you avoid this method. + ### Longhorn @@ -96,29 +98,32 @@ Follow the Post-Installation from official [documentation](https://openebs.io/do ### Piraeus / LINSTOR -* [Piraeus-Operator](https://piraeus.io/) -* [LINSTOR](https://linbit.com/drbd/) -* [DRBD Extension](https://github.com/siderolabs/extensions#storage) +[Piraeus](https://piraeus.io/) is a Kubernetes storage solution built on top of [LINSTOR](https://linbit.com/drbd/) and DRBD. It provides replicated block storage with low overhead and is a solid choice for clusters where performance and reliability matter. To use it with Talos, you will also need the [DRBD extension](https://github.com/siderolabs/extensions#storage). #### Install Piraeus Operator V2 -There is already a how-to for Talos: [Link](https://piraeus.io/docs/stable/how-to/talos/) +Piraeus maintains a dedicated how-to guide for Talos that covers the full installation process, including the necessary Talos-specific configuration. Follow it [here](https://piraeus.io/docs/stable/how-to/talos/) before proceeding. -#### Create first storage pool and PVC +#### Create a storage pool and PVC -Before proceeding, install linstor plugin for kubectl: -https://github.com/piraeusdatastore/kubectl-linstor +Once the operator is installed, you will need the `linstor` plugin for `kubectl` to manage storage resources. Install it directly from the [kubectl-linstor repository](https://github.com/piraeusdatastore/kubectl-linstor), or via [krew](https://krew.sigs.k8s.io/): -Or use [krew](https://krew.sigs.k8s.io/): `kubectl krew install linstor` +```sh +kubectl krew install linstor +``` + +With the plugin in place, create a device pool on a blank disk (one with no existing partition table) on your target node: ```sh -# Create device pool on a blank (no partition table!) disk on node01 -kubectl linstor physical-storage create-device-pool --pool-name nvme_lvm_pool LVM node01 /dev/nvme0n1 --storage-pool nvme_pool +kubectl linstor physical-storage create-device-pool \ + --pool-name nvme_lvm_pool LVM node01 /dev/nvme0n1 \ + --storage-pool nvme_pool ``` -piraeus-sc.yml +Next, define a StorageClass that uses the pool you just created. Save the following as `piraeus-sc.yml`: ```yaml +cat < piraeus-sc.yml apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: @@ -129,10 +134,12 @@ parameters: linstor.csi.linbit.com/storagePool: nvme_pool provisioner: linstor.csi.linbit.com volumeBindingMode: WaitForFirstConsumer +EOF ``` +Then apply it: + ```sh -# Create storage class kubectl apply -f piraeus-sc.yml ``` diff --git a/public/talos/v1.12/configure-your-talos-cluster/lifecycle-management/upgrading-talos.mdx b/public/talos/v1.12/configure-your-talos-cluster/lifecycle-management/upgrading-talos.mdx index 5a7b69df..007c164e 100644 --- a/public/talos/v1.12/configure-your-talos-cluster/lifecycle-management/upgrading-talos.mdx +++ b/public/talos/v1.12/configure-your-talos-cluster/lifecycle-management/upgrading-talos.mdx @@ -63,7 +63,7 @@ as: {` - $ talosctl upgrade --nodes 10.20.30.40 \\ + talosctl upgrade --nodes 10.20.30.40 \\ --image ghcr.io/siderolabs/installer:${release_v1_12} `} diff --git a/public/talos/v1.12/getting-started/quickstart.mdx b/public/talos/v1.12/getting-started/quickstart.mdx index ec1f4660..51c7c0c4 100644 --- a/public/talos/v1.12/getting-started/quickstart.mdx +++ b/public/talos/v1.12/getting-started/quickstart.mdx @@ -1,6 +1,5 @@ --- title: Quickstart -weight: 20 description: "A short guide on setting up a simple Talos Linux cluster locally with Docker." --- @@ -10,7 +9,7 @@ import { VersionWarningBanner } from "/snippets/version-warning-banner.jsx" import { release_v1_12, k8s_release } from '/snippets/custom-variables.mdx'; -The easiest way to run Talos is by using the CLI (`talosctl`) to create a cluster on a machine with `docker` installed. +This guide walks you through spinning up a local Talos Linux cluster using Docker and `talosctl` in a few minutes. + > -### Prerequisites +## Prerequisites -#### `talosctl` +Before you begin, make sure you have the following tools installed: -Download `talosctl` (macOS or Linux): +- `talosctl`: On macOS or Linux, install `talosctl` by running the command below. For other systems, see the [talosctl documentation](./talosctl): -```bash -brew install siderolabs/tap/talosctl -``` + ```bash + brew install siderolabs/tap/talosctl + ``` -#### `kubectl` +- `kubectl`: Refer to the [Install kubectl documentation](https://kubernetes.io/docs/tasks/tools/install-kubectl/) for installation instructions. -Download `kubectl` via one of the methods outlined in the [documentation](https://kubernetes.io/docs/tasks/tools/install-kubectl/). +## Create the cluster -### Create the cluster +With your prerequisites in place, run the following command to create a cluster on Docker: -Now run the following: - -```bash +````bash talosctl cluster create docker -``` +```` -If you are using Docker Desktop on a macOS computer, if you encounter the error: *Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?* you may need to manually create the link for the Docker socket: -```sudo ln -s "$HOME/.docker/run/docker.sock" /var/run/docker.sock``` - +If you are using Docker Desktop on macOS and encounter the error: *Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?*, you may need to manually create the Docker socket symlink: +````sudo ln -s "$HOME/.docker/run/docker.sock" /var/run/docker.sock``` -You can explore using Talos API commands: +Once the cluster is up, verify that you can reach Kubernetes by running: ```bash -talosctl dashboard --nodes 10.5.0.2 +kubectl get nodes -o wide ``` -Verify that you can reach Kubernetes: +You should see output similar to the following: -{` -kubectl get nodes -o wide - -NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME -talos-default-controlplane-1 Ready master 115s v${k8s_release} 10.5.0.2 Talos ${release_v1_12} containerd://1.5.5 -talos-default-worker-1 Ready 115s v${k8s_release} 10.5.0.3 Talos ${release_v1_12} containerd://1.5.5 -`} +{` NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME\ntalos-default-controlplane-1 Ready master 115s v${k8s_release} 10.5.0.2 Talos ${release_v1_12} containerd://2.1.5\ntalos-default-worker-1 Ready 115s v${k8s_release} 10.5.0.3 Talos ${release_v1_12} containerd://2.1.5 `} -### Destroy the cluster +From here, you can explore the Talos API using `talosctl`: + +```bash +talosctl --help +``` + +## Destroy the cluster -When you are all done, remove the cluster: +When you are done, run the following command to tear down the cluster and clean up all associated resources: ```bash talosctl cluster destroy diff --git a/public/talos/v1.12/getting-started/system-requirements.mdx b/public/talos/v1.12/getting-started/system-requirements.mdx index 03677ed6..9ed7424a 100644 --- a/public/talos/v1.12/getting-started/system-requirements.mdx +++ b/public/talos/v1.12/getting-started/system-requirements.mdx @@ -8,8 +8,12 @@ import { VersionWarningBanner } from "/snippets/version-warning-banner.jsx" +This page outlines the system requirements for running Talos Linux. Requirements are listed for both control plane and worker nodes, and are broadly similar to those of Kubernetes. + ## Minimum requirements +The following are the minimum system requirements for running Talos Linux on control plane and worker nodes: + @@ -35,7 +39,9 @@ import { VersionWarningBanner } from "/snippets/version-warning-banner.jsx"
-## Recommended +## Recommended requirements + +For production or heavier workloads, the following specifications are recommended: @@ -62,12 +68,10 @@ import { VersionWarningBanner } from "/snippets/version-warning-banner.jsx"
-These requirements are similar to that of Kubernetes. - -## Storage +## Storage requirement -Talos Linux itself only requires less than 100 MB of disk space, but the EPHEMERAL partition is used to store pulled images, container work directories, and so on. Thus a minimum of 10 GiB of disk space is required. 100 GiB is recommended. +While Talos Linux itself requires less than 100 MB of disk space, the EPHEMERAL partition is used to store pulled images, container working directories, and other runtime data — so a minimum of 10 GiB of total disk space is required, with 100 GiB recommended. -Talos manages disk partitioning automatically during installation, creating EFI, META, STATE, and EPHEMERAL partitions. The EPHEMERAL partition then expands to fill all the space left after the first three. That space can either remain entirely with EPHEMERAL or be divided into additional user volumes, depending on your needs. See [Disk Layout](../configure-your-talos-cluster/storage-and-disk-management/disk-management#disk-layout) for details. +During installation, Talos automatically manages disk partitioning, creating EFI, META, STATE, and EPHEMERAL partitions. The EPHEMERAL partition expands to fill all remaining space after the first three partitions. That space can either remain entirely with EPHEMERAL or be divided into additional user volumes, depending on your needs. See [Disk Layout](../configure-your-talos-cluster/storage-and-disk-management/disk-management#disk-layout) for details. -For production, it is often more efficient to dedicate a smaller disk for the Talos installation itself, and use additional disks for workload storage. Using a large, single disk for both system and workloads is supported, but may not be optimal depending on your environment. +For production environments, it is often more efficient to dedicate a smaller disk to the Talos installation and use separate disks for workload storage. Using a single large disk for both system and workloads is supported, but may not be optimal depending on your setup. diff --git a/public/talos/v1.12/getting-started/talosctl.mdx b/public/talos/v1.12/getting-started/talosctl.mdx index fd49bf12..61f3e057 100644 --- a/public/talos/v1.12/getting-started/talosctl.mdx +++ b/public/talos/v1.12/getting-started/talosctl.mdx @@ -7,37 +7,34 @@ import { VersionWarningBanner } from "/snippets/version-warning-banner.jsx" -## Recommended +Use `talosctl` to interact with the Talos API, manage nodes, and configure your Talos machines. Follow the instructions below to install it on your operating system. -The client can be installed and updated via the [Homebrew package manager](https://brew.sh/) for macOS and Linux. -You will need to install `brew` and then you can install `talosctl` from the Sidero Labs tap. +## Linux and macOS + +The recommended way to install `talosctl` on Linux and macOS is via the [Homebrew package manager](https://brew.sh/). This method automatically keeps `talosctl` up to date as new versions are released, without any manual intervention. ```bash brew install siderolabs/tap/talosctl ``` -This will also keep your version of `talosctl` up to date with new releases. -This homebrew tap also has formulae for `omnictl` if you need to install that package. +The same Homebrew tap also includes a formula for `omnictl`. -> Note: Your `talosctl` version should match the version of Talos Linux you are running on a host. -> To install a specific version of `talosctl` with `brew` you can follow [this github issue](https://github.com/siderolabs/homebrew-tap/issues/75). +Your `talosctl` version should match the version of Talos Linux running on your host. To install a specific version of `talosctl` with Homebrew, refer to [this GitHub issue](https://github.com/siderolabs/homebrew-tap/issues/75) for guidance. ## Alternative install -You can automatically install the correct version of `talosctl` for your operating system and architecture with an installer script. -This script won't keep your version updated with releases and you will need to re-run the script to download a new version. +If Homebrew is not available in your environment, you can use the installer script to automatically download the correct version of `talosctl` for your operating system and architecture: ```bash curl -sL https://talos.dev/install | sh ``` -This script will work on macOS, Linux, and WSL on Windows. -It supports amd64 and arm64 architecture. +This script supports macOS, Linux, and WSL on Windows, and works on both amd64 and arm64 architectures. -## Manual and Windows install +Note that this method does not update automatically, you will need to re-run the script to install a newer version. -All versions can be manually downloaded from the [talos releases page](https://github.com/siderolabs/talos/releases/) including Linux, macOS, and Windows. +## Windows -You will need to add the binary to a folder part of your executable `$PATH` to use it without providing the full path to the executable. +For Windows, all `talosctl` releases can be downloaded manually from the [Talos releases page](https://github.com/siderolabs/talos/releases/), which includes binaries for Linux, macOS, and Windows. -Updating the binary will be a manual process. +Once downloaded, add the binary to a folder on your `$PATH` so you can run it without specifying the full path to the executable. As with the alternative install method, updating `talosctl` on Windows is a manual process. diff --git a/public/talos/v1.12/platform-specific-installations/bare-metal-platforms/iso.mdx b/public/talos/v1.12/platform-specific-installations/bare-metal-platforms/iso.mdx index e3735746..9766a02e 100644 --- a/public/talos/v1.12/platform-specific-installations/bare-metal-platforms/iso.mdx +++ b/public/talos/v1.12/platform-specific-installations/bare-metal-platforms/iso.mdx @@ -7,19 +7,17 @@ import { VersionWarningBanner } from "/snippets/version-warning-banner.jsx" -Talos can be installed on bare-metal machine using an ISO image. -ISO images for `amd64` and `arm64` architectures are available on the [Talos releases page](https://github.com/siderolabs/talos/releases/latest/). +Talos can be installed on bare-metal machines using an ISO image. ISO images for `amd64` and `arm64` architectures are available on the [Talos releases page](https://github.com/siderolabs/talos/releases/latest/). -Talos doesn't install itself to disk when booted from an ISO until the machine configuration is applied. +When booted from an ISO, Talos does not install itself to disk until a machine configuration is applied. -Please follow the [getting started guide](../../getting-started/getting-started) for the generic steps on how to install Talos. +For steps on how to install Talos, follow the [getting started guide](../../getting-started/getting-started). -> Note: If there is already a Talos installation on the disk, the machine will boot into that installation when booting from a Talos ISO. -> The boot order should prefer disk over ISO, or the ISO should be removed after the installation to make Talos boot from disk. +If a Talos installation already exists on the disk, the machine will boot into that installation even when booting from a Talos ISO. The boot order should prefer disk over ISO, or the ISO should be removed after installation to ensure Talos boots from disk. -See [kernel parameters reference](../../reference/kernel) for the list of kernel parameters supported by Talos. +See the [kernel parameters reference](../../reference/kernel) for a list of kernel parameters supported by Talos. -There are two flavors of ISO images available: +Two flavors of ISO image are available: -* `metal-.iso` supports booting on BIOS and UEFI systems (for x86, UEFI only for arm64) -* `metal--secureboot.iso` supports booting on only UEFI systems in SecureBoot mode (via [Image Factory](../../learn-more/image-factory)) +* `metal-.iso` supports booting on BIOS and UEFI systems (UEFI only for arm64) +* `metal--secureboot.iso` supports booting on UEFI systems in SecureBoot mode only (via [Image Factory](../../learn-more/image-factory)) diff --git a/public/talos/v1.13/configure-your-talos-cluster/lifecycle-management/upgrading-talos.mdx b/public/talos/v1.13/configure-your-talos-cluster/lifecycle-management/upgrading-talos.mdx index 0bf985f6..90b3b576 100644 --- a/public/talos/v1.13/configure-your-talos-cluster/lifecycle-management/upgrading-talos.mdx +++ b/public/talos/v1.13/configure-your-talos-cluster/lifecycle-management/upgrading-talos.mdx @@ -94,7 +94,7 @@ as: {` - $ talosctl upgrade --nodes 10.20.30.40 \\ + talosctl upgrade --nodes 10.20.30.40 \\ --image ghcr.io/siderolabs/installer:${release_v1_13} `} diff --git a/public/talos/v1.13/getting-started/quickstart.mdx b/public/talos/v1.13/getting-started/quickstart.mdx index 77e2140a..f4d8fed0 100644 --- a/public/talos/v1.13/getting-started/quickstart.mdx +++ b/public/talos/v1.13/getting-started/quickstart.mdx @@ -1,6 +1,5 @@ --- title: Quickstart -weight: 20 description: "A short guide on setting up a simple Talos Linux cluster locally with Docker." --- @@ -10,7 +9,7 @@ import { VersionWarningBanner } from "/snippets/version-warning-banner.jsx" import { release_v1_13, k8s_release } from '/snippets/custom-variables.mdx'; -The easiest way to run Talos is by using the CLI (`talosctl`) to create a cluster on a machine with `docker` installed. +This guide walks you through spinning up a local Talos Linux cluster using Docker and `talosctl` in a few minutes. + > -### Prerequisites +## Prerequisites -#### `talosctl` +Before you begin, make sure you have the following tools installed: -Download `talosctl` (macOS or Linux): +- `talosctl`: On macOS or Linux, install `talosctl` by running the command below. For other systems, see the [talosctl documentation](./talosctl): -```bash -brew install siderolabs/tap/talosctl -``` + ```bash + brew install siderolabs/tap/talosctl + ``` -#### `kubectl` +- `kubectl`: Refer to the [Install kubectl documentation](https://kubernetes.io/docs/tasks/tools/install-kubectl/) for installation instructions. -Download `kubectl` via one of the methods outlined in the [documentation](https://kubernetes.io/docs/tasks/tools/install-kubectl/). +## Create the cluster -### Create the cluster +With your prerequisites in place, run the following command to create a cluster on Docker: -Now run the following: - -```bash +````bash talosctl cluster create docker -``` +```` -If you are using Docker Desktop on a macOS computer, if you encounter the error: *Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?* you may need to manually create the link for the Docker socket: -```sudo ln -s "$HOME/.docker/run/docker.sock" /var/run/docker.sock``` - +If you are using Docker Desktop on macOS and encounter the error: *Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?*, you may need to manually create the Docker socket symlink: +````sudo ln -s "$HOME/.docker/run/docker.sock" /var/run/docker.sock``` -You can explore using Talos API commands: +Once the cluster is up, verify that you can reach Kubernetes by running: ```bash -talosctl dashboard --nodes 10.5.0.2 +kubectl get nodes -o wide ``` -Verify that you can reach Kubernetes: +You should see output similar to the following: -{` -kubectl get nodes -o wide - -NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME -talos-default-controlplane-1 Ready master 115s v${k8s_release} 10.5.0.2 Talos ${release_v1_13} containerd://2.1.5 -talos-default-worker-1 Ready 115s v${k8s_release} 10.5.0.3 Talos ${release_v1_13} containerd://2.1.5 -`} +{` NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME\ntalos-default-controlplane-1 Ready master 115s v${k8s_release} 10.5.0.2 Talos ${release_v1_13} containerd://2.1.5\ntalos-default-worker-1 Ready 115s v${k8s_release} 10.5.0.3 Talos ${release_v1_13} containerd://2.1.5 `} -### Destroy the cluster +From here, you can explore the Talos API using `talosctl`: + +```bash +talosctl --help +``` + +## Destroy the cluster -When you are all done, remove the cluster: +When you are done, run the following command to tear down the cluster and clean up all associated resources: ```bash talosctl cluster destroy diff --git a/public/talos/v1.13/getting-started/system-requirements.mdx b/public/talos/v1.13/getting-started/system-requirements.mdx index 03677ed6..9ed7424a 100644 --- a/public/talos/v1.13/getting-started/system-requirements.mdx +++ b/public/talos/v1.13/getting-started/system-requirements.mdx @@ -8,8 +8,12 @@ import { VersionWarningBanner } from "/snippets/version-warning-banner.jsx" +This page outlines the system requirements for running Talos Linux. Requirements are listed for both control plane and worker nodes, and are broadly similar to those of Kubernetes. + ## Minimum requirements +The following are the minimum system requirements for running Talos Linux on control plane and worker nodes: + @@ -35,7 +39,9 @@ import { VersionWarningBanner } from "/snippets/version-warning-banner.jsx"
-## Recommended +## Recommended requirements + +For production or heavier workloads, the following specifications are recommended: @@ -62,12 +68,10 @@ import { VersionWarningBanner } from "/snippets/version-warning-banner.jsx"
-These requirements are similar to that of Kubernetes. - -## Storage +## Storage requirement -Talos Linux itself only requires less than 100 MB of disk space, but the EPHEMERAL partition is used to store pulled images, container work directories, and so on. Thus a minimum of 10 GiB of disk space is required. 100 GiB is recommended. +While Talos Linux itself requires less than 100 MB of disk space, the EPHEMERAL partition is used to store pulled images, container working directories, and other runtime data — so a minimum of 10 GiB of total disk space is required, with 100 GiB recommended. -Talos manages disk partitioning automatically during installation, creating EFI, META, STATE, and EPHEMERAL partitions. The EPHEMERAL partition then expands to fill all the space left after the first three. That space can either remain entirely with EPHEMERAL or be divided into additional user volumes, depending on your needs. See [Disk Layout](../configure-your-talos-cluster/storage-and-disk-management/disk-management#disk-layout) for details. +During installation, Talos automatically manages disk partitioning, creating EFI, META, STATE, and EPHEMERAL partitions. The EPHEMERAL partition expands to fill all remaining space after the first three partitions. That space can either remain entirely with EPHEMERAL or be divided into additional user volumes, depending on your needs. See [Disk Layout](../configure-your-talos-cluster/storage-and-disk-management/disk-management#disk-layout) for details. -For production, it is often more efficient to dedicate a smaller disk for the Talos installation itself, and use additional disks for workload storage. Using a large, single disk for both system and workloads is supported, but may not be optimal depending on your environment. +For production environments, it is often more efficient to dedicate a smaller disk to the Talos installation and use separate disks for workload storage. Using a single large disk for both system and workloads is supported, but may not be optimal depending on your setup. diff --git a/public/talos/v1.13/getting-started/talosctl.mdx b/public/talos/v1.13/getting-started/talosctl.mdx index fd49bf12..61f3e057 100644 --- a/public/talos/v1.13/getting-started/talosctl.mdx +++ b/public/talos/v1.13/getting-started/talosctl.mdx @@ -7,37 +7,34 @@ import { VersionWarningBanner } from "/snippets/version-warning-banner.jsx" -## Recommended +Use `talosctl` to interact with the Talos API, manage nodes, and configure your Talos machines. Follow the instructions below to install it on your operating system. -The client can be installed and updated via the [Homebrew package manager](https://brew.sh/) for macOS and Linux. -You will need to install `brew` and then you can install `talosctl` from the Sidero Labs tap. +## Linux and macOS + +The recommended way to install `talosctl` on Linux and macOS is via the [Homebrew package manager](https://brew.sh/). This method automatically keeps `talosctl` up to date as new versions are released, without any manual intervention. ```bash brew install siderolabs/tap/talosctl ``` -This will also keep your version of `talosctl` up to date with new releases. -This homebrew tap also has formulae for `omnictl` if you need to install that package. +The same Homebrew tap also includes a formula for `omnictl`. -> Note: Your `talosctl` version should match the version of Talos Linux you are running on a host. -> To install a specific version of `talosctl` with `brew` you can follow [this github issue](https://github.com/siderolabs/homebrew-tap/issues/75). +Your `talosctl` version should match the version of Talos Linux running on your host. To install a specific version of `talosctl` with Homebrew, refer to [this GitHub issue](https://github.com/siderolabs/homebrew-tap/issues/75) for guidance. ## Alternative install -You can automatically install the correct version of `talosctl` for your operating system and architecture with an installer script. -This script won't keep your version updated with releases and you will need to re-run the script to download a new version. +If Homebrew is not available in your environment, you can use the installer script to automatically download the correct version of `talosctl` for your operating system and architecture: ```bash curl -sL https://talos.dev/install | sh ``` -This script will work on macOS, Linux, and WSL on Windows. -It supports amd64 and arm64 architecture. +This script supports macOS, Linux, and WSL on Windows, and works on both amd64 and arm64 architectures. -## Manual and Windows install +Note that this method does not update automatically, you will need to re-run the script to install a newer version. -All versions can be manually downloaded from the [talos releases page](https://github.com/siderolabs/talos/releases/) including Linux, macOS, and Windows. +## Windows -You will need to add the binary to a folder part of your executable `$PATH` to use it without providing the full path to the executable. +For Windows, all `talosctl` releases can be downloaded manually from the [Talos releases page](https://github.com/siderolabs/talos/releases/), which includes binaries for Linux, macOS, and Windows. -Updating the binary will be a manual process. +Once downloaded, add the binary to a folder on your `$PATH` so you can run it without specifying the full path to the executable. As with the alternative install method, updating `talosctl` on Windows is a manual process. diff --git a/public/talos/v1.13/platform-specific-installations/bare-metal-platforms/iso.mdx b/public/talos/v1.13/platform-specific-installations/bare-metal-platforms/iso.mdx index e3735746..9766a02e 100644 --- a/public/talos/v1.13/platform-specific-installations/bare-metal-platforms/iso.mdx +++ b/public/talos/v1.13/platform-specific-installations/bare-metal-platforms/iso.mdx @@ -7,19 +7,17 @@ import { VersionWarningBanner } from "/snippets/version-warning-banner.jsx" -Talos can be installed on bare-metal machine using an ISO image. -ISO images for `amd64` and `arm64` architectures are available on the [Talos releases page](https://github.com/siderolabs/talos/releases/latest/). +Talos can be installed on bare-metal machines using an ISO image. ISO images for `amd64` and `arm64` architectures are available on the [Talos releases page](https://github.com/siderolabs/talos/releases/latest/). -Talos doesn't install itself to disk when booted from an ISO until the machine configuration is applied. +When booted from an ISO, Talos does not install itself to disk until a machine configuration is applied. -Please follow the [getting started guide](../../getting-started/getting-started) for the generic steps on how to install Talos. +For steps on how to install Talos, follow the [getting started guide](../../getting-started/getting-started). -> Note: If there is already a Talos installation on the disk, the machine will boot into that installation when booting from a Talos ISO. -> The boot order should prefer disk over ISO, or the ISO should be removed after the installation to make Talos boot from disk. +If a Talos installation already exists on the disk, the machine will boot into that installation even when booting from a Talos ISO. The boot order should prefer disk over ISO, or the ISO should be removed after installation to ensure Talos boots from disk. -See [kernel parameters reference](../../reference/kernel) for the list of kernel parameters supported by Talos. +See the [kernel parameters reference](../../reference/kernel) for a list of kernel parameters supported by Talos. -There are two flavors of ISO images available: +Two flavors of ISO image are available: -* `metal-.iso` supports booting on BIOS and UEFI systems (for x86, UEFI only for arm64) -* `metal--secureboot.iso` supports booting on only UEFI systems in SecureBoot mode (via [Image Factory](../../learn-more/image-factory)) +* `metal-.iso` supports booting on BIOS and UEFI systems (UEFI only for arm64) +* `metal--secureboot.iso` supports booting on UEFI systems in SecureBoot mode only (via [Image Factory](../../learn-more/image-factory)) diff --git a/public/talos/v1.13/platform-specific-installations/virtualized-platforms/proxmox.mdx b/public/talos/v1.13/platform-specific-installations/virtualized-platforms/proxmox.mdx index de7878ac..216004c8 100644 --- a/public/talos/v1.13/platform-specific-installations/virtualized-platforms/proxmox.mdx +++ b/public/talos/v1.13/platform-specific-installations/virtualized-platforms/proxmox.mdx @@ -6,303 +6,249 @@ aliases: --- import { VersionWarningBanner } from "/snippets/version-warning-banner.jsx" +import { release_v1_13 } from '/snippets/custom-variables.mdx'; -import { release_v1_13 } from '/snippets/custom-variables.mdx'; - -In this guide we will create a Kubernetes cluster using Proxmox. +In this guide you will create a Kubernetes cluster on Proxmox, from uploading the Talos ISO to bootstrapping etcd and retrieving your kubeconfig. Follow the steps in order — each one builds on the last. ## Video walkthrough -To see a live demo of this writeup, visit Youtube here: +To see a live demo of this guide, visit Youtube here: -## Installation +## Prerequisites + +Before you begin, make sure the following are in place. The QEMU guest agent section is optional, only follow it if you need guest VM shutdown support. -### How to get Proxmox +- **Proxmox**: This guide assumes you have already installed Proxmox on the server where you want to create Talos VMs. Visit the [Proxmox downloads page](https://www.proxmox.com/en/downloads) if you haven't done this yet. -It is assumed that you have already installed Proxmox onto the server you wish to create Talos VMs on. -Visit the [Proxmox](https://www.proxmox.com/en/downloads) downloads page if necessary. +- **talosctl**: Install `talosctl` on macOS or Linux with: -### Install talosctl + ```bash + brew install siderolabs/tap/talosctl + ``` -You can download `talosctl` on MacOS and Linux via: + For manual installation and other platforms, see the [talosctl installation guide](../../getting-started/talosctl). -```bash -brew install siderolabs/tap/talosctl -``` +- **ISO image**: Download the Talos ISO from [Image Factory](https://www.talos.dev/latest/talos-guides/install/boot-assets/#image-factory) with this command: -For manual installation and other platforms please see the [talosctl installation guide](../../getting-started/talosctl ). + ```bash + mkdir -p _out/ + curl https://factory.talos.dev/image/376567988ad370138ad8b2698212367b8edcb69b5fd68c80be1f2ec7d603b4ba//metal-.iso -L -o _out/metal-.iso + ``` -### Download ISO image + For example version, for linux platform: -In order to install Talos in Proxmox, you will need the ISO image from [Image Factory](https://www.talos.dev/latest/talos-guides/install/boot-assets/#image-factory). + + {`mkdir -p _out/\ncurl https://factory.talos.dev/image/376567988ad370138ad8b2698212367b8edcb69b5fd68c80be1f2ec7d603b4ba/${release_v1_13}/metal-amd64.iso -L -o _out/metal-amd64.iso`} + -```bash -mkdir -p _out/ -curl https://factory.talos.dev/image/376567988ad370138ad8b2698212367b8edcb69b5fd68c80be1f2ec7d603b4ba//metal-.iso -L -o _out/metal-.iso -``` +### Optional: QEMU guest agent ISO -For example version {release } for `linux` platform: +QEMU guest agent support for guest VM shutdowns requires a custom ISO and installer image. Skip this section if that doesn't apply to you. - - {` -mkdir -p _out/ -curl https://factory.talos.dev/image/376567988ad370138ad8b2698212367b8edcb69b5fd68c80be1f2ec7d603b4ba/${release_v1_13}/metal-amd64.iso -L -o _out/metal-amd64.iso - `} - +1. **Build the custom ISO**: Go to the [Image Factory](https://factory.talos.dev/) and complete these steps to build a custom ISO: + - Select your Talos version + - Check the box for `siderolabs/qemu-guest-agent` and submit. This action will provide you with a link similar to: + + {`Metal ISO\n\namd64 ISO\n https://factory.talos.dev/image/ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515/${release_v1_13}/metal-amd64.iso\narm64 ISO\n https://factory.talos.dev/image/ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515/${release_v1_13}/metal-arm64.iso\n\nInstaller Image\n\nFor the initial Talos install or upgrade, use the following installer image:\nfactory.talos.dev/installer/ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515:${release_v1_13}`} + + - Download the above ISO and take note of the installer image URL, you will need it later -### QEMU guest agent support (iso) -- If you need the QEMU guest agent so you can do guest VM shutdowns of your Talos VMs, then you will need a custom ISO -- To get this, navigate to https://factory.talos.dev/ -- Scroll down and select your Talos version ( {release_v1_13} for example) -- Then tick the box for `siderolabs/qemu-guest-agent` and submit -- This will provide you with a link to the bare metal ISO -- The lines we're interested in are as follows +2. **Enable QEMU Guest Agent in Proxmox:** Once your VM is created, go to **VM → Options** and set **QEMU Guest Agent** to **Enabled**. - - {` -Metal ISO + Only enable QEMU Guest Agent in Proxmox if you built the ISO with the `siderolabs/qemu-guest-agent` extension. Enabling it without the extension will only generate log spam and provide no functionality. See [Image Factory](../../learn-more/image-factory) for more on building custom ISOs. -amd64 ISO - https://factory.talos.dev/image/ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515/${release_v1_13}/metal-amd64.iso -arm64 ISO - https://factory.talos.dev/image/ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515/${release_v1_13}/metal-arm64.iso -Installer Image +### VM resource requirements -For the initial Talos install or upgrade use the following installer image: -factory.talos.dev/installer/ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515:{release_v1_13} - `} - +Before creating VMs, familiarise yourself with the [system requirements](../../getting-started/system-requirements) for Talos and use the following as a baseline for all Talos nodes on Proxmox: -- Download the above ISO (this will most likely be `amd64` for you) -- Take note of the `factory.talos.dev/installer` URL as you'll need it later +| Setting | Recommended value | Notes | +|---|---|---| +| **BIOS** | `ovmf` (UEFI) | Modern firmware, Secure Boot support, better hardware compatibility | +| **Machine** | `q35` | Modern PCIe-based machine type with better device support | +| **CPU type** | `host` | Enables advanced instruction sets (AVX-512, etc.). Use `kvm64` with feature flags for Proxmox < 8.0 | +| **CPU cores** | 2+ (control plane), 4+ (workers) | Minimum 2 cores required | +| **Memory** | 4GB+ (control plane), 8GB+ (workers) | Minimum 2GB required | +| **Disk controller** | VirtIO SCSI | Do NOT use **VirtIO SCSI Single** — causes bootstrap hangs ([#11173](https://github.com/siderolabs/talos/issues/11173)) | +| **Disk format** | Raw or QCOW2 | Raw preferred for performance; QCOW2 for snapshots | +| **Disk cache** | Write Through | Use None for clustered environments | +| **Network model** | `virtio` | Paravirtualized driver, best performance (up to 10 Gbit) | +| **EFI disk** | 4MB | Required for UEFI firmware, stores Secure Boot keys | +| **Ballooning** | Disabled | Talos does not support memory hotplug | +| **RNG device** | VirtIO RNG (optional) | Better entropy for cryptographic operations | -## Upload ISO +## Step 1: Upload the ISO -From the Proxmox UI, select the "local" storage and enter the "Content" section. -Click the "Upload" button: +From the Proxmox UI, select the **local** storage and enter the **Content** section. Click the **Upload** button: -Select the ISO you downloaded previously, then hit "Upload" +Select the ISO you downloaded in the prerequisites, then click **Upload**: -## Create VMs - -Before starting, familiarise yourself with the -[system requirements](../../getting-started/system-requirements) for Talos and assign VM -resources accordingly. - -### Recommended baseline VM configuration - -Use the following baseline settings for Proxmox VMs running Talos: - -| Setting | Recommended Value | Notes | -|---------|------------------|-------| -| **BIOS** | `ovmf` (UEFI) | Modern firmware, Secure Boot support, better hardware compatibility | -| **Machine** | `q35` | Modern PCIe-based machine type with better device support | -| **CPU Type** | `host` | Enables advanced instruction sets (AVX-512, etc.), best performance. Alternative: `kvm64` with feature flags for Proxmox < 8.0 | -| **CPU Cores** | 2+ (control plane), 4+ (workers) | Minimum 2 cores required | -| **Memory** | 4GB+ (control plane), 8GB+ (workers) | Minimum 2GB required | -| **Disk Controller** | **VirtIO SCSI** (NOT "VirtIO SCSI Single") | Single controller can cause bootstrap hangs (#11173) | -| **Disk Format** | Raw (performance) or QCOW2 (features/snapshots) | Raw preferred for performance | -| **Disk Cache** | Write Through (safe default) | Or None for clustered environments | -| **Network Model** | `virtio` | Paravirtualized driver, best performance (up to 10 Gbit) | -| **EFI Disk** | 4MB (for OVMF) | Required for UEFI firmware, stores Secure Boot keys | -| **Ballooning** | Disabled | Talos doesn't support memory hotplug | -| **RNG Device** | VirtIO RNG (optional) | Better entropy for security | - -> **Important**: When configuring the disk controller, use **VirtIO SCSI** (not "VirtIO SCSI Single"). -> Using "VirtIO SCSI Single" can cause bootstrap to hang or prevent disk discovery. -> See [issue #11173](https://github.com/siderolabs/talos/issues/11173) for details. +## Step 2: Create VMs -Create a new VM by clicking the "Create VM" button in the Proxmox UI: +Create a new VM by clicking **Create VM** in the Proxmox UI: -Fill out a name for the new VM: +This action would open 8 tabs, configure the tabs as follows: + +- **Name tab**: Fill out a name for the VM: -In the OS tab, select the ISO we uploaded earlier: +- **OS tab**; Select the ISO uploaded in Step 1: -In the "System" tab: -- Set **BIOS** to `ovmf` (UEFI) for modern firmware and Secure Boot support -- Set **Machine** to `q35` for modern PCIe-based machine type -- Add **EFI Disk** (4MB) for persistent UEFI settings and Secure Boot key storage - -In the "Hard Disk" tab: -- Set **Bus/Device** to `VirtIO SCSI` (NOT "VirtIO SCSI Single") -- Set **Storage** to your main storage pool -- Set **Format** to `Raw` (better performance) or `QCOW2` (features/snapshots) -- Set **Size** based on your workload requirements (adjust based on CSI and application needs) -- Set **Cache** to `Write Through` (safe default) or `None` for clustered environments -- Enable **Discard** (TRIM support) if using SSD storage -- Enable **SSD emulation** if using SSD storage - -> **Important**: When configuring the disk controller, use **VirtIO SCSI** (not "VirtIO SCSI Single"). -> Using "VirtIO SCSI Single" can cause bootstrap to hang or prevent disk discovery. -> See [issue #11173](https://github.com/siderolabs/talos/issues/11173) for details. - -In the "CPU" section: -- Set **Cores** to 2+ for control planes, 4+ for workers -- Set **Sockets** to 1 (keep simple) -- Set **Type** to `host` (best performance, enables advanced instruction sets) - - **Alternative for Proxmox < 8.0**: Use `kvm64` with feature flags by adding to `/etc/pve/qemu-server/.conf`: - ```text - args: -cpu kvm64,+cx16,+lahf_lm,+popcnt,+sse3,+ssse3,+sse4.1,+sse4.2 - ``` - - **Note**: `host` CPU type prevents live VM migration but provides best performance +- **System tab:** + - Set **BIOS** to `ovmf` (UEFI) + - Set **Machine** to `q35` + - Add an **EFI Disk** (4MB) for persistent UEFI settings and Secure Boot key storage + +- **Hard Disk tab:** + - Set **Bus/Device** to `VirtIO SCSI` (NOT **VirtIO SCSI Single**) + - Set **Storage** to your main storage pool + - Set **Format** to `Raw` (performance) or `QCOW2` (snapshots) + - Set **Cache** to `Write Through`, or `None` for clustered environments + - Enable **Discard** and **SSD emulation** if using SSD storage + +- **CPU tab:** + - Set **Cores** to 2+ for control planes, 4+ for workers + - Set **Type** to `host` for best performance + + For Proxmox < 8.0, use `kvm64` with feature flags instead. Add the following to `/etc/pve/qemu-server/.conf`: + ```text + args: -cpu kvm64,+cx16,+lahf_lm,+popcnt,+sse3,+ssse3,+sse4.1,+sse4.2 + ``` + + > **Note:** `host` CPU type prevents live VM migration. -In the "Memory" section: -- Set **Memory** to 4GB+ for control planes, 8GB+ for workers (minimum 2GB required) -- **Disable Ballooning** (can cause issues with Talos memory detection) +- **Memory tab:** + - Set **Memory** to 4GB+ for control planes, 8GB+ for workers + - **Disable Ballooning** — Talos does not support memory hotplug, and enabling it will prevent Talos from seeing all available memory - + -In the "Network" section: -- Set **Model** to `virtio` (paravirtualized driver, best performance) -- Set **Bridge** to your network bridge (e.g., `vmbr0`) -- Verify the VM is set to come up on the bridge interface +- **Network tab:** + - Set **Model** to `virtio` + - Set **Bridge** to your network bridge (e.g., `vmbr0`) - + -> **Tip**: Enable a serial console (ttyS0) in Proxmox VM settings to see early boot logs and troubleshoot network connectivity issues. -> This is especially helpful when debugging DHCP timing or bridge configuration problems. -> Set **Serial port** to `ttyS0` in Proxmox and add `console=ttyS0` if you're customizing kernel args. +Enable a serial console (`ttyS0`) in Proxmox VM settings to see early boot logs and troubleshoot network issues. This is especially useful when debugging DHCP timing or bridge configuration problems. -Finish creating the VM by clicking through the "Confirm" tab and then "Finish". +- **Confirm tab**: Finish creating the VM by clicking through the **Confirm** tab and then **Finish**. -Repeat this process for a second VM to use as a worker node. -You can also repeat this for additional nodes desired. +Repeat this process for each additional node, at minimum one control plane and one worker. -> Note: Talos doesn't support memory hot plugging, if creating the VM programmatically don't enable memory hotplug on your -> Talos VM's. -> Doing so will cause Talos to be unable to see all available memory and have insufficient memory to complete -> installation of the cluster. +## Step 3: Start the control plane node -## Start control plane node +Start the VM designated as the first control plane node. It will boot from the ISO and enter maintenance mode. -Once the VMs have been created and updated, start the VM that will be the first control plane node. -This VM will boot the ISO image specified earlier and enter "maintenance mode". + + -### With DHCP server +Once the machine enters maintenance mode, the console will display the IP address the node received. Take note of this IP — it will be referred to as `$CONTROL_PLANE_IP` for the rest of this guide: -Once the machine has entered maintenance mode, there will be a console log that details the IP address that the node received. -Take note of this IP address, which will be referred to as `$CONTROL_PLANE_IP` for the rest of this guide. -If you wish to export this IP as a bash variable, simply issue a command like `export CONTROL_PLANE_IP=1.2.3.4`. +```bash +export CONTROL_PLANE_IP=1.2.3.4 +``` -### Without DHCP server + + -To apply the machine configurations in maintenance mode, VM has to have IP on the network. -So you can set it on boot time manually. +To apply machine configuration in maintenance mode, the VM needs a static IP. Set it manually at boot time. + +1. Press `e` at the GRUB menu: -Press `e` on the boot time. -And set the IP parameters for the VM. -[Format is](https://www.kernel.org/doc/Documentation/filesystems/nfs/nfsroot.txt): +2. Append the IP parameters to the boot line using the following format ([kernel nfsroot format](https://www.kernel.org/doc/Documentation/filesystems/nfs/nfsroot.txt)): -```bash -ip=:::::: -``` + ```bash + ip=:::::: + ``` -For example $CONTROL_PLANE_IP will be 192.168.0.100 and gateway 192.168.0.1 + For example, with `$CONTROL_PLANE_IP` as `192.168.0.100` and gateway `192.168.0.1`: -```bash -linux /boot/vmlinuz init_on_alloc=1 slab_nomerge pti=on panic=0 consoleblank=0 printk.devkmsg=on earlyprintk=ttyS0 console=tty0 console=ttyS0 talos.platform=metal ip=192.168.0.100::192.168.0.1:255.255.255.0::eth0:off -``` + ```bash + linux /boot/vmlinuz init_on_alloc=1 slab_nomerge pti=on panic=0 consoleblank=0 printk.devkmsg=on earlyprintk=ttyS0 console=tty0 console=ttyS0 talos.platform=metal ip=192.168.0.100::192.168.0.1:255.255.255.0::eth0:off + ``` -Then press Ctrl-x or F10 +3. Press `Ctrl-x` or `F10` to boot with the updated parameters. + + + +## Step 4: Generate machine configurations -## Generate machine configurations +With `$CONTROL_PLANE_IP` set, generate the machine configurations for your cluster. -With the IP address above, you can now generate the machine configurations to use for installing Talos and Kubernetes. -Issue the following command, updating the output directory, cluster name, and control plane IP as you see fit: +Choose the tab that matches your ISO. + + + ```bash talosctl gen config talos-proxmox-cluster https://$CONTROL_PLANE_IP:6443 --output-dir _out ``` - -This will create several files in the `_out` directory: `controlplane.yaml`, `worker.yaml`, and `talosconfig`. - -> Note: The Talos config by default will install to `/dev/sda`. -> Depending on your setup the virtual disk may be mounted differently Eg: `/dev/vda`. -> You can check for disks running the following command: -> -> ```bash -> talosctl get disks --insecure --nodes $CONTROL_PLANE_IP -> ``` -> -> Update `controlplane.yaml` and `worker.yaml` config files to point to the correct disk location. - -### QEMU guest agent support - -For QEMU guest agent support, you can generate the config with the custom install image: + + +Use this if you built the custom ISO with the `siderolabs/qemu-guest-agent` extension in the prerequisites. The command is the same but passes the custom installer image URL you noted earlier. - {` -talosctl gen config talos-proxmox-cluster https://$CONTROL_PLANE_IP:6443 --output-dir _out --install-image factory.talos.dev/installer/ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515:${release_v1_13} - `} + {`talosctl gen config talos-proxmox-cluster https://$CONTROL_PLANE_IP:6443 --output-dir _out --install-image factory.talos.dev/installer/ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515:${release_v1_13}`} + + + +This command creates three files in `_out/`: `controlplane.yaml`, `worker.yaml`, and `talosconfig`. -> **Important**: Enable QEMU Guest Agent in Proxmox **only if** you built the ISO with the `siderolabs/qemu-guest-agent` extension in **Image Factory**. -> If you're using a standard Talos ISO without this extension, leave QEMU Guest Agent disabled in Proxmox VM settings. -> Enabling it without the extension will only generate log spam and won't provide any functionality. -> See: [Image Factory](../../learn-more/image-factory) for building a custom ISO with extensions. + +The default config installs Talos to `/dev/sda`. Depending on your setup, the virtual disk may be at a different path (e.g., `/dev/vda`). Check available disks with: +```bash +talosctl get disks --insecure --nodes $CONTROL_PLANE_IP +``` -- If you did include the extension, go to your VM → **Options** and set **QEMU Guest Agent** to **Enabled**. +Update `install.disk` in `controlplane.yaml` and `worker.yaml` if needed. + -## Create control plane node +## Step 5: Apply configuration to the control plane -Using the `controlplane.yaml` generated above, you can now apply this config using talosctl. -Issue: +Apply the control plane configuration to the node. If you are setting up an HA control plane, repeat this step for each additional control plane node. ```bash talosctl apply-config --insecure --nodes $CONTROL_PLANE_IP --file _out/controlplane.yaml ``` -You should now see some action in the Proxmox console for this VM. -Talos will be installed to disk, the VM will reboot, and then Talos will configure the Kubernetes control plane on this VM. -The VM will remain in stage `Booting` until the bootstrap is completed in a later step. - -> Note: This process can be repeated multiple times to create an HA control plane. - -## Create worker node +You should see activity in the Proxmox console. Talos will install to disk, the VM will reboot, and Talos will configure the Kubernetes control plane. The VM will remain in stage `Booting` until the cluster is bootstrapped in the next steps. -Create at least a single worker node using a process similar to the control plane creation above. -Start the worker node VM and wait for it to enter "maintenance mode". -Take note of the worker node's IP address, which will be referred to as `$WORKER_IP` +## Step 6: Apply configuration to worker nodes -Issue: +Start each worker VM and wait for it to enter maintenance mode. Repeat this step for each worker node, substituting `$WORKER_IP` with each node's IP address. ```bash talosctl apply-config --insecure --nodes $WORKER_IP --file _out/worker.yaml ``` -> Note: This process can be repeated multiple times to add additional workers. +## Step 7: Bootstrap the cluster -## Using the cluster +With the control plane and worker nodes configured, complete the cluster setup by running the following commands in order: -Once the cluster is available, you can make use of `talosctl` and `kubectl` to interact with the cluster. -For example, to view current running containers, run `talosctl containers` for a list of containers in the `system` namespace, or `talosctl containers -k` for the `k8s.io` namespace. -To view the logs of a container, use `talosctl logs ` or `talosctl logs -k `. - -First, configure talosctl to talk to your control plane node by issuing the following, updating paths and IPs as necessary: +1. Configure talosctl to talk to your control plane node: ```bash export TALOSCONFIG="_out/talosconfig" @@ -310,27 +256,35 @@ talosctl config endpoint $CONTROL_PLANE_IP talosctl config node $CONTROL_PLANE_IP ``` -### Bootstrap etcd +2. Bootstrap etcd: ```bash talosctl bootstrap ``` -### Retrieve the `kubeconfig` - -At this point we can retrieve the admin `kubeconfig` by running: +3. Retrieve the kubeconfig: ```bash talosctl kubeconfig . ``` +4. Verify that your cluster is ready: + +```bash +kubectl get nodes +``` + ## Troubleshooting +This section covers common issues you may encounter when setting up Talos VMs on Proxmox, along with steps to resolve them. + ### Cluster creation issues +If `talosctl cluster create` fails with disk controller errors, the most likely cause is an unsupported disk controller type. For example: If `talosctl cluster create` fails with disk controller errors: -- **"virtio-scsi-single disk controller is not supported"**: This disk controller type causes Talos bootstrap to hang. Use `virtio` or `scsi` instead: +- `virtio-scsi-single disk controller is not supported`: This disk controller type causes Talos bootstrap to hang. Use `virtio` or `scsi` instead: + ```bash # Wrong - will be rejected talosctl cluster create --disks virtio-scsi-single:10GiB @@ -342,12 +296,13 @@ If `talosctl cluster create` fails with disk controller errors: ### Network connectivity issues -If nodes fail to obtain IP addresses or show "network is unreachable" errors: +If nodes fail to obtain IP addresses or show "network is unreachable" errors, work through the following checks: -1. **Verify bridge interface**: Ensure the bridge interface (e.g., `vmbr0`) exists and is UP before starting VMs - ```bash - ip link show vmbr0 - ``` +1. **Verify bridge interface**: Ensure the bridge interface (e.g., `vmbr0`) exists and is UP before starting VMs: + + ```bash + ip link show vmbr0 + ``` 2. **Check DHCP server**: Ensure DHCP server is running and reachable from the bridge network @@ -358,7 +313,9 @@ If nodes fail to obtain IP addresses or show "network is unreachable" errors: 5. **Serial console**: Enable serial console to view early boot logs and network initialization messages -### Disk Controller Issues +### Disk controller issues + +If you run into disk-related problems during setup, the following may help: - **Configuration rejected**: If you see "virtio-scsi-single disk controller is not supported", use `--disks virtio:10GiB` instead of `--disks virtio-scsi-single:10GiB` - **Bootstrap hangs**: If bootstrap hangs or disks aren't discovered, verify you're using **VirtIO SCSI** (not "VirtIO SCSI Single") diff --git a/public/talos/v1.14/configure-your-talos-cluster/lifecycle-management/upgrading-talos.mdx b/public/talos/v1.14/configure-your-talos-cluster/lifecycle-management/upgrading-talos.mdx index baf65745..5070dde6 100644 --- a/public/talos/v1.14/configure-your-talos-cluster/lifecycle-management/upgrading-talos.mdx +++ b/public/talos/v1.14/configure-your-talos-cluster/lifecycle-management/upgrading-talos.mdx @@ -94,7 +94,7 @@ as: {` - $ talosctl upgrade --nodes 10.20.30.40 \\ + talosctl upgrade --nodes 10.20.30.40 \\ --image ghcr.io/siderolabs/installer:${release_v1_14} `} diff --git a/public/talos/v1.14/getting-started/quickstart.mdx b/public/talos/v1.14/getting-started/quickstart.mdx index e3bd0abe..211249aa 100644 --- a/public/talos/v1.14/getting-started/quickstart.mdx +++ b/public/talos/v1.14/getting-started/quickstart.mdx @@ -1,6 +1,5 @@ --- title: Quickstart -weight: 20 description: "A short guide on setting up a simple Talos Linux cluster locally with Docker." --- @@ -10,7 +9,7 @@ import { VersionWarningBanner } from "/snippets/version-warning-banner.jsx" import { release_v1_14, k8s_release } from '/snippets/custom-variables.mdx'; -The easiest way to run Talos is by using the CLI (`talosctl`) to create a cluster on a machine with `docker` installed. +This guide walks you through spinning up a local Talos Linux cluster using Docker and `talosctl` in a few minutes. + > -### Prerequisites +## Prerequisites -#### `talosctl` +Before you begin, make sure you have the following tools installed: -Download `talosctl` (macOS or Linux): +- `talosctl`: On macOS or Linux, install `talosctl` by running the command below. For other systems, see the [talosctl documentation](./talosctl): -```bash -brew install siderolabs/tap/talosctl -``` + ```bash + brew install siderolabs/tap/talosctl + ``` -#### `kubectl` +- `kubectl`: Refer to the [Install kubectl documentation](https://kubernetes.io/docs/tasks/tools/install-kubectl/) for installation instructions. -Download `kubectl` via one of the methods outlined in the [documentation](https://kubernetes.io/docs/tasks/tools/install-kubectl/). +## Create the cluster -### Create the cluster +With your prerequisites in place, run the following command to create a cluster on Docker: -Now run the following: - -```bash +````bash talosctl cluster create docker -``` +```` -If you are using Docker Desktop on a macOS computer, if you encounter the error: *Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?* you may need to manually create the link for the Docker socket: -```sudo ln -s "$HOME/.docker/run/docker.sock" /var/run/docker.sock``` - +If you are using Docker Desktop on macOS and encounter the error: *Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?*, you may need to manually create the Docker socket symlink: +````sudo ln -s "$HOME/.docker/run/docker.sock" /var/run/docker.sock``` -You can explore using Talos API commands: +Once the cluster is up, verify that you can reach Kubernetes by running: ```bash -talosctl dashboard --nodes 10.5.0.2 +kubectl get nodes -o wide ``` -Verify that you can reach Kubernetes: +You should see output similar to the following: -{` -kubectl get nodes -o wide - -NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME -talos-default-controlplane-1 Ready master 115s v${k8s_release} 10.5.0.2 Talos ${release_v1_14} containerd://2.1.5 -talos-default-worker-1 Ready 115s v${k8s_release} 10.5.0.3 Talos ${release_v1_14} containerd://2.1.5 -`} +{` NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME\ntalos-default-controlplane-1 Ready master 115s v${k8s_release} 10.5.0.2 Talos ${release_v1_14} containerd://2.1.5\ntalos-default-worker-1 Ready 115s v${k8s_release} 10.5.0.3 Talos ${release_v1_14} containerd://2.1.5 `} -### Destroy the cluster +From here, you can explore the Talos API using `talosctl`: + +```bash +talosctl --help +``` + +## Destroy the cluster -When you are all done, remove the cluster: +When you are done, run the following command to tear down the cluster and clean up all associated resources: ```bash talosctl cluster destroy diff --git a/public/talos/v1.14/getting-started/system-requirements.mdx b/public/talos/v1.14/getting-started/system-requirements.mdx index 03677ed6..9ed7424a 100644 --- a/public/talos/v1.14/getting-started/system-requirements.mdx +++ b/public/talos/v1.14/getting-started/system-requirements.mdx @@ -8,8 +8,12 @@ import { VersionWarningBanner } from "/snippets/version-warning-banner.jsx" +This page outlines the system requirements for running Talos Linux. Requirements are listed for both control plane and worker nodes, and are broadly similar to those of Kubernetes. + ## Minimum requirements +The following are the minimum system requirements for running Talos Linux on control plane and worker nodes: + @@ -35,7 +39,9 @@ import { VersionWarningBanner } from "/snippets/version-warning-banner.jsx"
-## Recommended +## Recommended requirements + +For production or heavier workloads, the following specifications are recommended: @@ -62,12 +68,10 @@ import { VersionWarningBanner } from "/snippets/version-warning-banner.jsx"
-These requirements are similar to that of Kubernetes. - -## Storage +## Storage requirement -Talos Linux itself only requires less than 100 MB of disk space, but the EPHEMERAL partition is used to store pulled images, container work directories, and so on. Thus a minimum of 10 GiB of disk space is required. 100 GiB is recommended. +While Talos Linux itself requires less than 100 MB of disk space, the EPHEMERAL partition is used to store pulled images, container working directories, and other runtime data — so a minimum of 10 GiB of total disk space is required, with 100 GiB recommended. -Talos manages disk partitioning automatically during installation, creating EFI, META, STATE, and EPHEMERAL partitions. The EPHEMERAL partition then expands to fill all the space left after the first three. That space can either remain entirely with EPHEMERAL or be divided into additional user volumes, depending on your needs. See [Disk Layout](../configure-your-talos-cluster/storage-and-disk-management/disk-management#disk-layout) for details. +During installation, Talos automatically manages disk partitioning, creating EFI, META, STATE, and EPHEMERAL partitions. The EPHEMERAL partition expands to fill all remaining space after the first three partitions. That space can either remain entirely with EPHEMERAL or be divided into additional user volumes, depending on your needs. See [Disk Layout](../configure-your-talos-cluster/storage-and-disk-management/disk-management#disk-layout) for details. -For production, it is often more efficient to dedicate a smaller disk for the Talos installation itself, and use additional disks for workload storage. Using a large, single disk for both system and workloads is supported, but may not be optimal depending on your environment. +For production environments, it is often more efficient to dedicate a smaller disk to the Talos installation and use separate disks for workload storage. Using a single large disk for both system and workloads is supported, but may not be optimal depending on your setup. diff --git a/public/talos/v1.14/getting-started/talosctl.mdx b/public/talos/v1.14/getting-started/talosctl.mdx index fd49bf12..61f3e057 100644 --- a/public/talos/v1.14/getting-started/talosctl.mdx +++ b/public/talos/v1.14/getting-started/talosctl.mdx @@ -7,37 +7,34 @@ import { VersionWarningBanner } from "/snippets/version-warning-banner.jsx" -## Recommended +Use `talosctl` to interact with the Talos API, manage nodes, and configure your Talos machines. Follow the instructions below to install it on your operating system. -The client can be installed and updated via the [Homebrew package manager](https://brew.sh/) for macOS and Linux. -You will need to install `brew` and then you can install `talosctl` from the Sidero Labs tap. +## Linux and macOS + +The recommended way to install `talosctl` on Linux and macOS is via the [Homebrew package manager](https://brew.sh/). This method automatically keeps `talosctl` up to date as new versions are released, without any manual intervention. ```bash brew install siderolabs/tap/talosctl ``` -This will also keep your version of `talosctl` up to date with new releases. -This homebrew tap also has formulae for `omnictl` if you need to install that package. +The same Homebrew tap also includes a formula for `omnictl`. -> Note: Your `talosctl` version should match the version of Talos Linux you are running on a host. -> To install a specific version of `talosctl` with `brew` you can follow [this github issue](https://github.com/siderolabs/homebrew-tap/issues/75). +Your `talosctl` version should match the version of Talos Linux running on your host. To install a specific version of `talosctl` with Homebrew, refer to [this GitHub issue](https://github.com/siderolabs/homebrew-tap/issues/75) for guidance. ## Alternative install -You can automatically install the correct version of `talosctl` for your operating system and architecture with an installer script. -This script won't keep your version updated with releases and you will need to re-run the script to download a new version. +If Homebrew is not available in your environment, you can use the installer script to automatically download the correct version of `talosctl` for your operating system and architecture: ```bash curl -sL https://talos.dev/install | sh ``` -This script will work on macOS, Linux, and WSL on Windows. -It supports amd64 and arm64 architecture. +This script supports macOS, Linux, and WSL on Windows, and works on both amd64 and arm64 architectures. -## Manual and Windows install +Note that this method does not update automatically, you will need to re-run the script to install a newer version. -All versions can be manually downloaded from the [talos releases page](https://github.com/siderolabs/talos/releases/) including Linux, macOS, and Windows. +## Windows -You will need to add the binary to a folder part of your executable `$PATH` to use it without providing the full path to the executable. +For Windows, all `talosctl` releases can be downloaded manually from the [Talos releases page](https://github.com/siderolabs/talos/releases/), which includes binaries for Linux, macOS, and Windows. -Updating the binary will be a manual process. +Once downloaded, add the binary to a folder on your `$PATH` so you can run it without specifying the full path to the executable. As with the alternative install method, updating `talosctl` on Windows is a manual process. diff --git a/public/talos/v1.14/platform-specific-installations/bare-metal-platforms/iso.mdx b/public/talos/v1.14/platform-specific-installations/bare-metal-platforms/iso.mdx index e3735746..9766a02e 100644 --- a/public/talos/v1.14/platform-specific-installations/bare-metal-platforms/iso.mdx +++ b/public/talos/v1.14/platform-specific-installations/bare-metal-platforms/iso.mdx @@ -7,19 +7,17 @@ import { VersionWarningBanner } from "/snippets/version-warning-banner.jsx" -Talos can be installed on bare-metal machine using an ISO image. -ISO images for `amd64` and `arm64` architectures are available on the [Talos releases page](https://github.com/siderolabs/talos/releases/latest/). +Talos can be installed on bare-metal machines using an ISO image. ISO images for `amd64` and `arm64` architectures are available on the [Talos releases page](https://github.com/siderolabs/talos/releases/latest/). -Talos doesn't install itself to disk when booted from an ISO until the machine configuration is applied. +When booted from an ISO, Talos does not install itself to disk until a machine configuration is applied. -Please follow the [getting started guide](../../getting-started/getting-started) for the generic steps on how to install Talos. +For steps on how to install Talos, follow the [getting started guide](../../getting-started/getting-started). -> Note: If there is already a Talos installation on the disk, the machine will boot into that installation when booting from a Talos ISO. -> The boot order should prefer disk over ISO, or the ISO should be removed after the installation to make Talos boot from disk. +If a Talos installation already exists on the disk, the machine will boot into that installation even when booting from a Talos ISO. The boot order should prefer disk over ISO, or the ISO should be removed after installation to ensure Talos boots from disk. -See [kernel parameters reference](../../reference/kernel) for the list of kernel parameters supported by Talos. +See the [kernel parameters reference](../../reference/kernel) for a list of kernel parameters supported by Talos. -There are two flavors of ISO images available: +Two flavors of ISO image are available: -* `metal-.iso` supports booting on BIOS and UEFI systems (for x86, UEFI only for arm64) -* `metal--secureboot.iso` supports booting on only UEFI systems in SecureBoot mode (via [Image Factory](../../learn-more/image-factory)) +* `metal-.iso` supports booting on BIOS and UEFI systems (UEFI only for arm64) +* `metal--secureboot.iso` supports booting on UEFI systems in SecureBoot mode only (via [Image Factory](../../learn-more/image-factory)) diff --git a/public/talos/v1.14/platform-specific-installations/virtualized-platforms/proxmox.mdx b/public/talos/v1.14/platform-specific-installations/virtualized-platforms/proxmox.mdx index 5e1d8956..99caaa75 100644 --- a/public/talos/v1.14/platform-specific-installations/virtualized-platforms/proxmox.mdx +++ b/public/talos/v1.14/platform-specific-installations/virtualized-platforms/proxmox.mdx @@ -6,303 +6,249 @@ aliases: --- import { VersionWarningBanner } from "/snippets/version-warning-banner.jsx" +import { release_v1_14 } from '/snippets/custom-variables.mdx'; -import { release_v1_14 } from '/snippets/custom-variables.mdx'; - -In this guide we will create a Kubernetes cluster using Proxmox. +In this guide you will create a Kubernetes cluster on Proxmox, from uploading the Talos ISO to bootstrapping etcd and retrieving your kubeconfig. Follow the steps in order — each one builds on the last. ## Video walkthrough -To see a live demo of this writeup, visit Youtube here: +To see a live demo of this guide, visit Youtube here: -## Installation +## Prerequisites + +Before you begin, make sure the following are in place. The QEMU guest agent section is optional, only follow it if you need guest VM shutdown support. -### How to get Proxmox +- **Proxmox**: This guide assumes you have already installed Proxmox on the server where you want to create Talos VMs. Visit the [Proxmox downloads page](https://www.proxmox.com/en/downloads) if you haven't done this yet. -It is assumed that you have already installed Proxmox onto the server you wish to create Talos VMs on. -Visit the [Proxmox](https://www.proxmox.com/en/downloads) downloads page if necessary. +- **talosctl**: Install `talosctl` on macOS or Linux with: -### Install talosctl + ```bash + brew install siderolabs/tap/talosctl + ``` -You can download `talosctl` on MacOS and Linux via: + For manual installation and other platforms, see the [talosctl installation guide](../../getting-started/talosctl). -```bash -brew install siderolabs/tap/talosctl -``` +- **ISO image**: Download the Talos ISO from [Image Factory](https://www.talos.dev/latest/talos-guides/install/boot-assets/#image-factory) with this command: -For manual installation and other platforms please see the [talosctl installation guide](../../getting-started/talosctl ). + ```bash + mkdir -p _out/ + curl https://factory.talos.dev/image/376567988ad370138ad8b2698212367b8edcb69b5fd68c80be1f2ec7d603b4ba//metal-.iso -L -o _out/metal-.iso + ``` -### Download ISO image + For example version, for linux platform: -In order to install Talos in Proxmox, you will need the ISO image from [Image Factory](https://www.talos.dev/latest/talos-guides/install/boot-assets/#image-factory). + + {`mkdir -p _out/\ncurl https://factory.talos.dev/image/376567988ad370138ad8b2698212367b8edcb69b5fd68c80be1f2ec7d603b4ba/${release_v1_14}/metal-amd64.iso -L -o _out/metal-amd64.iso`} + -```bash -mkdir -p _out/ -curl https://factory.talos.dev/image/376567988ad370138ad8b2698212367b8edcb69b5fd68c80be1f2ec7d603b4ba//metal-.iso -L -o _out/metal-.iso -``` +### Optional: QEMU guest agent ISO -For example version {release } for `linux` platform: +QEMU guest agent support for guest VM shutdowns requires a custom ISO and installer image. Skip this section if that doesn't apply to you. - - {` -mkdir -p _out/ -curl https://factory.talos.dev/image/376567988ad370138ad8b2698212367b8edcb69b5fd68c80be1f2ec7d603b4ba/${release_v1_14}/metal-amd64.iso -L -o _out/metal-amd64.iso - `} - +1. **Build the custom ISO**: Go to the [Image Factory](https://factory.talos.dev/) and complete these steps to build a custom ISO: + - Select your Talos version + - Check the box for `siderolabs/qemu-guest-agent` and submit. This action will provide you with a link similar to: + + {`Metal ISO\n\namd64 ISO\n https://factory.talos.dev/image/ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515/${release_v1_14}/metal-amd64.iso\narm64 ISO\n https://factory.talos.dev/image/ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515/${release_v1_14}/metal-arm64.iso\n\nInstaller Image\n\nFor the initial Talos install or upgrade, use the following installer image:\nfactory.talos.dev/installer/ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515:${release_v1_14}`} + + - Download the above ISO and take note of the installer image URL, you will need it later -### QEMU guest agent support (iso) -- If you need the QEMU guest agent so you can do guest VM shutdowns of your Talos VMs, then you will need a custom ISO -- To get this, navigate to https://factory.talos.dev/ -- Scroll down and select your Talos version ( {release_v1_14} for example) -- Then tick the box for `siderolabs/qemu-guest-agent` and submit -- This will provide you with a link to the bare metal ISO -- The lines we're interested in are as follows +2. **Enable QEMU Guest Agent in Proxmox:** Once your VM is created, go to **VM → Options** and set **QEMU Guest Agent** to **Enabled**. - - {` -Metal ISO + Only enable QEMU Guest Agent in Proxmox if you built the ISO with the `siderolabs/qemu-guest-agent` extension. Enabling it without the extension will only generate log spam and provide no functionality. See [Image Factory](../../learn-more/image-factory) for more on building custom ISOs. -amd64 ISO - https://factory.talos.dev/image/ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515/${release_v1_14}/metal-amd64.iso -arm64 ISO - https://factory.talos.dev/image/ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515/${release_v1_14}/metal-arm64.iso -Installer Image +### VM resource requirements -For the initial Talos install or upgrade use the following installer image: -factory.talos.dev/installer/ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515:{release_v1_14} - `} - +Before creating VMs, familiarise yourself with the [system requirements](../../getting-started/system-requirements) for Talos and use the following as a baseline for all Talos nodes on Proxmox: -- Download the above ISO (this will most likely be `amd64` for you) -- Take note of the `factory.talos.dev/installer` URL as you'll need it later +| Setting | Recommended value | Notes | +|---|---|---| +| **BIOS** | `ovmf` (UEFI) | Modern firmware, Secure Boot support, better hardware compatibility | +| **Machine** | `q35` | Modern PCIe-based machine type with better device support | +| **CPU type** | `host` | Enables advanced instruction sets (AVX-512, etc.). Use `kvm64` with feature flags for Proxmox < 8.0 | +| **CPU cores** | 2+ (control plane), 4+ (workers) | Minimum 2 cores required | +| **Memory** | 4GB+ (control plane), 8GB+ (workers) | Minimum 2GB required | +| **Disk controller** | VirtIO SCSI | Do NOT use **VirtIO SCSI Single** — causes bootstrap hangs ([#11173](https://github.com/siderolabs/talos/issues/11173)) | +| **Disk format** | Raw or QCOW2 | Raw preferred for performance; QCOW2 for snapshots | +| **Disk cache** | Write Through | Use None for clustered environments | +| **Network model** | `virtio` | Paravirtualized driver, best performance (up to 10 Gbit) | +| **EFI disk** | 4MB | Required for UEFI firmware, stores Secure Boot keys | +| **Ballooning** | Disabled | Talos does not support memory hotplug | +| **RNG device** | VirtIO RNG (optional) | Better entropy for cryptographic operations | -## Upload ISO +## Step 1: Upload the ISO -From the Proxmox UI, select the "local" storage and enter the "Content" section. -Click the "Upload" button: +From the Proxmox UI, select the **local** storage and enter the **Content** section. Click the **Upload** button: -Select the ISO you downloaded previously, then hit "Upload" +Select the ISO you downloaded in the prerequisites, then click **Upload**: -## Create VMs - -Before starting, familiarise yourself with the -[system requirements](../../getting-started/system-requirements) for Talos and assign VM -resources accordingly. - -### Recommended baseline VM configuration - -Use the following baseline settings for Proxmox VMs running Talos: - -| Setting | Recommended Value | Notes | -|---------|------------------|-------| -| **BIOS** | `ovmf` (UEFI) | Modern firmware, Secure Boot support, better hardware compatibility | -| **Machine** | `q35` | Modern PCIe-based machine type with better device support | -| **CPU Type** | `host` | Enables advanced instruction sets (AVX-512, etc.), best performance. Alternative: `kvm64` with feature flags for Proxmox < 8.0 | -| **CPU Cores** | 2+ (control plane), 4+ (workers) | Minimum 2 cores required | -| **Memory** | 4GB+ (control plane), 8GB+ (workers) | Minimum 2GB required | -| **Disk Controller** | **VirtIO SCSI** (NOT "VirtIO SCSI Single") | Single controller can cause bootstrap hangs (#11173) | -| **Disk Format** | Raw (performance) or QCOW2 (features/snapshots) | Raw preferred for performance | -| **Disk Cache** | Write Through (safe default) | Or None for clustered environments | -| **Network Model** | `virtio` | Paravirtualized driver, best performance (up to 10 Gbit) | -| **EFI Disk** | 4MB (for OVMF) | Required for UEFI firmware, stores Secure Boot keys | -| **Ballooning** | Disabled | Talos doesn't support memory hotplug | -| **RNG Device** | VirtIO RNG (optional) | Better entropy for security | - -> **Important**: When configuring the disk controller, use **VirtIO SCSI** (not "VirtIO SCSI Single"). -> Using "VirtIO SCSI Single" can cause bootstrap to hang or prevent disk discovery. -> See [issue #11173](https://github.com/siderolabs/talos/issues/11173) for details. +## Step 2: Create VMs -Create a new VM by clicking the "Create VM" button in the Proxmox UI: +Create a new VM by clicking **Create VM** in the Proxmox UI: -Fill out a name for the new VM: +This action would open 8 tabs, configure the tabs as follows: + +- **Name tab**: Fill out a name for the VM: -In the OS tab, select the ISO we uploaded earlier: +- **OS tab**; Select the ISO uploaded in Step 1: -In the "System" tab: -- Set **BIOS** to `ovmf` (UEFI) for modern firmware and Secure Boot support -- Set **Machine** to `q35` for modern PCIe-based machine type -- Add **EFI Disk** (4MB) for persistent UEFI settings and Secure Boot key storage - -In the "Hard Disk" tab: -- Set **Bus/Device** to `VirtIO SCSI` (NOT "VirtIO SCSI Single") -- Set **Storage** to your main storage pool -- Set **Format** to `Raw` (better performance) or `QCOW2` (features/snapshots) -- Set **Size** based on your workload requirements (adjust based on CSI and application needs) -- Set **Cache** to `Write Through` (safe default) or `None` for clustered environments -- Enable **Discard** (TRIM support) if using SSD storage -- Enable **SSD emulation** if using SSD storage - -> **Important**: When configuring the disk controller, use **VirtIO SCSI** (not "VirtIO SCSI Single"). -> Using "VirtIO SCSI Single" can cause bootstrap to hang or prevent disk discovery. -> See [issue #11173](https://github.com/siderolabs/talos/issues/11173) for details. - -In the "CPU" section: -- Set **Cores** to 2+ for control planes, 4+ for workers -- Set **Sockets** to 1 (keep simple) -- Set **Type** to `host` (best performance, enables advanced instruction sets) - - **Alternative for Proxmox < 8.0**: Use `kvm64` with feature flags by adding to `/etc/pve/qemu-server/.conf`: - ```text - args: -cpu kvm64,+cx16,+lahf_lm,+popcnt,+sse3,+ssse3,+sse4.1,+sse4.2 - ``` - - **Note**: `host` CPU type prevents live VM migration but provides best performance +- **System tab:** + - Set **BIOS** to `ovmf` (UEFI) + - Set **Machine** to `q35` + - Add an **EFI Disk** (4MB) for persistent UEFI settings and Secure Boot key storage + +- **Hard Disk tab:** + - Set **Bus/Device** to `VirtIO SCSI` (NOT **VirtIO SCSI Single**) + - Set **Storage** to your main storage pool + - Set **Format** to `Raw` (performance) or `QCOW2` (snapshots) + - Set **Cache** to `Write Through`, or `None` for clustered environments + - Enable **Discard** and **SSD emulation** if using SSD storage + +- **CPU tab:** + - Set **Cores** to 2+ for control planes, 4+ for workers + - Set **Type** to `host` for best performance + + For Proxmox < 8.0, use `kvm64` with feature flags instead. Add the following to `/etc/pve/qemu-server/.conf`: + ```text + args: -cpu kvm64,+cx16,+lahf_lm,+popcnt,+sse3,+ssse3,+sse4.1,+sse4.2 + ``` + + > **Note:** `host` CPU type prevents live VM migration. -In the "Memory" section: -- Set **Memory** to 4GB+ for control planes, 8GB+ for workers (minimum 2GB required) -- **Disable Ballooning** (can cause issues with Talos memory detection) +- **Memory tab:** + - Set **Memory** to 4GB+ for control planes, 8GB+ for workers + - **Disable Ballooning** — Talos does not support memory hotplug, and enabling it will prevent Talos from seeing all available memory - + -In the "Network" section: -- Set **Model** to `virtio` (paravirtualized driver, best performance) -- Set **Bridge** to your network bridge (e.g., `vmbr0`) -- Verify the VM is set to come up on the bridge interface +- **Network tab:** + - Set **Model** to `virtio` + - Set **Bridge** to your network bridge (e.g., `vmbr0`) - + -> **Tip**: Enable a serial console (ttyS0) in Proxmox VM settings to see early boot logs and troubleshoot network connectivity issues. -> This is especially helpful when debugging DHCP timing or bridge configuration problems. -> Set **Serial port** to `ttyS0` in Proxmox and add `console=ttyS0` if you're customizing kernel args. +Enable a serial console (`ttyS0`) in Proxmox VM settings to see early boot logs and troubleshoot network issues. This is especially useful when debugging DHCP timing or bridge configuration problems. -Finish creating the VM by clicking through the "Confirm" tab and then "Finish". +- **Confirm tab**: Finish creating the VM by clicking through the **Confirm** tab and then **Finish**. -Repeat this process for a second VM to use as a worker node. -You can also repeat this for additional nodes desired. +Repeat this process for each additional node, at minimum one control plane and one worker. -> Note: Talos doesn't support memory hot plugging, if creating the VM programmatically don't enable memory hotplug on your -> Talos VM's. -> Doing so will cause Talos to be unable to see all available memory and have insufficient memory to complete -> installation of the cluster. +## Step 3: Start the control plane node -## Start control plane node +Start the VM designated as the first control plane node. It will boot from the ISO and enter maintenance mode. -Once the VMs have been created and updated, start the VM that will be the first control plane node. -This VM will boot the ISO image specified earlier and enter "maintenance mode". + + -### With DHCP server +Once the machine enters maintenance mode, the console will display the IP address the node received. Take note of this IP — it will be referred to as `$CONTROL_PLANE_IP` for the rest of this guide: -Once the machine has entered maintenance mode, there will be a console log that details the IP address that the node received. -Take note of this IP address, which will be referred to as `$CONTROL_PLANE_IP` for the rest of this guide. -If you wish to export this IP as a bash variable, simply issue a command like `export CONTROL_PLANE_IP=1.2.3.4`. +```bash +export CONTROL_PLANE_IP=1.2.3.4 +``` -### Without DHCP server + + -To apply the machine configurations in maintenance mode, VM has to have IP on the network. -So you can set it on boot time manually. +To apply machine configuration in maintenance mode, the VM needs a static IP. Set it manually at boot time. + +1. Press `e` at the GRUB menu: -Press `e` on the boot time. -And set the IP parameters for the VM. -[Format is](https://www.kernel.org/doc/Documentation/filesystems/nfs/nfsroot.txt): +2. Append the IP parameters to the boot line using the following format ([kernel nfsroot format](https://www.kernel.org/doc/Documentation/filesystems/nfs/nfsroot.txt)): -```bash -ip=:::::: -``` + ```bash + ip=:::::: + ``` -For example $CONTROL_PLANE_IP will be 192.168.0.100 and gateway 192.168.0.1 + For example, with `$CONTROL_PLANE_IP` as `192.168.0.100` and gateway `192.168.0.1`: -```bash -linux /boot/vmlinuz init_on_alloc=1 slab_nomerge pti=on panic=0 consoleblank=0 printk.devkmsg=on earlyprintk=ttyS0 console=tty0 console=ttyS0 talos.platform=metal ip=192.168.0.100::192.168.0.1:255.255.255.0::eth0:off -``` + ```bash + linux /boot/vmlinuz init_on_alloc=1 slab_nomerge pti=on panic=0 consoleblank=0 printk.devkmsg=on earlyprintk=ttyS0 console=tty0 console=ttyS0 talos.platform=metal ip=192.168.0.100::192.168.0.1:255.255.255.0::eth0:off + ``` -Then press Ctrl-x or F10 +3. Press `Ctrl-x` or `F10` to boot with the updated parameters. + + + +## Step 4: Generate machine configurations -## Generate machine configurations +With `$CONTROL_PLANE_IP` set, generate the machine configurations for your cluster. -With the IP address above, you can now generate the machine configurations to use for installing Talos and Kubernetes. -Issue the following command, updating the output directory, cluster name, and control plane IP as you see fit: +Choose the tab that matches your ISO. + + + ```bash talosctl gen config talos-proxmox-cluster https://$CONTROL_PLANE_IP:6443 --output-dir _out ``` - -This will create several files in the `_out` directory: `controlplane.yaml`, `worker.yaml`, and `talosconfig`. - -> Note: The Talos config by default will install to `/dev/sda`. -> Depending on your setup the virtual disk may be mounted differently Eg: `/dev/vda`. -> You can check for disks running the following command: -> -> ```bash -> talosctl get disks --insecure --nodes $CONTROL_PLANE_IP -> ``` -> -> Update `controlplane.yaml` and `worker.yaml` config files to point to the correct disk location. - -### QEMU guest agent support - -For QEMU guest agent support, you can generate the config with the custom install image: + + +Use this if you built the custom ISO with the `siderolabs/qemu-guest-agent` extension in the prerequisites. The command is the same but passes the custom installer image URL you noted earlier. - {` -talosctl gen config talos-proxmox-cluster https://$CONTROL_PLANE_IP:6443 --output-dir _out --install-image factory.talos.dev/installer/ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515:${release_v1_14} - `} + {`talosctl gen config talos-proxmox-cluster https://$CONTROL_PLANE_IP:6443 --output-dir _out --install-image factory.talos.dev/installer/ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515:${release_v1_14}`} + + + +This command creates three files in `_out/`: `controlplane.yaml`, `worker.yaml`, and `talosconfig`. -> **Important**: Enable QEMU Guest Agent in Proxmox **only if** you built the ISO with the `siderolabs/qemu-guest-agent` extension in **Image Factory**. -> If you're using a standard Talos ISO without this extension, leave QEMU Guest Agent disabled in Proxmox VM settings. -> Enabling it without the extension will only generate log spam and won't provide any functionality. -> See: [Image Factory](../../learn-more/image-factory) for building a custom ISO with extensions. + +The default config installs Talos to `/dev/sda`. Depending on your setup, the virtual disk may be at a different path (e.g., `/dev/vda`). Check available disks with: +```bash +talosctl get disks --insecure --nodes $CONTROL_PLANE_IP +``` -- If you did include the extension, go to your VM → **Options** and set **QEMU Guest Agent** to **Enabled**. +Update `install.disk` in `controlplane.yaml` and `worker.yaml` if needed. + -## Create control plane node +## Step 5: Apply configuration to the control plane -Using the `controlplane.yaml` generated above, you can now apply this config using talosctl. -Issue: +Apply the control plane configuration to the node. If you are setting up an HA control plane, repeat this step for each additional control plane node. ```bash talosctl apply-config --insecure --nodes $CONTROL_PLANE_IP --file _out/controlplane.yaml ``` -You should now see some action in the Proxmox console for this VM. -Talos will be installed to disk, the VM will reboot, and then Talos will configure the Kubernetes control plane on this VM. -The VM will remain in stage `Booting` until the bootstrap is completed in a later step. - -> Note: This process can be repeated multiple times to create an HA control plane. - -## Create worker node +You should see activity in the Proxmox console. Talos will install to disk, the VM will reboot, and Talos will configure the Kubernetes control plane. The VM will remain in stage `Booting` until the cluster is bootstrapped in the next steps. -Create at least a single worker node using a process similar to the control plane creation above. -Start the worker node VM and wait for it to enter "maintenance mode". -Take note of the worker node's IP address, which will be referred to as `$WORKER_IP` +## Step 6: Apply configuration to worker nodes -Issue: +Start each worker VM and wait for it to enter maintenance mode. Repeat this step for each worker node, substituting `$WORKER_IP` with each node's IP address. ```bash talosctl apply-config --insecure --nodes $WORKER_IP --file _out/worker.yaml ``` -> Note: This process can be repeated multiple times to add additional workers. +## Step 7: Bootstrap the cluster -## Using the cluster +With the control plane and worker nodes configured, complete the cluster setup by running the following commands in order: -Once the cluster is available, you can make use of `talosctl` and `kubectl` to interact with the cluster. -For example, to view current running containers, run `talosctl containers` for a list of containers in the `system` namespace, or `talosctl containers -k` for the `k8s.io` namespace. -To view the logs of a container, use `talosctl logs ` or `talosctl logs -k `. - -First, configure talosctl to talk to your control plane node by issuing the following, updating paths and IPs as necessary: +1. Configure talosctl to talk to your control plane node: ```bash export TALOSCONFIG="_out/talosconfig" @@ -310,27 +256,35 @@ talosctl config endpoint $CONTROL_PLANE_IP talosctl config node $CONTROL_PLANE_IP ``` -### Bootstrap etcd +2. Bootstrap etcd: ```bash talosctl bootstrap ``` -### Retrieve the `kubeconfig` - -At this point we can retrieve the admin `kubeconfig` by running: +3. Retrieve the kubeconfig: ```bash talosctl kubeconfig . ``` +4. Verify that your cluster is ready: + +```bash +kubectl get nodes +``` + ## Troubleshooting +This section covers common issues you may encounter when setting up Talos VMs on Proxmox, along with steps to resolve them. + ### Cluster creation issues +If `talosctl cluster create` fails with disk controller errors, the most likely cause is an unsupported disk controller type. For example: If `talosctl cluster create` fails with disk controller errors: -- **"virtio-scsi-single disk controller is not supported"**: This disk controller type causes Talos bootstrap to hang. Use `virtio` or `scsi` instead: +- `virtio-scsi-single disk controller is not supported`: This disk controller type causes Talos bootstrap to hang. Use `virtio` or `scsi` instead: + ```bash # Wrong - will be rejected talosctl cluster create --disks virtio-scsi-single:10GiB @@ -342,12 +296,13 @@ If `talosctl cluster create` fails with disk controller errors: ### Network connectivity issues -If nodes fail to obtain IP addresses or show "network is unreachable" errors: +If nodes fail to obtain IP addresses or show "network is unreachable" errors, work through the following checks: -1. **Verify bridge interface**: Ensure the bridge interface (e.g., `vmbr0`) exists and is UP before starting VMs - ```bash - ip link show vmbr0 - ``` +1. **Verify bridge interface**: Ensure the bridge interface (e.g., `vmbr0`) exists and is UP before starting VMs: + + ```bash + ip link show vmbr0 + ``` 2. **Check DHCP server**: Ensure DHCP server is running and reachable from the bridge network @@ -358,7 +313,9 @@ If nodes fail to obtain IP addresses or show "network is unreachable" errors: 5. **Serial console**: Enable serial console to view early boot logs and network initialization messages -### Disk Controller Issues +### Disk controller issues + +If you run into disk-related problems during setup, the following may help: - **Configuration rejected**: If you see "virtio-scsi-single disk controller is not supported", use `--disks virtio:10GiB` instead of `--disks virtio-scsi-single:10GiB` - **Bootstrap hangs**: If bootstrap hangs or disks aren't discovered, verify you're using **VirtIO SCSI** (not "VirtIO SCSI Single")