Skip to content

Commit 1b107b9

Browse files
garloffgarloff
authored
Add optional availabilityZone.from:Machine to disks. (#263)
* Add optional availabilityZone.from:Machine to disks. Previously capo defaulted to creating disks in the failureDomain (availability zone) of the virtual machine that is created. In some clouds this is a requirement for good performance or to be able to attach the disk at all. Other clouds have a global AZ for disks, so we need a pararmeter to control this. So we introduce three new boolean variables, all defaulting to false: controlPlaneRootDiskPin workerRootDiskPin workerAdditionalBlockDevicesPin which control whether we set availabilityZone/from: Machine for the respective disks. Signed-off-by: Kurt Garloff <kurt@garloff.de> * Use one setting diskPinAZ to control all volumes. This was suggested by @Nils98Ar. There are few imaginable reasons why one would want to pin volumes into the AZs of the VMs they are attached to for some types but not for others. So have one setting, controlling all of controlPlaneNodes root disks, workerMachines root disks and worker additionalBlockDevices. Also add a bit of documentation. Signed-off-by: Kurt Garloff <kurt@garloff.de> --------- Signed-off-by: Kurt Garloff <kurt@garloff.de>
1 parent a931cdb commit 1b107b9

2 files changed

Lines changed: 89 additions & 6 deletions

File tree

‎docs/providers/openstack/configuration.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,5 +80,6 @@ topology:
8080
| `oidc_config.username_prefix` | string | oidc: | oidc: | Prefix prepended to username claims to prevent cla shes with existing names (such as system: users). For example, the value oid c: will create usernames like oidc:jane.doe. If this flag isn't provided and --o idc-username-claim is a value other than email the prefix defaults to ( Iss uer URL )# where ( Issuer URL ) is the value of --oidc-issuer-url. The value - c an be used to disable all prefixing. | |
8181
| `oidc_config.groups_prefix` | string | oidc: | oidc: | Prefix prepended to group claims to prevent clashes wit h existing names (such as system: groups). For example, the value oidc: will cre ate group names like oidc:engineering and oidc:infra. | |
8282
| `network_mtu` | integer | | 1500 | NetworkMTU sets the maximum transmission unit (MTU) value to address fragmentation for the private network ID. | False |
83+
| `diskPinAZ` | boolean | | False | This tells capo to create volumes in the same AZ as the VM that they are attached to. This affects cinder volumes used as controlPlane and worker machine root disks as well as additionalBlockDevices. Only works for clouds where volume AZs are matched with compute AZs. There it might be a performance optimization or a requirement.
8384
| `controlPlaneAvailabilityZones` | array | | ['nova'] | ControlPlaneAvailabilityZones is the set of availability zones which control plane machines may be deployed to. | False |
8485
| `controlPlaneOmitAvailabilityZone` | boolean | | True | ControlPlaneOmitAvailabilityZone causes availability zone to be omitted when creating control plane nodes, allowing the Nova scheduler to make a decision on which availability zone to use based on other scheduling constraints. | False |

‎providers/openstack/scs2/cluster-class/templates/cluster-class.yaml‎

Lines changed: 88 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -110,6 +110,21 @@ spec:
110110
Cluster actuator will create a network, a subnet with nodeCIDR,
111111
and a router connected to this subnet.
112112
If you leave this empty, no network will be created.
113+
- name: diskPinAZ
114+
required: false
115+
schema:
116+
openAPIV3Schema:
117+
type: boolean
118+
example: true
119+
default: false
120+
description: |-
121+
Pass availabilityZone (failureDomain) of the machine to its (root) disk volumes.
122+
Some clouds have a 1:1 relationship of storage availability zones with compute AZs.
123+
In these, attaching a disk from another AZ often comes with a performance penalty
124+
or is not possible at all. You can set this to true to tell capo to create the root
125+
disk in the stated failureDomain of this machine. This setting applies to controlPlane
126+
root disks (if they use root volumes), worker machine root disks (if they use root
127+
volumes) and workerAdditionalBlockDevices (if any are configured).
113128
# Control plane
114129
- name: controlPlaneFlavor
115130
required: false
@@ -119,8 +134,9 @@ spec:
119134
default: "SCS-2V-4"
120135
example: "SCS-2V-4-20s"
121136
description: |-
122-
OpenStack instance flavor for control plane nodes.
123-
(Default: SCS-2V-4, specify a controlPlanRootDisk or better replace by SCS-2V-4-20s.)
137+
OpenStack instance flavor for control plane nodes. (The default, SCS-2V-4, is not ideal;
138+
recommendation is to replace this by SCS-2V-4-20s and set controlPlaneRootDisk to 0
139+
or specify a non-zero controlPlaneRootDisk with a diskless flavor.)
124140
- name: controlPlaneRootDisk
125141
required: false
126142
schema:
@@ -132,7 +148,7 @@ spec:
132148
description: |-
133149
Root disk size in GiB for control-plane nodes.
134150
OpenStack volume will be created and used instead of an ephemeral disk defined in flavor.
135-
Should only be used for the diskless flavors (>= 20), otherwise set to 0.
151+
Should only be used for the diskless flavors (and then be set >= 20), otherwise set to 0.
136152
- name: controlPlaneServerGroupID
137153
required: false
138154
schema:
@@ -193,6 +209,9 @@ spec:
193209
schema:
194210
openAPIV3Schema:
195211
type: array
212+
description: |-
213+
You may pass an array with additional volumes (disks) here, consisting of
214+
name, sizeGiB (default: 20) and type (aka storage class, default: __DEFAULT__).
196215
default: []
197216
items:
198217
type: object
@@ -570,7 +589,7 @@ spec:
570589
variable: controlPlaneFlavor
571590
- name: controlPlaneRootDisk
572591
description: "Sets the root disk size in GiB for control-plane nodes."
573-
enabledIf: {{ `'{{ if .controlPlaneRootDisk }}true{{end}}'` }}
592+
enabledIf: {{ `'{{ if and .controlPlaneRootDisk (not .diskPinAZ) }}true{{end}}'` }}
574593
definitions:
575594
- selector:
576595
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
@@ -583,6 +602,23 @@ spec:
583602
valueFrom:
584603
template: |
585604
sizeGiB: {{ `{{ .controlPlaneRootDisk }}` }}
605+
- name: controlPlaneRootDiskPinned
606+
description: "Sets the root disk size in GiB for control-plane nodes and pins them to the failureDomain of the machine."
607+
enabledIf: {{ `'{{ if and .controlPlaneRootDisk .diskPinAZ }}true{{end}}'` }}
608+
definitions:
609+
- selector:
610+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
611+
kind: OpenStackMachineTemplate
612+
matchResources:
613+
controlPlane: true
614+
jsonPatches:
615+
- op: add
616+
path: "/spec/template/spec/rootVolume"
617+
valueFrom:
618+
template: |
619+
sizeGiB: {{ `{{ .controlPlaneRootDisk }}` }}
620+
availabilityZone:
621+
from: Machine
586622
- name: controlPlaneServerGroupID
587623
description: "Sets the server group to assign the control plane nodes to."
588624
enabledIf: {{ `'{{ ne .controlPlaneServerGroupID "" }}'` }}
@@ -697,7 +733,7 @@ spec:
697733
variable: workerFlavor
698734
- name: workerRootDisk
699735
description: "Sets the root disk size in GiB for worker nodes."
700-
enabledIf: {{ `'{{ if .workerRootDisk }}true{{end}}'` }}
736+
enabledIf: {{ `'{{ if and .workerRootDisk (not .diskPinAZ) }}true{{end}}'` }}
701737
definitions:
702738
- selector:
703739
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
@@ -713,6 +749,26 @@ spec:
713749
valueFrom:
714750
template: |
715751
sizeGiB: {{ `{{ .workerRootDisk }}` }}
752+
- name: workerRootDiskPinned
753+
description: "Sets the root disk size in GiB for worker nodes and pins the to the AZ of the machine."
754+
enabledIf: {{ `'{{ if and .workerRootDisk .diskPinAZ }}true{{end}}'` }}
755+
definitions:
756+
- selector:
757+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
758+
kind: OpenStackMachineTemplate
759+
matchResources:
760+
controlPlane: false
761+
machineDeploymentClass:
762+
names:
763+
- default-worker
764+
jsonPatches:
765+
- op: add
766+
path: "/spec/template/spec/rootVolume"
767+
valueFrom:
768+
template: |
769+
sizeGiB: {{ `{{ .workerRootDisk }}` }}
770+
availabilityZone:
771+
from: Machine
716772
- name: workerServerGroupID
717773
description: "Sets the server group to assign the worker nodes to."
718774
enabledIf: {{ `'{{ ne .workerServerGroupID "" }}'` }}
@@ -732,7 +788,31 @@ spec:
732788
template: |
733789
id: {{ `{{ .workerServerGroupID }}` }}
734790
- name: workerAdditionalBlockDevices
735-
enabledIf: {{ `'{{ if .workerAdditionalBlockDevices }}true{{end}}'` }}
791+
enabledIf: {{ `'{{ if and .workerAdditionalBlockDevices (not .diskPinAZ) }}true{{end}}'` }}
792+
definitions:
793+
- selector:
794+
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
795+
kind: OpenStackMachineTemplate
796+
matchResources:
797+
controlPlane: false
798+
machineDeploymentClass:
799+
names:
800+
- default-worker
801+
jsonPatches:
802+
- op: add
803+
path: /spec/template/spec/additionalBlockDevices
804+
valueFrom:
805+
template: |
806+
{{ `{{- range .workerAdditionalBlockDevices }}` }}
807+
- name: {{ `{{ .name }}` }}
808+
sizeGiB: {{ `{{ .sizeGiB }}` }}
809+
storage:
810+
type: Volume
811+
volume:
812+
type: {{ `{{ .type }}` }}
813+
{{ `{{- end }}` }}
814+
- name: workerAdditionalBlockDevicesPinned
815+
enabledIf: {{ `'{{ if and .workerAdditionalBlockDevices .diskPinAZ }}true{{end}}'` }}
736816
definitions:
737817
- selector:
738818
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
@@ -750,6 +830,8 @@ spec:
750830
{{ `{{- range .workerAdditionalBlockDevices }}` }}
751831
- name: {{ `{{ .name }}` }}
752832
sizeGiB: {{ `{{ .sizeGiB }}` }}
833+
availabilityZone:
834+
from: Machine
753835
storage:
754836
type: Volume
755837
volume:

0 commit comments

Comments
 (0)