Skip to content

Commit bf6e42f

Browse files
openshift-merge-bot[bot]openshift-merge-bot[bot]
authored
Merge pull request #1342 from cschwede/swift-conversion
Add procedure for converting external Swift storage nodes
2 parents 6d0a6b5 + 9fdeaab commit bf6e42f

3 files changed

Lines changed: 298 additions & 3 deletions

File tree

‎docs_user/assemblies/assembly_migrating-the-object-storage-service.adoc‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,15 @@ ifdef::context[:parent-context: {context}]
88
= Migrating the {object_storage} to {rhos_long_noacro} nodes
99

1010
[role="_abstract"]
11-
If you are using the {rhos_prev_long} {object_storage_first_ref} as an Object Storage service, you must migrate your Object Storage service to {rhos_long_noacro} nodes.
11+
If you are using the {rhos_prev_long} {object_storage_first_ref} as an Object Storage service, you must migrate your Object Storage service to {rhos_long} nodes.
1212

1313
If you are using the Object Storage API of the Ceph Object Gateway (RGW), you can skip this chapter and migrate your Red Hat Ceph Storage cluster. For more information, see "Migrating the {Ceph} cluster". If you are not planning to migrate Ceph daemons from Controller nodes, you must perform the steps that are described in "Deploying a Ceph ingress daemon" and "Create or update the Object Storage service endpoints".
1414

15+
You have two options for handling the existing {object_storage} storage nodes:
16+
17+
* *Migrate data to new {rhos_acro} nodes* : Provision new storage nodes on {rhocp_long} and gradually move data by using the `swift-ring-tool`.
18+
* *Convert existing nodes to data plane nodes*: Keep the data on the same nodes and convert them to data plane nodes that are managed by the data plane operator.
19+
1520
The data migration happens replica by replica. For example, if you have 3 replicas, move them one at a time to ensure that the other 2 replicas are still operational, which enables you to continue to use the {object_storage} during the migration.
1621

1722
[NOTE]
@@ -22,6 +27,8 @@ include::../modules/proc_migrating-object-storage-data-to-rhoso-nodes.adoc[level
2227

2328
include::../modules/con_troubleshooting-object-storage-migration.adoc[leveloffset=+1]
2429

30+
include::../modules/proc_converting-object-storage-nodes.adoc[leveloffset=+1]
31+
2532
[role="_additional-resources"]
2633
== Additional resources
2734
* xref:ceph-migration_adopt-control-plane[Migrating the {Ceph} cluster]

‎docs_user/modules/proc_adopting-the-object-storage-service.adoc‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -132,5 +132,10 @@ $ openstack object save test obj --file -
132132
Hello World!
133133
----
134134

135-
[NOTE]
136-
The Object Storage data is still stored on the existing {OpenStackShort} nodes. For more information about migrating the actual data from the {OpenStackShort} deployment to the {rhos_acro} deployment, see xref:migrating-object-storage-data-to-rhoso-nodes_migrate-object-storage-service[Migrating the {object_storage_first_ref} data from {OpenStackShort} to {rhos_long} nodes].
135+
.Next steps
136+
137+
After the initial adoption, the {object_storage} data is still stored on the existing {OpenStackShort} nodes. You have two options for handling these storage nodes:
138+
139+
* *Migrate data to new {rhos_acro} nodes*: Provision new storage nodes on {rhocp_long} and gradually move data by using the `swift-ring-tool`. For more information, see xref:migrating-object-storage-data-to-rhoso-nodes_migrate-object-storage-service[Migrating the {object_storage_first_ref} data from {OpenStackShort} to {rhos_long} nodes].
140+
141+
* *Convert existing nodes to dataplane nodes*: Keep the data on the same nodes and convert them to data plane nodes that are managed by the `openstack-operators`. For more information, see xref:converting-object-storage-nodes[Converting {object_storage} storage nodes].

‎docs_user/modules/proc_converting-object-storage-nodes.adoc‎

Lines changed: 283 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,283 @@
1+
:_mod-docs-content-type: PROCEDURE
2+
[id="converting-object-storage-nodes"]
3+
4+
= Converting {object_storage} storage nodes
5+
6+
[role="_abstract"]
7+
If you used external {object_storage} storage nodes, you can convert these
8+
nodes after the initial {object_storage_first_ref} adoption is complete. This
9+
step reconfigures the nodes in-place while keeping the {object_storage}
10+
available throughout the process.
11+
12+
Before you begin, review the following key concepts:
13+
14+
Pre-provisioned nodes:: Setting `preProvisioned: true` in the
15+
`OpenStackDataPlaneNodeSet` custom resource (CR) skips provisioning the node from scratch and does not reinstall the operating system.
16+
{object_storage} disk and mount preservation:: `edpm_swift_disks: []` must be
17+
set in both the `nodeTemplate` and in the individual node definition under
18+
`nodes` to ensure that no disks are formatted or mounted differently.
19+
Existing disk mounts are untouched. The `SwiftRawDisks` parameter from
20+
{OpenStackPreviousInstaller} is not migrated or converted. Disks on
21+
converted nodes must be managed manually.
22+
Rolling conversion with multiple node sets:: In production, do not convert
23+
all {object_storage} nodes at once. Instead, create separate
24+
`OpenStackDataPlaneNodeSet` custom resources and deploy them sequentially. Align
25+
groups with ring zones or regions so that no more than one replica of any
26+
partition is affected during the conversion.
27+
28+
.Prerequisites
29+
30+
* The initial {object_storage} adoption is complete. For more information, see
31+
xref:adopting-the-object-storage-service_hsm-integration[Adopting the
32+
{object_storage}].
33+
* Nodes cannot be members of more than one `OpenStackDataPlaneNodeSet` CR at
34+
the same time. If the nodes you want to convert were part of the node set
35+
that you created during the control plane adoption, delete that node set
36+
before proceeding:
37+
+
38+
----
39+
$ oc delete openstackdataplanenodeset <adoption-nodeset-name>
40+
----
41+
+
42+
where:
43+
44+
`<adoption-nodeset-name>`::
45+
Specifies the node set that you want to delete.
46+
47+
.Procedure
48+
49+
. Create a patch file to configure the {object_storage} data plane storage
50+
ports:
51+
+
52+
[NOTE]
53+
The {OpenStackPreviousInstaller} deployment used different default
54+
ports: 6002 for the account server, 6001 for the container server, 6000 for
55+
the object server. The converted nodes must continue to use these same
56+
ports to ensure connectivity between all {object_storage} storage nodes by using
57+
the unmodified rings.
58+
+
59+
----
60+
cat > swift-config-patch.yaml << EOF
61+
spec:
62+
swift:
63+
template:
64+
swiftStorage:
65+
defaultConfigOverwrite:
66+
01-account-server.conf: |
67+
[DEFAULT]
68+
bind_port = 6002
69+
01-container-server.conf: |
70+
[DEFAULT]
71+
bind_port = 6001
72+
01-object-server.conf: |
73+
[DEFAULT]
74+
bind_port = 6000
75+
EOF
76+
----
77+
+
78+
[NOTE]
79+
If you customized any {object_storage} service settings in the
80+
{OpenStackPreviousInstaller} deployment, include those settings as well. For
81+
example, if you changed the workers count or added additional configuration options
82+
for the account, container, or object services, add them to the
83+
corresponding section in the patch file.
84+
85+
. Apply the patch to the `OpenStackControlPlane` CR:
86+
+
87+
----
88+
$ oc patch openstackcontrolplane openstack --type=merge --patch-file=swift-config-patch.yaml
89+
----
90+
91+
. Create a `OpenStackDataPlaneService` CR to remove remaining
92+
{OpenStackPreviousInstaller} artifacts, such as containers, `systemd` units, and
93+
configuration, from the node:
94+
+
95+
----
96+
$ oc apply -f - <<EOF
97+
apiVersion: dataplane.openstack.org/v1beta1
98+
kind: OpenStackDataPlaneService
99+
metadata:
100+
name: tripleo-cleanup
101+
namespace: openstack
102+
spec:
103+
playbook: osp.edpm.tripleo_cleanup
104+
EOF
105+
----
106+
107+
ifeval::["{build}" == "upstream"]
108+
. Create a `OpenStackDataPlaneService` CR to configure RPM repositories on the node:
109+
+
110+
----
111+
$ oc apply -f - <<EOF
112+
apiVersion: dataplane.openstack.org/v1beta1
113+
kind: OpenStackDataPlaneService
114+
metadata:
115+
name: repo-setup
116+
namespace: openstack
117+
spec:
118+
addCertMounts: false
119+
caCerts: combined-ca-bundle
120+
edpmServiceType: repo-setup
121+
playbookContents: |
122+
- hosts: all
123+
strategy: linear
124+
tasks:
125+
- name: Enable podified-repos
126+
become: true
127+
ansible.builtin.shell: |
128+
# This needs rhos-release.rpm to be downloaded before or an
129+
# alternative way to use upstream repos
130+
dnf install -y /tmp/rhos-release.rpm
131+
rhos-release -x
132+
rhos-release 18.0
133+
EOF
134+
----
135+
endif::[]
136+
137+
. Create the `OpenStackDataPlaneNodeSet` CR that defines which nodes to convert and how to configure them:
138+
+
139+
[subs="+quotes"]
140+
----
141+
apiVersion: dataplane.openstack.org/v1beta1
142+
kind: OpenStackDataPlaneNodeSet
143+
metadata:
144+
name: openstack-edpm-ipam
145+
namespace: openstack
146+
spec:
147+
networkAttachments:
148+
- ctlplane
149+
- storage
150+
nodeTemplate:
151+
ansible:
152+
ansibleUser: cloud-admin
153+
ifeval::["{build}" == "downstream"]
154+
ansibleVarsFrom:
155+
- secretRef:
156+
name: subscription-manager
157+
- secretRef:
158+
name: redhat-registry
159+
endif::[]
160+
ansibleVars:
161+
edpm_nodes_validation_validate_controllers_icmp: false
162+
edpm_nodes_validation_validate_gateway_icmp: false
163+
edpm_service_removal_skip_list:
164+
- tripleo-container-shutdown.service
165+
edpm_sshd_allowed_ranges:
166+
- 192.168.122.0/24
167+
edpm_swift_disks: []
168+
enable_debug: false
169+
gather_facts: false
170+
timesync_ntp_servers:
171+
- hostname: pool.ntp.org
172+
ifeval::["{build}" != "downstream"]
173+
edpm_bootstrap_command: |
174+
# This is a hack to deploy RDO Delorean repos to RHEL as if it were Centos 9 Stream
175+
set -euxo pipefail
176+
curl -sL https://github.com/openstack-k8s-operators/repo-setup/archive/refs/heads/main.tar.gz | tar -xz
177+
python3 -m venv ./venv
178+
PBR_VERSION=0.0.0 ./venv/bin/pip install ./repo-setup-main
179+
# This is required for FIPS enabled until trunk.rdoproject.org
180+
# is not being served from a centos7 host, tracked by
181+
# https://issues.redhat.com/browse/RHOSZUUL-1517
182+
dnf -y install crypto-policies
183+
update-crypto-policies --set FIPS:NO-ENFORCE-EMS
184+
./venv/bin/repo-setup current-podified -b antelope -d centos9 --stream
185+
dnf -y upgrade openstack-selinux
186+
rm -f /run/virtlogd.pid
187+
rm -rf repo-setup-main
188+
endif::[]
189+
ifeval::["{build}" == "downstream"]
190+
rhc_release: 9.2
191+
rhc_repositories:
192+
- {name: "*", state: disabled}
193+
- {name: "rhel-9-for-x86_64-baseos-eus-rpms", state: enabled}
194+
- {name: "rhel-9-for-x86_64-appstream-eus-rpms", state: enabled}
195+
- {name: "rhel-9-for-x86_64-highavailability-eus-rpms", state: enabled}
196+
- {name: "rhoso-18.0-for-rhel-9-x86_64-rpms", state: enabled}
197+
endif::[]
198+
ansibleSSHPrivateKeySecret: dataplane-adoption-secret
199+
managementNetwork: ctlplane
200+
nodes:
201+
edpm-swift-0:
202+
ansible:
203+
ansibleHost: 192.168.122.100
204+
ansibleVars:
205+
edpm_swift_disks: []
206+
hostName: edpm-swift-0
207+
networks:
208+
- defaultRoute: true
209+
fixedIP: 192.168.122.100
210+
name: ctlplane
211+
subnetName: subnet1
212+
- name: internalapi
213+
subnetName: subnet2
214+
- name: storage
215+
subnetName: subnet3
216+
- name: tenant
217+
subnetName: subnet4
218+
preProvisioned: true
219+
services:
220+
- bootstrap
221+
- download-cache
222+
- install-os
223+
- configure-os
224+
- ssh-known-hosts
225+
- run-os
226+
- reboot-os
227+
- install-certs
228+
- tripleo-cleanup
229+
- swift
230+
tlsEnabled: true
231+
----
232+
233+
. Apply the `OpenStackDataPlaneNodeSet` CR:
234+
+
235+
----
236+
$ oc apply -f nodeset.yaml
237+
----
238+
239+
. Create an `OpenStackDataPlaneDeployment` CR to trigger the Ansible pipeline on the node:
240+
+
241+
----
242+
$ oc apply -f - <<EOF
243+
apiVersion: dataplane.openstack.org/v1beta1
244+
kind: OpenStackDataPlaneDeployment
245+
metadata:
246+
name: openstack-edpm-ipam
247+
namespace: openstack
248+
spec:
249+
nodeSets:
250+
- openstack-edpm-ipam
251+
EOF
252+
----
253+
254+
. Wait for the deployment to complete:
255+
+
256+
----
257+
$ oc wait --for condition=Ready openstackdataplanedeployment/openstack-edpm-ipam --timeout=30m
258+
----
259+
260+
.. Monitor the progress:
261+
+
262+
----
263+
$ watch oc get pod -l app=openstackansibleee
264+
----
265+
+
266+
----
267+
$ oc logs -l app=openstackansibleee -f --max-log-requests 40
268+
----
269+
270+
.Verification
271+
272+
* Verify that the Object Storage services are running on the converted node:
273+
+
274+
----
275+
$ ssh cloud-admin@192.168.122.100 "sudo systemctl status edpm_swift_*"
276+
----
277+
278+
* Verify that the Object Storage services are functional after conversion, for
279+
example by uploading an object:
280+
+
281+
----
282+
$ oc rsh openstackclient /bin/sh -c "touch obj && openstack container create cont && openstack object create cont obj"
283+
----

0 commit comments

Comments
 (0)