diff --git a/docs_user/modules/proc_adopting-compute-services-with-dcn-backend.adoc b/docs_user/modules/proc_adopting-compute-services-with-dcn-backend.adoc new file mode 100644 index 000000000..db3a2f094 --- /dev/null +++ b/docs_user/modules/proc_adopting-compute-services-with-dcn-backend.adoc @@ -0,0 +1,320 @@ +:_mod-docs-content-type: PROCEDURE +[id="adopting-compute-services-with-dcn-backend_{context}"] + += Adopting {compute_service} services with multiple {Ceph} back ends (DCN) + +[role="_abstract"] +In a Distributed Compute Node (DCN) deployment where {image_service_first_ref} and {block_storage_first_ref} services run on edge Compute nodes, each site has its own {CephCluster} cluster. The {compute_service_first_ref} nodes at each site must be configured with the {Ceph} connection details and {image_service} endpoint for their local site. Because the {image_service} has a separate API endpoint at each site, each site's `OpenStackDataPlaneNodeSet` custom resource (CR) must use a different `OpenStackDataPlaneService` CR that points to the correct {image_service}. + +In a DCN deployment, all node sets belong to a single {compute_service} cell. The central site and each edge site are separate `OpenStackDataPlaneNodeSet` resources within that cell. The per-site `OpenStackDataPlaneService` resources deliver different {Ceph} and {image_service} configurations to each node set while sharing the same cell-level {compute_service} configuration. + +.Prerequisites + +* You have adopted the {image_service} with multiple {Ceph} back ends. For more information, see xref:adopting-image-service-with-dcn-backend_image-service[Adopting the Image service with multiple Ceph back ends]. +* You have adopted the {block_storage} with multiple {Ceph} back ends. For more information, see xref:adopting-block-storage-service-with-dcn-backend_hsm-integration[Adopting the Block Storage service with multiple Ceph back ends]. +* The `ceph-conf-files` secret contains the configuration and keyrings for all {Ceph} clusters in your DCN deployment. +* Retrieve the `fsid` for each {Ceph} cluster: ++ +---- +$ oc get secret ceph-conf-files -o json | jq -r '.data | to_entries[] | select(.key | endswith(".conf")) | "\(.key): \(.value | @base64d)"' | grep fsid +---- + +.Procedure + +. Set the cell name variable. In a DCN deployment, all node sets belong to a single cell: ++ +---- +$ DEFAULT_CELL_NAME="cell1" +---- + +. Retrieve the `fsid` for each {Ceph} cluster and store them in shell variables: ++ +[subs="+quotes"] +---- +$ CEPH_FSID_CENTRAL=$(oc get secret ceph-conf-files -o json | jq -r '.data.""' | base64 -d | grep fsid | sed -e 's/fsid = //') +$ CEPH_FSID_DCN1=$(oc get secret ceph-conf-files -o json | jq -r '.data.""' | base64 -d | grep fsid | sed -e 's/fsid = //') +$ CEPH_FSID_DCN2=$(oc get secret ceph-conf-files -o json | jq -r '.data.""' | base64 -d | grep fsid | sed -e 's/fsid = //') +---- ++ +where: + +``:: +Specifies the name of the {Ceph} configuration file for the central site in the `ceph-conf-files` secret. + +``:: +Specifies the name of the {Ceph} configuration file for an edge site in the `ceph-conf-files` secret. + +``:: +Specifies the name of the {Ceph} configuration file for an additional edge site in the `ceph-conf-files` secret. + +. Create a `ConfigMap` for each site. Each `ConfigMap` contains the {Ceph} and {image_service} configuration specific to that site. ++ +The following example creates `ConfigMap` resources for a central site and two edge sites. ++ +.. Create the `ConfigMap` for the central site: ++ +---- +$ oc apply -f - <`:: +Specifies the path to your SSH key for each `DistributedComputeHCI` edge Compute node on each DCN edge site. + +``, ``, ``:: +Specifies the IP address for each `DistributedComputeHCI` edge Compute node within the DCN1 edge site. + +``, ``, ``:: +Specifies the IP address for each `DistributedComputeHCI` edge Compute node within the DCN2 edge site. + + +.. Stop the storage services on all `DistributedComputeHCI` nodes: ++ +---- +# Services to stop on DistributedComputeHCI edge compute nodes +DCN_HCI_SERVICES=("tripleo_glance_api_internal.service" + "tripleo_cinder_volume.service" + "tripleo_etcd.service") + +# List of all DistributedComputeHCI node SSH commands +DCN_HCI_NODES=("$DCN1_HCI0_SSH" + "$DCN1_HCI1_SSH" + "$DCN1_HCI2_SSH" + "$DCN2_HCI0_SSH" + "$DCN2_HCI1_SSH" + "$DCN2_HCI2_SSH") + +echo "Stopping storage services on DistributedComputeHCI nodes" +for node_ssh in "${DCN_HCI_NODES[@]}"; do + [ -z "$node_ssh" ] && continue + echo "Processing node: $node_ssh" + for service in "${DCN_HCI_SERVICES[@]}"; do + if $node_ssh sudo systemctl is-active $service 2>/dev/null; then + echo "Stopping $service" + $node_ssh sudo systemctl stop $service + fi + done +done + +echo "Checking storage services on DistributedComputeHCI nodes" +for node_ssh in "${DCN_HCI_NODES[@]}"; do + [ -z "$node_ssh" ] && continue + for service in "${DCN_HCI_SERVICES[@]}"; do + if ! $node_ssh systemctl show $service 2>/dev/null | grep ActiveState=inactive >/dev/null; then + echo "ERROR: Service $service still running on $node_ssh" + else + echo "OK: Service $service is not running on $node_ssh" + fi + done +done +---- ++ +[NOTE] +==== +* On edge sites, the {image_service} runs with the service name `tripleo_glance_api_internal.service`, which is different from the `tripleo_glance_api.service` on the central controller. +* The {block_storage} volume service, `tripleo_cinder_volume.service`, uses the same service name on both edge sites and the central controller. +* The etcd service, `tripleo_etcd.service`, is used as a distributed lock manager for the {block_storage} volume service running in active/active mode on edge sites. +==== + +. If your DCN deployment includes `DistributedComputeHCIScaleOut` nodes, stop the HAProxy service on those nodes: ++ +[NOTE] +==== +The `DistributedComputeHCIScaleOut` role is used to scale Compute and storage capacity beyond the initial three `DistributedComputeHCI` nodes at each site. These nodes run `HAProxyEdge`, which proxies {image_service} requests to the `GlanceApiEdge` instances on `DistributedComputeHCI` nodes. Skip this step if your DCN deployment does not include `DistributedComputeHCIScaleOut` nodes. For non-HCI deployments, the equivalent role is `DistributedComputeScaleOut`, which runs the same `HAProxyEdge` service. +==== ++ +.. Define shell variables for your `DistributedComputeHCIScaleOut` edge Compute nodes: ++ +[subs=+quotes] +---- +# DCN1 edge site DistributedComputeHCIScaleOut nodes +DCN1_SCALEOUT0_SSH="ssh -i ** root@**" +DCN1_SCALEOUT1_SSH="ssh -i ** root@**" + +# DCN2 edge site DistributedComputeHCIScaleOut nodes +DCN2_SCALEOUT0_SSH="ssh -i ** root@**" +DCN2_SCALEOUT1_SSH="ssh -i ** root@**" +---- ++ +where: + +``, ``:: +Specifies the IP address for each `DistributedComputeHCIScaleOut` node within the DCN1 edge site. + +``, ``:: +Specifies the IP address for each `DistributedComputeHCIScaleOut` node within the DCN2 edge site. + +.. Stop the services on all `DistributedComputeHCIScaleOut` nodes: ++ +---- +# Services to stop on DistributedComputeHCIScaleOut edge compute nodes +DCN_SCALEOUT_SERVICES=("tripleo_haproxy_edge.service") + +# List of all DistributedComputeHCIScaleOut node SSH commands +DCN_SCALEOUT_NODES=("$DCN1_SCALEOUT0_SSH" + "$DCN1_SCALEOUT1_SSH" + "$DCN2_SCALEOUT0_SSH" + "$DCN2_SCALEOUT1_SSH") + +echo "Stopping services on DistributedComputeHCIScaleOut nodes" +for node_ssh in "${DCN_SCALEOUT_NODES[@]}"; do + [ -z "$node_ssh" ] && continue + echo "Processing node: $node_ssh" + for service in "${DCN_SCALEOUT_SERVICES[@]}"; do + if $node_ssh sudo systemctl is-active $service 2>/dev/null; then + echo "Stopping $service" + $node_ssh sudo systemctl stop $service + fi + done +done + +echo "Checking services on DistributedComputeHCIScaleOut nodes" +for node_ssh in "${DCN_SCALEOUT_NODES[@]}"; do + [ -z "$node_ssh" ] && continue + for service in "${DCN_SCALEOUT_SERVICES[@]}"; do + if ! $node_ssh systemctl show $service 2>/dev/null | grep ActiveState=inactive >/dev/null; then + echo "ERROR: Service $service still running on $node_ssh" + else + echo "OK: Service $service is not running on $node_ssh" + fi + done +done +---- ++ +[NOTE] +==== +The HAProxy edge service, `tripleo_haproxy_edge.service`, provided a local {image_service} endpoint on `DistributedComputeHCIScaleOut` nodes, proxying requests to the `GlanceApiEdge` instances on `DistributedComputeHCI` nodes. During adoption, {rhocp_long} Kubernetes service endpoints backed by MetalLB replace HAProxy. +====