CLVM enhancements and fixes (#12617)

This PR enhances the existing CLVM implementation which was based on the deprecated CLVM technology which was based on corosync/pacemaker. With RHEL 7 having reached EOL, CLVM seems to be broken. CLVM supports RAW volumes on LVM , where as CLVM_NG support QCOW2 on LVM.

Further details: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modernized+CLVM%3A+Enhancements+and+CLVM_NG+support

NOTE: On testing - it was identified that incremental snapshots for clvm-ng do not work as expected. As of now it's been removed from scope. So, CLVM and CLVM_NG would only support full snapshots.


* add support for proper cleanup of snapshots and prevent vol snapshot of running vm

* remove snap vol restriction for sunning vms

* refactor clvm code

* add support for live migration

* add support for migrating lvm lock

* clvm deletion called explicitly

* made necessary changes to allow migration of lock and deletion of detached volumes

* fix create vol from snap and attach

* add support to revert snapshot for clvm

* add support to revert snapshot for clvm

* make zero fill configurable

* make setting non-dynamic & fix test

* fix locking at vol/vm creation

* fix revert snapshot format type and handle revert snapshot functionality for clvm

* 1. Create clvmlockmanager and move common code \n
2. handle attaching volumes to stopped VMs \n
3. Handle lock transfer when VM is started on another host

* add license

* remove command/answer classes from sonar coverage check

* add support for new gen clvm with template (qcow2) backing

* Add support for clvm_ng - which allows qcow2 on block storage , linked clones, etc

* fix test and use physical size + 50% of virtual size for backing file, while virtual size + pe for disk

* migrate clvm volumes as full clone and allow migration from clvm to nfs

* fix clvm_ng to nfs migration, and handle overhead calc

* support live migration from clvm_ng to nfs and vice-versa

* add support to migrate to and from clvm to nfs

* fix creation of volume on destination host during migration to clvm/clvm-ng

* support live vm migration between clvm -> clvm-ng (vice-versa), nfs -> clvm (vice-versa) and nfs->clvm-ng (vice-versa)

* add unit tests for clvm/clvm_ng operations

* Add support for incremental volume snapshots for clvm_ng

* prevent snapshot backup for incremental clvm_ng snaps, fix build failure, add unit tests

* fix lockhost on creation of volumes from snap and fix bitmap issue when migrating a vol with incremental snap

* restrict pre and post migration commands to only kvm hosts where vm has CLVM/CLVM-NG volumes

* evist lock tracking - use lvs command to get lock host than DB

* add test for pre/post migration

* Create a CLVM storage adaptor

* update existing clvm get stats method

* fix precommit check failure

* Apply suggestions from code review

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* Apply suggestions from code review

Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

* improve lock host retrieval logic and quicker retrival using db host as first check point and then fanning out

* add proper support for resizing of clvm_ng which calculated PE correctly for qcow2 metadata

* fallback to full snapshots for clvm-ng - incremental not supported in 4.23

* expunge volume detail of lock host on vm expunge

* if vmmigration with volume is done to the same clvm volume group, then dont do data transfer, just lock transfer and vm

* add clvm pools with deterministic uuid , so as to prevent adding the same pool twic

* added a small improvement to factor in a senario when lv is inactive on all hosts, could happen in storage outage issue

* address comment - extract common code for endpoint identification if clvm pool type

* Address comments - add early return guard to reduce indentation

* minor improvement - when migrating vm with volumes, if there's a failures, change the clvm vols to exclusive on source from shared, and on success, change dest vol to exclusive only for cross-pool migration

* cleanup unused code and tests for incremental snaps for clvmng and other cleanups

* allow storage browser to list lv in clvm, fix clvm shrink, overprovisioning factor isnt used for clvm pools - so set it to 1 and prevented display of provisioning type for clvm

* no need to have locktransfercommand to execute in sequence

* increase lv cmd timeouts to consider cluster load

---------

Co-authored-by: Pearl Dsilva <pearl1954@gmail.com>
Co-authored-by: Suresh Kumar Anaparti <sureshkumar.anaparti@gmail.com>

Author: 3 people

api/src/main/java/com/cloud/storage/Storage.java

@@ -170,6 +170,7 @@ public static enum StoragePoolType {
         ISO(false, false, EncryptionSupport.Unsupported), // for iso image
         LVM(false, false, EncryptionSupport.Unsupported), // XenServer local LVM SR
         CLVM(true, false, EncryptionSupport.Unsupported),
+        CLVM_NG(true, false, EncryptionSupport.Hypervisor),
         RBD(true, true, EncryptionSupport.Unsupported), // http://libvirt.org/storage.html#StorageBackendRBD
         SharedMountPoint(true, true, EncryptionSupport.Hypervisor),
         VMFS(true, true, EncryptionSupport.Unsupported), // VMware VMFS storage

core/src/main/java/com/cloud/agent/api/MigrateCommand.java

@@ -26,6 +26,7 @@
 
 import com.cloud.agent.api.to.DpdkTO;
 import com.cloud.agent.api.to.VirtualMachineTO;
+import com.cloud.storage.Storage;
 
 public class MigrateCommand extends Command {
     private String vmName;
@@ -42,6 +43,7 @@ public class MigrateCommand extends Command {
     private Map<String, DpdkTO> dpdkInterfaceMapping = new HashMap<>();
 
     private int newVmCpuShares;
+    private boolean clvmCrossPoolMigration;
 
     Map<String, Boolean> vlanToPersistenceMap = new HashMap<>();
 
@@ -149,6 +151,14 @@ public void setNewVmCpuShares(int newVmCpuShares) {
         this.newVmCpuShares = newVmCpuShares;
     }
 
+    public boolean isClvmCrossPoolMigration() {
+        return clvmCrossPoolMigration;
+    }
+
+    public void setClvmCrossPoolMigration(boolean clvmCrossPoolMigration) {
+        this.clvmCrossPoolMigration = clvmCrossPoolMigration;
+    }
+
     public static class MigrateDiskInfo {
         public enum DiskType {
             FILE, BLOCK;
@@ -184,6 +194,8 @@ public String toString() {
         private final String sourceText;
         private final String backingStoreText;
         private boolean isSourceDiskOnStorageFileSystem;
+        private Storage.StoragePoolType sourcePoolType;
+        private Storage.StoragePoolType destPoolType;
 
         public MigrateDiskInfo(final String serialNumber, final DiskType diskType, final DriverType driverType, final Source source, final String sourceText) {
             this.serialNumber = serialNumber;
@@ -232,6 +244,22 @@ public boolean isSourceDiskOnStorageFileSystem() {
         public void setSourceDiskOnStorageFileSystem(boolean isDiskOnFileSystemStorage) {
             this.isSourceDiskOnStorageFileSystem = isDiskOnFileSystemStorage;
         }
+
+        public Storage.StoragePoolType getSourcePoolType() {
+            return sourcePoolType;
+        }
+
+        public void setSourcePoolType(Storage.StoragePoolType sourcePoolType) {
+            this.sourcePoolType = sourcePoolType;
+        }
+
+        public Storage.StoragePoolType getDestPoolType() {
+            return destPoolType;
+        }
+
+        public void setDestPoolType(Storage.StoragePoolType destPoolType) {
+            this.destPoolType = destPoolType;
+        }
     }
 
     @Override

core/src/main/java/com/cloud/agent/api/PostMigrationAnswer.java

@@ -0,0 +1,42 @@
+//
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+
+package com.cloud.agent.api;
+
+/**
+ * Answer for PostMigrationCommand.
+ * Indicates success or failure of post-migration operations on the destination host.
+ */
+public class PostMigrationAnswer extends Answer {
+
+    protected PostMigrationAnswer() {
+    }
+
+    public PostMigrationAnswer(PostMigrationCommand cmd, String detail) {
+        super(cmd, false, detail);
+    }
+
+    public PostMigrationAnswer(PostMigrationCommand cmd, Exception ex) {
+        super(cmd, ex);
+    }
+
+    public PostMigrationAnswer(PostMigrationCommand cmd) {
+        super(cmd, true, null);
+    }
+}

core/src/main/java/com/cloud/agent/api/PostMigrationCommand.java

@@ -0,0 +1,59 @@
+//
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+
+package com.cloud.agent.api;
+
+import com.cloud.agent.api.to.VirtualMachineTO;
+
+/**
+ * PostMigrationCommand is sent to the destination host after a successful VM migration.
+ * It performs post-migration tasks such as:
+ * - Claiming exclusive locks on CLVM volumes (converting from shared to exclusive mode)
+ * - Other post-migration cleanup operations
+ */
+public class PostMigrationCommand extends Command {
+    private VirtualMachineTO vm;
+    private String vmName;
+
+    protected PostMigrationCommand() {
+    }
+
+    public PostMigrationCommand(VirtualMachineTO vm, String vmName) {
+        this.vm = vm;
+        this.vmName = vmName;
+    }
+
+    public VirtualMachineTO getVirtualMachine() {
+        return vm;
+    }
+
+    public String getVmName() {
+        return vmName;
+    }
+
+    @Override
+    public boolean executeInSequence() {
+        return true;
+    }
+
+    @Override
+    public boolean isBypassHostMaintenance() {
+        return true;
+    }
+}

core/src/main/java/com/cloud/agent/api/PreMigrationCommand.java

@@ -0,0 +1,61 @@
+//
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+
+package com.cloud.agent.api;
+
+import com.cloud.agent.api.to.VirtualMachineTO;
+
+/**
+ * PreMigrationCommand is sent to the source host before VM migration starts.
+ * It performs pre-migration tasks such as:
+ * - Converting CLVM volume exclusive locks to shared mode so destination host can access them
+ * - Other pre-migration preparation operations on the source host
+ *
+ * This command runs on the SOURCE host before PrepareForMigrationCommand runs on the DESTINATION host.
+ */
+public class PreMigrationCommand extends Command {
+    private VirtualMachineTO vm;
+    private String vmName;
+
+    protected PreMigrationCommand() {
+    }
+
+    public PreMigrationCommand(VirtualMachineTO vm, String vmName) {
+        this.vm = vm;
+        this.vmName = vmName;
+    }
+
+    public VirtualMachineTO getVirtualMachine() {
+        return vm;
+    }
+
+    public String getVmName() {
+        return vmName;
+    }
+
+    @Override
+    public boolean executeInSequence() {
+        return true;
+    }
+
+    @Override
+    public boolean isBypassHostMaintenance() {
+        return true;
+    }
+}

core/src/main/java/org/apache/cloudstack/storage/clvm/command/ClvmLockTransferAnswer.java

@@ -0,0 +1,90 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.cloudstack.storage.clvm.command;
+
+import com.cloud.agent.api.Answer;
+
+/**
+ * Answer for ClvmLockTransferCommand, containing lock state information.
+ * This answer includes the current lock holder information when querying lock state.
+ */
+public class ClvmLockTransferAnswer extends Answer {
+
+    private String currentLockHostname;
+    private boolean isActive;
+    private boolean isOpen;
+    private String lvAttributes;
+
+    public ClvmLockTransferAnswer(ClvmLockTransferCommand cmd, boolean result, String details) {
+        super(cmd, result, details);
+    }
+
+    public ClvmLockTransferAnswer(ClvmLockTransferCommand cmd, boolean result, String details,
+                                  String currentLockHostname, boolean isActive, boolean isOpen,
+                                  String lvAttributes) {
+        super(cmd, result, details);
+        this.currentLockHostname = currentLockHostname;
+        this.isActive = isActive;
+        this.isOpen = isOpen;
+        this.lvAttributes = lvAttributes;
+    }
+
+    /**
+     * Get the hostname from lv_host. Retained for diagnostics only —
+     * do NOT use this to determine lock holder identity.
+     */
+    public String getCurrentLockHostname() {
+        return currentLockHostname;
+    }
+
+    public void setCurrentLockHostname(String currentLockHostname) {
+        this.currentLockHostname = currentLockHostname;
+    }
+
+    /**
+     * Whether the LV is locally active on the queried host (lv_attr[4]=='a').
+     * This is the authoritative signal for lock holder discovery via fan-out.
+     */
+    public boolean isActive() {
+        return isActive;
+    }
+
+    public void setActive(boolean active) {
+        isActive = active;
+    }
+
+    /**
+     * Whether a process has the device file open on the queried host (lv_attr[5]=='o').
+     * true means a VM is actively doing I/O on this host right now — do NOT deactivate.
+     */
+    public boolean isOpen() {
+        return isOpen;
+    }
+
+    public void setOpen(boolean open) {
+        isOpen = open;
+    }
+
+    public String getLvAttributes() {
+        return lvAttributes;
+    }
+
+    public void setLvAttributes(String lvAttributes) {
+        this.lvAttributes = lvAttributes;
+    }
+}