Replication Cross-Cluster delete origin disk vm folder

[depedro-ai]

Hey there! 👋 Thanks for taking the time to report this bug — we appreciate it!

Please keep in mind: PegaProx is developed and maintained entirely by volunteers in our free time. We do our best, but we can't work magic 🪄 — please be patient, we'll get to your issue as soon as we can.

Want to help keep PegaProx alive?

• ⭐ Star the project — it's free and helps a lot!
• 💖 Become a Sponsor — helps us dedicate more time to development
• 🤝 Contributing code or docs is always welcome too!

---

Describe the bug

Replication Cross-Cluster delete origin disk vm folder.

Steps to Reproduce

1. create a Replicatin Cross-Cluster job and start it
2. When replicatin end, origin vm fail deleting snapshot
3. VM appear bloked

Expected behavior

origin vm should be in production mode

Environment

• PegaProx Version: Beta 0.9.10.2
• Installation Method: CT Template
• OS: Debian 13 LXC Container
• Browser: Chromium 146.0.7680.22
• Behind Reverse Proxy? No
• UI? Corporate

Logs

snapshotting 'drive-scsi0' (netapp:105/vm-105-disk-0.qcow2) internal qemu snapshot TASK OK
creating a clone of VM 105 with ID 127
create full clone of drive scsi0 (netapp:105/vm-105-disk-0.qcow2)
Formatting '/mnt/pve/netapp/images/127/vm-127-disk-0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=off compression_type=zlib size=42949672960 lazy_refcounts=off refcount_bits=16
transferred 40.0 GiB of 40.0 GiB (100.00%) TASK OK
2026-05-15 14:10:12 remote: started tunnel worker 'UPID:pve04:00219114:0E7ADED2:6A070D24:qmtunnel:105:root@pam!xcrepl-7511d85d-47007:'
tunnel: -> sending command "version" to remote
tunnel: <- got reply
2026-05-15 14:10:13 local WS tunnel version: 2
2026-05-15 14:10:13 remote WS tunnel version: 2
2026-05-15 14:10:13 minimum required WS tunnel version: 2
websocket tunnel started
2026-05-15 14:10:13 starting migration of VM 127 to node 'pve04' (10.0.0.77)
tunnel: -> sending command "bwlimit" to remote
tunnel: <- got reply
2026-05-15 14:10:13 found local disk 'netapp:127/vm-127-disk-0.qcow2' (attached)
2026-05-15 14:10:13 copying local disk images
tunnel: -> sending command "disk-import" to remote
tunnel: <- got reply
tunnel: accepted new connection on '/run/pve/127.storage'
tunnel: requesting WS ticket via tunnel
tunnel: established new WS for forwarding '/run/pve/127.storage'
31847612416 bytes (32 GB, 30 GiB) copied, 272.496 s, 117 MB/s
tunnel: -> sending command "query-disk-import" to remote
tunnel: done handling forwarded connection from '/run/pve/127.storage'
tunnel: <- got reply
2026-05-15 14:14:46 disk-import: Formatting '/mnt/pve/replica/images/105/vm-105-disk-0.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off preallocation=metadata compression_type=zlib size=42949672960 lazy_refcounts=off refcount_bits=16
tunnel: -> sending command "query-disk-import" to remote
tunnel: <- got reply
2026-05-15 14:14:51 disk-import: 838+1954674 records in
tunnel: -> sending command "query-disk-import" to remote
tunnel: <- got reply
2026-05-15 14:14:52 disk-import: 838+1954674 records out
tunnel: -> sending command "query-disk-import" to remote
tunnel: <- got reply
2026-05-15 14:14:53 disk-import: 31847612416 bytes (32 GB, 30 GiB) copied, 276.456 s, 115 MB/s
tunnel: -> sending command "query-disk-import" to remote
tunnel: <- got reply
2026-05-15 14:14:55 volume 'netapp:127/vm-127-disk-0.qcow2' is 'replica:105/vm-105-disk-0.qcow2' on the target
2026-05-15 14:14:55 mapped: net2 from vmbr0 to vmbr0
2026-05-15 14:14:55 mapped: net1 from vmbr0 to vmbr0
2026-05-15 14:14:55 mapped: net0 from vmbr0 to vmbr0
tunnel: -> sending command "config" to remote
tunnel: <- got reply
tunnel: -> sending command "unlock" to remote
tunnel: <- got reply
tunnel: -> sending command "quit" to remote
tunnel: <- got reply
2026-05-15 14:14:57 migration finished successfully (duration 00:04:45)
TASK OK
TASK ERROR: VM 105 qmp command 'blockdev-snapshot-delete-internal-sync' failed - Failed to remove snapshot from snapshot list: Stale file handle

Screenshots

Checklist

• I have searched existing issues to make sure this is not a duplicate
• I am using the latest version of PegaProx

[mkellermann97]

Hey @depedro-ai — looking at your trace, two distinct things going on:

1. Migration of the clone (VM 127) succeeded — the TASK OK at the end of the migration block is the cluster-side leg working fine. Disks copied, network mapped, VM unlocked on the target.

2. The post-migration snapshot cleanup on the ORIGIN VM (105) failed with Failed to remove snapshot from snapshot list: Stale file handle. ESTALE / "Stale file handle" is a well-known NFS condition — the storage server invalidated the file handle while QEMU still held it (typical after a rotate/remount). The underlying error is a PVE+NFS interaction, not something PegaProx triggers directly. But: our replication workflow needs to surface that cleanup-fail and leave VM 105 in a recoverable state instead of "blocked" — handling the partial-cleanup-failure path is the gap pointing back to us.

That replication-state-machine piece sits with @MrMasterbay; he's on holiday until May 18, so the analysis on that lands after he's back.

Quick recovery for the blocked VM in the meantime — on the origin node, check qm config 105 for the lock: field, then qm unlock 105 if it's still set. That should clear the lock without touching anything else, and the leftover internal snapshot (if any) can be cleaned up after via qm delsnapshot 105 <snapname> when you're ready. If you have a support bundle (Settings → Support Bundle), drop it here and I'll bundle it into the queue.

https://opencollective.com/pegaprox — donations keep the kind of multi-week replication-debug work going if any of it earns its keep.

— Marcus

[depedro-ai]

[pegaprox_support_20260515_145407.zip](https://github.com/user-attachments/files/27806086/pegaprox_support_20260515_145407.zip) The issue is not only that the snapshot cleanup fails with ESTALE. In my case, during the replication workflow, the original VM disk directory is removed from the shared NFS storage. The original VM configuration still points to: netapp:105/vm-105-disk-0.qcow2 but the physical file and even the VM disk directory disappeared from: /mnt/pve/netapp/images/105/ The VM config still contains the xcrepl snapshots and parent references, but the actual disk file no longer exists. From what I can see, the workflow creates a snapshot, creates/clones a temporary VM, migrates that clone, and then performs cleanup. Somewhere in that cleanup phase, the directory belonging to the original VM is deleted instead of only removing the temporary clone/snapshot. This happened because the replication target used the same shared NFS storage and the same VMID as the original VM. The cleanup logic appears to be using the VMID/storage path in a way that can collide with the origin VM path. This is data destructive: it removed the original VM disk from the origin storage. Expected behavior: - Never delete the origin VM disk directory. - Never delete images/<origin_vmid>/ during cleanup. - If cleanup fails, leave the origin VM untouched. - Use a separate temporary VMID/path for clone artifacts. - Refuse replication if origin and target resolve to the same physical storage path and VMID. DavidDePedro El El vie, 15 may 2026 a las 15:42, mkellermann97 ***@***.***> escribió:

…

*mkellermann97* left a comment (PegaProx/project-pegaprox#422) <#422 (comment)> Hey @depedro-ai <https://github.com/depedro-ai> — looking at your trace, two distinct things going on: 1. *Migration of the clone (VM 127) succeeded* — the TASK OK at the end of the migration block is the cluster-side leg working fine. Disks copied, network mapped, VM unlocked on the target. 2. *The post-migration snapshot cleanup on the ORIGIN VM (105) failed* with Failed to remove snapshot from snapshot list: Stale file handle. ESTALE / "Stale file handle" is a well-known NFS condition — the storage server invalidated the file handle while QEMU still held it (typical after a rotate/remount). The underlying error is a PVE+NFS interaction, not something PegaProx triggers directly. But: our replication workflow needs to surface that cleanup-fail and leave VM 105 in a recoverable state instead of "blocked" — handling the partial-cleanup-failure path is the gap pointing back to us. That replication-state-machine piece sits with @MrMasterbay <https://github.com/MrMasterbay>; he's on holiday until May 18, so the analysis on that lands after he's back. Quick recovery for the blocked VM in the meantime — on the origin node, check qm config 105 for the lock: field, then qm unlock 105 if it's still set. That should clear the lock without touching anything else, and the leftover internal snapshot (if any) can be cleaned up after via qm delsnapshot 105 <snapname> when you're ready. If you have a support bundle (Settings → Support Bundle), drop it here and I'll bundle it into the queue. https://opencollective.com/pegaprox — donations keep the kind of multi-week replication-debug work going if any of it earns its keep. — Marcus — Reply to this email directly, view it on GitHub <#422 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BPN6HVGUBUAGZJ522WRMQT3424NCVAVCNFSM6AAAAACY7OFMHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DINRQGI2DGNBQGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[depedro-ai]

I am testing and, some times, the first réplica is ok, but the second happen the same. Another times, the first réplica fails with the same.
When the first réplica work any of the recovery forms, test, planed or recovery, fails with no error

[mkellermann97]

@depedro-ai — thanks for the extra runs. Two patterns to separate out:

1. First-replica intermittent: same Stale file handle even on the very first run sometimes. Flaky-then-deterministic patterns usually point more at storage-side behaviour than at orchestration — worth checking on the NetApp side whether snapshot ops on images/<vmid>/ paths are racing against any automated rotation.

2. "Fails with no error" on the failover forms after a successful replication: that's the symptom I want a support bundle on. Silent failures are usually one of: a swallowed exception in the orchestrator, a frontend not surfacing the backend status, or a task that's still running in the background — the bundle disambiguates which. Settings → Support Bundle from the UI gives the slice we need.

Combined diagnosis stays with @MrMasterbay (replication state-machine is his lane). He's back from holiday May 18, so the deep-dive picks up after that.

https://opencollective.com/pegaprox if multi-pass triage rounds like this one are worth a contribution — entirely voluntary.

— Marcus

[depedro-ai]

i attach last logs

pegaprox_support_20260515_183258.zip

[mkellermann97]

@depedro-ai — apologies for the delay in getting back on the bundle. Got around to it just now; the recent_tasks in your bundle nailed the full chain. Posting the diagnosis + fix + immediate workaround.

Root cause (visible directly in your bundle's recent_tasks):

qmdelsnapshot   status=command '/usr/bin/qemu-img snapshot -d xcrepl-ac43d91a-... \
                /mnt/pve/netapp/images/319/vm-319-disk-0.qcow2' failed: exit code 1
qmshutdown      status=VM is locked (snapshot-delete) id=319    [× 9 in the log]

So the chain is:

1. Cross-cluster replication created the snapshot xcrepl-ac43d91a-1778856448 on VM 319
2. Replication itself succeeded
3. PegaProx issued the post-replication snapshot cleanup
4. PVE's qemu-img snapshot -d failed on NetApp NFS with ESTALE / exit code 1
5. PVE left the VM at lock = snapshot-delete and did NOT release the lock
6. Every subsequent operation (your failover attempts included) bounced off "VM is locked (snapshot-delete)" — which is exactly the "fails with no error" symptom you reported, because PegaProx wasn't surfacing the lock-status to the UI

Immediate workaround to unblock VM 319 right now, on the source cluster node:

# Clear the snapshot-delete lock
qm unlock 319
# Force-delete the snapshot config entry (the actual on-disk snapshot may
# stay orphaned on NetApp, but the VM gets out of lock state)
qm delsnapshot 319 xcrepl-ac43d91a-1778856448 --force

After that the VM should accept shutdown / failover / migrate again.

Fix landed on Testing as c45390d — _cleanup_snapshot in our cross-cluster-replication code now:

• Always passes force=1 to the DELETE — PVE removes the config entry even when the on-disk qemu-img delete fails. We own xcrepl snapshots end to end; an orphan internal .qcow2 slot on storage is harmless, a stuck lock is not.
• Defence in depth: after the DELETE, re-reads the VM config and explicitly clears lock = snapshot-delete if it's still set (with audit-log entry so operators can see why the lock disappeared).

After this rolls into a release, the "fails with no error on failover" symptom should disappear — the lock that was silently blocking everything will get cleaned as part of the standard replication cleanup path.

The intermittent ESTALE itself is a NetApp-NFS-side behaviour, not something PegaProx triggers, but our cleanup needs to tolerate it gracefully. That's what this fix does.

Donations at https://opencollective.com/pegaprox if recent_tasks-driven root-causing earns its keep.

— Marcus

[depedro-ai]

Thanks for the detailed diagnosis. The snapshot-delete lock issue makes sense and matches the qemu-img ESTALE failure we saw. However, there is still another concern from my side. In one of the affected cases, the problem was not only that the VM remained locked. The original VM config still pointed to: netapp:100/vm-100-disk-0.qcow2 but the physical disk file and the images/100 directory contents were gone from the shared NFS storage. This happened after testing cross-cluster replication where source and target had access to the same NetApp NFS storage and the replicated VM used the same VMID as the original one. So my concern is whether the cleanup logic can accidentally remove the origin VM disk directory when source and target resolve to the same physical storage path and VMID. The lock cleanup fix is helpful, but can you also confirm whether PegaProx now detects/refuses this unsafe topology? Specifically: - source and target cluster using the same NFS export - source VMID and target VMID being identical - source disk path and target disk path resolving to the same images/<VMID> directory Expected behavior would be to abort replication before any cleanup if source and target disk paths collide. DavidDePedro El El mar, 19 may 2026 a las 14:49, mkellermann97 ***@***.***> escribió:

…

*mkellermann97* left a comment (PegaProx/project-pegaprox#422) <#422 (comment)> @depedro-ai <https://github.com/depedro-ai> — apologies for the delay in getting back on the bundle. Got around to it just now; the recent_tasks in your bundle nailed the full chain. Posting the diagnosis + fix + immediate workaround. *Root cause* (visible directly in your bundle's recent_tasks): qmdelsnapshot status=command '/usr/bin/qemu-img snapshot -d xcrepl-ac43d91a-... \ /mnt/pve/netapp/images/319/vm-319-disk-0.qcow2' failed: exit code 1 qmshutdown status=VM is locked (snapshot-delete) id=319 [× 9 in the log] So the chain is: 1. Cross-cluster replication created the snapshot xcrepl-ac43d91a-1778856448 on VM 319 2. Replication itself succeeded 3. PegaProx issued the post-replication snapshot cleanup 4. PVE's qemu-img snapshot -d failed on NetApp NFS with ESTALE / exit code 1 5. *PVE left the VM at lock = snapshot-delete* and did NOT release the lock 6. Every subsequent operation (your failover attempts included) bounced off "VM is locked (snapshot-delete)" — which is exactly the "fails with no error" symptom you reported, because PegaProx wasn't surfacing the lock-status to the UI *Immediate workaround to unblock VM 319 right now*, on the source cluster node: # Clear the snapshot-delete lock qm unlock 319# Force-delete the snapshot config entry (the actual on-disk snapshot may# stay orphaned on NetApp, but the VM gets out of lock state) qm delsnapshot 319 xcrepl-ac43d91a-1778856448 --force After that the VM should accept shutdown / failover / migrate again. *Fix landed on Testing as c45390d* — _cleanup_snapshot in our cross-cluster-replication code now: - Always passes force=1 to the DELETE — PVE removes the config entry even when the on-disk qemu-img delete fails. We own xcrepl snapshots end to end; an orphan internal .qcow2 slot on storage is harmless, a stuck lock is not. - Defence in depth: after the DELETE, re-reads the VM config and explicitly clears lock = snapshot-delete if it's still set (with audit-log entry so operators can see why the lock disappeared). After this rolls into a release, the "fails with no error on failover" symptom should disappear — the lock that was silently blocking everything will get cleaned as part of the standard replication cleanup path. The intermittent ESTALE itself is a NetApp-NFS-side behaviour, not something PegaProx triggers, but our cleanup needs to tolerate it gracefully. That's what this fix does. Donations at https://opencollective.com/pegaprox if recent_tasks-driven root-causing earns its keep. — Marcus — Reply to this email directly, view it on GitHub <#422 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BPN6HVFM4AYLAFMDLCUMYTD43RJ43AVCNFSM6AAAAACY7OFMHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DIOBXHA4DKMBQGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[depedro-ai]

i test, as you now, cross-replication in different storage and the same error. The folder of the vm disk delete El mar, 19 may 2026 a las 15:17, David De Pedro ***@***.***>) escribió:

…

Thanks for the detailed diagnosis. The snapshot-delete lock issue makes sense and matches the qemu-img ESTALE failure we saw. However, there is still another concern from my side. In one of the affected cases, the problem was not only that the VM remained locked. The original VM config still pointed to: netapp:100/vm-100-disk-0.qcow2 but the physical disk file and the images/100 directory contents were gone from the shared NFS storage. This happened after testing cross-cluster replication where source and target had access to the same NetApp NFS storage and the replicated VM used the same VMID as the original one. So my concern is whether the cleanup logic can accidentally remove the origin VM disk directory when source and target resolve to the same physical storage path and VMID. The lock cleanup fix is helpful, but can you also confirm whether PegaProx now detects/refuses this unsafe topology? Specifically: - source and target cluster using the same NFS export - source VMID and target VMID being identical - source disk path and target disk path resolving to the same images/<VMID> directory Expected behavior would be to abort replication before any cleanup if source and target disk paths collide. DavidDePedro El El mar, 19 may 2026 a las 14:49, mkellermann97 < ***@***.***> escribió: > *mkellermann97* left a comment (PegaProx/project-pegaprox#422) > <#422 (comment)> > > @depedro-ai <https://github.com/depedro-ai> — apologies for the delay in > getting back on the bundle. Got around to it just now; the recent_tasks in > your bundle nailed the full chain. Posting the diagnosis + fix + immediate > workaround. > > *Root cause* (visible directly in your bundle's recent_tasks): > > qmdelsnapshot status=command '/usr/bin/qemu-img snapshot -d xcrepl-ac43d91a-... \ > /mnt/pve/netapp/images/319/vm-319-disk-0.qcow2' failed: exit code 1 > qmshutdown status=VM is locked (snapshot-delete) id=319 [× 9 in the log] > > So the chain is: > > 1. Cross-cluster replication created the snapshot > xcrepl-ac43d91a-1778856448 on VM 319 > 2. Replication itself succeeded > 3. PegaProx issued the post-replication snapshot cleanup > 4. PVE's qemu-img snapshot -d failed on NetApp NFS with ESTALE / exit > code 1 > 5. *PVE left the VM at lock = snapshot-delete* and did NOT release > the lock > 6. Every subsequent operation (your failover attempts included) > bounced off "VM is locked (snapshot-delete)" — which is exactly the "fails > with no error" symptom you reported, because PegaProx wasn't surfacing the > lock-status to the UI > > *Immediate workaround to unblock VM 319 right now*, on the source > cluster node: > > # Clear the snapshot-delete lock > qm unlock 319# Force-delete the snapshot config entry (the actual on-disk snapshot may# stay orphaned on NetApp, but the VM gets out of lock state) > qm delsnapshot 319 xcrepl-ac43d91a-1778856448 --force > > After that the VM should accept shutdown / failover / migrate again. > > *Fix landed on Testing as c45390d* — _cleanup_snapshot in our > cross-cluster-replication code now: > > - Always passes force=1 to the DELETE — PVE removes the config entry > even when the on-disk qemu-img delete fails. We own xcrepl snapshots end to > end; an orphan internal .qcow2 slot on storage is harmless, a stuck lock is > not. > - Defence in depth: after the DELETE, re-reads the VM config and > explicitly clears lock = snapshot-delete if it's still set (with > audit-log entry so operators can see why the lock disappeared). > > After this rolls into a release, the "fails with no error on failover" > symptom should disappear — the lock that was silently blocking everything > will get cleaned as part of the standard replication cleanup path. > > The intermittent ESTALE itself is a NetApp-NFS-side behaviour, not > something PegaProx triggers, but our cleanup needs to tolerate it > gracefully. That's what this fix does. > > Donations at https://opencollective.com/pegaprox if recent_tasks-driven > root-causing earns its keep. > > — Marcus > > — > Reply to this email directly, view it on GitHub > <#422 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/BPN6HVFM4AYLAFMDLCUMYTD43RJ43AVCNFSM6AAAAACY7OFMHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DIOBXHA4DKMBQGA> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >

-- DePedro,

[mkellermann97]

@depedro-ai — you're right, I was too narrow with c45390d. That commit fixes the lock-stuck-after-cleanup half (the symptom you spent the most time on in the previous comments — silent failover failures), but it does NOT address the more serious data-loss half you're flagging here.

Re-reading the chain you described:

• Source cluster A, VM 100, on netapp: (shared NFS)
• Target cluster B, same netapp: mount, no other VMs → cluster/nextid on B returns 100 (because B's local view doesn't know about A's VMIDs)
• xcrepl clone lands at netapp:100/vm-100-disk-0.qcow2 on B
• Same physical directory as A's VM 100
• Cleanup on one side removes images/100/ → both lose data

This is a real same-shared-storage + VMID-collision scenario that our xcrepl flow doesn't currently guard against. We use cluster/nextid on the target cluster, which only checks the target's own VMID namespace — it has no view of the source's VMIDs when they share physical storage.

The lock-cleanup fix in c45390d does NOT add detection for this. So as the ticket stands, I'd consider the lock-half addressed and the data-loss-half still open.

To confirm whether PegaProx now detects/refuses this unsafe topology: no, not yet. The detection needs:

1. Resolve source-storage and target-storage to their physical mount (PVE-side pvesm path or storage UUID comparison).
2. If they share the same physical mount AND cluster/nextid on target collides with source's VMID → refuse with a clear error before kicking off the snapshot, OR shift the target VMID range to avoid collision.
3. Defence in depth: even with separate VMIDs, if source and target share physical storage, the xcrepl cleanup needs to verify the disk path is unambiguously the clone's before deleting.

That's a multi-layer fix and I won't ship it in the same slot as the lock-cleanup. Tracking it as the second piece of this ticket.

Workaround until the topology-check ships:

• Easiest: ensure source and target clusters point at different physical NFS mounts even if they're on the same NetApp head (different exports / different storage IDs that don't share /mnt/pve/<id> after symlink resolution).
• OR: if they MUST share the storage, manually pre-allocate VMIDs on the target so cluster/nextid returns something outside the source's used range.

Apologies for not catching this in the first read — you flagged it explicitly in the original report title ("delete origin disk vm folder") and I anchored on the lock-symptom instead.

— Marcus

[depedro-ai]

Thanks, that matches exactly what I observed. For now I will stop using xcrepl with shared NFS storage between clusters. The dangerous part is not only the VMID collision, but the fact that both clusters can resolve the same storage path to the same physical directory. In my case, source VM 100 on cluster A and target VM 100 on cluster B both resolved to: /mnt/pve/netapp/images/100/ When cleanup ran, the original VM disk directory was removed. For the final fix, I think PegaProx should refuse the operation if: - source and target storage resolve to the same physical NFS export/path - and the target VMID equals the source VMID Also, I would strongly suggest allowing a configured target VMID range for cross-cluster replication, for example 900-1299, instead of relying only on cluster/nextid. Even with different VMIDs, cleanup should never delete a directory unless it has positively identified it as belonging to the temporary clone created by the current xcrepl job. DavidDePedro El El mar, 19 may 2026 a las 17:18, mkellermann97 ***@***.***> escribió:

…

*mkellermann97* left a comment (PegaProx/project-pegaprox#422) <#422 (comment)> @depedro-ai <https://github.com/depedro-ai> — you're right, I was too narrow with c45390d. That commit fixes the *lock-stuck-after-cleanup* half (the symptom you spent the most time on in the previous comments — silent failover failures), but it does NOT address the more serious *data-loss* half you're flagging here. Re-reading the chain you described: - Source cluster A, VM 100, on netapp: (shared NFS) - Target cluster B, same netapp: mount, no other VMs → cluster/nextid on B returns 100 (because B's local view doesn't know about A's VMIDs) - xcrepl clone lands at netapp:100/vm-100-disk-0.qcow2 on B - *Same physical directory as A's VM 100* - Cleanup on one side removes images/100/ → both lose data This is a real same-shared-storage + VMID-collision scenario that our xcrepl flow doesn't currently guard against. We use cluster/nextid on the target cluster, which only checks the target's own VMID namespace — it has no view of the source's VMIDs when they share physical storage. The lock-cleanup fix in c45390d does NOT add detection for this. So as the ticket stands, I'd consider the *lock-half* addressed and the *data-loss-half* still open. *To confirm whether PegaProx now detects/refuses this unsafe topology: no, not yet.* The detection needs: 1. Resolve source-storage and target-storage to their physical mount (PVE-side pvesm path or storage UUID comparison). 2. If they share the same physical mount AND cluster/nextid on target collides with source's VMID → refuse with a clear error before kicking off the snapshot, OR shift the target VMID range to avoid collision. 3. Defence in depth: even with separate VMIDs, if source and target share physical storage, the xcrepl cleanup needs to verify the disk path is unambiguously the clone's before deleting. That's a multi-layer fix and I won't ship it in the same slot as the lock-cleanup. Tracking it as the second piece of this ticket. *Workaround until the topology-check ships:* - Easiest: ensure source and target clusters point at *different* physical NFS mounts even if they're on the same NetApp head (different exports / different storage IDs that don't share /mnt/pve/<id> after symlink resolution). - OR: if they MUST share the storage, manually pre-allocate VMIDs on the target so cluster/nextid returns something outside the source's used range. Apologies for not catching this in the first read — you flagged it explicitly in the original report title ("delete origin disk vm folder") and I anchored on the lock-symptom instead. — Marcus — Reply to this email directly, view it on GitHub <#422 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BPN6HVDYKN3BI5AJQVV7NUT43R3MDAVCNFSM6AAAAACY7OFMHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DIOBZGIZDSNZYHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[mkellermann97]

@depedro-ai — v0.9.11 includes c45390d. Lock-stuck-after-cleanup is handled (force=1 + ESTALE unlock). The shared-NFS VMID-collision risk you flagged in the last comment is a broader replication-design question — noted on our side. Closing the original.

— Marcus

(donations at https://opencollective.com/pegaprox if PegaProx earns its keep)