Is there an existing request for this?
Is your feature request related to a problem?
It would be great if Midnight Commander supported an additional overwrite mode for copy/move operations: updating an existing destination file in place by writing only the blocks that differ from the source file.
At the moment, when copying or moving a file over an existing file, MC offers actions such as overwrite, append, and reget. Reget is useful when the destination file is a shorter prefix of the source file, but it only handles the simple "resume from the end" case. It does not help when the destination file is mostly identical to the source file but has changed blocks somewhere in the middle.
The proposed mode is similar in spirit to what rsync --inplace --no-whole-file is useful for: avoid rewriting the whole destination file when only parts of it differ.
Suggested UI changes:
For a single existing file:
[ Yes ] [ No ] [ Append ] [ Reget ] [ Update* ]
For the “Overwrite all files?” section:
Overwrite all files?
[ ] Don't overwrite with zero length file
[ ] Update files in place when possible*
[ All ] [ Older ] [ None ] [ Smaller ] [ Size differs ]
The single-file [Update] button would mean: make the destination file identical to the source file, but compare source and destination contents block-by-block and write only changed blocks when possible.
The all-files checkbox would modify the selected overwrite policy. For example:
[All] + “Update files in place when possible” updates every existing destination file in place where possible.
[Older] + the checkbox updates in place only when the source is newer.
[Size differs] + the checkbox updates in place only when the source and destination sizes differ.
This keeps the existing overwrite policies intact and avoids adding another overloaded "all files" button.
Rationale:
This is useful for large files where the destination is already mostly correct and only a small part changed, for example:
- virtual machine images;
- disk images;
- large archives;
- database dumps;
- partially copied files;
- game/resource files;
- large project/media files;
- files on flash storage (USB flash drives, SSD, NVMe) where avoiding unnecessary writes is desirable.
Currently, overwriting such a file rewrites the entire destination even when most of the destination is already byte-identical to the source. That is wasteful in both time and writes. Reget helps only when the missing/different part is strictly at the end of the file. The proposed mode generalizes that idea to changed ranges anywhere in the file.
The attached patch implements this conservatively:
- adds a new
REPLACE_UPDATE action;
- adds
ctx->do_update_in_place;
- adds an
[Update] button to the single-file overwrite dialog;
- adds an "Update files in place when possible" checkbox to the all-files overwrite dialog;
- compares source and destination in 128 KiB chunks;
- compares those chunks in 4 KiB blocks;
- writes only destination blocks that differ;
- appends the missing tail when the source file is larger than the destination;
- falls back to the usual overwrite path when the destination is larger than the source.
The fallback for destination-larger-than-source is intentional. MC's VFS layer does not appear to expose a generic truncate operation suitable for this code path. Since making the final destination byte-identical to the source would require truncation in that case, the patch falls back to the existing overwrite behavior instead of leaving an incorrect longer file.
The mode is deliberately described as "when possible" because it is only appropriate for existing regular files where MC can seek and update the destination safely. It should not replace normal overwrite semantics.
Important caveat:
This mode is not atomic. Unlike a safe overwrite implementation that writes a temporary file and renames it over the old destination, in-place update mutates the existing destination file. If the operation is interrupted, the destination may contain a mixture of old and new blocks.
That is the same basic tradeoff as other in-place update mechanisms: fewer writes and better resume-like behavior, but weaker crash/interruption semantics. For that reason this should remain an explicit user-selected mode, not a replacement for normal overwrite.
The help/man page should document this clearly. The patch adds help text for the new [Update] button and the all-files checkbox, including the non-atomic behavior.
Describe the solution you'd like
See above.
Describe alternatives you've considered
There's rsync's --inplace --no-whole-file mode but it's extremely user-unfriendly and prone to catastrophic errors (the tailing slash issue).
Additional context
mc-update-inplace-overwrite.patch
Is there an existing request for this?
Is your feature request related to a problem?
It would be great if Midnight Commander supported an additional overwrite mode for copy/move operations: updating an existing destination file in place by writing only the blocks that differ from the source file.
At the moment, when copying or moving a file over an existing file, MC offers actions such as
overwrite,append, andreget.Regetis useful when the destination file is a shorter prefix of the source file, but it only handles the simple "resume from the end" case. It does not help when the destination file is mostly identical to the source file but has changed blocks somewhere in the middle.The proposed mode is similar in spirit to what
rsync --inplace --no-whole-fileis useful for: avoid rewriting the whole destination file when only parts of it differ.Suggested UI changes:
For a single existing file:
For the “Overwrite all files?” section:
The single-file
[Update]button would mean: make the destination file identical to the source file, but compare source and destination contents block-by-block and write only changed blocks when possible.The all-files checkbox would modify the selected overwrite policy. For example:
[All]+ “Update files in place when possible” updates every existing destination file in place where possible.[Older]+ the checkbox updates in place only when the source is newer.[Size differs]+ the checkbox updates in place only when the source and destination sizes differ.This keeps the existing overwrite policies intact and avoids adding another overloaded "all files" button.
Rationale:
This is useful for large files where the destination is already mostly correct and only a small part changed, for example:
Currently, overwriting such a file rewrites the entire destination even when most of the destination is already byte-identical to the source. That is wasteful in both time and writes.
Regethelps only when the missing/different part is strictly at the end of the file. The proposed mode generalizes that idea to changed ranges anywhere in the file.The attached patch implements this conservatively:
REPLACE_UPDATEaction;ctx->do_update_in_place;[Update]button to the single-file overwrite dialog;The fallback for destination-larger-than-source is intentional. MC's VFS layer does not appear to expose a generic truncate operation suitable for this code path. Since making the final destination byte-identical to the source would require truncation in that case, the patch falls back to the existing overwrite behavior instead of leaving an incorrect longer file.
The mode is deliberately described as "when possible" because it is only appropriate for existing regular files where MC can seek and update the destination safely. It should not replace normal overwrite semantics.
Important caveat:
This mode is not atomic. Unlike a safe overwrite implementation that writes a temporary file and renames it over the old destination, in-place update mutates the existing destination file. If the operation is interrupted, the destination may contain a mixture of old and new blocks.
That is the same basic tradeoff as other in-place update mechanisms: fewer writes and better resume-like behavior, but weaker crash/interruption semantics. For that reason this should remain an explicit user-selected mode, not a replacement for normal overwrite.
The help/man page should document this clearly. The patch adds help text for the new
[Update]button and the all-files checkbox, including the non-atomic behavior.Describe the solution you'd like
See above.
Describe alternatives you've considered
There's rsync's --inplace --no-whole-file mode but it's extremely user-unfriendly and prone to catastrophic errors (the tailing slash issue).
Additional context
mc-update-inplace-overwrite.patch