You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/chunking/fixed_size.md
+9-5Lines changed: 9 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -83,16 +83,21 @@ For most use cases, `fixed_128k` or CDC variants are preferred. Consider `fixed_
83
83
84
84
!!! warning "Alpha"
85
85
86
-
`fixed_auto` is in alpha status and is not recommended for production use
86
+
`fixed_auto` is in alpha status and is not well optimized for performance
87
87
88
-
`fixed_auto` walks the file in 128 KiB windows.
89
-
For each full window, it checks the previous backup's same-path chunk layout at the same offset:
88
+
`fixed_auto` walks the file in 128 KiB windows. For each full window, it checks the previous backup's same-path chunk layout at the same offset:
90
89
91
90
- if the previous window was one 128 KiB chunk and the current content is unchanged, it keeps one 128 KiB chunk
92
91
- if the previous window was one 128 KiB chunk and the current content changed, it stores the current window as thirty-two 4 KiB chunks
93
92
- if the previous window was thirty-two 4 KiB chunks, it compares the 4 KiB hashes first; when none changed, it stores one 128 KiB chunk, otherwise it keeps thirty-two 4 KiB chunks
94
93
95
-
Missing previous data, direct blobs, irregular previous layouts, and incomplete tail windows are stored as one chunk for that window.
94
+
Missing previous data, direct blobs, irregular previous layouts, and incomplete tail windows are stored as one chunk for that window
95
+
96
+
With this, `fixed_auto` can achieve the following effect: for parts of a file that keep changing, it performs chunk-level deduplication at 4 KiB granularity;
97
+
for other parts, it performs chunk-level deduplication at 128 KiB granularity
98
+
99
+
Since region files (`.mca`) in Minecraft saves are modified at 4 KiB granularity,
100
+
`fixed_auto` is expected to achieve deduplication close to `fixed_4k` without introducing excessive metadata overhead
96
101
97
102
## Poor Candidates
98
103
@@ -106,4 +111,3 @@ Fixed-size chunking is a poor choice for:
106
111
107
112
Fixed-size chunking has no additional Python dependency requirements.
108
113
It is available as long as Prime Backup is installed
0 commit comments