Skip to content

Commit 7e56264

Browse files
committed
docs: fixed_auto chunking method update
1 parent 94db2b0 commit 7e56264

2 files changed

Lines changed: 18 additions & 9 deletions

File tree

docs/chunking/fixed_size.md

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -83,16 +83,21 @@ For most use cases, `fixed_128k` or CDC variants are preferred. Consider `fixed_
8383

8484
!!! warning "Alpha"
8585

86-
`fixed_auto` is in alpha status and is not recommended for production use
86+
`fixed_auto` is in alpha status and is not well optimized for performance
8787

88-
`fixed_auto` walks the file in 128 KiB windows.
89-
For each full window, it checks the previous backup's same-path chunk layout at the same offset:
88+
`fixed_auto` walks the file in 128 KiB windows. For each full window, it checks the previous backup's same-path chunk layout at the same offset:
9089

9190
- if the previous window was one 128 KiB chunk and the current content is unchanged, it keeps one 128 KiB chunk
9291
- if the previous window was one 128 KiB chunk and the current content changed, it stores the current window as thirty-two 4 KiB chunks
9392
- if the previous window was thirty-two 4 KiB chunks, it compares the 4 KiB hashes first; when none changed, it stores one 128 KiB chunk, otherwise it keeps thirty-two 4 KiB chunks
9493

95-
Missing previous data, direct blobs, irregular previous layouts, and incomplete tail windows are stored as one chunk for that window.
94+
Missing previous data, direct blobs, irregular previous layouts, and incomplete tail windows are stored as one chunk for that window
95+
96+
With this, `fixed_auto` can achieve the following effect: for parts of a file that keep changing, it performs chunk-level deduplication at 4 KiB granularity;
97+
for other parts, it performs chunk-level deduplication at 128 KiB granularity
98+
99+
Since region files (`.mca`) in Minecraft saves are modified at 4 KiB granularity,
100+
`fixed_auto` is expected to achieve deduplication close to `fixed_4k` without introducing excessive metadata overhead
96101

97102
## Poor Candidates
98103

@@ -106,4 +111,3 @@ Fixed-size chunking is a poor choice for:
106111

107112
Fixed-size chunking has no additional Python dependency requirements.
108113
It is available as long as Prime Backup is installed
109-

docs/chunking/fixed_size.zh.md

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -84,16 +84,21 @@ title: '固定大小分块'
8484

8585
!!! warning "Alpha"
8686

87-
`fixed_auto` 处于 alpha 阶段,不建议在生产环境中使用
87+
`fixed_auto` 处于 alpha 阶段,性能相对其他分块算法较低
8888

89-
`fixed_auto` 会按 128 KiB 窗口遍历文件。
90-
对于每个完整窗口,它会检查上一次备份中同路径文件在相同 offset 的分块布局:
89+
`fixed_auto` 会按 128 KiB 窗口遍历文件。 对于每个完整窗口,它会检查上一次备份中同路径文件在相同 offset 的分块布局:
9190

9291
- 如果上一版窗口是 1 个 128 KiB chunk,且当前内容未变化,则继续使用 1 个 128 KiB chunk
9392
- 如果上一版窗口是 1 个 128 KiB chunk,但当前内容已变化,则将当前窗口切成 32 个 4 KiB chunk
9493
- 如果上一版窗口是 32 个 4 KiB chunk,则先比较 4 KiB hash;当变化数量为 0 时,存成 1 个 128 KiB chunk,否则继续使用 32 个 4 KiB chunk
9594

96-
上一版数据缺失、上一版是 direct blob、上一版布局不规则,或当前窗口是不完整尾块时,该窗口会作为单个 chunk 存储。
95+
上一版数据缺失、上一版是 direct blob、上一版布局不规则,或当前窗口是不完整尾块时,该窗口会作为单个 chunk 存储
96+
97+
借此,`fixed_auto` 可达成这样的效果:对于文件持续变化的部分,以 4KiB 为粒度进行分块去重;
98+
对于其他部分则以 128KiB 的粒度进行分块去重
99+
100+
由于 Minecraft 存档中的的 region 文件(.mca)是以 4KiB 为粒度进行修改的,
101+
`fixed_auto` 预期可在不产生过多元数据开销的前提下,达到接近 `fixed_4k` 的去重效果
97102

98103
## 不适用场景
99104

0 commit comments

Comments
 (0)