Efficient sparse file handling

[andreygursky]

Daniel,
what's about support of variable block length? Perhaps this could be used to mark (very) large holes in files, thus making them to be patched/transferred at once.

P.S. By the way, do you plan to continue working on go-sync or maybe you are about to switch it into an unmaintained project? Or a third alternative: no active development anymore, only if there are patches from others. Maybe you could update https://github.com/Redundancy/go-sync#current-state with additional infos about how you see the further development process of go-sync. You could also clarify, what are TODOs, that you're personally about to finish, and what are nice things to implement additionally, but rather a long term ones and/or welcome to do by others.

[Redundancy]

go-sync uses a fixed block size like rsync, but will combine continuous missing ranges when it analyses the file. It can then decide the maximum range length that it will do in a single blocksource request.

You can decrease the block size and get more accurate diffs, but this will be at the cost of an increased index size and significantly more comparisons that are needed.

[jsilhan]

By the way, do you plan to continue working on go-sync or maybe you are about to switch it into an unmaintained project?

I also wonder what would the future be of this project. @Redundancy, do you know who is already using go-sync in production?

[bwmarrin]

Same here :)

[Redundancy]

I don't know of anyone who is using gosync in production, and still wouldn't recommend it personally.

I'd like to continue to poke at it, but it hasn't been easy to find the free time and energy recently with work commitments, so I also don't want to promise anything. I was hoping that some people might start contributing, because I don't think it's something that I could really support solo anyway. At the very least I'd like to separate the commandline tool from the underlying library with something like gb.

Apologies if that's disappointing anyone :)

[bwmarrin]

It is a bit sad :) I'd love and would be willing to help, but my knowledge of rsync inner workings is pretty non existent. But having a cross platform rsync compatible alternative that runs natively on Windows would be a godsend and I always hoped someday this package would lead the way.

[Redundancy]

Noted - I'll try and pick up the effort again once things are a little less crazy.