Just to express my gratefulness for implementing this!
I’ve been waiting for this feature for a long time now.
In one of my setups, I have 3 nodes in separate locations that are syncing sync about 3TB of data using syncthing.
Most of data is not changed often, but small updates are quite frequent.
Up until the delta-indexes implementation, initial index exchange was resulting in over 1GB of data transmission for each client. And would a connection drop for a second - and here goes another 1GB.
Given poor upload speeds, each such reconnect would take sometimes up to few hours, even if the actual data change would be a 500KB text file update if any.
So I had to jump through some loops and run several separate ST instances, to keep small and big repositories separate.
But that’s no longer the case! No more “useless” GBs flying around! No more waiting for hours just to establish a connection!
Some numbers:
Without delta-indexes: >1GB send + >1GB received by each client, initial connection takes hours.
With delta-indexes: <10KB send and received in total by all clients, connection established in seconds!!!
Also slow calculators (raspberry pi 1 etc.) got a lot more usable by this. Index exchange was several minutes of 100% CPU usage, now it’s <5 seconds for my data.
Similar experience, synology NAS devices, with “hilariously ridiculous” CPU’s and slow DSL connexions.
Previously with one folder of 600Gb/50.000 files it could take up to two hours “starting”,exchanged each time ~320Mb with cpu >90%.
Now it takes seconds.
Amazing improvement. Forever grateful.
Yep, great stuff! Now I don’t know if this is because of delta indexes, but the syncing of big files with little change (VM images) is now lighting fast:) It improved at least 4x. For example to sync my 9GB outlook pst files - now it takes less than hour, before 4-5hours. So it seems that reusing existing block is way faster than before, thanks @calmh
Any device running syncthing version 0.14.1 and up, support delta-indexes. And when communicating to other devices that support it as well, delta-indexes will be used by default.
