I’ve been troubleshooting extremely slow transfer speeds, and I’ve finally pinpointed it to block indexing. Disabling it makes speeds go up from 350kB/s to the expected 16MB/s. This is when transferring a large set of 8MiB files (created with random data for testing). The endpoints are running 4.5GHz Skylake and 3.7GHz Zen3. The former seems to be slightly faster despite the latter having much more cores. And sure enough, with the slow transfers, only a couple of CPU cores are maxed out.
Even after disabling indexing on most of my shares, so that the total amount of block indexes to compare is smaller, the speed with block indexing on the share doing the transfer is still on the order of 800kB/s.
The conclusion is to just not use block indexing. But from my understanding, that’s exactly the feature that allows Syncthing to “rename/move” files. Is there really no other way for Syncthing to detect renames and moves? The shares where renames and moves are regular enough still contain about 300k files and 1TiB, which appears to be too much for Syncthing’s block indexing to handle at any reasonable speed. Do I just have to live with re-transferring a file when a simple rename happens?
Another option would be to disable block comparisons between shares, so each share has its own private pool of block indexes. This would greatly reduce the number of comparisons. But Syncthing doesn’t appear to have this option.
Is it the actual block comparing that’s slowing you down? Or is it the actual rehashing which is re-reading all the files and slowing down the disk access?
In my experience renaming/moving doesn’t really work at all. In my experience the best case scenario is the side originating the move or rename re-indexes the files, sends the hashes and the remote duplicates the file with the new name or location and then deletes the original.
In the end you get what you’re supposed to have at the remote end. You can say it doesn’t work because everything is copied but you still get the right result in the end. (with the negative side effect if you’re using versioning that a copy is placed in the versions folder.)
Oh, then I did something wrong when I tested. Moves and renames do indeed work now without block indexing. My bad, and I guess this whole topic was a waste of my time =). Although maybe it can help someone else troubleshoot their slow transfer speeds, hopefully.
Not sure why it would re-read all files, but all shares reported “Up to date” and none were scanning.
Disabling block indexing on the share I was transferring to instantly increased transfer speeds from 350kB/s to whatever my internet speed is, with CPU usage going from 200% to 10% on the receiving end.
Renaming and moving seems to work fine after all. Although the receiver does not actually rename or move, it creates a copy of the original, which is a problem with ZFS snapshots, but I’m not sure what can be done about this.