Large files sent through Syncthing have very slow reads from disks?


(Ben) #1

I’ve been using Syncthing for awhile and recently came into a serious problem which is basically going to require me to wipe a 100TB backup server. I noticed most of the files I’ve synced in the last few months have very low reads from the disks (~11.7MB/s). It seems that it might only affect very large files (50GB+) but I haven’t been able to confirm. However, every single 50+ GB file is affected.

  • Around 400 50GB+ files I sent through Syncthing can only be read at ~11.7MB/s from the disks on the backup server.

  • 22 different disks are affected, so it’s not the disks themselves.

  • Files on the disks that were not sent via Syncthing read at 150-200MB/s.

  • Transferring the data off the disks at ~11.7MB/s, deleting the slow files, and then transferring it back via SMB/Windows results in 150-200MB/s reads again.

  • The disks are not splicing or sharing data and are on 3 different SATA controllers.

  • Both the main server and backup server are on unRAID and using XFS filesystem.

The only thing I can think of is Syncthing is causing extreme fragmentation when dealing with large files however I feel like it wouldn’t be a consistent ~11.7MB/s if that were the case. This honestly is making no sense to me, so any ideas why this could have happened would be helpful.


(lf) #2

Syncthing transfers files in chunks, in a random order. If you are using spinning disks, the resulting fragmentation will be quite annoying.

I don’t have experience with XFS, but it seems like there are tools for measuring fragmentation, and for defragmenting (found with google):

Relevant prior discussion with some advice:


(Ben) #3

I’m going to try what is suggested on that thread, enabling “Large Blocks” and setting Copiers to 1. Thanks.


(Audrius Butkevicius) #4

The blocks will still be delivered in random order they arrive, but yes, large blocks will help.

My understanding is that fragmentation should not happen at the disk level, as I believe spinning disk seeks to sector representing that offset to write data.