I’ve been using syncthing for about 8 months and I love it. It does a great job. Large folders with lots of files and subdirectories, no problem. Lots of folders being synchronized across a few devices, no problem.
But lately I’ve noticed that if the folder contains large files (like 6GB) then it slows down to a crawl. CPU usage is minimal, HDD r/w is minimal, and network throughput is minimal. I’ve read other posts and it seems to be related to the time taken for hashing the file blocks.
I’m mainly hoping for some advice here. The machine I’m using is powerful. It seems as if there’s one or a couple processes running and when it comes to these large files they take a while to do their thing. In the meantime, other smaller files start to queue up. Can I increase the number of processes somehow? Each folder is set as “Smallest first” but some folders seem to wait for other folders to finish.
The following screenshots might help to explain my problem and what’s going on. The AppdataBackup folder contains the large files and the Pictures folder has lots of smaller files but it seems as if the Pictures folder is waiting for the AppdataBackup folder.
Have you checked whether the devices have a direct connection, or is there a relay involved? The latter might limit throughput to way less than your own network connection could sustain.
Maybe if someone can help me to understand the architecture a bit. For example, does each folder get a process to handle that folder? When syncthing starts up it scans each folder but some folders say “Waiting”. I assume it’s because there’s a limit to how much it does at one time.
Once a folder has been scanned does it add its files to a queue? I can see that a few files are in progress. It seems that there’s a set number of files that can be handled at one time. Is this file queue shared across the app (hence maybe the reason why large files take a while and hold up the processing)?
The reason I ask all this and want to understand what options are out there is because my servers are meant to be file servers and syncthing is the main, important app running on them. I want syncthing to use all the resources it can to do its thing.
I’ll keep tabs on this day by day. I think stuff is happening but it’s hard to see. It’s like watching paint dry. If you stare at it nothing seems to be happening but if you walk away and come back it’s dry.
One question: In my screenshots I show the syncthing03 server which is connected to syncthing01 and syncthing02. In the most recent screenshots I show the device info for syncthing01 (from the syncthing03 server). It shows an upload of 102 KiB. This would be upload from syncthing03 to syncthing01 (upload to that device)?
Without trying to hijack this thread too, could Tim do a speed test and also set different lower transfer values. I’m assuming you have a send and receive setup. It’s possible that due to issues i’m finding, there’s not enough bandwidth to transfer files.
I found that setting a lower upload speed actually helped with download speeds
