i wasn’t sure if i should open up this as a feature request or just a simple support thread as there may be a solution to this already.
Today i discovered that Syncthing runs extremely slow on my ds213j synology NAS as a result of both scanning a huge folder while also not pausing syncing. So i added about 700GB of large files to my folder (r/w in both directions) and syncthing started scanning these files. It did scan those files for as slowly as 500kB - 1 MB/sec resulting in an approximate time remaining of 3 days or longer. Yet as soon as i paused the sync with my remote folder, the process sped up to about 14 MB/sec having scanned those 700GB in about 4 hrs.
And as for syncing, speeds were below 1 MB/sec while scanning, so no bargain there too.
CPU utilisation was in both cases 100% and RAM usage did not change a lot. There were no other activities on the NAS while testing. Even though my HD speeds are up to 90MB/sec read, it is most likely that the CPU is bottlenecking when simultaniously scanning and syncing at the same time.
Now my proposal would be to get an option to auto-pause all syncs while folders are being scanned for low end devices.
Setup 1 was: syncing a folder from another disc (not where syncthing is installed) with a remote location
Setup 2 was: syncing a folder from the same disc where syncthing is installed
Setup 3, as in: syncfolder is in the same directory as the database was never the case.
So i actually changed to setup 2 because i found out, setup 1 was slower. This might be because (and this is my assumption) syncthing actually grabs parts of the files and copies it to its home folder and then scans or syncs it. Because whenever i was syncing and scanning, both my discs had the same read and write speeds at the same time, while i thought it would only need the one disc where the data was stored and RAM and CPU for the rest of the work. But it seems as if syncthing is caching all the data in its own homefolder. Thats at least my observation.
Setup 2 is better in terms of performance and Setup 3 was never used. I never used the default directories and setup my own.
Why not the other way around? What’s a low end device?
Why not the other way around: Might be dependent on for what you use those shares of course. For me, there is nothing to transfer if the folders don’t get scanned prior, thats why i suggested it.
Low end devices aren’t actually extremely uncommon, but just rare. Your own statistic provides the data for that. About 5% of all users do have similar computing power as to my setup. Yet i have to admit, they handle way less data than i am.
But that brings me back to my second answer. Other than this hiccup, syncthing works perfect even though my low end hardware. Having an opt-in to a more conditionalised approach in terms of “what job to do first” would be nice in my case.
I’m not suggesting low end devices don’t exist, I’m sure they are rather common, just wondering what criteria you’re considering. In terms of scanning+syncing folders on the same physical disk, anything with rotating disks is low end and will behave much the same I think.
CPU wise I don’t think low end vs high end matters much - doing A plus B concurrently will in total take roughly as long as doing first A then B. If there are lots of cores then doing the tasks concurrently will be faster, but even with just the one core the difference isn’t night and day and it’s not something I think it’s worth optimizing for. (We already de-optimize CPU performance on Windows and Mac to avoid hurting system interactivity too much.)
Disks have the other property, that doing A and B concurrently can take ten times as long as doing first A then B.
I am not suggesting you bring this upon certain users which low end devices. I am merely suggesting to think about an opt-in in the advanced folder section menu. Helppage would vaguely say that this might improve performance on low and devices such as embedded devices with lots of data to handle.
I am not sure if you want to tell me to let it be or if my problem is not clear enough. I’m fine either way as i at least tried If you think there is definitively no room for that, it’s ok for me. Maybe i’ll stick with it, maybe i’ll check if rclone performes better on my embedded device. I merely wanted to bring up the issue and suggest a work around for it.