I love syncthing but the truth is that I end up disabling it most of the time.
My laptop is often on heavy loads and syncthing seems to add up significantly and disproportionally.
On 99% of the situations I really don’t care how quick the Sync takes place, as long as it eventually gets there.
With regards to HD usage, I backup my mobile with Syncthing and you end up with huge videos that can actually be deleted, however, my laptop will be drained right in the middle of the working day… which effectively forces me to stop my work and start watching and selecting videos before I can continue working.
I find these problems a real deal breaker in terms of user friendliness and could potentially have a lot of impact on mass adoption by the general public.
New feature description
Syncthing should be capable to throttle the resource usage and adapt it to current system workload.
For example never go over 20% CPU and disable syncing when general system load goes over 50% CPU, things like that. Different rules could be applied if the user is present of if the computer is idle.
Similarly Syncing should not drain HardDisk down to 0 bytes left, but stop the sync on when fiesystem free space is less than a given %.
The order of the sync folders in the UI could also be used to give some sense of priority to the tasks. Even some tasks could be ticket as low priority.
I could be possible to boost resource usage for that 1% of situations in which you actually need a sync file to arrive. In that case the boost period could be automatically finished when sync is complete or after a given period. Boosting particular syncs could also be good.
Syncthing already sets its own priority lower than the standard, so the OS should already throttle it if resources are maxed out and needed by other applications. Monitoring CPU usage and changing what Syncthing is doing based on that is clearly out of scope. If you want to limit its cpu usage, there probably tools for that on windows (I know there are on linux (systemd, cpulimit, …)).
There already are disk usage limits, see minDiskFree for folder limit (UI: folder editign) and minHomeDiskFree (UI: actions>settings): https://docs.syncthing.net/users/config.html
You mean you want to assign priority to shared folders, and then when multiple folders need to sync/scan, they will do so in order of priority?
The question is weather the intended goal is been achieved. In my case it is clearly not. I disable syncthing everyday on my i7-12GB linux laptop. In github searching for CPU brings 232 issues (open/closed).
Perhaps it is only me. Ultimately if there is a large % of users currently turning on/off syncthing daily that means that the use case is already out there and would make sense to facilitate it.
If monitoring the system is clearly out of scope, perhaps there is other way to avoid that users have to turn on/off syncthing. I could only think of monitoring system load.
Syncthing is clearly a very advanced background service. I don’t see why it would not benefit form having more insights on how the system is working and adapt to it. (CPU, Disk, Removable Drives, etc)
Thanks! I don’t know how I missed it!
Yes. If a folder with “work documents”, is way more important than “mobile videos”, for a given users they could signal it somehow via UI. The resources such as CPU could be assigned accordingly. Honestly I would mark some folders to be synched only while my system is idle if I could.
I have an i7 with 16 GB RAM and so far I have not been able to bring the CPU with Syncthing or Resilio, which I use in parallel, to full capacity. Maybe you can say something about the applications that make your laptop so busy. And also how many devices and peers you have in Syncthing.
On my productive Windows 10 computer, I connected a total of 20 peers and 10 devices, and on average, the CPU is loaded with about 1%. But that’s relative again.
The “out of scope” argument is contestable, however there’s another one: That’s complex and a lot of work, and there’s bigger fish to fry - so it’s unlikely to happen. Also there’s an easy workaround: Set a hard limit on cpu usage externally (if that isn’t easy, it’s a problem of your OS).
The *_PRIORITY_CLASS values affect the CPU scheduling priority of the process. For processes that perform background work such as file I/O, network I/O, or data processing, it is not sufficient to adjust the CPU scheduling priority; even an idle CPU priority process can easily interfere with system responsiveness when it uses the disk and memory. Processes that perform background work should use the PROCESS_MODE_BACKGROUND_BEGIN […]
Sorry, but process scheduling is a problem of the operating system, not a syncthing problem.
Yes, syncthing uses CPU, it’s explained in the FAQ why, and it’s usually a temporary state that doesn’t last forever (unless you are syncing something crazy like forever changing log files which are being synced permanently, but in that case you are asking for it).
Perhaps the slowness you are seeing is slowness caused by IO, but there is very little you can do there either, you should just move to an ssd.
I don’t think any sensible monitoring system is feasible with the number of platforms we support.
I don’t have a strong feeling about this, but I believe users can already adjust this externally, so whichever default we go for is not that important.
Saying the default isn’t important because users can set it externally is somewhat self-contradicting, as the entire point of having any default value is to provide a specific behaviour when the user doesn’t specify anything else, isn’t it?
Firefox, Telegram and Syncthing are supposed to be ON pretty much always. Zoom for RT conferencing often but as required.
I use the Jetbrains dev suites, sometimes also Visual studio. Sometimes also some docker containers for testing, etc. When documenting it will be LibreOffice.
Currently I synchronize 12 folders that include my Development folder and Mobile phone multimedia, around 60GB.
So, it is likely for me to experience times in which my system is quite busy.
RT services, like Zoom, Are a trigger that forces me to close syncthing. You cannot be on a teleconference with customers and experience cuts. On the other hand syncthing is non-critical.
Then you forget to put it back on… for some time. I use GTK with tray icon, which makes it easy to turn it on/off.
I don’t want to open a heated discussion about what Syncthing is and isn’t.
I just want it to succeed and become massively adopted.
In my particular case, this is a big issue. Honestly I don’t know if we are many or not. But if we are, this could be an area that would benefit a lot.
Number of Issues in github searching by CPU signal that it is an area of interest.
I would just open a pool among users asking: how often do you turn off syncthing due to load? never, rarely, weekly, daily… or something. I get some facts before committing to anything.
Sorry I dont see this:
if (System.load() > 3) syncthing.pause();
Sorry for the bad joke. But a nice part of engineering is when we find simple solutions to big problems. Sometimes, there are.
I agree, but most services tweak system settings when installed.
You know, like: increase the number of inodes before running this service, etc…
As said, I think we should find out first if users turn of syncthing due to performance and how often. And then, decide if something should be done about it.
I’m syncing ~100GB in 25k files to my laptop (Core i5) and never turn off Syncthing. It’s designed to be a continuous file sync service, and that’s how I use it. I don’t want to have to “boost” it or manually give it priority, I want it to just work in the background.
If you’re getting excessive CPU and/or I/O then you need to find out why. This should only be happening if a lot of data is being synced, e.g. lots of files being added or one large file being modified. If this is happening frequently, perhaps a different copying solution would be more suitable?
Instead of using Syncthing to sync a frequently-busy folder, why not script rsync to operate when the system is on AC power and/or idle, or at a set time? Or you could do similar to pause/unpause devices and/or folders when you need to preserve battery - which could be done automatically.
Again, lean and targeted is the reason we don’t want feature bloat.
Adding this would open a can of worms justifying why cron scheduling should be part of syncthing etc, and after that people would feel they have the mandate to ask for any feature, like shutting down computers and stuff.
I am sure you are clever enough to write a little script that monitors whatever metric you are interested in (even if zoom is currently in foreground) and calls pause/unpause rest end points.
The beauty of syncthing that it’s easy to integrate and interact with it, so more focus should be there, instead of bloating core product.
Heck, even inotify support started off as an external application, and everyone was happy with that.
Be the hero that we need, write a small utility that does what you are asking for, see if users use it, and if they really do, perhaps then make an argument why it should make it to core.
Trying to do polls in a random forum thread is never going to be a good enough indicator that people want this
None of my servers or clients ever stopped syncthing. I don’t want to do that in the future either, because I don’t want any manual intervention. I am a friend of the automatic and reliable processes.
