No, obviously, I do have a lot of files - and there is a lot of syncing still going on - but I’d be grateful to know if there are any other measures I can take to reduce my memory footprint.
I’ve grabbed a couple of heap profiles - are they safe to upload publicly without exposing private data?
Just a tip - whatever you do, don’t split your one massive folder into lots of small folders. It makes things worse.
I have ~150gb of backups that I’m syncing around spread over 36 folders, and when you have multiple folders on the same slow disk, SyncThing scans all the folders simultaneously, resulting in long disk queues and a complete inability to get anything done.
Also, this isn’t just my development machine. If anything goes wrong on the production database server, my first step is to stop SyncThing because 90% of the problems I’ve had with it have been traced back to SyncThing either using all the disk bandwidth or all the CPU or all the memory. And then of course, when I restart SyncThing it does a scan. So I have to restart SyncThing, pause all the folders, and then restart them one at a time to get it back on its feet.
I was actually considering changing the configuration to “one huge folder” rather than “lots of big folders” but I guess I won’t do that now.
I just want SyncThing to run in the background moving databases off the database servers and builds onto the web servers and to never get in the way of any of my foreground activities. I don’t want it to slow down either my production database server or my development machine whilst it does its thing. I don’t care if it takes twice as long to sync, as long as it does so reliably, and without affecting performance.
[Note - of course on my production server the backups are on a different physical drive to the databases and the logs and the system so SyncThing causing massive disk queues shouldn’t affect performance. But it does. Not sure how.]
A significant chunk is ignore caching. You should turn that off (it’s off by default as well). Another big chunk is file metadata while scanning. I’m not sure why that is, will look into it.
Can you post your config? The whole thing looks a bit odd to me, you may be running into some corner/edge case we haven’t thought of.
