High memory usage

I’ve recently started using Syncthing to sync between a Synology NAS with 2x Seagate IronWolf 12 TB HDDs configured as RAID-1 and a Raspberry Pi 4B with 2x Seagate Backup Plus 5 TB portable drives configured as a single disk with striping using LVM. There are six shared folders with a total of 6.15TB in over 333,000 files.

I use the stable Syncthing Debian repo and just upgraded both to 1.9.0. They both started doing to do lots of scanning and syncing with the Synology NAS doing an excessive amount of disk grinding and actually going a lot slower than the Raspberry Pi. It turned out that that was due to the system running out of physical memory and swapping a lot. The syncthing process seemed to be using nearly 2 GB of memory. Is that to be expected given the amount of data I’m sharing?

I am not an expert, but the RAM usage seems pretty normal to me. For comparison, mine is using ~500 MB of RAM for ~300GB of data (667,623 files). It also probably depends on the number of folders, files, and connected devices.

This is with the 64-bit version, as the 32-bit one will probably use less (but also may crash if the RAM usage gets too high).

Post upgrade it does a ton of database work. So yes, its possible that for 6TB of data, it needs more than 2GB.

@markshep, can you please try what I wrote here (pause devices while scanning to prevent transfers during scan) and see if it helps.

Sounds like this memory usage is expected. Luckily my particular model of NAS has upgradable memory so I’ll order some more.

When this was happening there were periods where it was transferring and scanning at the same time, like uok describes, which wouldn’t have been helping on top of the swap usage too!

Also I’ve seen Syncthing scanning multiple folders at once so I’ve now set the “Max Folder Concurrency” to 1 too, which is a good tip.

Thanks to everyone for the help.

1.9.0 has increased memory usage for folders with many files, due to case sensitivity checks. We’re bringing that down a bit in 1.10.0. You might also set the case sensitivity option. (If your file systems are all in fact case sensitive.)

I found an advanced setting Case Sensitive FS for each folder, is this what needs to be checked to return to prior 1.9.0 (or similar) behavior? Or is there a central option? Thanks!

It does return to the older behaviour, and I do not think that there is a central option. You could mass-edit the config.xml file though.